DTrace BoF at LISA10

Interested in DTrace ? If so, you have to watch these videos and see how smart Brendan Gregg used mdb and dtrace to hack those TCP DTrace scripts without looking at the kernel source code.

Merger of the Merger

I was given 50+ EMC Navisphere performance data and asked to merge them into a single data file for Navisphere to import. The bad news is the existing EMC Navisphere Analyzer Command Line Interface (naviseccli) can only allow to merge 2 files at a time.
naviseccli analyzer -archivemerge -data archive1 archive2 [-out outputarchive] [-overwrite y|n]

Working with this limitation, I developed a merger shell to merge all these performance data. Assuming you have the following performance files with timestamp builtin in the filename, my approach is to generate the 1st merger file based on the first two files. After that, I will loop through the rest (starting from the 3rd file) of them together with the merger file to form a new archive. You will need to have a temporary file to house the intermediate archive and swap it before the next loop.

# ls -1
CKM12345678901_SPA_2010-10-24_17-00-41-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-24_19-00-41-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-24_21-00-41-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-26_20-00-55-GMT_P08-00.nar
...
CKM12345678901_SPA_2010-10-29_10-00-46-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-29_12-00-46-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-29_14-00-47-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-29_16-00-46-GMT_P08-00.nar
CKM12345678901_SPA_2010-10-29_18-00-46-GMT_P08-00.nar

Here is the merger.sh script

#! /bin/ksh


if [ $# -ne 1 ]; then
        echo "Usage: $0 <mergefile>"
        exit 1
fi
mergefile="$1"
if [ -f $mergefile ]; then
        echo "Error. $file already exist"
        exit 2
fi


export PATH=/opt/Navisphere/bin:/usr/bin:/bin:/usr/sbin:/usr/local/bin
export LD_LIBRARY_PATH=/opt/Navisphere/lib:/usr/lib:/lib:/usr/local/lib


tmpls=`mktemp`
tmpmerge=`mktemp`


ls -1 CKM*.nar > $tmpls


#
# do the first 2 files
#
file1=`head -1 $tmpls`
file2=`head -2 $tmpls | tail -1`
echo "Processing $file1 $file2 ... \c"
naviseccli analyzer -archivemerge -data $file1 $file2 -out $mergefile
echo "Completed."


#
# do the rest of the files
#
awk 'NR>2&&NR<=10' $tmpls | while read fileN
do
        echo "Processing $fileN ... \c"
        naviseccli analyzer -archivemerge -data $mergefile $fileN -out $tmpmerge
        mv $tmpmerge $mergefile
        echo "Completed."
done


rm -f $tmpls $tmpmerge

See merger.sh in action:

# ./merger.sh new.nar
Processing CKM12345678901_SPA_2010-10-24_17-00-41-GMT_P08-00.nar CKM12345678901_SPA_2010-10-24_19-00-41-GMT_P08-00.nar ... Completed merging archives.
Completed.
Processing CKM12345678901_SPA_2010-10-24_21-00-41-GMT_P08-00.nar ... Completed merging archives.
Completed.
Processing CKM12345678901_SPA_2010-10-26_20-00-55-GMT_P08-00.nar ... Completed merging archives.
Completed.
...
Processing CKM12345678901_SPA_2010-10-29_10-00-46-GMT_P08-00.nar ... Completed merging archives.
Completed.
Processing CKM12345678901_SPA_2010-10-29_12-00-46-GMT_P08-00.nar ... Completed merging archives.
Completed.
Processing CKM12345678901_SPA_2010-10-29_14-00-47-GMT_P08-00.nar ... Completed merging archives.
Completed.
Processing CKM12345678901_SPA_2010-10-29_16-00-46-GMT_P08-00.nar ... Completed merging archives.
Completed.
Processing CKM12345678901_SPA_2010-10-29_18-00-46-GMT_P08-00.nar ... Completed merging archives.
Completed.