Finding Newer Files

Although find provides flags to locate files newer than certain days (-mtime -2), it is not fine grain enough to allow user to specify based on user-defined timestamp. Below script will touch a temporary file with specific timestamp and find will make use of it as a reference by using the "-newer" flag. FYI, this has been tested on Solaris

#! /bin/ksh

if [ $# -ne 2 ]; then
        echo "Usage: $0 directory time"
        echo "       time - a decimal number of the form: [[CC]YY]MMDDhhmm[.SS]"
        exit 1
fi
directory=$1
timestamp=$2


if [ ! -d $directory ]; then
        echo "Error. Directory $directory does not exist"
        exit 2
fi


tmpfile="/tmp/${0##*/}-$$"
touch -t $timestamp $tmpfile > /dev/null 2>&1
if [ $? -ne 0 ]; then
        echo "Error. Incorrect time format [[CC]YY]MMDDhhmm[.SS]"
        exit 2
fi


trap "rm -f $tmpfile; exit 0" EXIT


find $directory -newer $tmpfile -local -mount -type f

What Is Personal Leadership Brand

If you have 10 minutes to spare, you should watch this video from Harvard Business in YouTube

Thumbnail Your RRDtool Graphs

If you have lots of RRDtool graphs you need to go through everyday, you may want to consider creating thumbnail dynamically from the RRD files. As CPU and memory utilisations are plotted with same scale (100%), we can put them together for ease of viewing and analysis.

A simple CGI script is able to do the job. Basically it ignores x- & y-grids/ticks/labels and legends

echo "Content-tyep: image/png"
echo ""
/usr/bin/rrdtool graph - \
        --start -${2:-1d} \
        --imgformat PNG \
        --lower-limit 0 --upper-limit 100 --rigid \
        --width=125 --height=30 \
        --x-grid none \
        --y-grid none \
        --title "$host" \
        --color=BACK#ffffff \
        --color=CANVAS#ffeee5 \
        --color=GRID#7F7F7F \
        --color=MGRID#B8B8B8 \
        --color=ARROW#FF0000 \
        DEF:usr=$rrd_cpu:usr:AVERAGE \
        DEF:sys=$rrd_cpu:sys:AVERAGE \
        DEF:wio=$rrd_cpu:wio:AVERAGE \
        DEF:idle=$rrd_cpu:idle:AVERAGE \
        DEF:mtotal=$rrd_mem:total:AVERAGE \
        DEF:mused=$rrd_mem:used:AVERAGE \
        CDEF:gbtotal=mtotal,1024,/ \
        CDEF:gbused=mused,1024,/ \
        CDEF:gbfree=mtotal,mused,-,1024,/ \
        CDEF:percent=gbfree,gbtotal,/,100,* \
        AREA:usr#dddd00:\
        STACK:sys#dd0000:\
        STACK:wio#ff8a60:\
        STACK:idle#e2e2f2:\
        LINE1:percent#0000ff: \
        GPRINT:gbfree:LAST:"Mem\:%5.1lf/" \
        GPRINT:gbtotal:LAST:"%5.1lfGB"

New DTrace Resources

Some new resources from Sun on DTrace

• DTrace Toolkit
• DTraceQuick StartGuide, Observing Native andWeb Applications in Production
• Observing Applications with DTrace (video)

Scripting Hustle

I was given this compliment:
不愧是万中无一的scripting guy
+ learn a lifetime skill of scripting for $10

Watch this movie clip (fast forward to 08:40) to find out the resemblance

Too Big An Audit Log To Handle

Have you ever encounter Solaris audit log that has not been rotated for months ? It is just far too big for some of the down stream programs to handle. Click here for a sample of the output from praudit -l. Here is a sample script to 'chop' them into individual files based on day. With this simple script, now you can handle audit log with ease.

IFS_orig=$IFS
cd /var/audit
praudit -l 2009052803069.20091009095022.myhost | while read line
do
        IFS=","
        set -- $line
        d=$7
        IFS=$IFS_orig
        set -- $d
        ymd=$1
        echo $line >> $ymd.txt
done

Number of Days in a Month

If you have to loop through everyday in a month to generate daily summary, you may want to know how many days in a particular month. It may not be as straightforward as you think because you need to take into account of leap year, short months, and so on. Here is a simple trick that can help you on this. Run the cal command to display the calendar and pipe it to awk to pick up the last number from the last non-empty line.

$ cal
    October 2009
Su Mo Tu We Th Fr Sa
             1  2  3
 4  5  6  7  8  9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

$ cal | awk 'NF>0{n=$NF}END{print n}'
31

$ cal 2 2012
    February 2012
Su Mo Tu We Th Fr Sa
          1  2  3  4
 5  6  7  8  9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29

$ cal 2 2012 | awk 'NF>0{n=$NF}END{print n}'
29

Date Format, N Days Ago

In UNIX shell script, it is very difficult to manipulate date/time using standard UNIX commands. Modern programming languages such as Perl, Python, Tcl, ..., are designed to be a general programming/scripting language that can handle this type of task with ease.

Below shows shell functions that wrap around these general scripting languages to determine the date format that is N days ago.

ago_perl()
{
        # localtime return:
        #       Seconds past the minute
        #       Minutes past the hour
        #       Hours past midnight
        #       Day of the month
        #       Months past the start of the year
        #       Number of years since 1900
        #       Number of days since the start of the week (Sunday)
        #       Number of days since the start of the year
        #       Whether or not daylight savings is active
        days=${1:-1}
        perl -e "@t=localtime(time-86400*$days);\
                printf('%4d-%02d-%02d', @t[5]+1900, @t[4]+1, @t[3])"

}
ago_tcl()
{
        days=${1:-1}
        echo "puts [clock format [expr [clock seconds] - $days*86400] \
                -format {%Y-%m-%d}]" | tclsh
}
ago_python()
{
        days=${1:-1}
        python -c "import time, datetime
t=datetime.datetime.fromtimestamp(time.time()-86400*$days)
print '%4d-%02d-%02d' % (t.year,t.month,t.day)"
}

Here is how you can use any of these functions

$ ago=`ago_perl 45`

$ echo "45 days ago is $ago"
45 days ago is 2009-08-12

Monitoring of LTO Tape Drives via FC Switch

I was trying to monitor the throughput of individual LTO tape drive but not getting any useful information from the tape library system. The only way to overcome this is to monitor it via the FC switch. In my case, it is a Brocade switch. Online SNMP MIBs for Brocade provides a very detailed description of the OIDs (swFCPortRxWords:1.3.6.1.4.1.1588.2.1.1.1.6.2.1.12 and swFCPortTxWords: 1.3.6.1.4.1.1588.2.1.1.1.6.2.1.11).

Due to the fact that the counter reset so frequent, I need to poll the FC switch every minute. Data is stored in RRDtool format.

Below shows two ways to visualise the LTO throughput. Line grpah shows individual drive throughput. Area/Stack graph shows the overall throughput of the backup:

John Ousterhout is in Stanford

The creator of Tcl/Tk is now in Stanford University. Some links for those who want to find out more:

• His Home Page in Stanford
• His Personal Home Page
• Chairman of the Board, Electric Cloud