Wednesday, November 28, 2007

Change Your Way In Looking At Web Application Performance

Google TechTalks, High Performance Web Sites and YSlow

This video will definitely change your way in looking at web application performance.

Labels: ,

Saturday, November 24, 2007

If everybody gets the same answer, it has to be right. Is it ? Part 2

Why I bothered to blog about this ? Reasons:
  1. In the school examinations, students are given back the examin papers. However, it is not the case for the PSLE (Primary School Leaving Examination. That means if the kids are given this type of questions, it is definitely not fair to those who spend the effort but mark it wrong. It is the examination committee's responsibility to ensure the correctness of the questions as well as the corresponding answers. In PSLE, every mark counts. That extra mark will determine whether you get selected to a particular secondary school of your choice. I hope whatever examination committee will take extra effort to ensure everything in the exam paper is in order
  2. I wanted to find out what kind of tolerance can we give with this type of exam question. Suppose we allow tolerance of ± 1 minor marking on the ruler and protractor, we will be talking about 6.9-7.1 for 7cm, 9.9-10.1 for 10cm, and 29.9-30.1 for 30°. With Sun Studio 12 from Sun Microsystems, you can write a program to work that out based on Interval Arithmetic. Instead of providing the exact value to the variable, you can give them a range. The output will be given in range too. Below shows the Fortran 95 code and result:
    $ cat a1.f95
    INTERVAL :: ANG = [29.9, 30.1]
    INTERVAL :: EF = [6.9, 7.1]
    INTERVAL :: FG = [9.9, 10.1]
    INTERVAL :: GW
    INTERVAL :: EW
    INTERVAL :: E
    INTERVAL :: F
    REAL :: PI = 3.14159265
    
    GW=FG*sin(ANG*PI/180)
    EW=FG*cos(ANG*PI/180)-EF
    ALPHA=atan(GW/EW)
    ANSWER=(PI-ALPHA)*180/PI
    
    PRINT *, ANSWER
    
    END
    
    $ f95 -xia -xarch=amd64 a1.f95
    
    $ ./a.out
     [106.1311886713034,110.60714019410637]
    

So the answer should be between 106 to 111, if we round it off to the nearest integer

Think about it. If everybody is doing the same thing, it has to be right. Is it ?

Labels:

If everybody gets the same answer, it has to be right. Is it ?

I was on leave yesterday and happened to have time to go through my son's final mathematics examination paper (2nd Semester Assessment in 2007, Primary 5) . He scored 86 out of 100 and I can see he improved quite a lot. Normally I will go through his assessments and focused on those that are incorrect. One of the questions in that paper got my attention. It seemed everybody is correct, however in reality it is not. My son was telling me that most of the students got the 'correct answer' which is 110. So, the question is: Is my son not careful enough to measure it correctly and that lead him to an incorrect answer ? Let's do the CIS work.

The question is:
Draw a triangle EFG in which EF=7cm, angle EFG=30 degree and FG=10cm. Measure angle FEG.
The student supposed to use ruler and protractor to accomplish the task. Below showed the examine question with my son's working on the left and the 'model answer' in green copied from the whiteboard. Obviously the red color cross is marked by the teacher. My son was telling me that any answer between 109 and 110 is considered to be correct, ie. with 1 degree tolerance. Fair enough.

So I took the ruler and protractor to work it out on my own to see whether I can get the answer within the same tolerance level. I got the same answer as my son. Wait a minute, this can't be right 'cos I took extreme precaution to ensure I measure extactly 7cm, 10cm and 30 degree.

Okay, why not use trigonometry to work out the exact answer. See below for proof.

Hey, the correct answer should be 108.37. If we were to round to the nearest degree, it should be 108. If we were to based on the tolerance level given by the teacher, the answer should be between 107 and 109. Definitely 110 is not the correct answer.

I even used Geogebra to verify it.

Labels:

Friday, November 23, 2007

Script to Download

Today, my colleague came to me when I was about to knock off (5 minutes before 6 !). He was telling me that his account in the Sun Downloads cannot allow him to download the Application Server. I have no idea why his account cannot work, but mine worked perfectly. The "Application Server 7 Update 9 SE - Solaris Sparc Bundle, English" is about 126.52MB and it is going to take a while to download. Our office's internet connection is really slow, and it will take you at least an hour to download, if you are lucky.

Of course I wanted to help my colleague, on the other hand I do not want to stay back either. Also, I do not want to leave my notebook unattended in the office. If I would capture the HTTP headers at the final click, I should be able replay that as a script. With this approach, the script does not have to worry about session. I launched LiveHTTPheaders to assist me to capture the HTTP headers. Below is the screen capture:

To prove that my concept works, I captured all these HTTP headers and put them in a script to program Wget to download it for me in a Solaris server. Instead of running the script with nohup and puting it as a background job, I ran that as a batch process using "at now". If you were to run with 'nohup command ... &', you wouldn't be able to logout cleanly from your connection. Here is how I scripted the whole thing

#! /bin/sh

h1='Host: sdlc-esd.sun.com'
h2='User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9'
h3='Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5'
h4='Accept-Language: en-us,en;q=0.5'
h5='Accept-Encoding: gzip,deflate'
h6='Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7'
h7='Cookie: s_cc=true; s_sq=; SessionCredentials=6664883992855580291'
url='http://sdlc-esd.sun.com/ESD24/Java_Systems/AS7U9/sun-appserver7-solaris-sparc-ur9.tar.gz?AuthParam=1195746732_f8c9c901546df8b1985446441554100c&TUrl=an1npDpbKod7kSYrROhENTonIuc5W0D1Lc4nXz+pGFFranixdCdgxDTPbW4=&TicketId=dVN4Og5LN+k6+w==&GroupName=SDLC&BHost=sdlc3h.sun.com&FilePath=/ESD24/Java_Systems/AS7U9/sun-appserver7-solaris-sparc-ur9.tar.gz&File=sun-appserver7-solaris-sparc-ur9.tar.gz'

wget \
--header="$h1" \
--header="$h2" \
--header="$h3" \
--header="$h4" \
--header="$h5" \
--header="$h6" \
--header="$h7" \
-O sun-appserver7-solaris-sparc-ur9.tar.gz \
"$url"

Shell script saved the day.

Labels: ,

Saturday, November 17, 2007

find, An Interview Question

A colleague of mine was one of the job interviewers and he posted the following question to the interviewee: "Can you tell me how you can remove files that have not been modified for more than 30 days?"

Sad to say that the interviewee was not able to provide the exact command. Sometime it is just not possible to remember those cryptic syntax of UNIX commands. What if the interviewee is able to answer the question, does it mean that he/she know 'find' command well enough ? Also, is the interviewee capable of doing other system administrative jobs? If I were the interviewer, I would follow up with more questions:

  1. Can you tell me how you can remove files that have not been modified for more than 30 days? (original question)
  2. Can you tell me how to find those files that was modified between the last 30 and 60 days?
  3. Can you locate those files that are more than 200 days old, but last modified on the weekends?
  4. Can you locate those files that was last modified on 4 July 2007 ?

Let's create tonnes of files (365 of them) for our experiment using this Tcl script

set now [clock seconds]
for { set i 1 } { $i <= 365 } { incr i } {
    set time [clock format [expr {$now-86400*$i}] -format {%Y%m%d%H%M}]
    exec touch -t $time day-$i
}

    My answers to the above questions:
  1. find . -type f -mtime +30 -exec rm -i {} \; In this case, I include -i in 'rm' to let the system to prompt me before I delete the files. If you are dealing with a lot of files, you may want to use 'rm -f'. Make sure you are deleting the files that you are suppose to delete.
  2. find . -type f \( -mtime +30 -a -mtime -60 \) -ls
  3. find . -type f -mtime +200 | ./wk 'wk' is a simple Tcl script to determine whether the file is last modified on weekend
    while { [gets stdin line] >= 0 } {
          file stat $line fstat
          set wk [clock format $fstat(mtime) -format %a]
          if { [string equal $wk Sun] || [string equal $wk Sat] } {
                  puts $line
          }
    }
    
  4. You may think that 'ls -lR | grep "4 July"' is the solution. However, this may not be fool proof 'cos if the file is more than 180 days old, 'ls -l' will show the "4 July 2007" time format. If not, it will show "4 July 17:11" time format. That means, you may have more than one occurrence of "4 July" in the 'ls -lR'. Worse still, you can pick up files modified on "14 July" or "24 July"

    What you need is a program to calculate the number of days old and use the output as the input to find. A sample Tcl program (dayold) like this will do the job

    set t [clock scan [lindex $argv 0]]
    set now [clock seconds]
    set days [expr ($now-$t)/86400]
    puts $days
    
    Below shows you how it can be combined with find
    $ ./etime 2007-07-04
    136
    
    $ find . -mtime `./etime 2007-07-04`
    ./day-136
    
    $ ls -l ./day-136
    -rw-------   1 chihung  gdz            0 Jul  4 17:11 ./day-136
    

If I were the interviewer, I would expect the interviewee (if he/she is apply for a system admin job) to be able to program in scripting language. It really does not matter what language as long as he/she can master that language well.

Labels: ,

Friday, November 16, 2007

Solaris Operating System Package List

We often want to find out what packages are included in a particular Solaris version and update. Sometime, you would like to know what is the difference between different distributions. In Solaris, you have a choice to install any of the distributions: "Reduced Network" (new in Solaris 10), "Core", "End User", "Developer", "Entire" or "Entire OEM" distribution.

Below are the links to the various updates of Solaris 10

Labels:

Tuesday, November 06, 2007

Is it 32-bit or 64-bit

How to find whether an executable or a running process is 32-bit or 64-bit. I will show you in Solaris as well as in Linux. It will be based on this simple C program.
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
        unsigned int t;
        t=(unsigned int)atoi(argv[1]);
        sleep(t);
        exit(0);
}

In Solaris, you can differentiate it from either the dynamic dependencies of the executable or the address space of a running process

$ uname -a
SunOS myserver 5.10 Generic_118833-36 sun4u sparc SUNW,UltraSPARC-IIi-cEngine

$ isainfo -v
64-bit sparcv9 applications
        vis
32-bit sparc applications
        vis v8plus div32 mul32

$ which gcc
/usr/sfw/bin/gcc

$ gcc --version
gcc (GCC) 3.4.3 (csl-sol210-3_4-branch+sol_rpath)
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ gcc -o a64 -m64 a.c

$ gcc -o a32 -m32 a.c

$ ldd a32
        libc.so.1 =>     /lib/libc.so.1
        libm.so.2 =>     /lib/libm.so.2

$ ldd a64
        libc.so.1 =>     /lib/64/libc.so.1
        libm.so.2 =>     /lib/64/libm.so.2

$ ./a32 100 &
[1] 3392

$ pmap 3392
3392:   ./a32 100
00010000       8K r-x--  /export/home/chihung/tmp/a32
00020000       8K rwx--  /export/home/chihung/tmp/a32
FF280000     864K r-x--  /lib/libc.so.1
FF368000      32K rwx--  /lib/libc.so.1
FF370000       8K rwx--  /lib/libc.so.1
FF3A0000      24K rwx--    [ anon ]
FF3B0000     184K r-x--  /lib/ld.so.1
FF3EE000       8K rwx--  /lib/ld.so.1
FF3F0000       8K rwx--  /lib/ld.so.1
FFBFE000       8K rwx--    [ stack ]
 total      1152K

$ ./a64 100 &
[2] 3394

$ pmap 3394
3394:   ./a64 100
0000000100000000          8K r-x--  /export/home/chihung/tmp/a64
0000000100100000          8K rwx--  /export/home/chihung/tmp/a64
FFFFFFFF7F000000          8K rwx--    [ anon ]
FFFFFFFF7F100000         24K rwx--    [ anon ]
FFFFFFFF7F200000        920K r-x--  /lib/sparcv9/libc.so.1
FFFFFFFF7F3E6000         64K rwx--  /lib/sparcv9/libc.so.1
FFFFFFFF7F3F6000          8K rwx--  /lib/sparcv9/libc.so.1
FFFFFFFF7F500000          8K rwx--    [ anon ]
FFFFFFFF7F600000        176K r-x--  /lib/sparcv9/ld.so.1
FFFFFFFF7F72C000         16K rwx--  /lib/sparcv9/ld.so.1
FFFFFFFF7FFFE000          8K rw---    [ stack ]
         total         1248K

In Linux, you can do the same as what we did for Solaris.
CentOS-5 32-bit

$ uname -a
Linux myserver 2.6.18-8.1.8.el5 #1 SMP Tue Jul 10 06:50:22 EDT 2007 i686 i686 i386 GNU/Linux

$ gcc --version
gcc (GCC) 4.1.1 20070105 (Red Hat 4.1.1-52)
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ gcc -m32 -o a32 a.c

$ ./a32 100 &
[1] 4305

$ ldd a32
        linux-gate.so.1 =>  (0x008ef000)
        libc.so.6 => /lib/libc.so.6 (0x0070a000)
        /lib/ld-linux.so.2 (0x006e8000)

$ cat /proc/4305/maps
00477000-00478000 r-xp 00477000 00:00 0          [vdso]
006e8000-00701000 r-xp 00000000 fd:00 7307266    /lib/ld-2.5.so
00701000-00702000 r-xp 00018000 fd:00 7307266    /lib/ld-2.5.so
00702000-00703000 rwxp 00019000 fd:00 7307266    /lib/ld-2.5.so
0070a000-00841000 r-xp 00000000 fd:00 7310310    /lib/libc-2.5.so
00841000-00843000 r-xp 00137000 fd:00 7310310    /lib/libc-2.5.so
00843000-00844000 rwxp 00139000 fd:00 7310310    /lib/libc-2.5.so
00844000-00847000 rwxp 00844000 00:00 0
08048000-08049000 r-xp 00000000 fd:00 1444299    /home/chihung/a32
08049000-0804a000 rw-p 00000000 fd:00 1444299    /home/chihung/a32
b7fa6000-b7fa7000 rw-p b7fa6000 00:00 0
b7fbf000-b7fc0000 rw-p b7fbf000 00:00 0
bfb33000-bfb48000 rw-p bfb33000 00:00 0          [stack]

RHEL3 64-bit
$ gcc --version
gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-52)
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ uname -a
Linux myserver 2.4.21-32.ELsmp #1 SMP Fri Apr 15 21:03:28 EDT 2005 x86_64 x86_64 x86_64 GNU/Linux

$ gcc -m64 -o a64 a.c

$ ./a64 100 &
[1] 21632

$ ldd a64
        libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9567c000)
        /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x0000002a95556000)

$ cat /proc/21632/maps
0000000000400000-0000000000401000 r-xp 0000000000000000 08:02 1255888  /home/chihung/a64
0000000000500000-0000000000501000 rw-p 0000000000000000 08:02 1255888  /home/chihung/a64
0000002a95556000-0000002a9566b000 r-xp 0000000000000000 08:02 537723  /lib64/ld-2.3.2.so
0000002a9566b000-0000002a9566c000 rw-p 0000000000015000 08:02 537723  /lib64/ld-2.3.2.so
0000002a9566c000-0000002a9566d000 rw-p 0000000000000000 00:00 0
0000002a9567c000-0000002a957b7000 r-xp 0000000000000000 08:02 944733  /lib64/tls/libc-2.3.2.so
0000002a957b7000-0000002a958b7000 ---p 000000000013b000 08:02 944733  /lib64/tls/libc-2.3.2.so
0000002a958b7000-0000002a958bc000 rw-p 000000000013b000 08:02 944733  /lib64/tls/libc-2.3.2.so
0000002a958bc000-0000002a958c1000 rw-p 0000000000000000 00:00 0
0000007fbfff9000-0000007fc0000000 rw-p ffffffffffffb000 00:00 0

Labels: ,

Thursday, November 01, 2007

CentOS 5 in openSolaris xVM

As you already know that I installed openSolaris build 75 recently and promised to blog more about the xVM. Good news, I managed to install CentOS 5 after a lot of attempts based on CDROM, Hard Disk, NFS, ...

I copied ISO images of CentOS5, openSUSE 10.3, and Solaris 10 U4 to the SunFire X4600. I tried to install a guest xvm on top of the openSolaris using DVD and ISO, but no luck in any of these. Same for NFS installation, the "virt-install" command just complained that it cannot find the NFS server. However, I was able to mount it manually.

Today, I tried the HTTP method and CentOS 5 worked but the others just failed. The CentOS 5 ISO was mounted using the LOFI ("lofiadm -a") followed by "mount -F hsfs". The mount point will be the "DocumentRoot" for the Apache web server. Below shows some of the details:

Make sure you boot up openSolaris xVM (this is not the default)

root@opensolaris# uname -a
SunOS opensolaris 5.11 snv_75 i86pc i386 i86xpv

root@opensolaris# svcs -a | grep xvm
online         Oct_30   svc:/system/xvm/store:default
online         Oct_30   svc:/system/xvm/xend:default
online         Oct_30   svc:/system/xvm/console:default
online         Oct_30   svc:/system/xvm/domains:default

root@opensolaris# ps -ef | grep xen
    root   336     1   0   Oct 30 ?           0:01 /usr/lib/xenstored --pid-file=/var/run/xenstore.pid
    root   418     1   0   Oct 30 ?           0:01 /usr/lib/xenconsoled start
    root   366     1   0   Oct 30 ?           0:30 python /usr/lib/xend start

For installation, you need "virt-install" (install guest operating system on xVM system) and "virsh" (management user interface for guest domains). You can either supply all the necessary arguments to virt-install. If insufficient details in creating the domain, it will prompt you for the details. By default, 1 CPU will be allocated to the guest domain. In this exercise, I forced it to use 4 'cos I have 16 CPUs in the server. You can always shutdown the domain and change the number of vcpu. In this domain, I allocated 4 CPUs, 4 GB memory and 10GB of disk space. The whole domain of 10GB will be housed in 1 file and you will see that this file has been used as lofi (loop back file system).

root@opensolaris# virt-install -vcpus=4
What is the name of your virtual machine? centos5
 How much RAM should be allocated (in megabytes)? 4096
 What would you like to use as the disk (path)? /data/xvm/centos5.xvm
 How large would you like the disk (/data/xvm/centos5.xvm) to be (in gigabytes)? 10
 Would you like to enable graphics support? (yes or no) no
 What is the install location? http://10.0.12.251/centos5/
....
....
[answer all the questions to setup the CentOS5]

root@opensolaris# xm list
Name                                      ID   Mem VCPUs      State   Time(s)
Domain-0                                   0 27759    16     r-----   7321.9
centos5                                    7  4096     4     -b----      7.3

root@opensolaris# xm list -l centos5
(domain
    (domid 7)
    (on_crash restart)
    (memory 4096)
    (uuid 3816e1f7-5e95-1841-0adc-5fc4a5360aa5)
    (bootloader_args )
    (name centos5)
    (maxmem 4096)
    (on_reboot restart)
    (on_poweroff destroy)
    (localtime 0)
    (vcpus 4)
    (bootloader /usr/lib/xen/bin/pygrub)
    (shadow_memory 0)
    (cpu_weight 256)
    (cpu_cap 0)
    (features )
    (on_xend_start ignore)
    (on_xend_stop shutdown)
    (start_time 1193889884.81)
    (cpu_time 7.283464146)
    (online_vcpus 4)
    (image (linux (kernel )))
    (status 2)
    (memory_dynamic_min 4096)
    (memory_dynamic_max 4096)
    (state -b----)
    (store_mfn 7498403)
    (console_mfn 7498402)
    (device
        (vif
            (bridge )
            (mac 00:16:3e:14:84:e5)
            (uuid b7628569-99c7-6b55-d120-13c4e0b5630d)
        )
    )
    (device
        (vbd
            (uname file:/data/xvm/centos5.xvm)
            (driver paravirtualised)
            (mode w)
            (dev xvda)
            (uuid 6666b6c5-6667-a8cf-17fd-52a7374881c7)
        )
    )
)

root@opensolaris# ls -l /data/xvm/centos5.xvm
-rwxr-xr-x   1 root     root     10737418240 Nov  1 13:26 /data/xvm/centos5.xvm

root@opensolaris# lofiadm
Block Device             File
/dev/lofi/1              /data/xvm/centos5.xvm

Now the guest domain is ready, let us login via the "xm" command and explore whether the new guest domain has been configured properly. Ctrl-] is the keyboard sequence to break out from the xvm.

root@opensolaris# xm console centos5
Bootdata ok (command line is ro root=/dev/VolGroup00/LogVol00 console=xvc0)
Linux version 2.6.18-8.el5xen (mockbuild@builder6.centos.org) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP Thu Mar 15 19:56:43 EDT 2007
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000100800000 (usable)
No mptable found.
Built 1 zonelists.  Total pages: 1050624
Kernel command line: ro root=/dev/VolGroup00/LogVol00 console=xvc0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Xen reported: 2613.400 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Software IO TLB disabled
Memory: 4096768k/4202496k available (2321k kernel code, 96972k reserved, 1312k data, 168k init)
Calibrating delay using timer specific routine.. 6538.09 BogoMIPS (lpj=13076195)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU: Physical Processor ID: 7
CPU: Processor Core ID: 1
(SMP-)alternatives turned off
Brought up 1 CPUs
checking if image is initramfs... it is
Grant table initialized
NET: Registered protocol family 16
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread 377A0 could not acquire Mutex [2] [20060707]
Initializing CPU#1
migration_cost=1074
migration_cost=1074Initializing CPU#2

migration_cost=1074
Brought up 4 CPUs
Initializing CPU#3
PCI: setting up Xen PCI frontend stub
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI: disabled
xen_mem: Initialising balloon driver.
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
IA-32 Microcode Update Driver: v1.14-xen <tigran@veritas.com>
audit: initializing netlink socket (disabled)
audit(1193918686.777:1): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
SELinux:  Registering netfilter hooks
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key 991108A46E99ED4B
- User ID: CentOS (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
rtc: IRQ 8 is not free.
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
Xen virtual console successfully installed as xvc0
Bootdata ok (command line is ro root=/dev/VolGroup00/LogVol00 console=xvc0)
Linux version 2.6.18-8.el5xen (mockbuild@builder6.centos.org) (gcc version 4.1.1 20070105 (Red Hat 4.1.1-52)) #1 SMP Thu Mar 15 19:56:43 EDT 2007
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000100800000 (usable)
No mptable found.
Built 1 zonelists.  Total pages: 1050624
Kernel command line: ro root=/dev/VolGroup00/LogVol00 console=xvc0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Xen reported: 2613.400 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Software IO TLB disabled
Memory: 4096768k/4202496k available (2321k kernel code, 96972k reserved, 1312k data, 168k init)
Calibrating delay using timer specific routine.. 6538.09 BogoMIPS (lpj=13076195)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU: Physical Processor ID: 7
CPU: Processor Core ID: 1
(SMP-)alternatives turned off
Brought up 1 CPUs
checking if image is initramfs... it is
Grant table initialized
NET: Registered protocol family 16
ACPI Exception (utmutex-0262): AE_BAD_PARAMETER, Thread 377A0 could not acquire Mutex [2] [20060707]
Initializing CPU#1
migration_cost=1074
migration_cost=1074Initializing CPU#2

migration_cost=1074
Brought up 4 CPUs
Initializing CPU#3
PCI: setting up Xen PCI frontend stub
ACPI: Interpreter disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI: disabled
xen_mem: Initialising balloon driver.
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: System does not support PCI
PCI: System does not support PCI
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
IA-32 Microcode Update Driver: v1.14-xen <tigran@veritas.com>
audit: initializing netlink socket (disabled)
audit(1193918686.777:1): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
SELinux:  Registering netfilter hooks
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key 991108A46E99ED4B
- User ID: CentOS (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
rtc: IRQ 8 is not free.
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
Xen virtual console successfully installed as xvc0
Event-channel device installed.
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 50MHz system bus speed for PIO modes; override with idebus=xx
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: No PS/2 controller found. Probing ports directly.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
XENBUS: Device with no driver: device/vbd/51712
XENBUS: Device with no driver: device/vif/0
Write protecting the kernel read-only data: 439k
Red Hat nash version 5.1.19.6 starting
Mounting proc filesystem
Mounting sysfs filesystem
Creating /dev
Creating initial device nodes
Setting up hotplug.
Creating block device nodes.
Loading uhci-hcd.ko module
USB Universal Host Controller Interface driver v3.0
Loading ohci-hcd.ko module
Loading ehci-hcd.ko module
Loading jbd.ko module
Loading ext3.ko module
Loading xenblk.ko module
Registering block device major 202
 xvda: xvda1 xvda2
Loading dm-mod.ko module
device-mapper: ioctl: 4.11.0-ioctl (2006-09-14) initialised: dm-devel@redhat.com
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Making device-mapper control node
Scanning logical volumes
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2
Activating logical volumes
  2 logical volume(s) in volume group "VolGroup00" now active
Creating root device.
Mounting root filesystem.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Setting up other filesystems.
Setting up new root fs
no fstab.sys, mounting internal defaults
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
SELinux:  Disabled at runtime.
SELinux:  Unregistering netfilter hooks
audit(1193918690.801:2): selinux=0 auid=4294967295
INIT: version 2.86 booting
                Welcome to  CentOS release 5 (Final)
                Press 'I' to enter interactive startup.
Cannot access the Hardware Clock via any known method.
Use the --debug option to see the details of our search for an access method.
Setting clock  (utc): Thu Nov  1 20:04:50 SGT 2007 [  OK  ]
Starting udev: [  OK  ]
Setting hostname centos5:  [  OK  ]
Setting up Logical Volume Management:   2 logical volume(s) in volume group "VolGroup00" now active
[  OK  ]
Checking filesystems
Checking all file systems.
[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/VolGroup00/LogVol00
/dev/VolGroup00/LogVol00: clean, 42848/2080768 files, 369765/2080768 blocks
[/sbin/fsck.ext3 (1) -- /boot] fsck.ext3 -a /dev/xvda1
/boot: clean, 36/26104 files, 15441/104388 blocks
[  OK  ]
Remounting root filesystem in read-write mode:  [  OK  ]
Mounting local filesystems:  [  OK  ]
Enabling local filesystem quotas:  [  OK  ]
Enabling /etc/fstab swaps:  [  OK  ]
INIT: Entering runlevel: 3
Entering non-interactive startup
Starting background readahead: [  OK  ]
Checking for hardware changes [  OK  ]
Bringing up loopback interface:  [  OK  ]
Bringing up interface eth0:  [  OK  ]
Starting auditd: [  OK  ]
Starting system logger: [  OK  ]
Starting kernel logger: [  OK  ]
Starting irqbalance: [  OK  ]
Starting portmap: [  OK  ]
Starting NFS statd: [  OK  ]
Starting RPC idmapd: [  OK  ]
Starting system message bus: [  OK  ]
[  OK  ] Bluetooth services:[  OK  ]
Mounting other filesystems:  [  OK  ]
Starting PC/SC smart card daemon (pcscd): [  OK  ]
Starting hidd: [  OK  ]
Starting autofs:  Loading autofs4: [  OK  ]
Starting automount: [  OK  ]
[  OK  ]
Starting cups: [  OK  ]
Starting sshd: [  OK  ]
Starting xinetd: [  OK  ]
Starting sendmail: [  OK  ]
Starting sm-client: [  OK  ]
Starting console mouse services: [  OK  ]
Starting crond: [  OK  ]
Starting anacron: [  OK  ]
Starting atd: [  OK  ]
Starting yum-updatesd: [  OK  ]
Starting Avahi daemon... [  OK  ]
Starting HAL daemon: [  OK  ]
Starting smartd: [  OK  ]

CentOS release 5 (Final)
Kernel 2.6.18-8.el5xen on an x86_64

centos5 login: chihung
Password:
Last login: Thu Nov  1 19:55:29 from 10.0.12.33
[chihung@centos5 ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      7.7G  1.2G  6.2G  16% /
/dev/xvda1             99M   12M   82M  13% /boot
tmpfs                 2.0G     0  2.0G   0% /dev/shm

[chihung@centos5 ~]$ which tclsh
/usr/bin/tclsh
[chihung@centos5 ~]$ tclsh
% parray tcl_platform
tcl_platform(byteOrder) = littleEndian
tcl_platform(machine)   = x86_64
tcl_platform(os)        = Linux
tcl_platform(osVersion) = 2.6.18-8.el5xen
tcl_platform(platform)  = unix
tcl_platform(threaded)  = 1
tcl_platform(user)      = chihung
tcl_platform(wordSize)  = 8
% set tcl_patchLevel
8.4.13
% exit

[chihung@centos5 ~]$ exit

[chihung@centos5 ~]$ su -

[root@centos5 proc]# xentop
xentop - 12:13:56   Xen 3.0.4-1-xvm
2 domains: 1 running, 1 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdown
Mem: 33029676k total, 33028064k used, 1612k free    CPUs: 16 @ 2613MHz
      NAME  STATE   CPU(sec) CPU(%)     MEM(k) MEM(%)  MAXMEM(k) MAXMEM(%) VCPUS NETS NETTX(k) NETRX(k) VBDS   VBD_OO   VBD_
RD   VBD_WR SSID
   centos5 --b---          8    0.0    4194072   12.7    4194304      12.7     4    1        0        0    1        0
 0        0    0
  Domain-0 -----r       7350    0.0   28424612   86.1   no limit       n/a    16    0        0        0    0        0
 0        0    0

[root@centos5 proc]# cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 33
model name      : Dual Core AMD Opteron(tm) Processor 885
stepping        : 2
cpu MHz         : 2613.400
cache size      : 1024 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips        : 6538.09
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor       : 1
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 33
model name      : Dual Core AMD Opteron(tm) Processor 885
stepping        : 2
cpu MHz         : 2613.400
cache size      : 1024 KB
physical id     : 1
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips        : 6538.09
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor       : 2
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 33
model name      : Dual Core AMD Opteron(tm) Processor 885
stepping        : 2
cpu MHz         : 2613.400
cache size      : 1024 KB
physical id     : 2
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips        : 6538.09
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor       : 3
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 33
model name      : Dual Core AMD Opteron(tm) Processor 885
stepping        : 2
cpu MHz         : 2613.400
cache size      : 1024 KB
physical id     : 3
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips        : 6538.09
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

[root@centos5 proc]# cat /proc/meminfo
MemTotal:      4194304 kB
MemFree:       3911712 kB
Buffers:         11708 kB
Cached:         117648 kB
SwapCached:          0 kB
Active:          51032 kB
Inactive:       105476 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      4194304 kB
LowFree:       3911712 kB
SwapTotal:     2031608 kB
SwapFree:      2031608 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:       27168 kB
Mapped:           8720 kB
Slab:            11084 kB
PageTables:       2612 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   4128760 kB
Committed_AS:    66944 kB
VmallocTotal: 34359738367 kB
VmallocUsed:      1080 kB
VmallocChunk: 34359737267 kB

[root@centos5 proc]# uname -a
Linux centos5 2.6.18-8.el5xen #1 SMP Thu Mar 15 19:56:43 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

[root@centos5 proc]# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:16:3E:14:84:E5
          inet addr:10.0.12.34  Bcast:10.0.12.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fe14:84e5/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:138 errors:0 dropped:0 overruns:0 frame:0
          TX packets:63 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:9518 (9.2 KiB)  TX bytes:9536 (9.3 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:560 (560.0 b)  TX bytes:560 (560.0 b)

sit0      Link encap:IPv6-in-IPv4
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

[root@centos5 proc]# top 
top - 20:10:23 up 5 min,  1 user,  load average: 0.00, 0.04, 0.02
Tasks:  78 total,   1 running,  77 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4194304k total,   281992k used,  3912312k free,    11796k buffers
Swap:  2031608k total,        0k used,  2031608k free,   117720k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 1654 root      15   0 12568 1212  916 R    0  0.0   0:00.03 top
    1 root      18   0 10308  712  592 S    0  0.0   0:00.06 init [3]
    2 root      RT   0     0    0    0 S    0  0.0   0:00.00 [migration/0]
    3 root      34  19     0    0    0 S    0  0.0   0:00.00 [ksoftirqd/0]
    4 root      RT   0     0    0    0 S    0  0.0   0:00.00 [watchdog/0]
    5 root      10  -5     0    0    0 S    0  0.0   0:00.00 [events/0]
    6 root      10  -5     0    0    0 S    0  0.0   0:00.00 [khelper]
    7 root      10  -5     0    0    0 S    0  0.0   0:00.00 [kthread]
    9 root      10  -5     0    0    0 S    0  0.0   0:00.00 [xenwatch]
   10 root      10  -5     0    0    0 S    0  0.0   0:00.00 [xenbus]
   14 root      RT  -5     0    0    0 S    0  0.0   0:00.00 [migration/1]
   15 root      34  19     0    0    0 S    0  0.0   0:00.00 [ksoftirqd/1]
   16 root      RT  -5     0    0    0 S    0  0.0   0:00.00 [watchdog/1]
   17 root      10  -5     0    0    0 S    0  0.0   0:00.00 [events/1]

[root@centos5 proc]# exit

[chihung@centos5 ~]$ exit

CentOS release 5 (Final)
Kernel 2.6.18-8.el5xen on an x86_64

centos5 login: root@opensolaris#

To manage those guest domains, you need to use "virsh"

root@opensolaris# virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # help
Commands:

    capabilities    capabilities
    connect         (re)connect to hypervisor
    console         connect to the guest console
    create          create a domain from an XML file
    start           start a (previously defined) inactive domain
    destroy         destroy a domain
    define          define (but don't start) a domain from an XML file
    domid           convert a domain name or UUID to domain id
    domuuid         convert a domain name or id to domain UUID
    dominfo         domain information
    domname         convert a domain id or UUID to domain name
    domstate        domain state
    dumpxml         domain information in XML
    help            print help
    list            list domains
    nodeinfo        node information
    quit            quit this interactive terminal
    reboot          reboot a domain
    restore         restore a domain from a saved state in a file
    resume          resume a domain
    save            save a domain state to a file
    schedinfo       show/set scheduler parameters
    dump            dump the core of a domain to a file for analysis
    shutdown        gracefully shutdown a domain
    setmem          change memory allocation
    setmaxmem       change maximum memory limit
    setvcpus        change number of virtual CPUs
    suspend         suspend a domain
    undefine        undefine an inactive domain
    vcpuinfo        domain vcpu information
    vcpupin         control domain vcpu affinity
    version         show version
    vncdisplay      vnc display
    attach-device   attach device from an XML file
    detach-device   detach device from an XML file

virsh # vcpuinfo centos5
VCPU:           0
CPU:            7
State:          blocked
CPU time:       4.3s
CPU Affinity:   yyyyyyyyyyyyyyyy

VCPU:           1
CPU:            0
State:          blocked
CPU time:       1.9s
CPU Affinity:   yyyyyyyyyyyyyyyy

VCPU:           2
CPU:            3
State:          blocked
CPU time:       1.4s
CPU Affinity:   yyyyyyyyyyyyyyyy

VCPU:           3
CPU:            12
State:          blocked
CPU time:       1.2s
CPU Affinity:   yyyyyyyyyyyyyyyy

virsh # nodeinfo
CPU model:           i86pc
CPU(s):              16
CPU frequency:       2613 MHz
CPU socket(s):       8
Core(s) per socket:  2
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:         33029120 kB

virsh # version
Compiled against library: libvir 0.2.3
Using library: libvir 0.2.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.0

virsh # dumpxml centos5
<domain type='xen' id='8'>
  <name>centos5</name>
  <uuid>3816e1f75e9518410adc5fc4a5360aa5</uuid>
  <bootloader>/usr/lib/xen/bin/pygrub</bootloader>
  <os>
    <type>linux</type>
  </os>
  <memory>4194304</memory>
  <vcpu>4</vcpu>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <interface type='ethernet'>
      <mac address='00:16:3e:14:84:e5'/>
    </interface>
    <disk type='file' device='disk'>
      <driver name='file'/>
      <source file='/data/xvm/centos5.xvm'/>
      <target dev='xvda'/>
    </disk>
    <console tty='/dev/pts/3'/>
  </devices>
</domain>

virsh # dumpxml Domain-0
<domain type='xen' id='0'>
  <name>Domain-0</name>
  <uuid>00000000000000000000000000000000</uuid>
  <memory>28423168</memory>
  <vcpu>16</vcpu>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
  </devices>
</domain>

virsh # quit

BTW, the xm, virt-install and virsh frontend to xVM are mostly implemented in Python.

root@opensolaris# pwd
/usr/lib/python2.4/vendor-packages/xen

root@opensolaris# ls
__init__.py   __init__.pyc  lowlevel      sv            util          web           xend          xm

root@opensolaris# pwd
/usr/lib/python2.4/vendor-packages/xen/xm

root@opensolaris# ls
XenAPI.py          cfgbootpolicy.pyc  dumppolicy.py      labels.pyc         migrate.py         resources.pyc
XenAPI.pyc         console.py         dumppolicy.pyc     loadpolicy.py      migrate.pyc        rmlabel.py
__init__.py        console.pyc        getlabel.py        loadpolicy.pyc     new.py             rmlabel.pyc
__init__.pyc       create.py          getlabel.pyc       main.py            new.pyc            shutdown.py
addlabel.py        create.pyc         help.py            main.pyc           opts.py            shutdown.pyc
addlabel.pyc       dry-run.py         help.pyc           makepolicy.py      opts.pyc
cfgbootpolicy.py   dry-run.pyc        labels.py          makepolicy.pyc     resources.py

root@opensolaris# head -1 /usr/sbin/xm
#!/usr/bin/env python

Hope those output helps you to 'interact' with the xvm in openSolaris.

Labels: , , ,