Thursday, September 18, 2008

Beauty of UNIX Sub-Shell

UNIX sub-shell "(...)" is extremely useful in a couple of situations:
  1. group the output of a few commands and have it to be processed by another command
    (cat /etc/hosts; uname -a; df -k) | wc -l
  2. execute a command in a different directory without having to 'cd' back to your original directory
    (cd /some/other/directory; ./run-a-command > output.txt)
  3. run a group of commands in background
    (run1.sh; run2.sh; run3.sh) &

By applying some of these principles, you can do quite a few amusing things. Do you know that you can combine sub-shell and 'tar' to copy directory recursively:

$ (cd /usr/local; tar cpf - .) | (cd $HOME/local; tar xpf -)

Suppose you need to 'talk SMTP' protocol with the mail server directly:

$ telnet mail.some.mail.server.com 25
Trying 1.2.3.4...
Connected to mail.some.mail.server.com.
Escape character is '^]'.
220-mailsrv.some.mail.server.com -- Server ESMTP (SomeServer Mail Service)
220 Authorised Use Only
HELO chihungchan.blogspot.com
250 mailsrv.some.mail.server.com OK, [203.166.139.133].
MAIL FROM: chihung@some.mail.server.com
250 2.5.0 Address Ok.
RCPT TO: chihung@some.mail.server.com
250 2.1.5 chihung@some.mail.server.com OK.
DATA
354 Please start mail input.
From: chihung@some.mail.server.com
To: chihung@some.mail.server.com
Subject: sub shell

testing sub shell
.
250 Mail queued for delivery.
QUIT
221 Closing connection. Good bye.
Connection closed by foreign host.

You may think that by 'echo' all these input commands and have it piped to the remote smtp port will work, think again. When you echo something, the system immediate output the text to the standard output and finish off the command. However, telnet to some.mail.server.com may take a while. First it requires to resolve the hostname to IP address, follow by the establishing TCP connection to the remote server. All these may take more than a second depending on how busy is your DNS server, how far is the remote host and how busy is the network. So by the time some.mail.server.com is available to talk SMTP, the data is gone. To avoid such situation, you can echo the SMTP commands and sleep for a while in a sub-shell while telnet is trying to establish the connection.

Below will NOT be able to send email because of the above reason:

$ echo "HELO chihungchan.blogspot.com
MAIL FROM: chihung@some.mail.server.com
RCPT TO: chihung@some.mail.server.com
DATA
From: chihung@some.mail.server.com
To: chihung@some.mail.server.com
Subject: sub shell

testing sub shell
.
QUIT" | telnet mail.some.mail.server.com 25
Trying 1.2.3.4...
Connected to mail.some.mail.server.com.
Escape character is '^]'.
Connection closed by foreign host.

This will work by applying the sub-shell trick:

$ (echo "HELO chihungchan.blogspot.com
MAIL FROM: chihung@some.mail.server.com
RCPT TO: chihung@some.mail.server.com
DATA
From: chihung@some.mail.server.com
To: chihung@some.mail.server.com
Subject: sub shell

testing sub shell
.
QUIT"; sleep 5) | telnet mail.some.mail.server.com 25
Trying 1.2.3.4...
Connected to mail.some.mail.server.com.
Escape character is '^]'.
220-mailsrv.some.mail.server.com -- Server ESMTP (SomeServer Mail Service)
220 Authorised Use Only
250 mailsrv.some.mail.server.com OK, [1.2.3.4].
250 2.5.0 Address Ok.
250 2.1.5 chihung@some.mail.server.com OK.
354 Please start mail input.
250 Mail queued for delivery.
221 Closing connection. Good bye.
Connection closed by foreign host.

You can do the same trick for HTTP protocol.

$ (echo "GET / HTTP/1.1
Host: chihungchan.blogspot.com
"; sleep 5) | telnet chihungchan.blogspot.com 80 > /tmp/aa 2>&1

$ head /tmp/aa
Trying 209.85.133.191...
Connected to chihungchan.blogspot.com.
Escape character is '^]'.
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Last-Modified: Mon, 15 Sep 2008 02:19:49 GMT
Cache-Control: max-age=0 private
ETag: "05983cb0-d1a4-47e0-bce1-37b9f2e233db"
Transfer-Encoding: chunked
Date: Thu, 18 Sep 2008 09:23:29 GMT
Server: GFE/1.3

58a
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/x
html1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <title>Chi Hung Chan</title>

<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">

If you are running Linux, you can explore the use of nc command. Doing the 'nc' way as suggested in the man page does not work on my office mail server. It gives "220 Authorised Use Only" error message. If you know why, appreciate if you can leave me a comment. BTW, Solaris does not have nc command installed.

Labels: ,

2 Comments:

Blogger Andre Misc said...

Thanks for the information. Just what I needed to whip up a nice telnet script.

2:36 AM  
Blogger chihungchan said...

Glad to hear that.

7:55 AM  

Post a Comment

<< Home