SHELLdorado Newsletter 1/2002 - February 13th, 2002
================================================================
The "SHELLdorado Newsletter" covers UNIX shell script related
topics. To subscribe to this newsletter, leave your e-mail
address at the SHELLdorado home page:
http://www.shelldorado.com/
View previous issues at the following location:
http://www.shelldorado.com/newsletter/
"Heiner's SHELLdorado" is a place for UNIX shell script
programmers providing
Many shell script examples,
shell scripting tips & tricks + more...
================================================================
This issue focuses on Korn Shell 93
Contents
o Editorial
o How to use associative arrays
o How to get string lengths and sub-strings
o How to create easy counting loops with the new "for" syntax
o How to access variables by name using "typeset -n"
o How to use the built-in printf command
o Q&A: Where can I find more articles about ksh93?
o Q&A: How can I write my own POP3 e-mail client using ksh?
-----------------------------------------------------------------
>> Editorial
-----------------------------------------------------------------
Much has changed since Stephen "Steve" R. Bourne wrote his
command line interpreter the "Bourne Shell", or just "sh".
David Korn rewrote it from scratch, adding new features on
his way. The result, KornShell 88 (named after the year it
was published) is available on almost any UNIX system today.
This issue of the SHELLdorado Newsletter focuses on the
latest version of the KornShell: KornShell 93 (or "ksh93").
Many people already know and value the shell as the base of
the "dtksh" shell, which comes as part of the "Common
Desktop Environment" CDE for many UNIX systems, e.g. Solaris
or AIX (search for /usr/dt/bin/dtksh).
ksh93 is not only a new version of the old KornShell with
many new features particularly for shell script programmers:
it could become the new standard shell for all new UNIX
systems, including Linux.
The source code is now available, together with pre-compiled
binaries for many UNIX systems, including MacOS, Linux,
FreeBSD, HP-UX, AIX, Linux, Solaris, SCO UnixWare and
even Windows (for use with U/WIN emulation):
http://www.research.att.com/sw/download/
If you are not interested in ksh93, you may still find the
small POP3 e-mail client written in "conventional" KornShell
(ksh88) interesting.
If you have comments or suggestions for this newsletter, or
even want to write an article by your own, please write me
an e-mail.
Heiner Steven, Editor
<heiner.steven@shelldorado.com>
-----------------------------------------------------------------
>> How to use associative arrays
-----------------------------------------------------------------
An associative array is an array indexed by a string. This
lets us easily create lookup tables, e.g. a table containing
the full names of all users given their login id:
typeset -A name # "name" is an array
# Input consists of fields separated by a colon (':')
OIFS=$IFS; IFS=:
while read login pw uid gid fullname ignore
do
name[$login]=$fullname
done < /etc/passwd
IFS=$OIFS
# $login sequentially contains all indices from name[]
for login in ${!name[@]}
do
print "$login: ${name[$login]}"
done
This prints e.g.
root: root
nobody: nobody
lp: Printing daemon
heiner: Heiner Steven
[...]
Within the script, print ${name[heiner]} could be used to
print the full name of the user with the login id "heiner".
-----------------------------------------------------------------
>> How to get string lengths and sub-strings
-----------------------------------------------------------------
Some frequent string manipulations get easier with ksh93.
The length of each string can be printed in the following
way:
a="Heiner's SHELLdorado"
$ print ${#a} # string length
20
$ print ${a:9} # string starting with position 9
SHELLdorado
$ print ${a:9:5} # starting with position 9; 5 characters
SHELL
Even "sed" like substitutions are available. The following
command replaces all "U" characters in the variable "a" with
an "X" in the output:
$ print ${a//U/X}
These command make string manipulation not just easier,
but also much faster.
-----------------------------------------------------------------
>> How to create easy counting loops with the new "for" syntax
-----------------------------------------------------------------
The old Bourne shell forced us to write code like the
following for simple tasks as counting from 1 to 10:
i=1
while [ $i -le 10 ]
do
echo $i
i=`expr $i + 1`
done
Compare this to the new ksh93 syntax:
integer i
for (( i=1; i<=10; i++ ))
do
echo $i
done
-----------------------------------------------------------------
>> How to access variables by name using "typeset -n"
-----------------------------------------------------------------
Sometimes it's useful to specify the name of a variable as
an argument to a function, e.g.
getstring firstname "First Name:"
The function "getstring" should read a string from the user,
and return the result in our variable "firstname". This used
to be solved with "eval", but ksh93 has an easier solution:
function getstring { # varname promptstring
typeset -n vname=$1
typeset prompt=${2-"?"}
print -u2 "$prompt\c"
read vname
}
Since "vname" contains the name of a variable (not its
value), all manipulations on "vname" change the value of the
variable specified as an argument to the function.
Without "typeset -n" we would have written the function
similar to the following example:
getstring () { # varname promptstring
vname=$1
prompt=${2-"?"}
echo -n "$prompt" >&2
read answer
eval "$vname=\"$answer\""
}
-----------------------------------------------------------------
>> How to use the built-in printf command
-----------------------------------------------------------------
Why should we use yet another version of the "printf"
command? Many systems already have a version in /usr/bin.
Well, one advantage of printf being a built-in is, that the
behaviour of the function is system-independent, and ksh93
programmers can rely on its existence.
But this built-in version of printf is special in some other
ways, too:
o If there are more arguments than formats in the format
string, the format string is reused. This can be used
e.g. to replace "cut" in some places:
cut -c1,10 < /etc/passwd
to list the first 10 characters of each line in
/etc/passwd can be rewritten in the following way:
printf ".10s\n" /etc/passwd
o It can convert regular expressions to shell patterns,
and vice versa:
$ printf "%P\n" "a(b|c)x"
*a@(b|c)x*
$ printf "%R\n" "*a@(b|c)x*"
a(b|c)x
o "printf" can help quoting:
$ printf "%q\n" "Heiner's SHELLdorado"
$'Heiner\'s SHELLdorado'
...and printf even knows something about HTML and XML, and
how to expand special characters in a way suitable for HTML
text:
$ printf "%H\n" "<Tips & Tricks>"
<Tips & Tricks>
Well, that does not look too readable, but we are not the
interpreters targeted ;-) A web browser knows how to convert
these "character entities" to a readable representation.
-----------------------------------------------------------------
>> Q&A: Where can I find more articles about ksh93?
-----------------------------------------------------------------
The KornShell 93 is available free of charge from AT&T
Labs-Research:
http://www.research.att.com/sw/download/
A concise list of the new features of ksh93 is listed
in the following article:
http://www.cs.princeton.edu/~jlk/ksh93.html
The author of the shell, David Korn, maintains a web site
with many more articles:
http://www.kornshell.com/
A description of the new features of ksh93 from the Linux
Journal:
http://www.linuxjournal.com/article.php?sid=1273
A manual page
http://www.cs.princeton.edu/~jlk/kornshell/doc/man93.html
Frequently Asked Questions (FAQ):
http://www.kornshell.com/doc/faq.html
Many more links to shell scripts and shell scripting related
articles are available at the SHELLdorado Links section:
http://www.shelldorado.com/links/
-----------------------------------------------------------------
>> Q&A: How can I write my own POP3 e-mail client using ksh?
-----------------------------------------------------------------
Sometimes these viruses (like "SirCam") can really be
annoying. They make completely strangers send you 3 MB
holiday pictures, or some 5 MB audio file you did not ask
for. Most POP3 e-mail clients (like Netscape) will download
the files from the server to your local file system before
allowing the relieving click on the "delete" button.
In these cases the following script is useful: it can delete
files directly on the POP3 e-mail server, before they reach
the local hard disk. It can also list the e-mails, and
display them one by one.
The script will certainly not replace your e-mail
application of choice, but it may be useful for the purpose
stated above, or just as an example of KornShell
co-processes.
Note that the script does NOT require ksh93, it should run
with any KornShell dialect.
#! /usr/bin/ksh
# popc.ksh - example of a POP3 e-mail client written in KornShell
# Heiner Steven, heiner@shelldorado.com
#
# This example implements the "LIST" command to list all messages,
# "RETR" to retrieve a message text by number, and "DELE" to delete
# a message by number.
#
# Refer to RFC 1939 (http://www.ietf.org/rfc/rfc1939.txt) for a full
# description of the POP3 protocol.
#
# Needs the non-standard program "netcat" (aka "nc") to establish a
# TCP connection. You could use "socket(1)" instead, if installed.
host=localhost # Name or address of POP3 server
port=110 # POP3 port (standard is 110)
user=${USER:-$LOGNAME}
pass=dontsay
typeset -u pop_status # Global last status { "+OK" | "-ERR" }
typeset pop_msg # Global last server status message
function Fatal { print -u2 "$@"; exit 1; }
# sendline - send one line to co-process
function sendline { print -p -- "$@"; }
# getack - get positive or negative acknowledgement from server
#
# The server will answer with either "+OK ..." or "-ERR ...". We
# evaluate the first word read from the server, and set the
# return value accordingly
function getack {
read -p -r pop_status pop_msg; print -u2 "DEBUG: $pop_status $pop_msg"
[[ $pop_status == '+OK' ]] && return 0
print -u2 -- "$pop_status $pop_msg"
return 1
}
# getlist - get multi-line response from server
#
# Multi-line responses are terminated by a line consisting only of
# a period '.'
function getlist {
typeset line
while read -p -r line
do
line=${line%+(?)} # Remove trailing CR (ASCII 13)
[[ $line == '.' ]] && break # End of list
print -- "<$line>"
done
}
function ListCmd {
sendline "LIST"
getack && getlist
}
function DeleteCmd {
integer msgno
read msgno?"DELETE message Number: "
sendline "DELE $msgno" && getack
}
function RetrieveCmd {
integer msgno
read msgno?"Retrieve message Number: "
sendline "RETR $msgno"
getack && getlist
}
# Startup the server as a co-process. We will use "print -p"
# and "read -p" to write and read data.
netcat "$host" "$port" |& popdpid=$!
# Check if the server is ready
getack || Fatal "cannot start POP3 server $host:$port"
# Authentication
set -e # First error terminates script
sendline "USER $user" && getack
sendline "PASS $pass" && getack
set +e
print -u2 "Authentication successful"
PS3="POP3 command (RETURN prints menu): "
select choice in List Retrieve Delete Quit
do
case "$choice" in
(List) ListCmd ;;
(Retrieve) RetrieveCmd ;;
(Delete) DeleteCmd ;;
(Quit) break;;
esac
done
# Shut down co-process
exec 3<&p 3>&p # redirect co-process fd to fd 3
exec 3<&- 3>&- # close fd 3
kill $popdpid >/dev/null 2>&1 # if the above does not work...
exit 0
[Download location for "netcat":
ftp://ftp.uni-stuttgart.de/pub/org/uni-s/rus/security/unix/hobbit/nc110.tgz
Check your local system first; many systems (e.g. Linux)
already have this program installed.
The script was tested with pdksh v5.2.14 99/07/13.2
(Linux) and ksh93 Version M 1993-12-28 m (Linux).]
----------------------------------------------------------------
The examples were tested using Linux 2.4 and KornShell 93
Version M 1993-12-28 m. Use "print ${.sh.version}" to find your
ksh93 revision number.
----------------------------------------------------------------
If you want to comment on the newsletter, have suggestions for
new topics to be covered in one of the next issues, or even want
to submit an article of your own, send an e-mail to
mailto:heiner.steven@shelldorado.com
================================================================
To unsubscribe send a mail with the body "unsubscribe" to
newsletter@shelldorado.com
================================================================