SHELLdorado Newsletter 3/2002 - August 18th, 2002
================================================================
The "SHELLdorado Newsletter" covers UNIX shell script related
topics. To subscribe to this newsletter, leave your e-mail
address at the SHELLdorado home page:
http://www.shelldorado.com/
View previous issues at the following location:
http://www.shelldorado.com/newsletter/
"Heiner's SHELLdorado" is a place for UNIX shell script
programmers providing
Many shell script examples,
shell scripting tips & tricks + more...
================================================================
Contents
o What's new at the SHELLdorado?
o Shell Tip: KornShell built-in string manipulation functions
o Shell Tip: Calculating with large numbers
o Shell Tip: Better (and larger) random numbers with /dev/urandom
o Shell Tip: Automated remote command execution using TELNET
-----------------------------------------------------------------
>> What's new at the SHELLdorado?
-----------------------------------------------------------------
o The "Links" page has been extended: it now lists more than
200 pointers to shell script examples, scripting tutorials
and references and many more shell script related resources:
http://www.shelldorado.com/links/
If you have comments or suggestions for this newsletter, or
even want to write an article on your own, please write me
an e-mail.
Heiner Steven, Editor
<heiner.steven@shelldorado.com>
-----------------------------------------------------------------
>> Shell Tip: KornShell built-in string manipulation functions
-----------------------------------------------------------------
Shell script programmers are used to work with "cut", "sed",
and "awk" for string manipulations. But sometimes these
programs are not needed, at least for the lucky programmers
using ksh, ksh93, or BASH, because these shells have powerful
build-in string manipulation functions.
If we had the following PATH settings:
PATH=/usr/local/bin:/usr/bin:/opt/kde3/bin
we could get different parts of this variable using special
expressions:
${variable#pattern}, ${variable##pattern}
remove (first/all) substring matching pattern from the
START of the variable's content:
$ echo ${PATH#*:}
/usr/bin:/opt/kde3/bin
$ echo ${PATH##*:}
/opt/kde3/bin
${variable%pattern}, ${variable%%pattern}
remove the first (all) sub-strings from the END of the
variable's content:
$ echo ${PATH%:*}
/usr/local/bin:/usr/bin
$ echo ${PATH%%:*}
/usr/local/bin
ksh93 and BASH go even a step further and implement string
substitution facilities resembling "sed"'s "s/../../"
command. Instead of writing
$ echo "$PATH" | sed 's/:/ /g'
or
$ echo "$PATH" | tr : " "
we can use
$ echo "${PATH//:/ }"
to print $PATH with all ':' characters replaced by a blank:
/usr/local/bin /usr/bin /opt/kde3/bin
Of course the regular expressions are not limited to constant
characters and an asterisk (as in the examples above). All of
the extended file name generation expressions can be used:
* ? . [...]
?(...) *(...) +(...) @(...) !(...)
If the second line looks unfamiliar, I suggest to RTFM, read
the "fine" manual page, to learn the whole story.
-----------------------------------------------------------------
>> Shell Tip: Calculating with large numbers using "bc"
-----------------------------------------------------------------
Many shell script programmers know, that the KornShell
and BASH have built-in arithmetic, e.g.
$ integer a=6 b=7
$ integer theanswer=$((a*b))
$ echo $theanswer
42
But sometimes these calculations are not too impressive:
$ integer pi=3.14159256
ksh: 3.14159256: unexpected `.'
-- ksh(88) has no floating point values
# lifetime (in seconds) of a 72 year old person:
$ echo $((60*60*24*7*52*72))
-2030596096
-- integer overflow with pdksh, Linux
When we need precision, or calculate with large numbers, "bc"
(an "arbitrary precision calculator language") can be a
better option:
$ echo "2*3" | bc
6
$ echo "5.98 * 10^24" | bc # weight of the earth [in kg]
5980000000000000000000000.00
$ echo "10/3" | bc
3 # should be 3.3333333..., shouldn't it?
Well, the last result can certainly be improved on. We need
to tell "bc" that we are not interested in integer
calculations, but want two decimal digits, too:
$ echo "scale=2; 10/3" | bc
3.33
Advanced programmers can even start "bc" as a (ksh)
co-process, speeding up scripts that use "bc" extensively.
Examples on scripts using "bc" co-processes are available at
the SHELLdorado:
http://www.shelldorado.com/scripts/cmds/base
http://www.shelldorado.com/scripts/cmds/calc
[ Further reading: results of a survey on the integer
processing capabilities of different shells on different
operating systems:
http://groups.google.de/groups?threadm=aasch6012ap%40enews3.newsguy.com
]
-----------------------------------------------------------------
>> Shell Tip: Better (and larger) random numbers with /dev/urandom
-----------------------------------------------------------------
Random numbers are very useful in shell scripts. They can be
used to print changing "fortune cookie" every day, or select
a random MP3 file from a play list.
A common way to get a random number is to use the special
$RANDOM environment variable (ksh, BASH, zsh). Unfortunately
this will only give us values in the range 0..32767. This is
disturbing, the more so because some systems (Linux, Solaris
9) have a high quality random source: /dev/random, or
/dev/urandom.
The following example script will use /dev/urandom to print
random numbers in the range [1.. number specified on the
command line], or [1..18446744073709551616] if called without
arguments:
:
# rand - create large random number using /dev/urandom
# usage: rand [maxvalue]
RandomDevice=/dev/urandom
MaxRand=18446744073709551616 # 2^64
[ $# -lt 1 ] && set -- $MaxRand
# Read 8 (binary) bytes, convert them to 8 upper-case hex
# numbers using "od", remove whitespace:
hex=`dd if=/dev/urandom bs=1 count=8 2>/dev/null |
od -tx1 | head -1 | cut -d' ' -f2- |
tr -d ' ' | tr '[a-f]' '[A-F]'`
# convert from hexadecimal to decimal:
dec=`echo "ibase=16; $hex" | bc`
echo >&2 "DEBUG: hex=<$hex>; dec=<$dec>"
echo "$dec % $1 + 1" | bc
A more sophisticated script allowing for even larger numbers
is available at the SHELLdorado:
http://www.shelldorado.com/scripts/cmds/rand
[ The example script was tested with Linux (PD-KSH, ksh93,
BASH, ash, ZSH) and Solaris 9 (ksh, ksh93, BASH, ZSH) ]
-----------------------------------------------------------------
>> Shell Tip: Automated remote command execution using TELNET
-----------------------------------------------------------------
It's often useful to have a script that can log into a remote
host, and automatically start commands there. Although "ssh",
"rsh" and even "expect" <http://expect.nist.gov/> can (and
should) be used for this purpose, there is an easy way to use
the "telnet" command for simple scripts.
The first try does not work as expected:
echo "myname\nmypassword" | telnet remotehost
When the "telnet" command finally is connected to the host,
and starts to read its standard input, the "echo" command
already has completed. The following script uses "sleep" to
force breaks between the different steps. It is not elegant
(but error-prone), but can be useful in times of need:
# rtelnet - use telnet to remotely execute a command
# usage: rtelnet [command]
[ $# -lt 1 ] && set -- pwd # default command
user=${USER:-$LOGNAME}
passwd=secret
host=localhost
(
sleep 2 # give "telnet" time to connect
echo "$user"; sleep 2
echo "$passwd"; sleep 2
echo "$*" # execute command!
sleep 2 # give time to print results
) | telnet "$host"
----------------------------------------------------------------
If you want to comment on this newsletter, have suggestions for
new topics to be covered in one of the next issues, or even want
to submit an article of your own, send an e-mail to
mailto:heiner.steven@shelldorado.com
================================================================
To unsubscribe send a mail with the body "unsubscribe" to
newsletter@shelldorado.com
================================================================