SHELLdorado Newsletter 3/2002 - August 18th, 2002

The "SHELLdorado Newsletter" covers UNIX shell script related
topics. To subscribe to this newsletter, leave your e-mail
address at the SHELLdorado home page:

View previous issues at the following location:

"Heiner's SHELLdorado" is a place for UNIX shell script
programmers providing

     Many shell script examples,
     shell scripting tips & tricks + more...


 o  What's new at the SHELLdorado?
 o  Shell Tip: KornShell built-in string manipulation functions
 o  Shell Tip: Calculating with large numbers
 o  Shell Tip: Better (and larger) random numbers with /dev/urandom
 o  Shell Tip: Automated remote command execution using TELNET

>> What's new at the SHELLdorado?

 o  The "Links" page has been extended: it now lists more than
    200 pointers to shell script examples, scripting tutorials
    and references and many more shell script related resources:

    If you have comments or suggestions for this newsletter, or
    even want to write an article on your own, please write me
    an e-mail.
    Heiner Steven, Editor

>> Shell Tip: KornShell built-in string manipulation functions

    Shell script programmers are used to work with "cut", "sed",
    and "awk" for string manipulations. But sometimes these
    programs are not needed, at least for the lucky programmers
    using ksh, ksh93, or BASH, because these shells have powerful
    build-in string manipulation functions.

    If we had the following PATH settings:


    we could get different parts of this variable using special

	${variable#pattern}, ${variable##pattern}
	remove (first/all) substring matching pattern from the
	START of the variable's content:

		$ echo ${PATH#*:}
		$ echo ${PATH##*:}

	${variable%pattern}, ${variable%%pattern}
	remove the first (all) sub-strings from the END of the
	variable's content:

		$ echo ${PATH%:*}
		$ echo ${PATH%%:*}

    ksh93 and BASH go even a step further and implement string
    substitution facilities resembling "sed"'s "s/../../"
    command. Instead of writing

    	$ echo "$PATH" | sed 's/:/ /g'
    	$ echo "$PATH" | tr : " "

    we can use

	$ echo "${PATH//:/ }"

    to print $PATH with all ':' characters replaced by a blank:

    	/usr/local/bin /usr/bin /opt/kde3/bin

    Of course the regular expressions are not limited to constant
    characters and an asterisk (as in the examples above). All of
    the extended file name generation expressions can be used:

    	*	?	.	[...]
	?(...)	*(...)	+(...)	@(...)	!(...)

    If the second line looks unfamiliar, I suggest to RTFM, read
    the "fine" manual page, to learn the whole story.

>> Shell Tip: Calculating with large numbers using "bc"

    Many shell script programmers know, that the KornShell
    and BASH have built-in arithmetic, e.g.

    	$ integer a=6 b=7
	$ integer theanswer=$((a*b))
	$ echo $theanswer

    But sometimes these calculations are not too impressive:

    	$ integer pi=3.14159256
	ksh: 3.14159256: unexpected `.'

	-- ksh(88) has no floating point values

	# lifetime (in seconds) of a 72 year old person:
	$ echo $((60*60*24*7*52*72))

	-- integer overflow with pdksh, Linux

    When we need precision, or calculate with large numbers, "bc"
    (an "arbitrary precision calculator language") can be a
    better option:

    	$ echo "2*3" | bc

    	$ echo "5.98 * 10^24" | bc # weight of the earth [in kg]

        $ echo "10/3" | bc
	3		# should be 3.3333333..., shouldn't it?

    Well, the last result can certainly be improved on. We need
    to tell "bc" that we are not interested in integer
    calculations, but want two decimal digits, too:

        $ echo "scale=2; 10/3" | bc

    Advanced programmers can even start "bc" as a (ksh)
    co-process, speeding up scripts that use "bc" extensively.
    Examples on scripts using "bc" co-processes are available at
    the SHELLdorado:

   [ Further reading: results of a survey on the integer
     processing capabilities of different shells on different
     operating systems:

>> Shell Tip: Better (and larger) random numbers with /dev/urandom

    Random numbers are very useful in shell scripts. They can be
    used to print changing "fortune cookie" every day, or select
    a random MP3 file from a play list.

    A common way to get a random number is to use the special
    $RANDOM environment variable (ksh, BASH, zsh). Unfortunately
    this will only give us values in the range 0..32767. This is
    disturbing, the more so because some systems (Linux, Solaris
    9) have a high quality random source: /dev/random, or

    The following example script will use /dev/urandom to print
    random numbers in the range [1.. number specified on the
    command line], or [1..18446744073709551616] if called without

	# rand - create large random number using /dev/urandom
	# usage: rand [maxvalue]

	MaxRand=18446744073709551616	# 2^64

	[ $# -lt 1 ] && set -- $MaxRand

	# Read 8 (binary) bytes, convert them to 8 upper-case hex
	# numbers using "od", remove whitespace:

	hex=`dd if=/dev/urandom bs=1 count=8 2>/dev/null |
		od -tx1 | head -1 | cut -d' ' -f2- |
		tr -d ' ' | tr '[a-f]' '[A-F]'`

		# convert from hexadecimal to decimal:
		dec=`echo "ibase=16; $hex" | bc`
		echo >&2 "DEBUG: hex=<$hex>; dec=<$dec>"

		echo "$dec % $1 + 1" | bc

    A more sophisticated script allowing for even larger numbers
    is available at the SHELLdorado:

    [ The example script was tested with Linux (PD-KSH, ksh93,
      BASH, ash, ZSH) and Solaris 9 (ksh, ksh93, BASH, ZSH) ]

>> Shell Tip: Automated remote command execution using TELNET

    It's often useful to have a script that can log into a remote
    host, and automatically start commands there. Although "ssh",
    "rsh" and even "expect" <> can (and
    should) be used for this purpose, there is an easy way to use
    the "telnet" command for simple scripts.

    The first try does not work as expected:

    	echo "myname\nmypassword" | telnet remotehost

    When the "telnet" command finally is connected to the host,
    and starts to read its standard input, the "echo" command
    already has completed. The following script uses "sleep" to
    force breaks between the different steps. It is not elegant
    (but error-prone), but can be useful in times of need:

	# rtelnet - use telnet to remotely execute a command
	# usage: rtelnet [command]

	[ $# -lt 1 ] && set -- pwd      # default command


	    sleep 2     # give "telnet" time to connect
	    echo "$user";   sleep 2
	    echo "$passwd"; sleep 2
	    echo "$*"   # execute command!
	    sleep 2     # give time to print results
	) | telnet "$host"

If you want to comment on this newsletter, have suggestions for
new topics to be covered in one of the next issues, or even want
to submit an article of your own, send an e-mail to

To unsubscribe send a mail with the body "unsubscribe" to