CSE271 Lab 7: Shell Programming

Introduction

On Friday and Monday, we introduced shell scripting and regular expressions. Today we will work through some scripting exercises, and in the process, also learn about the ability to schedule jobs in unix systems.

A simple shell script

Let's begin by writing a simple shell script. Edit a file called stamp.sh, which should be a shell script that creates (or overwrites) a file called timestamp with the current date and time. You can run it with bash stamp.sh. Emacs, like any editor, will create a file that is readable and editable, but not executable by default. To make the file containing the script executable, you'll have to change the permissions on it. The chmod command lets you reset the permissions of a file. Let's just use

chmod +x stamp.sh

to make it executable. Remember that you can use ls -l to see the permissions associated with a file.

Now we should change the script to always write to the same file. Perhaps your script already does that, but most scripts would just write to a file named 'timestamp', which would create a different file when the script is executed while in a different directory. So to make the script always write to the same file, either the script needs to use cd to get into the right directory, or else the script should specify the target file with an absolute path. So for me, I would need to write to the file /home/bdd3/timestamp. Once that change has been made, I can execute the file from anywhere on the filesystem, and it will always write to the same timestamp file.

Getting a script to run on schedule

UNIX has great support for getting scripts to run on a specific schedule. There are two common mechanisms. The first is the at(1) command, which schedules a particular job (often a script) to execute once at a particular time in the future. We'll come back to that one shortly. The second is the cron system, which allows you to schedule jobs to run repeatedly, perhaps every day, or every hour, or every Monday at 6:14am, etc. The cron system uses a set of crontables -- one per account on a given machine, that lists the jobs to be run, and when. Any user can set up jobs in their own crontab(1) file---type crontab -e to edit yours (which should be empty). (Note that crontab -e will automatically open an editor for you, which will be vi unless you have set the EDITOR environment variable to be a different value, such as with a export EDITOR=emacs statement.) Here is an example of the crontab for the printing subsystem:

# At 03:13am on Sundays: # Move a weeks worth of 'requests' to 'requests.1'. # If there was a 'requests.1' move it to 'requests.2'. # If there was a 'requests.2' then it is lost. # 13 3 * * 0 cd /var/lp/logs; if [ -f requests ]; then if [ -f requests.1 ]; then /bin/mv requests.1 requests.2; fi; /usr/bin/cp requests requests.1; >requests; fi

The first five fields express when to run, and the remainder is the command (or set of commands, in this case) to run. Details of how to express a time are available in the crontab man page. Looking at the rest, you can see that there are multiple shell commands separated by semicolons. In fact, the sequence of commands are only separated into multiple lines for easy presentation here---in the actual crontab entry, they are all on a single line. The only expression that might be new is >file, which is simply a shorthand for touch file.

Here is another crontab, running on the system in my lab that makes backups of important files:

1 */3 * * * /bin/nice /root/make_snapshot.sh 2 2 * * * /bin/nice /root/daily_snapshot_rotate.sh 2 1 * * 0 /bin/nice /root/weekly_snapshot_rotate.sh 2 4 * * * /bin/nice /root/daily_mysql_dump.sh

The first entry runs at 1 minute past the hour, every third hour of the day (e.g., 12:01, 3:01, 6:01, etc.). It is a script that creates a snapshot of all user files. The daily script take the oldest hourly snapshot and adds it to the daily snapshots, every day at 2:02am. The weekly script is similar, taking the oldest daily snapshot and moving it to the weekly snapshots, but only runs on Sunday mornings. The mysql dump script makes a backup of the mysql database every day at 4:02am.

OK, back to the first, simpler method of scheduling a single job. This is the approach that I use to collect your homeworks and projects. I use at to schedule a script to run just after midnight of the day a project is due.

Let's now use at to schedule your stamp.sh script to be run. Check the current time on your system, and then schedule your job to run in a few minutes. Here is how I scheduled mine:

europa:~% at 15:16 at> ./stamp.sh at> <EOT> commands will be executed using /bin/bash job 1173640560.a at Sun Mar 11 15:16:00 2007 europa:~%

The only thing that does not show up above is that in order to finish the at job, I typed CTRL-D on a line by itself (to which the system responded with <EOT>). Later, after the time had passed, I checked it:

europa:~% more timestamp Sun Mar 11 15:16:00 EDT 2007 europa:~%

Don't be too worried about making mistakes in scheduling or in the command to run. If you mistakenly create an at job that you would like to delete, you can do so. First, list all your pending at jobs with atq. Then use atrm to remove the job you identify (using the job number). Here is an example:

io 2% atq Rank Execution Date Owner Job Queue Job Name 1st Mar 12, 2007 14:02 brian 1173722520.a a stdin io 2% atrm 1173722520.a 1173722520.a: removed io 2% atq no files in queue. io 2%

Finally, after the scheduled time, verify that your job has run (make sure it is no longer in the at queue) and see that the timestamp file has the correct timestamp in it.

Regular expressions in emacs

On Monday we introduced globbing and regular expressions. We saw a few regular expressions used in grep. Today we see how regular expressions are also used in editors.

UNIX editors such as vi and emacs also support regular expressions to search and transform files. In emacs, M-C-S will allow you to type a regular expression and interactively find the next one (C-S will move to the next, as with regular search). M-X replace-regexp will prompt for a regular expression for matching, and then prompt for the replacement string.

Now try it yourself. Start emacs, and place five lines of text "Lehigh X" (without the quotes and where X is the line number, 1-5) into a file. Now use the regex "L.*igh [0-9]" and C-S to see that it matches each one.

Regular expressions in grep

Grep can find lots of useful information for us in a file. Given the file datebook.txt, determine the various grep commands that will

Print all lines containing the string San.
Print all lines where the person's first name starts with J.
Print all lines ending in 700.
Print all lines that don't contain 834.
Print all lines where birthdays are in December (dates are written M/D/Y).
Print all lines containing an uppercase letter, followed by four lowercase letters, a comma, a space, and one uppercase letter.
Print lines where the last name begins with a K or k.
Print those lines that do not have phone numbers in the ###-###-#### format.
Print lines preceded by a line number where the salary (last field) is a six-figure number.

Finally, write a short shell script that will ask the user for the name of the person to look up in the databook.txt file, and then output the line or lines that match.

Last revised: 27 February 2013, Prof. Davison.