Learning the Mac OS X Terminal, Part 5
Pages: 1, 2
The User Crontab
In Part 1, you learned how to modify the system crontab by simply opening it in a text editor. For this procedure, however, you will be creating a user crontab. User crontabs (or "cron tables") run under regular user accounts instead of root, therefore their scripts can only access those directories and applications that the user can. This arrangement allows any user to create a personal schedule of automated commands without risk to essential system files.
User crontabs differ from the system crontab in that you cannot edit user crontabs directly. For one thing, user crontabs and the directory in which they reside are owned by root, thus are inaccessible to non-root users. The proper way to edit user crontabs is with the crontab utility (not to be confused with the crontab files it creates). Known as a "setuid root program," crontab has had its permissions set so it will always run as root, a functionality that provides you the link to the otherwise restricted locations. The crontab program also checks that your crontab is formatted correctly before installing it as /private/var/cron/tabs/username.
Before you begin with the crontab utility, however, you'll first need to formulate and test the command line you'll use in your cron job. Base your command line on this, which is what I would use on my machine:
30 18 * * * sh ~/bin/backup.sh 2>&1 | mail -s "Daily Backup Report" chris
|
Previously in the Series
Learning the Mac OS X Terminal: Part 1 Learning the Mac OS X Terminal, Part 2 Learning the Mac OS X Terminal, Part 3 Learning the Mac OS X Terminal, Part 4 |
This line should be easy to understand if you remember the system crontab in Part 1. What does differ, however, is the lack of the sixth "user" field in the user crontab. This field in the system crontab identifies which user account the job should run under. Since, of course, the user crontab will always run under that user's account, that field is unnecessary in the user crontab.
The next field holds the actual command line to be run. To roughly paraphrase:
First, use the Bourne shell to run the script we've created as ~/bin/backup.sh:
sh ~/bin/backup.sh
Next, send all of the output from that script, including error messages, on to the next command using the pipe character ("|"):
2>&1 |
Finally, have the mail utility receive the input from the pipe, use it as the body of a new mail message with the subject "Daily Backup Report" and send it to user "chris":
mail -s "Daily Backup Report" chris
The line you should enter will differ only in the scheduling fields and the account name used at the end. You probably want to schedule for a time when you're not usually busy on your Mac, but if you do happen to be working when the job starts, you won't notice much, if any, disruption from ditto running in the background (depending on your machine's performance, of course).
To test the command from the prompt, first change to the Bourne shell (just type sh and a return):
[localhost:~] chris% sh
localhost%
Then enter the command line (not the scheduling fields or the sh command):
localhost% ~/bin/backup.sh 2>&1 | mail -s "Daily Backup Report" chris
localhost%
If all went well, you should just receive a new prompt, and then in a few moments the mailed report should arrive looking something like this:
From root Tue Jun 25 21:17:54 2002
Date: Tue, 25 Jun 2002 21:17:54 -0700 (PDT)
From: Chris <chris>
To: chris
Subject: Daily Backup Report
Results of the daily backup:
>>> Copying /Users/chris/Documents/Microsoft User Data/Office X Identities
copying file ./.DS_Store ... 6148 bytes
copying file ./Main Identity/Database ... 10047232 bytes
copying file ./Main Identity/Database Cache ... 17092 bytes
copying file ./Main Identity/Mailing Lists ... 20784 bytes
copying file ./Main Identity/Rules ... 20784 bytes
copying file ./Main Identity/Signatures ... 12560 bytes
copying file ./Newsgroup Cache ... 8 bytes
To leave the Bourne shell and return to tcsh, just type in exit, and you'll get a new tcsh prompt:
localhost% exit
[localhost:~] chris%
Once you've checked that the files have been copied correctly, you're ready to use the crontab utility, which actually hands off much of the job to the text editor of your choice. By default, this is the vi editor. If you are already familiar with vi and would like to use it to edit your crontab, skip this next command. Otherwise, since by now you're probably most comfortable with pico, set it as your editor with this command:
setenv EDITOR pico
In this case, the setenv command is setting an environment variable called EDITOR to the value pico. This setting is only temporary, however, lasting only for the current shell session (that is, until you close that Terminal window). Therefore, you'll need to issue this command during each session in which you edit your crontab. It's not difficult to make this setting permanent, but I'll have to save that procedure for a future article.
Finally, enter this command to create and edit your user crontab:
crontab -e
(The other crontab options are -l, which displays your crontab, and -r, which removes it.)
Add your line to your crontab in pico as you would in any other file. My example cron job, of course, was set to run everyday at 6:30 p.m., but you might first want to schedule yours to run just a few minutes from when you edit your crontab so you can soon know if it works or not.
Be sure to follow that line with a new empty line, which cron requires. This is what my pico session looks like:

Finally, save the file with the temporary name given, and then close pico as usual. Once you do, you'll see a final line from crontab reporting that your new crontab was installed:
crontab: installing new crontab
[localhost:~] chris%
You can confirm that it has run by just waiting for the email report to come, or you could see it run using the command-line process watcher top. Much like the GUI Process Watcher application inside /Applications/Utilities, top lists the running processes one per line. If you run top with its -u flag, you'll see the list dynamically ordered, with the most active processes at top:
[localhost:~] chris% top -u
Processes: 52 total, 2 running, 2 stuck, 48 sleeping... 137 threads 13:03:21
Load Avg: 0.91, 0.59, 0.41 CPU usage: 6.8% user, 29.9% sys, 63.2% idle
SharedLibs: num = 89, resident = 22.7M code, 1.52M data, 5.87M LinkEdit
MemRegions: num = 3185, resident = 91.2M + 8.32M private, 59.7M shared
PhysMem: 79.0M wired, 58.2M active, 340M inactive, 477M used, 419M free
VM: 2.08G + 44.6M 6656(0) pageins, 0(0) pageouts
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
434 ditto 11.9% 0:02.31 1 16 17 424K 296K 648K 1.79M
436 top 7.6% 0:00.38 1 14 17 268K 320K 524K 1.82M
383 Terminal 5.1% 0:13.65 8 117 243 2.89M 10.7M 8.84M 109M
375 Microsoft 3.4% 6:44.00 2 79 292 12.8M 18.0M 19.7M 119M
0 kernel_tas 2.5% 1:14.20 27 0 - - - 64.8M- 733M-
382 CPU Monito 2.5% 0:32.75 1 67 88 1.21M 6.29M 3.07M 100.0
356 Window Man 1.7% 0:59.54 3 156 152 2.01M 20.0M 21.8M 79.5M
376 Microsoft 0.8% 2:14.07 8 128 283 14.3M 31.0M 33.8M 142M
367 Finder 0.0% 0:35.43 2 93 365 21.0M 15.9M 26.1M 130M
377 TextEdit 0.0% 0:34.01 2 93 130 7.56M 9.22M 11.6M 109M
0 idle_threa 63.9% 24:26.99
|
Related Reading Mac OS X Pocket Reference |
There at the top of the list you can see ditto getting started. Once it has finished, that process will leave the list. To stop top and return to the prompt, just press q. As you can see, top tells you a lot more about your system, so start with its man page (man top), to learn all about it. Also, here's a good page from Apple that describes top.
Finally, once you're sure it's working, don't forget to go back and reset your crontab to the time you want it to run regularly.
This article introduced you to the basics of shell scripting and the user crontab, both very powerful features of Mac OS X's Unix. Stay tuned for future articles that show still more ways to put this power to use.
Chris Stone is a Senior Macintosh Systems Administrator for O'Reilly, coauthor of Mac OS X in a Nutshell and contributing author to Mac OS X: The Missing Manual, which provides over 40 pages about the Mac OS X Terminal.
Return to the Mac DevCenter.
You must be logged in to the O'Reilly Network to post a talkback.
Showing messages 1 through 24 of 24.
-
Perl Scripts
2006-01-09 14:57:28 Cruzapete [Reply | View]
-
Perl Scripts
2006-01-18 06:43:21 tankl [Reply | View]
OS X has built-in PERL support. Run "which perl" to show the path of your PERL interpretor.
G4-Cube:/private/etc kltan$ which perl
/usr/bin/perl
Another note is Windows PERL script does not require the shebangs line, but is required for your UNIX/OSX PERL script. Just enter the shebangs line at the top of your script as below.
#!/usr/bin/perl
...your script..
good luck.
-
rehash
2004-12-03 14:36:17 ayemose [Reply | View]
i was reading this tutorial because i want to back up some files automatically but when i got to the point where i enter the command rehash it says command not found,is it because i am using 10.3.5.please help
-
This is almost exactly what I've been looking for.
2004-08-19 05:49:04 mcbeewl [Reply | View]
Hello Chris,
Thank you very much for the very helpful article. I have an Entourage Database that I need to backup on a regular basis. I've got a few questions about the solution in the tutorial.
1) I see this article was written in 2002, should it still work in Panther 10.3.x? Or have too many things changed?
2) Will this solution work weather the user is logged in or not?
3) If the user is logged in, is there a way to make sure that Entourage is not running. And, if so, that it is properly quit saving open windows. I ask because I could imagine that the copy could go wrong if that user receives and email during the copy.
4) I think I will use hdiutil instead of ditto, or I will use ditto to a disk image. Would hdiutil work in place of ditto?
5) I'd like for this to run daily, but I only want last two days to be saved. So for instance, if it ran on Mon. and Tues. it would result in two files (i.e. "Backup [Mon. date]" and "Backup [Tues. date]"). Then on Wed. Monday's backup would be deleted and Wednesday's written.
Your help or comments on any of this will be greatly appreciated.
Thank You.
-
ditto over the Network
2003-01-03 05:08:28 anonymous2 [Reply | View]
Thank you Chris !!!!!
I would like to have the copy on a other Mac in the Network. With Finder mounted Disk it works fine but without monted disk?
What about to Backup al local Volumes to a external Disk?
what i like to do is some thing like:
ditto rsrcFork /Volumes ssh root@remoteMac /Volumes/Backup/
OK ,Ok i know there is retrospect.
just let me know if someone has a way to do that.
paul.dusby@bama.ch
-
What about tar?
2002-11-26 18:14:51 anonymous2 [Reply | View]
The article makes an interesting point (which I didn't know) that the unix cp command does not copy everything. What about tar? My current back up scripts us tar, but I haven't tested whether the OS X specific file attributes are preserved with tar. (backing up with tar is more convenient because one can easily split and compress the the tar file to fit on b/u media)
-
What about psync?
2002-10-28 12:42:38 anonymous2 [Reply | View]
Psync does what you are recommending better than any other command line app. You can learn more at www.macosxhints.com (do a search on psync).
-
The (missing) -rsrc flag
2002-08-11 01:16:17 lfransson [Reply | View]
Is it just me, or does the man page for ditto not say anything about a -rsrc flag? The man page on my system is dated August of 1994.
-
Change many file names?
2002-08-05 11:52:17 Subhash [Reply | View]
Can anybody tell me if there is a way to add a suffix to a lot of - let's say - "*.fp5"-files?
I know there is an AppleScript, but I would like to write the command in my backup.sh
Thanks!
-
Yery useful!
2002-08-05 01:34:24 Subhash [Reply | View]
I'm very pleased to learn this lesson!
Thank you very much Chris for this useful utility!
Subhash
(Austria)
-
ditto and find
2002-07-24 04:20:17 sperling [Reply | View]
Thank you for another informative article. I learned of the existence of ditto!
Why not use ditto with the Unix tool 'find'? This will eliminate need for rsync.
An excellent introduction to find is found in AEleen Frisch's book about Essential System Administration, and in fact the bit about find is in the free sample chapter (which is so good that you will run out to buy the book):
http://www.oreilly.com/catalog/esa2/chapter/ch03.html
It is fairly straightforward to write a shell script that combines ditto and find to back things up in quite a tailored fashion (for example, excluding Library or other directories). The only problem, mentioned in many of the other messages, is the ditto filename length problem.
-
Long filename problem
2002-07-23 07:01:34 gavinaiken [Reply | View]
Thanks for the article - I'm a long-term Mac and Unix user but learnt loads anyway - I'd never heard of ditto for example.
I've tried putting some of it into practice to backup my OSX system to a Sun server running ethershare, which I was doubtful about at first. However, ditto seems to be smart enough even to make that work. I did run into problems with long file names though, e.g. I tested backing up a few apps but got the following:
[localhost:~] gavin% ditto -rsrc -V bin /Volumes/Mac\ Backups\ 2-1/bin
>>> Copying bin
copying file ./.DS_Store ... 6148 bytes
copying file ./beep ... 9116 bytes
copying file ./hello ... 9116 bytes
copying file ./notepad.app/Contents/Info.plist ... /Volumes/Mac Backups 2-1/bin/notepad.app/Contents/Info.plist.__XXCOPIERXX__.1267.3: File name too long
/Volumes/Mac Backups 2-1/bin/notepad.app/Contents/Info.plist: File name too long
Any idea if ditto will ever support long(er) filenames? I also tried with rsync, after downloading rsyncX, and got similar problems:
[localhost:~] gavin% rsync -ae /usr/bin/ssh bin /Volumes/Mac\ Backups\ 2-1
fopen: No such file or directory
/bin/rm: notepad.app/Contents/Contents-dirattr.145757-rsynckbattr: No such file or directory
fopen: No such file or directory
/bin/rm: notepad.app/Contents/MacOS/MacOS-dirattr.145757-rsynckbattr: No such file or directory
mkstemp notepad.app/Contents/Resources/English.lproj/.English.lproj-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/English.lproj/.English.lproj-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/English.lproj/.English.lproj-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/English.lproj/MainMenu.nib/.MainMenu.nib-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/English.lproj/MainMenu.nib/.MainMenu.nib-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/English.lproj/MainMenu.nib/.MainMenu.nib-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/.Resources-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/.Resources-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/Contents/Resources/.Resources-dirattr.145757.001996 failed: File name too long
fopen: No such file or directory
/bin/rm: notepad.app/Contents/Resources/Scripts/Scripts-dirattr.145757-rsynckbattr: No such file or directory
mkstemp notepad.app/.notepad.app-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/.notepad.app-dirattr.145757.001996 failed: File name too long
mkstemp notepad.app/.notepad.app-dirattr.145757.001996 failed: File name too long
rsync error: partial transfer (code 23) at main.c(578)
-
Me too
2002-07-23 15:27:37 mike_dowe [Reply | View]
I got the long filename problem too, eg. "copying file ./Acrobat User Data/.FBCLockFolder/.FBCSemaphoreFile ... /Volumes/Backup/Documents/Acrobat User Data/.FBCLockFolder/.FBCSemaphoreFile.__XXCOPIERXX__.327.2: File name too long"
This happened when I backed up to a Zip Disk. I dragged these files to their destination using the Finder with no problems.
-
Sudo and ditto
2002-07-08 02:26:01 tjj [Reply | View]
Works very well, thank you.
I like to back up my ~folder, once in a while. And if I run a script with ditto from my crontab it chokes:
copying file ./Library/Preferences/com.apple.NetInfoManager.plist ... com.apple.NetInfoManager.plist: Permission denied
/Volumes/FirewireDrive2/HomeBackUp/tjj/Library/Preferences/com.apple.NetInfoManager.plist: Permission denied
If the script is run from terminal: sudo backup.sh, no problem. And ditto IS much faster copying 1.7 Gb than RsyncX sync'ing 1.4 Gb. But how do I get around the sudo issue with ditto?
Regards
Thomas Jon
-
Nice!
2002-07-05 00:12:51 tjj [Reply | View]
Thank you once again for a very informative piece!
I have been using a rather inelegant solution to back up. An apple script with a terminal command using rsync. The script is started from system's crontab. I gather I will have problems with resource forks..I back up my entire ~ folder. If I ditto this (1.5 Gb) it takes about seven minutes. I haven't yet tried the suggested RsyncX. Ditto's man page states that it must be run as root, does this imply that you will have troubles ditto'ing files that is not owned by root? I also use an apple script to start a terminal session that in turn starts the command line scan engine from Virex. This script is started from root's crontab. If I start this from a user crontab will it still be able to scan system files?
Obviously I'm going to try to write shell scripts to do these jobs, but I'm not sure if they should be run from root's crontab or a user's.
Keep the articles comin';)
Regards Thomas Jon
-
Nice!
2002-07-10 13:33:20 Chris Stone |
[Reply | View]
You're welcome, Thomas. Glad you liked the article.
Contrary to what the man page states, you don't need to be root to run ditto.
Also, if you need access to system files, or other files your account hasn't acces to withough sudo , you'll want to use a root crontab (not to be confused with the system crontab we looked at earlier).
To do this, first get a root shell session with sudo -s, and then follow the same steps you did to create a user crontab. When you're done, type exit to leave the root shell session.
BE CAREFUL, though....a root crontab can really make a mess of things if you've not set it up right, and it will do so at regularly scheduled intervals...;-)
--Chris
-
Cronning for non 24 hour computers
2002-07-03 13:41:56 kwidholm [Reply | View]
Chris, another excellent article! However, since a lot of Macs aren't used as 24 hour servers, and are put to sleep often, using Cron the old fashioned way doesn't really hold up for regular maintenance to be performed. I have written a Perl script that gets executed every hour which simply checks to see how long it's been since a particular task has been performed, compares the time with how often that task should be performed, and, if necessary executes the task. The script is extensible to any maintenance tasks that have output to a log file (and any task can be made to do so). The script also sends an e-mail only if the task(s) have been executed, including full script output.
Here's the script. You can also download it from <http://www.theapotek.com/teknotes/XJanitor.pl.sit>
#!/usr/bin/perl -w
use strict;
# Janitor script. Run through root cron on a daily or hourly basis. The script
# will check for daily, monthly, weekly maintenance, and, if necessary
# will perform the maintenance needed.
# By Kristofer Widholm, <http://www.theapotek.com>
# A script like this should always be in the public domain 'cause anybody
# can do it. Accordlingly, so is this script.
# SCRIPT VARIABLES
my($summary) = "";
my($sinceMod,$i);
my($done) = 0;
my(@fileinfo,@tests);
# END SCRIPT VARIABLES
# YOUR CONFIGURATION VARIABLES
my($admin) = "admin"; # Local mail address (usually username) of sysadmin
my($hostname) = `hostname`; # Or whatever you call your machine
my($sleep) = 60; # How many seconds should the script pause between each executed script?
# Check modified time on out files
# Pass the name of the test, the logfile, the script name,
# and the interval at which the script should run, expressed in seconds
# Array of: test name -Location of log -Script -Interval
$tests[0] = (["daily","/var/log/daily.out","/etc/daily",86400]);#1 day
$tests[1] = (["weekly","/var/log/weekly.out","/etc/weekly",604800]);#7 days
$tests[2] = (["monthly","/var/log/monthly.out","/etc/monthly",2419200]);#28 days
# Add other scripts you want to run here:
$tests[3] = (["quarterly","/var/log/quarterly.out","/Users/admin/Library/Scripts/quarterly",7889400]);#91.3 days
#$tests[4] = (["test","/Users/kit/Desktop/test.out","/Users/kit/Desktop/test",60]);
# END YOUR CONFIGURATION
for ($i = 0; $i <= $#tests; $i++) {
@fileinfo = stat($tests[$i][1]); #[$i][1] is the output file
$sinceMod = time() - $fileinfo[9]; #Last modified is index 9 in file info array returned by stat
if ( $sinceMod > ($tests[$i][3]) ) { #index [$i][3] is the running interval
if (system("$tests[$i][2] >& $tests[$i][1]") > -1) {
$summary .= ($i+1) . "> XJanitor ran the $tests[$i][0] script:\n";
$summary .= `cat $tests[$i][1]`;
$summary .= "\n\n";
$done++;
if ($i < $#tests) { sleep($sleep); } # Sleep between scripts, but not after last one.
}
else {
$summary .= ($i+1) . "> ERROR: XJanitor could not run the $tests[$i][0] script!!!\n";
}
}
else {
$summary .= ($i+1) . "> XJanitor did not need to run the $tests[$i][0] script.\n";
}
}
if ($done > 0) { # Don't bother me with mail if scripts weren't run
chop($hostname);
open(MAIL,"| mail -s \"XJanitor output ($hostname)\" $admin");
print MAIL `date` . "\n";
print MAIL "$summary";
close(MAIL);
}
exit;
-
Cronning for non 24 hour computers
2002-07-12 21:32:41 dm2243 [Reply | View]
I got an error
Use of uninitialized value in subtraction (-) at /etc/XJanitor.pl line 38.
and the script stalled.
thoughts?
dm -
Cronning for non 24 hour computers: Addendum on permissions
2002-07-03 13:55:24 kwidholm [Reply | View]
If you want to use the above script for the maintenance tasks, you'll need to be sure that the user running the script has permission to run the daily, weekly, monthly scripts as well as permission to output to the log directory. I have set up a separate admin user on my machine, and granted rights only to this user to these files. The /var/log directory is owned by admin, and in general I've let admin take over many of the tasks normally performed by root, in order to minimize the amount of processes someone could hijack.
Alternatively, for system-wide administration, run XJanitor.pl from the system cron, and for personal ones, run a different copy of XJanitor.pl, with different parameters set to run scripts and output that require no administrative privileges (as in the backing up example above). Each user could have his or her own copy of XJanitor.pl in her ~/Library/Scripts folder, for example. -
Cronning for non 24 hour computers: URL Fix
2002-07-03 13:47:46 kwidholm [Reply | View]
The actual URL for the script mentioned above is <http://www.theapotek.com/teknotes/XJanitor.sit>. Evidently, my Web host won't let anybody download files with '.pl' in them unless they're in the cgi-bin directory.
-
Thanks, Chris!
2002-07-03 08:48:14 jeb1 [Reply | View]
I'm just learning the ins and outs of the Terminal.app so articles like this are a great resource for newbies like myself.
I printed out this article and went throught the entire tutorial and felt pretty good when everything went exactly as shown.
Keep them coming!







I was a windows user but moved to Mac OS X. I have some perl scripts I run on MS-DOS cmd.exe but have been having problems running those scripts on Mac OS X terminal.
Is there any command used for running perl scripts or how do I go about it.
Thanks for responding.
Cruzapete