Cron Job Best Practices

Cron is a timer, not a job runner. It has no retries, no locking, no alerting, no history and no notion of a job having failed — it starts a process at a time and forgets about it. Everything that makes scheduled work survivable in production has to be added by you, in the script or around it. These are the practices worth adding first, roughly in the order they stop hurting.

Stop jobs overlapping themselves

Cron starts a new instance on schedule whether or not the previous one has finished. A five-minute job that occasionally takes twelve minutes will, on a bad day, have three copies running against the same database. The usual symptoms are duplicated rows, lock contention that makes the job slower still, and a pile-up that ends in an out-of-memory kill.

flock solves this in one line and belongs on essentially every recurring job:

# -n: exit immediately if the lock is held
*/5 * * * * /usr/bin/flock -n /var/lock/sync.lock /usr/local/bin/sync.sh

# -w 60: wait up to 60s for the lock, then give up
0 * * * * /usr/bin/flock -w 60 /var/lock/report.lock /usr/local/bin/report.sh

The choice between -n and -w is a real decision. -n is right when a skipped run is harmless because the next one will catch up — polling, syncing, cache warming. -w is right when every run must happen eventually, and you would rather it be late than missing. What you almost never want is the default: waiting indefinitely, which converts an overlap into an unbounded queue of stalled processes.

For a distributed setup, a file lock is not enough — each host has its own filesystem. Use a shared lock in a database or key-value store with a TTL, so a host that dies mid-job releases the lock rather than blocking every future run. On Kubernetes, concurrencyPolicy: Forbid does this for you, and defaults to Allow if you do not set it.

Make jobs idempotent

No scheduler gives you exactly-once execution. Cron can run a job twice across a daylight saving transition; Kubernetes documents that a CronJob may create two Jobs for one schedule; EventBridge delivers at-least-once; Quartz replays misfires after an outage. A job that is only correct when run exactly once is a job that is intermittently incorrect.

Idempotency is usually cheaper than it sounds:

• Key the work on a business date, not on "now". A job that processes "yesterday" computes the same result whether it runs once or three times.
• Upsert instead of insert. A unique constraint plus ON CONFLICT DO NOTHING turns a duplicate run into a no-op.
• Record what was done. A small table of completed periods lets the job check before it starts and exit cleanly if the work is already there.
• Write to a temporary file and rename. mv within a filesystem is atomic, so a half-written output file is never visible to readers.

The test is simple: run the job twice by hand, back to back, and check that the second run changes nothing. If it does, fix that before adding monitoring — because your monitoring will eventually cause a retry.

Stagger schedules off the top of the hour

Left to their own devices, everyone writes 0 * * * * and 0 0 * * *. On a single host this means every job starts in the same second, competing for CPU, disk and database connections while the machine sits idle the rest of the time. Across a fleet, it means a synchronised thundering herd against whatever shared service they all call.

# Instead of three jobs at 03:00
0 3 * * * /usr/local/bin/backup.sh
0 3 * * * /usr/local/bin/rotate-logs.sh
0 3 * * * /usr/local/bin/send-report.sh

# Spread them out
7  3 * * * /usr/local/bin/backup.sh
23 3 * * * /usr/local/bin/rotate-logs.sh
41 3 * * * /usr/local/bin/send-report.sh

On managed platforms this is not just tidiness. GitHub Actions explicitly deprioritises scheduled runs at peak times, and the top of the hour is the peak — an offset of a few minutes measurably improves how promptly workflows start. Jenkins has H for exactly this purpose, and systemd has RandomizedDelaySec. Plain cron has neither, so the offset has to be written by hand.

Exit codes, logging and output

Cron's only feedback channel is email, and on most modern hosts there is no mail transport, so output is discarded with a log line most people never read. Replace it with something deliberate.

Start every shell script with strict mode, so a failing command actually fails the job:

#!/usr/bin/env bash
set -euo pipefail

Without -e, a script continues past errors and exits 0, and cron reports success. Without -o pipefail, generate | upload succeeds whenever upload does, even if generate died. Without -u, an unset variable becomes an empty string, and rm -rf "$DIR/" becomes rm -rf /.

Then log with timestamps, to a file that is rotated:

0 3 * * * /usr/local/bin/nightly.sh >> /var/log/myapp/nightly.log 2>&1

Add a logrotate rule, or the disk-full incident arrives on its own schedule:

# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    copytruncate
}

Have the job log a start line and an end line with the exit code and the duration. When someone asks "did last night's run work", that is the answer, and it costs two echo statements.

Monitoring: alert on absence, not failure

The failure mode that costs the most is not a job that errors — it is a job that stops running entirely. A crontab deleted by a stray crontab -r, a host replaced without its cron config, a container image rebuilt without the daemon, a GitHub Actions schedule auto-disabled after 60 days of repo inactivity. In every case the job produces no error, because it produces nothing at all. Error-based alerting is silent by construction.

The fix is a dead man's switch: the job checks in when it succeeds, and the monitor alerts when a check-in does not arrive on time.

#!/usr/bin/env bash
set -euo pipefail

/usr/local/bin/do-the-work.sh
curl -fsS --retry 3 https://monitor.example.com/ping/nightly-report > /dev/null

Put the ping after the work, not before, and never in a trap that fires on any exit — it should only run when the job genuinely succeeded. Several hosted services do this (Healthchecks.io, Cronitor, Better Stack among them), and it is a couple of dozen lines to build against your own Prometheus Pushgateway or alerting system if you would rather not add a dependency.

Pair it with two other signals: alert on non-zero exit codes, and alert on runtime — a job that normally takes 90 seconds and today took 40 minutes has a problem, even though it succeeded.

Retries and backoff

Cron does not retry. For a job that talks to a network, a single transient failure means a missed run until the next schedule — acceptable for a five-minute poll, not for a nightly report.

Retry inside the job, with backoff, and only for errors worth retrying:

for attempt in 1 2 3; do
  if /usr/local/bin/upload.sh; then
    exit 0
  fi
  echo "attempt $attempt failed, retrying in $((attempt * 30))s" >&2
  sleep $((attempt * 30))
done
echo "all attempts failed" >&2
exit 1

Two constraints. Keep the total retry window shorter than the schedule interval, or runs will overlap — which is what the lock is for, but overlapping-and-locked is still a job that silently stops doing work. And do not retry non-transient failures: a 401, a malformed config or a missing file will fail identically three times and only delay the alert.

Secrets and least privilege

Cron jobs accumulate privilege quietly, because the easy fix for any permission error is to run the job as root. A few habits prevent that:

• Run as a dedicated unprivileged user. In a system crontab the sixth field names it; use a service account with access to exactly the paths the job needs.
• Never put secrets in the crontab. crontab -l is readable by anyone who can become that user, and the spool file often ends up in backups. Read secrets from a file with mode 600, or from a secrets manager at run time.
• Never put secrets on the command line. Arguments are visible in ps to every user on the host.
• Check the permissions of the script itself. A world-writable script run by a privileged cron job is a straightforward privilege escalation.
• Scope credentials to the job. A backup job needs read on the database and write on one bucket, not an administrator role.

Version-control the crontab

A crontab edited by hand on a production host has no history, no review and no backup. When the host is replaced, the schedule goes with it, and nobody can say what it contained.

Keep the authoritative copy in the repository and install it as a unit:

# Deploy the whole crontab atomically
crontab deploy/crontab.txt

# Verify what is actually installed
crontab -l | diff -u deploy/crontab.txt -

Better still, let configuration management own /etc/cron.d/ so the schedule is declared alongside the rest of the host's state. Either way, put a comment above each line saying what the job is for, who owns it, and which time zone the schedule is written in — the next person to read it at 3 AM will not have that context otherwise.

When you have outgrown cron

Cron is excellent at one thing: starting a process at a time, on one machine, with no dependencies. Several requirements push past that, and bolting them onto cron with lock files and sentinel files tends to cost more than adopting the right tool:

• Dependencies between jobs. "Run B after A succeeds" is a workflow engine's job — Airflow, Dagster, Temporal, Step Functions. Encoding it as "A at 02:00, B at 02:30 and hope" is a scheduled race condition.
• Run-once-per-fleet semantics. If the same crontab is on ten hosts, you need a distributed lock or a leader election, at which point a scheduler with that built in is simpler.
• Visibility and history. If people routinely ask whether a job ran and what it did, you want a system with a run history and a UI, not grep over rotated logs.
• Backfills. Re-running last Tuesday because the upstream data was wrong is a first-class operation in a workflow engine and a manual exercise under cron.
• Sub-minute or interval-accurate scheduling. Cron's floor is one minute and its steps reset each hour; a long-running process with its own timer is a better fit.

None of this means cron is the wrong default. For a single self-contained task on a single host — a backup, a cleanup, a report — it is the simplest thing that works, and the practices above are what keep it working. When you do write the next schedule, compose it in the builder and read the computed run times before it ships; that thirty seconds catches the class of mistake that no amount of monitoring will.