Cron Job Best Practices

Cron is a timer. It starts a process at a set time and then forgets about it. It does not retry, lock, alert or keep a history, and it has no concept of a job failing. Anything that makes scheduled work dependable in production has to be added by you, in the script or around it. The practices below are listed roughly in the order worth adopting them, starting with the ones that prevent the most damage.

Stop jobs overlapping themselves

Cron starts a new instance on schedule whether or not the previous one has finished. A five-minute job that occasionally takes twelve minutes will, on a bad day, have three copies running against the same database. The usual symptoms are duplicated rows, lock contention that makes the job slower still, and a pile-up that ends in an out-of-memory kill.

flock fixes this with one line, and nearly every recurring job should use it:

# -n: exit immediately if the lock is held
*/5 * * * * /usr/bin/flock -n /var/lock/sync.lock /usr/local/bin/sync.sh

# -w 60: wait up to 60s for the lock, then give up
0 * * * * /usr/bin/flock -w 60 /var/lock/report.lock /usr/local/bin/report.sh

Choose between -n and -w based on what a skipped run costs. -n fits jobs where a skipped run is harmless because the next one will catch up, such as polling, syncing or cache warming. -w fits jobs where every run has to happen eventually and a late run is better than a missing one. With neither flag, flock waits indefinitely, which turns an overlap into an unbounded queue of stalled processes. That default is almost never what you want.

A file lock does not work across several hosts, because each host has its own filesystem. Use a shared lock in a database or key-value store with a TTL, so that a host which dies mid-job releases the lock instead of blocking every future run. On Kubernetes, concurrencyPolicy: Forbid handles this for you. If you do not set it, the policy defaults to Allow.

Make jobs idempotent

No scheduler guarantees exactly-once execution. Cron can run a job twice across a daylight saving transition. The Kubernetes documentation says a CronJob may create two Jobs for one schedule. EventBridge delivers at least once, and Quartz replays misfires after an outage. If a job is only correct when it runs exactly once, it will be wrong some of the time.

Making a job idempotent usually takes less work than people expect:

• Key the work on a business date, not on "now". A job that processes "yesterday" computes the same result whether it runs once or three times.
• Upsert instead of insert. A unique constraint plus ON CONFLICT DO NOTHING turns a duplicate run into a no-op.
• Record what was done. A small table of completed periods lets the job check before it starts and exit cleanly if the work is already there.
• Write to a temporary file and rename. mv within a filesystem is atomic, so a half-written output file is never visible to readers.

To test it, run the job twice by hand, back to back, and check that the second run changes nothing. If it does change something, fix that before you add monitoring, because the monitoring will eventually cause a retry.

Stagger schedules off the top of the hour

Most people write 0 * * * * and 0 0 * * * by default. On a single host, that means every job starts in the same second and competes for CPU, disk and database connections, while the machine sits idle the rest of the time. Across a fleet, every host calls the same shared service at the same moment, in a synchronised thundering herd.

# Instead of three jobs at 03:00
0 3 * * * /usr/local/bin/backup.sh
0 3 * * * /usr/local/bin/rotate-logs.sh
0 3 * * * /usr/local/bin/send-report.sh

# Spread them out
7  3 * * * /usr/local/bin/backup.sh
23 3 * * * /usr/local/bin/rotate-logs.sh
41 3 * * * /usr/local/bin/send-report.sh

On managed platforms the offset has a practical effect. GitHub Actions explicitly deprioritises scheduled runs at peak times, and the top of the hour is the peak, so moving a workflow a few minutes off it measurably improves how promptly it starts. Jenkins provides H for this purpose, and systemd provides RandomizedDelaySec. Plain cron has neither, so you write the offset by hand.

Exit codes, logging and output

Cron reports back only by email. Most modern hosts have no mail transport, so the output is discarded and cron writes a log line that most people never read. Set up logging on purpose instead.

Start every shell script with strict mode, so that a failing command fails the job:

#!/usr/bin/env bash
set -euo pipefail

Without -e, a script continues past errors and exits 0, and cron reports success. Without -o pipefail, generate | upload succeeds whenever upload does, even if generate died. Without -u, an unset variable becomes an empty string, and rm -rf "$DIR/" becomes rm -rf /.

Then log with timestamps, to a file that is rotated:

0 3 * * * /usr/local/bin/nightly.sh >> /var/log/myapp/nightly.log 2>&1

Add a logrotate rule as well, otherwise the log will eventually fill the disk:

# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    copytruncate
}

Have the job log a start line, plus an end line with the exit code and the duration. That answers the question "did last night's run work?" and costs two echo statements.

Monitoring: alert on absence, not failure

The most expensive failure is usually a job that stops running entirely, rather than one that errors. Typical causes are a crontab deleted by a stray crontab -r, a host replaced without its cron config, a container image rebuilt without the daemon, or a GitHub Actions schedule that was auto-disabled after 60 days of repo inactivity. In each case the job produces no error because it does not run, so alerting that waits for an error never fires.

The fix is a dead man's switch: the job checks in when it succeeds, and the monitor alerts when a check-in does not arrive on time.

#!/usr/bin/env bash
set -euo pipefail

/usr/local/bin/do-the-work.sh
curl -fsS --retry 3 https://monitor.example.com/ping/nightly-report > /dev/null

Put the ping after the work, and never inside a trap that fires on any exit. It should run only when the job has succeeded. Several hosted services offer this, including Healthchecks.io, Cronitor and Better Stack. If you would rather not add a dependency, building the same thing against your own Prometheus Pushgateway or alerting system takes a couple of dozen lines.

Pair it with two other alerts: one on non-zero exit codes and one on runtime. A job that normally takes 90 seconds and today took 40 minutes has a problem, even though it succeeded.

Retries and backoff

Cron does not retry. When a job that talks to a network hits a single transient failure, that run is lost and nothing happens until the next scheduled run. That is acceptable for a five-minute poll but not for a nightly report.

Retry inside the job, with backoff, and only for errors worth retrying:

for attempt in 1 2 3; do
  if /usr/local/bin/upload.sh; then
    exit 0
  fi
  echo "attempt $attempt failed, retrying in $((attempt * 30))s" >&2
  sleep $((attempt * 30))
done
echo "all attempts failed" >&2
exit 1

Retries come with two constraints. First, keep the total retry window shorter than the schedule interval, or runs will overlap. The lock stops them running at the same time, but a job that keeps colliding with its own lock still silently stops doing work. Second, do not retry non-transient failures. A 401, a malformed config or a missing file will fail the same way three times and only delay the alert.

Secrets and least privilege

Cron jobs tend to gain privileges over time without anyone noticing, because the easy fix for any permission error is to run the job as root. These habits prevent that:

• Run as a dedicated unprivileged user. In a system crontab the sixth field names it; use a service account with access to exactly the paths the job needs.
• Never put secrets in the crontab. crontab -l is readable by anyone who can become that user, and the spool file often ends up in backups. Read secrets from a file with mode 600, or from a secrets manager at run time.
• Never put secrets on the command line. Arguments are visible in ps to every user on the host.
• Check the permissions of the script itself. A world-writable script run by a privileged cron job is a straightforward privilege escalation.
• Scope credentials to the job. A backup job needs read on the database and write on one bucket, not an administrator role.

Version-control the crontab

A crontab edited by hand on a production host has no history, nobody has reviewed it, and it is not backed up. When the host is replaced, the schedule disappears with it and nobody can say what it contained.

Keep the authoritative copy in the repository and install it as a unit:

# Deploy the whole crontab atomically
crontab deploy/crontab.txt

# Verify what is actually installed
crontab -l | diff -u deploy/crontab.txt -

A better option is to let configuration management own /etc/cron.d/, so that the schedule is declared alongside the rest of the host's state. In both cases, put a comment above each line that says what the job is for, who owns it, and which time zone the schedule is written in. Without it, the next person reading the file at 3 AM will not have that context.

When you have outgrown cron

Cron does one thing well: it starts a process at a set time on one machine, with no dependencies. Some requirements go beyond that, and adding them to cron with lock files and sentinel files usually costs more than switching to a tool built for them:

• Dependencies between jobs. "Run B after A succeeds" is a job for a workflow engine such as Airflow, Dagster, Temporal or Step Functions. Writing it as "A at 02:00, B at 02:30 and hope" builds a race condition into the schedule.
• Run-once-per-fleet semantics. If the same crontab is on ten hosts, you need a distributed lock or a leader election, at which point a scheduler with that built in is simpler.
• Visibility and history. If people routinely ask whether a job ran and what it did, a system with a run history and a UI will serve you better than grep over rotated logs.
• Backfills. Re-running last Tuesday because the upstream data was wrong is a first-class operation in a workflow engine and a manual exercise under cron.
• Sub-minute or interval-accurate scheduling. Cron's floor is one minute and its steps reset each hour; a long-running process with its own timer is a better fit.

Cron is still a sensible default. For a single self-contained task on a single host, such as a backup, a cleanup or a report, it is the simplest option that works, and the practices above keep it working. When you write the next schedule, compose it in the builder and read the computed run times before it ships. Those thirty seconds catch a class of mistake that monitoring cannot.