Cron jobs don't announce themselves. They run quietly in the background, doing their thing — backing up databases, rotating logs, sending emails, running reports. Most of the time they're fine. But when two jobs happen to run at the same time and compete for the same resources, things get ugly fast.
The frustrating part is that the problem is invisible in the logs. You see high CPU usage, maybe some slow queries or timeouts, and then everything goes back to normal. Fifteen minutes later it's fine. Nothing in the error logs. No clear cause. Just a recurring mystery that gets worse over time as you add more jobs.
What actually happens when cron jobs overlap
Take a simple example. You have a database backup that runs every night at midnight and takes about 20 minutes. You also have a report generation job that runs at midnight. And a log cleanup script — also midnight, because that's the obvious "run it when no one's around" time.
At 00:00 all three jobs start simultaneously. The backup is hammering disk I/O. The report generator is running heavy queries against the same database. The log cleanup is scanning the filesystem. The server CPU spikes. The database slows down. If any of your services have health checks or timeouts, they start failing. Users hitting your site at 12:01 AM get slow responses or errors.
This is the midnight pile-up. It's the most common cron scheduling problem on the internet and it's almost entirely avoidable.
The hidden version: jobs that run too long
The midnight pile-up is obvious once you know to look for it. The harder problem is when a single job occasionally runs longer than its interval.
You have a job scheduled every 15 minutes. Most of the time it runs in 3 minutes and exits. But occasionally — when the database is under load, or the network is slow, or there's just more data than usual — it takes 18 minutes. At minute 15, cron fires a second instance of the same job. Now you have two copies running simultaneously, both competing for the same database connections, both writing to the same output files.
# This looks innocent: */15 * * * * /usr/bin/process-queue.sh # But if process-queue.sh takes 18 minutes, you get this: # 00:00 — instance 1 starts # 00:15 — instance 2 starts (instance 1 still running) # 00:18 — instance 1 finishes # 00:30 — instance 3 starts (instance 2 still running) # ... and so on
This compounds. Under heavy load the job takes longer. A longer job means more overlap. More overlap means more load. More load means the job takes even longer. It's a death spiral that's nearly impossible to debug after the fact because by the time you look at it, the jobs have already finished.
Why this is hard to see in your crontab
Look at a typical crontab:
0 2 * * * /usr/bin/db-backup.sh */30 * * * * /usr/bin/sync-data.sh 0 * * * * /usr/bin/health-check.sh 30 1 * * * /usr/bin/generate-report.sh 0 0 * * * /usr/bin/cleanup-logs.sh */5 * * * * /usr/bin/process-queue.sh
Reading this and mentally computing all the overlaps is genuinely hard. The */30 job runs at :00 and :30 of every hour. The 0 * * * * job runs at the top of every hour. So they overlap at :00 of every hour. The cleanup job at midnight and the health check at midnight also overlap. The report job at 1:30 runs for how long? If it takes more than 30 minutes it will still be running when the 2am backup starts.
Six jobs and you already need a spreadsheet to track the interactions. Real crontabs have 15-30 jobs. The mental model breaks down completely.
How to find every overlap
The right approach is to visualize the schedule — render every job as a bar on a timeline and see where they overlap visually.
# Step 1: export your crontab crontab -l # Step 2: paste into a visualizer and look for overlapping bars
Paste your crontab and see every job on a 24-hour timeline. Overlaps are flagged in red with exact conflict windows and severity ratings.
Open Cron Visualiser →Fixing overlaps: stagger the schedule
The simplest fix for the midnight pile-up is to spread jobs across different minutes:
# Before — three jobs colliding at midnight: 0 0 * * * /usr/bin/cleanup-logs.sh 0 0 * * * /usr/bin/generate-report.sh 0 0 * * * /usr/bin/db-backup.sh # After — staggered by 10 minutes: 0 0 * * * /usr/bin/cleanup-logs.sh 10 0 * * * /usr/bin/generate-report.sh 20 0 * * * /usr/bin/db-backup.sh
This doesn't prevent overlap if jobs run longer than their stagger window — but it eliminates the simultaneous start problem.
Fixing concurrent runs: flock
For jobs that might run longer than their interval, use flock to prevent concurrent execution:
# Without flock — two instances can run simultaneously: */15 * * * * /usr/bin/process-queue.sh # With flock — second instance exits immediately if first is still running: */15 * * * * flock -n /tmp/process-queue.lock /usr/bin/process-queue.sh
The -n flag means non-blocking — if the lock is held, exit immediately rather than waiting. The lock file is automatically released when the first process finishes, even if it crashes.
What to check in your crontab right now
Three patterns to look for:
Jobs scheduled at exactly 0 0 * * * — midnight is the most over-scheduled minute in every crontab. Count how many of your jobs run at midnight and spread them out.
Jobs with short intervals doing heavy work — anything running every 1, 5, or 15 minutes that touches a database or makes HTTP requests should have flock protection.
Jobs with no runtime estimate — if you don't know how long a job takes, find out. Run it manually with time /usr/bin/yourscript.sh and compare against its interval.