If a Linux automation job sometimes lasts longer than its scheduling interval, wrap its command with flock so another cooperating invocation skips instead of running simultaneously. Use a stable lock path and nonblocking acquisition. This prevents overlap on that host; it does not make repeated work idempotent or coordinate different servers.
A typical symptom is two processes rewriting the same report or submitting the same request. Before adding a lock, decide whether a busy invocation should skip, wait, or alert. This tutorial uses skip-on-busy behaviour because an unlimited waiting queue can conceal a slow job.
Choose a private, stable lock location
The util-linux flock command creates the named lock file if permissions allow. Use a directory owned by the automation account, preferably on a local filesystem. Avoid a shared world-writable location where other users can replace the path. For a user-owned learning example in Bash:
mkdir -p -- "$HOME/.local/state/report-job"
chmod 700 -- "$HOME/.local/state/report-job"Use the same absolute location in every entry point: scheduled runs, manual launches and recovery procedures. A scheduler may not inherit your interactive environment, so resolve the service account home explicitly in the eventual configuration. A second filename means a second independent lock, even if both commands operate on the same data.
Wrap the foreground job and preserve its result
This Bash example assumes that /opt/report-job/report.py is an existing reviewed script and /usr/bin/python3 is its intended interpreter. Replace those paths after verifying them. Do not run an unknown script merely to try the wrapper.
lock_path="$HOME/.local/state/report-job/run.lock"
flock -n -E 75 "$lock_path" /usr/bin/python3 /opt/report-job/report.py
rc=$?
if [ "$rc" -eq 75 ]; then
printf '%s\n' 'Skipped: another run holds the lock'
fi
exit "$rc"The default lock is exclusive. The -n option refuses to wait, and -E 75 assigns the lock-conflict exit code. When the child command executes, flock returns its exit status. Reserve 75 by agreement: if your script also returns 75, the wrapper cannot distinguish its failure from contention. Record busy skips separately from successful reports in monitoring.
Keep the work in the foreground until it finishes. Background children can complicate lock lifetime through inherited descriptors. The manual documents descriptor-closing options, but adding one without understanding the process tree can release protection while work continues. Start with a simple foreground command and review subprocess behaviour.
Test contention without production data
Open two Bash terminals as the same account. In the first, hold the example lock briefly:
flock -n -E 75 "$HOME/.local/state/report-job/run.lock" sleep 20While that command is still running, use the second terminal:
flock -n -E 75 "$HOME/.local/state/report-job/run.lock" printf '%s\n' 'Lock acquired'
printf 'Exit status: %s\n' "$?"The expected result during contention is no acquisition message and status 75. After the first command exits, repeat the second command; it should print the acquisition message and status zero. These are expected demonstration results, not evidence about your scheduler. Test manual and scheduled entry points before enabling the wrapper for real work.
Understand what the lock does not protect
The lock is cooperative: another program that ignores it can still change the report. All participating commands must use the same locking convention. The manual warns that NFS and CIFS implementations or mount options can limit flock support. A local demonstration does not validate a network filesystem or provide a distributed lock.
Do not delete a lock file just because it exists. The lock is associated with an open file, not a text flag saying busy. Removing and recreating the pathname while a process holds the old file can allow processes to lock different files. Let normal completion release the lock; investigate a stuck process before intervening.
Deployment checklist and common questions
- Confirm executable paths under the scheduled account.
- Verify directory ownership and identical lock paths across entry points.
- Test contention, normal completion and a deliberate child failure.
- Alert on persistent skips instead of silently losing work.
Does this prevent partial output? No. Combine overlap control with atomic JSON state-file writes when readers must never see half-written state. How do I roll back? Restore the previous launch command only after the active run ends and after checking whether concurrent execution is safe.
Reference: util-linux flock manual, including options, exit statuses and filesystem limitations.
This tutorial is for educational purposes. Test carefully before production use. The site owner is not responsible for misuse, damage, data loss or production issues.
