To avoid leaving a half-written JSON state file, serialize the new state first, write it to a temporary file in the destination directory, and replace the destination only after that write succeeds. This lets a small automation job keep its previous state if preparation fails. It does not, by itself, stop two workers from overwriting each other’s updates.
This pattern suits a local script recording a completed report or a last-successful checkpoint. For a shared multi-worker queue, use a transactional store or a carefully designed locking protocol instead of treating a JSON file as a database.
Why opening the live file for writing is risky
Opening an existing file with mode w truncates it before the new contents are complete. If serialization, disk space or the process fails afterward, the next run may find empty or incomplete JSON. Catching the exception does not restore the old bytes.
Python’s official os.replace documentation states that an existing destination file can be replaced silently when permissions permit, and that a successful rename is atomic. It also warns that a move between filesystems may fail. Keeping the temporary file beside the target avoids that particular cross-filesystem problem.
A small implementation for a trusted local directory
The following function assumes the parent directory already exists and is controlled by your account. Use it with a new practice directory and dummy data before adapting it to a real workflow. The replacement file receives the temporary file’s permissions; it does not automatically preserve the previous file’s ownership, mode or access-control entries.
import json
import os
import tempfile
from pathlib import Path
def save_state(path, state):
path = Path(path)
payload = json.dumps(state, indent=2, ensure_ascii=False) + '\n'
temporary = None
try:
with tempfile.NamedTemporaryFile(
mode='w', encoding='utf-8',
dir=path.parent, prefix=path.name + '.',
suffix='.tmp', delete=False
) as handle:
temporary = Path(handle.name)
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
temporary = None
finally:
if temporary is not None:
temporary.unlink(missing_ok=True)
save_state('practice-state.json', {'last_report': 'example.csv'})Serialization happens before creating the temporary file, so an unsupported value cannot truncate the destination. A unique temporary filename avoids collisions between temporary names. Closing the handle before replacement also avoids a common source of portability problems when renaming open files.
Check failures without touching production state
Begin with a practice file containing a recognizable value. Read it with json.load after a successful save and check the expected key. Then pass an unserializable value such as a set: the function should raise an exception before replacement, leaving the earlier file readable. These are suggested checks, not claimed results from your environment.
To examine interruption behavior, use a disposable copy of the function with an intentional exception immediately before os.replace. Confirm the old destination remains readable and cleanup removes the temporary file. Do not simulate disk exhaustion or kill processes on a production machine merely to test this example.
After an abrupt termination, an abandoned temporary file may remain because cleanup code never ran. Do not automatically promote it to the live checkpoint. Validate its contents, understand which operation it represents, and reconcile the underlying work first.
Separate atomic replacement from durability and concurrency
Atomic replacement addresses visibility of a partial file, not every storage failure. Flushing and syncing the temporary file is useful, but guaranteeing the directory entry survives a power failure can require directory syncing and filesystem-specific handling. Network filesystems and unusual storage arrangements need their own testing and operational assumptions.
Two writers can each read the same old state, calculate different updates, and then replace the file in turn. Both writes may succeed while one update is lost. A unique temporary name does not solve this. Use one writer, a lock covering the entire read-modify-write transaction, or a transactional database.
Also separate saved state from external effects. If sending a message succeeds but saving its checkpoint fails, retrying blindly can send a duplicate. Use stable operation identifiers and read-before-retry reconciliation. For controlling another failure mode, see subprocess timeouts for hanging Python automation.
Quick checklist and common question
- Keep the target in a trusted, backed-up directory.
- Serialize before touching the live path.
- Create the temporary file on the same filesystem.
- Check permissions after replacement.
- Plan for lost updates and ambiguous external actions.
Is this a backup? No. A valid but incorrect new state replaces the old file successfully. Keep versioned backups or an audit log when recovery requires earlier checkpoints.
Safety: This tutorial is for educational purposes. Test carefully before production use. The site owner is not responsible for misuse, damage, data loss or production issues.
