Skip to main content

Manage backup retention

A backup schedule that never deletes anything eventually fills its disk or its bucket, and the failure usually surfaces as a failed backup rather than as a full volume. A retention policy bounds that growth: you declare how many recovery points to keep, and infrahub-backup deletes the rest.

Follow this guide to preview a policy against your existing archives, apply it on demand, and attach it to a scheduled backup so the directory stays bounded without a wrapper script. You can also extend the same policy to S3-compatible storage.

Prerequisites​

Before applying a retention policy:

  • Ensure infrahub-backup is installed and configured. See Install Infrahub Backup.
  • Know which directory holds your archives (--backup-dir, ./infrahub_backups by default).
  • Confirm your recovery point objectives, so the policy you write matches what your organization requires.
  • For object storage, confirm your credentials can list and delete under the prefix. See Grant list and delete permissions.

How retention decides what to keep​

Two independent rules define a policy, and you can set either one or both:

FlagRuleEnvironment variable
--retention-days NKeep backups newer than N daysINFRAHUB_RETENTION_DAYS
--retention-count NKeep the N most recent backupsINFRAHUB_RETENTION_COUNT

Flags and environment variables are the only two ways to configure retention. Where both supply the same rule, the flag on the command you ran wins.

Both rules accept whole numbers of 1 or greater. To leave a rule inactive, omit its flag and leave its variable unset. Any other value either channel supplies — an explicit 0, a negative number, 7.5, 7d — is a validation error that aborts the run before a single archive is examined, rather than being read as "disabled". A schedule configured wrongly fails loudly instead of quietly keeping everything or deleting more than you intended.

A variable that is present but empty or whitespace-only counts as unset rather than as a bad value, because that is what Docker Compose and Kubernetes produce when a variable they were told to substitute has no value. Whitespace around a real value is tolerated.

Four properties govern every deletion decision.

A backup survives if either rule claims it. The rules combine as a union, not an intersection, so the more generous rule always wins. This is the property operators most often read backwards:

infrahub-backup prune --retention-days 7 --retention-count 14 --force

That policy keeps every archive from the last seven days and the 14 most recent archives, whichever set is larger. On a daily schedule it settles at 14 archives. On an hourly schedule the age rule dominates and roughly 168 archives survive. If you want a hard cap on the number of archives, set --retention-count alone.

Pair the two rules when the schedule that produces the backups might stall. An age rule alone answers "how old may an archive be", not "how many recovery points must remain", so once every archive is older than the window the age rule claims nothing and only the keep-newest floor below applies — a scheduled prune erodes the location to a single archive. Adding a count rule raises that floor to the number you name, whether or not new backups keep arriving:

# Thirty days of history, and never fewer than 14 recovery points
infrahub-backup prune --retention-days 30 --retention-count 14 --force

Retention attached to create is already protected from this: the policy is applied only after the backup is written and its checksum recorded, so a schedule that has started failing deletes nothing at all. The pairing matters for a standalone prune on its own schedule, which applies the policy whether or not a recent backup succeeded.

The most recent backup always survives. Even a policy that matches every archive on disk keeps the newest one. There is no flag to override this, so a mistyped policy can never leave you with zero recovery points.

Each location is evaluated on its own. The local directory and the S3 prefix are separate populations: each keeps its own newest archive, and each applies the rules to the archives it actually holds. Deleting a local copy never deletes the uploaded one.

Only recognized archives are considered. Retention acts on files named infrahub_backup_<YYYYMMDD_HHMMSS>.tar.gz or infrahub_backup_<YYYYMMDD_HHMMSS>.tar.gz.enc, and it reads each backup's age from the timestamp embedded in that name rather than from the file modification time. Anything else in the directory — notes, unrelated archives, renamed backups — is invisible to retention and is never deleted.

warning

Because age comes from the filename, renaming an archive changes when retention considers it expired, and touching a file does not extend its life. Keep the generated names if you want the policy to behave predictably.

That embedded timestamp carries no timezone. infrahub-backup writes it in the local time of the host taking the backup, reads it in the local time of the host applying retention, and measures a day as a fixed 24 hours. Archives written and pruned on the same host therefore always agree. If you prune from a container or scheduler host whose timezone differs, the age boundary moves by that offset and an archive can be deleted up to a day earlier or later than you expect. Give both hosts the same TZ if you split the work between them.

Step 1: Preview what a policy would delete​

Run prune with --dry-run before trusting any new policy. It reports the exact set a real run would delete, deletes nothing, and never prompts:

infrahub-backup prune --backup-dir /data/backups/infrahub --retention-days 7 --dry-run
INFO[0000] Applying retention policy (days: 7, count: 0) to 1 location(s)
INFO[0000] Would prune backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub (dry run)
INFO[0000] Would prune backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub (dry run)
INFO[0000] Would prune backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub (dry run)
INFO[0000] Retention at local:/data/backups/infrahub: 3 backup(s) kept, 3 candidate(s) to prune

Compare that list against what you expect to keep. A real run deletes exactly this set.

Step 2: Apply the policy​

Run the same command without --dry-run. On an interactive terminal, prune reports the candidates and asks once before deleting anything:

infrahub-backup prune --backup-dir /data/backups/infrahub --retention-days 7
INFO[0000] Applying retention policy (days: 7, count: 0) to 1 location(s)
INFO[0000] Would prune backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Would prune backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Would prune backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Retention at local:/data/backups/infrahub: 3 backup(s) kept, 3 candidate(s) to prune
Prune 3 backup(s) listed above? [y/N]: y
INFO[0000] Pruned backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Pruned backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Pruned backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Pruned 3 of 3 confirmed backup(s)

The set you confirm is the set that is deleted: the candidates are chosen once, before the question, and nothing is listed again afterwards. A candidate another process removed in the meantime counts as already satisfied rather than as a failure, so the closing line can report fewer deletions than candidates without the run failing.

Anything other than y or yes declines. A declined run deletes nothing and exits successfully, so an operator who changes their mind has not caused a failure.

For scheduled and scripted runs, pass --force to skip both the prompt and the candidate listing. Every deletion is still logged as it happens, so a forced run remains auditable after the fact:

infrahub-backup prune --backup-dir /data/backups/infrahub --retention-days 7 --force
INFO[0000] Applying retention policy (days: 7, count: 0) to 1 location(s)
INFO[0000] Retention at local:/data/backups/infrahub: 3 backup(s) kept, 3 candidate(s) to prune
INFO[0000] Pruned backup infrahub_backup_20260715_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Pruned backup infrahub_backup_20260710_073229.tar.gz from local:/data/backups/infrahub
INFO[0000] Pruned backup infrahub_backup_20260705_073229.tar.gz from local:/data/backups/infrahub
info

Without --force, a run whose standard input is not a terminal refuses to delete anything and tells you to pass --force. This keeps a prune that was written for a terminal from deleting silently when it later runs under cron. --dry-run and --force together are rejected as contradictory: a dry run never deletes and never prompts, so there is nothing to force.

warning

Without --force, a run that cannot list one of its locations exits non-zero without prompting and without deleting anything at any location. Confirming a partial preview would mean answering for a set the tool does not fully know, and a bucket it cannot list is not the same as a bucket with nothing to prune. A mistyped --backup-dir and an unreachable S3 endpoint both land here. --force behaves differently by design: it prunes every location it could list and still exits non-zero naming the one that failed.

Step 3: Attach retention to scheduled backups​

Passing a retention flag to create applies the policy automatically after the backup finishes, with no prompt. Retention runs only once the archive is written and its checksum recorded, so a failed backup never triggers a deletion:

infrahub-backup create --retention-days 7 --retention-count 14

Retention on create logs what a forced prune logs — the policy it applied, a Retention at <location> summary of how many archives it kept and how many it selected, and one line per deletion — so an unattended nightly run accounts for what it removed at the default log level.

This replaces the find-based cleanup that backup scripts often carry, and it prunes only recognized archives instead of everything matching a glob:

# Daily backup at 2 AM, keeping 30 days of archives
0 2 * * * /usr/local/bin/infrahub-backup create --retention-days 30 >> /var/log/infrahub-backup.log 2>&1

Running create with no retention rule configured behaves exactly as it always has: nothing is pruned, and no archive is examined.

note

--dry-run, --force, and --s3 belong to prune alone and are deliberately per-invocation switches, with no environment variable behind them. A persistent force setting would disarm the confirmation that makes pruning safe, and a persistent s3 would delete objects an operator only configured for upload.

If the backup succeeds but a deletion fails, the run exits non-zero with a message that states the backup succeeded and names the new archive, so you can tell a retention problem from a lost backup. Within each location, deletion is best-effort: one failure does not strand the remaining candidates.

Step 4: Extend retention to object storage​

Retention treats object storage as a second location, and it never touches that location unless you ask:

  • On create, the S3 leg runs exactly when that run uploaded an archive, which means --s3-upload was passed and the upload succeeded.
  • On prune, the S3 leg runs only when you pass --s3. Without it, a fully configured bucket is left alone.
note

--s3-upload without --s3-keep-local deletes the local copy of the new archive as soon as the upload succeeds, before retention runs. The keep-newest floor still protects each location's newest archive, but in the local directory that is the previous archive, not the one this run just made. Pass --s3-keep-local if you want the fresh archive in both places.

# Prune both the local directory and the configured bucket/prefix
infrahub-backup prune --retention-days 30 --s3 --force \
  --s3-bucket my-backups --s3-prefix infrahub/prod

# Prune the local directory only; the bucket is untouched
infrahub-backup prune --retention-days 30 --force

Only objects whose name matches the backup pattern are considered, and only under the configured prefix. Objects your team stores alongside the backups are never candidates.

Grant list and delete permissions​

Uploading a backup needs write permission. Pruning one also needs to list the prefix and delete objects under it. Scope those rights to the backup prefix rather than the whole bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListInfrahubBackupPrefix",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::my-backups",
      "Condition": {
        "StringLike": {
          "s3:prefix": "infrahub/prod/*"
        }
      }
    },
    {
      "Sid": "ManageInfrahubBackupObjects",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::my-backups/infrahub/prod/*"
    }
  ]
}

With upload-only credentials the S3 leg fails and the run exits non-zero naming that location. What happens to the local directory depends on which right is missing and whether you passed --force:

Missing right--dry-runInteractive (no --force)--force
s3:ListBucketNothing deletedNothing deleted anywhere; you are not askedLocal directory pruned
s3:DeleteObject onlyNothing deletedCandidates listed for both locations; on y, local deletions succeed and S3 deletions failLocal directory pruned

Without list permission the S3 leg cannot be previewed at all, and a preview the tool cannot complete stops the whole non-forced run before it deletes anything. Grant both rights together, and use --dry-run to confirm the bucket is being listed before you rely on a scheduled --force run.

Combine with bucket lifecycle rules​

A bucket lifecycle rule and a retention policy solve overlapping problems from different sides, and running both is a reasonable belt-and-braces arrangement:

  • Retention keeps the two locations consistent with one policy you can preview with --dry-run, and it guarantees the newest archive survives.
  • A lifecycle rule enforces expiry inside the storage provider even if the tool never runs again, and it can also transition older archives to colder storage classes.

If you use both, set the lifecycle expiry window wider than the retention window. A lifecycle rule has no notion of "keep the newest", so a window narrower than your backup interval can expire your last remaining archive.

Validation​

Confirm your retention setup works:

  • ✓ --dry-run lists exactly the archives you expect to lose
  • ✓ A real run deletes that set and nothing else
  • ✓ Unrelated files in the backup directory are still present
  • ✓ At least one archive remains at every location
  • ✓ For object storage, the bucket reflects the same policy after a --s3 run

Troubleshooting​

An archive I expected to be deleted is still there​

Check whether it is the newest at its location — the floor keeps it unconditionally. Otherwise, confirm the name still matches infrahub_backup_<YYYYMMDD_HHMMSS>.tar.gz; a renamed archive is invisible to retention. Remember also that the two rules combine as a union, so an archive older than --retention-days still survives while it sits within --retention-count.

An archive I copied back from the bucket became a deletion candidate​

An archive you place in the backup directory yourself — copying one down from the bucket with aws s3 cp or mc cp, or restoring one from tape — keeps its original name and therefore its original timestamp. Retention reads it as an archive of that age, not as one that arrived today, so the next run may well delete it. Keep such a copy outside the backup directory, or rename it, if you need it to survive.

infrahub-backup restore s3://… does not leave you in this situation: it downloads into the backup directory but removes its own copy once the restore finishes.

Retention did nothing on the plakar backend​

Retention is not yet supported for the plakar backend. A create run with --backend plakar and retention flags completes the backup, warns that retention is not yet supported, prunes nothing, and exits successfully. A prune run against that backend fails with a validation error rather than doing nothing quietly. Use the default tarball backend if you need retention today.