photocli/USERGUIDE.md

photoscli User Guide

This guide explains how to use photoscli safely for Apple Photos exports and backups. It is written for day-to-day users who want reliable commands, clear recovery steps, and predictable backup behavior.

For a compact command reference, see README.md or run:

photoscli help

What photoscli Is For

photoscli exports assets from Apple Photos on macOS.

It is especially useful when you want to:

• Back up the whole Photos album tree to normal folders.
• Export one album for sharing or archiving.
• Resume a large export without starting over.
• Keep a manifest of already-exported assets.
• Retry transient iCloud failures.
• Verify that files referenced by a manifest exist on disk.
• Detect missing, zero-byte, and size-mismatched manifest files.
• Store optional SHA-256 checksums in manifests.
• Inspect or clear deduplicated failure records.
• Write optional XMP sidecar metadata for archival workflows.
• Add optional reverse-geocoded address metadata to XMP sidecars for GPS assets.
• Script Photos exports with stable exit codes.

It is not intended to replace Apple Photos, iCloud Photos, or Time Machine. Think of it as an additional file-based export and backup tool.

Before You Start

You need:

• An Apple Silicon Mac, M1/M2/M3/M4 or newer. The prebuilt release binary is darwin/arm64 only.
• macOS.
• Apple Photos library available on the machine.
• Photos permission granted to the terminal app or binary.
• Enough disk space for the export.
• A built photoscli binary.

Build it from the repo:

make build

Run the binary directly:

./bin/photoscli version

If you install or copy it somewhere else, replace ./bin/photoscli in examples with photoscli.

Intel Macs are not currently a supported release target. If Intel support is added later, release assets will include a separate architecture-specific package.

First Run And Permissions

Start with a harmless command:

./bin/photoscli albums

macOS may ask for Photos access. Allow it.

If access is denied, open:

System Settings > Privacy & Security > Photos

Then enable access for the terminal application you use, such as Terminal, iTerm, or another launcher. Depending on how you start the binary, macOS may associate the permission with the terminal app rather than the binary itself.

Verify access:

./bin/photoscli tree

Recommended Backup Strategy

For most users, the safest full-library backup command is:

./bin/photoscli backup-all \
  --out /Volumes/BackupDrive/PhotosExport \
  --manifest sqlite \
  --log \
  --retry 3 \
  --concurrency 4

This gives you:

• Full album-tree export.
• SQLite manifest for fast resumable backups.
• Structured logs in the same database.
• Automatic retries for transient failures.
• Moderate concurrency that is usually safe for large libraries.

Before running it for real, run a dry-run:

./bin/photoscli backup-all \
  --out /Volumes/BackupDrive/PhotosExport \
  --manifest sqlite \
  --dry-run \
  --json

Preview Export vs Original Export

By default, photoscli exports optimized preview images.

Preview export:

• Produces smaller files.
• Uses --size and --quality.
• Is good for browsing, sharing, and lightweight archives.
• May not preserve original file bytes or all original metadata.

Original export:

• Uses --originals.
• Exports original asset resources where PhotoKit exposes them.
• Ignores --size and --quality.
• Uses more disk space.
• Is usually the better choice for archival backups.

Preview example:

./bin/photoscli export --album-id "Vacation" --out ./VacationPreview --size 2048 --quality 90

Original example:

./bin/photoscli export --album-id "Vacation" --out ./VacationOriginals --originals

Export One Album

List albums first:

./bin/photoscli albums

Export by title:

./bin/photoscli export --album-id "Vacation" --out ./Vacation

Export by PhotoKit local identifier:

./bin/photoscli export --album-id "5E9F.../L0/001" --out ./Vacation

Use the local identifier when album names are duplicated or ambiguous.

Recommended one-album archival export:

./bin/photoscli export \
  --album-id "Vacation" \
  --out ./VacationOriginals \
  --originals \
  --manifest sqlite \
  --log \
  --retry 3

Back Up The Whole Library

Use backup-all to export every album into a folder tree matching Photos folders and albums.

./bin/photoscli backup-all --out ./PhotosBackup

Recommended full backup:

./bin/photoscli backup-all \
  --out ./PhotosBackup \
  --manifest sqlite \
  --log \
  --retry 3 \
  --concurrency 4

Recommended full original backup:

./bin/photoscli backup-all \
  --out ./PhotosOriginals \
  --originals \
  --manifest sqlite \
  --log \
  --retry 3 \
  --concurrency 4

If two sibling albums have the same name, photoscli adds the album ID to the generated folder name so the folders do not collide.

Dry Runs

Dry-runs are strongly recommended before large exports.

./bin/photoscli backup-all --out ./PhotosBackup --dry-run

With JSON output:

./bin/photoscli backup-all --out ./PhotosBackup --dry-run --json

Dry-run mode does not write exported files, manifests, logs, or failure files. It prints what would be exported.

Use dry-run when:

• You are testing filters.
• You are checking output paths.
• You are preparing a large first-time backup.
• You are validating an exclude list.

Incremental Backups

Incremental backups are enabled by manifests.

Run the same backup command repeatedly:

./bin/photoscli backup-all \
  --out ./PhotosBackup \
  --manifest sqlite \
  --log \
  --retry 3

Already-exported assets are skipped based on the manifest.

For new assets since a date:

./bin/photoscli backup-all \
  --out ./PhotosBackup \
  --manifest sqlite \
  --since 2024-01-01

The --since flag accepts:

• YYYY-MM-DD
• RFC3339, for example 2024-01-01T00:00:00Z

Manifests

Manifests track exported asset IDs and allow safe resume behavior.

JSONL manifest:

downloads.jsonl

SQLite manifest:

downloads.db

Use JSONL when:

• You want a simple append-friendly text file.
• Your library is small or medium-sized.
• You want easy inspection with normal text tools.

Use SQLite when:

• Your library is large.
• You want faster lookup behavior.
• You want logs stored in the same database.
• You plan to run repeated backups over time.

Recommended for serious backups:

--manifest sqlite

Disable manifests only for temporary exports:

./bin/photoscli export --album-id "Scratch" --out ./Scratch --no-manifest

Without a manifest, photoscli cannot use manifest-based resume behavior.

Logging

Enable logs with:

--log

For JSONL or no-manifest exports, logs go to:

export.log

For SQLite manifests, logs go to the logs table in:

downloads.db

Logged events include:

• Session start.
• Session end.
• Export completed.
• Export skipped.
• Export failed.

Use logs when:

• You are running unattended backups.
• You need auditability.
• You want details about failed or skipped assets.

Failed Exports And Retries

Failed exports are written to:

failures.jsonl

Failure records are deduplicated by asset ID. Repeated failures update the existing record and increment the attempt count.

For transient failures, first use retries during normal export:

./bin/photoscli backup-all --out ./PhotosBackup --retry 3

If failures remain, retry later:

./bin/photoscli retry-failed --out ./PhotosBackup

Clear successful retry records automatically:

./bin/photoscli retry-failed --out ./PhotosBackup --clear-on-success

List or clear failure records:

./bin/photoscli failures list --out ./PhotosBackup
./bin/photoscli failures clear --out ./PhotosBackup

Typical reasons for failures:

• iCloud asset not currently downloadable.
• Network interruption.
• Photos permission issue.
• Destination disk unavailable.
• Destination disk full.
• Invalid or changing Photos library state.

After retrying, run verification:

./bin/photoscli verify --out ./PhotosBackup --manifest sqlite

Verification

Run verification after large backups:

./bin/photoscli verify --out ./PhotosBackup --manifest sqlite

Verification checks that manifest entries point to files on disk. It also reports zero-byte files and size mismatches when the manifest has a recorded size.

It exits with:

• 0 when all checked files exist.
• 2 when files are missing.
• 1 for argument or runtime errors.

You can also verify immediately after export:

./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --verify

Reporting

Show manifest and failure counts:

./bin/photoscli report --out ./PhotosBackup --manifest sqlite

Example output:

entries	12345
failures	2

Use this after long runs to quickly check whether work was recorded and whether failures occurred.

Diffing An Album Against A Backup

Use diff to find album assets that are not in the manifest:

./bin/photoscli diff --album-id "Vacation" --out ./PhotosBackup --manifest sqlite

Missing assets are printed as:

<asset-id>	<filename>

Exit code is 2 if missing assets are found.

Filtering

Favorites Only

./bin/photoscli export --album-id "Favorites" --out ./Favorites --only-favorites

Media Type

Photos only, the default:

./bin/photoscli backup-all --out ./PhotosBackup --media photos

Videos only:

./bin/photoscli backup-all --out ./VideoBackup --media videos --include-videos

Everything:

./bin/photoscli backup-all --out ./FullBackup --media all --include-videos

Exclude Albums

Exclude exact album names:

./bin/photoscli backup-all --out ./PhotosBackup --exclude-album "Recently Deleted"

Exclude by glob:

./bin/photoscli backup-all --out ./PhotosBackup --exclude-album "Temp*"

Use multiple exclusions:

./bin/photoscli backup-all \
  --out ./PhotosBackup \
  --exclude-album "Recently Deleted" \
  --exclude-album "Temp*" \
  --exclude-album "Screenshots"

Date Filter

./bin/photoscli backup-all --out ./PhotosBackup --since 2024-01-01

Estimated Size Filter

--min-size and --max-size currently filter by estimated pixel count from dimensions, not encoded file size.

./bin/photoscli export --album-id "Large" --out ./LargeOnly --min-size 12000000

Date-Based Folder Layouts

Use --date-template to append date folders based on asset creation date.

Example:

./bin/photoscli export \
  --album-id "Vacation" \
  --out ./Vacation \
  --date-template YYYY/MM/DD

Possible output:

Vacation/
  2024/
    07/
      15/
        0000_....jpg

Supported tokens:

• YYYY
• MM
• DD

Assets without parseable creation dates stay in the base output path.

XMP Sidecars

Use XMP sidecars when you want portable metadata next to exported files:

./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json

Sidecars are disabled by default. When enabled, they use the exported file basename:

IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json

Use XMP for standards-oriented metadata workflows and JSON when you want structured photoscli metadata for scripts or audits.

The XMP includes photoscli archive metadata such as asset ID, original filename, exported filename, album, manifest path, media type, dimensions, favorite state, hidden state, cloud state, export mode, version, exported time, size, creation date, modification date, duration, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP date fields, EXIF GPS fields, favorite rating, and album/folder keywords when PhotoKit exposes them.

Control location privacy in generated sidecars:

./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-address
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-location

keep is the default. strip-address omits reverse-geocoded address fields while preserving GPS coordinates. strip-location omits both GPS coordinates and address fields.

Control generated keywords and ratings:

./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords album
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords none --xmp-rating none

--xmp-keywords album-path is the default and writes album/folder keywords. album writes only the album name. none omits generated dc:subject keywords. --xmp-rating favorite is the default and maps favorite assets to xmp:Rating="5"; none omits that generated rating.

For address metadata from GPS coordinates, opt in to Apple's reverse geocoder:

./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --reverse-geocode

Reverse geocoding uses Apple MapKit and requires macOS 26 or newer. It can require network access and may be rate-limited by Apple, so results are cached in .photoscli/geocode-cache.jsonl under the backup root. On older macOS versions, --reverse-geocode is treated as unavailable: the export continues, no address fields are added, and the XMP still contains GPS coordinates.

Verify sidecars after an export:

./bin/photoscli verify --out ./PhotosBackup --sidecar

This reports missing, zero-byte, unreadable, or asset-ID mismatched .xmp files.

Use strict verification for sidecars generated by recent photoscli versions:

./bin/photoscli verify --out ./PhotosBackup --sidecar --strict

Strict mode also checks photoscli schema metadata, generator metadata, and the exported filename recorded inside the sidecar.

Inspect one generated sidecar when troubleshooting or scripting:

./bin/photoscli sidecar inspect ./PhotosBackup/IMG_0001.xmp
./bin/photoscli sidecar inspect ./PhotosBackup/IMG_0001.xmp --json

Refresh metadata only for an existing manifest-backed backup:

./bin/photoscli backup-all --out ./PhotosBackup --sidecar xmp --metadata-only

Metadata-only mode does not re-export media files. It uses manifest paths to find existing files and rewrites generated .xmp sidecars next to them. It requires --sidecar xmp and an enabled manifest.

If you explicitly request --sidecar xmp and the XMP file cannot be written, the asset is counted as failed.

Configuration File

You can store default values in:

~/.photoscli.toml

Example:

size = 2048
quality = 90
concurrency = 4
manifest = "sqlite"
sort = "newest"
media = "photos"
retry = 3
log = true
sidecar = "xmp"
reverse-geocode = true

Use a custom config path:

PHOTOSCLI_CONFIG=./photoscli.toml ./bin/photoscli backup-all --out ./PhotosBackup

Command-line flags override config-file defaults.

Recommended config for recurring backups:

manifest = "sqlite"
log = true
retry = 3
concurrency = 4
sort = "oldest"
media = "photos"
quality = 90
size = 2048

Then run:

./bin/photoscli backup-all --out ./PhotosBackup

Automation Examples

Weekly Incremental Backup

#!/bin/sh
set -eu

OUT="/Volumes/BackupDrive/PhotosExport"
BIN="$HOME/bin/photoscli"

"$BIN" backup-all \
  --out "$OUT" \
  --manifest sqlite \
  --log \
  --retry 3 \
  --concurrency 4 \
  --json

"$BIN" verify --out "$OUT" --manifest sqlite

Export Favorites For Sharing

./bin/photoscli export \
  --album-id "Favorites" \
  --out ./FavoritesShare \
  --only-favorites \
  --size 2048 \
  --quality 88 \
  --no-manifest

Archive Originals From One Album

./bin/photoscli export \
  --album-id "Family Archive" \
  --out ./FamilyArchiveOriginals \
  --originals \
  --manifest sqlite \
  --log \
  --retry 3

Backup Everything Including Videos

./bin/photoscli backup-all \
  --out ./FullPhotosBackup \
  --media all \
  --include-videos \
  --manifest sqlite \
  --log \
  --retry 3

Interpreting Exit Codes

photoscli uses stable exit codes for scripts:

• 0: success.
• 1: invalid arguments, runtime error, or all exports failed.
• 2: partial failure, diff found missing manifest entries, or verify found missing files.
• 3: Photos access denied.

Example shell handling:

./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log
rc=$?

case "$rc" in
  0) echo "backup succeeded" ;;
  2) echo "backup partially succeeded; inspect failures.jsonl" ;;
  3) echo "Photos access denied; check macOS privacy settings" ;;
  *) echo "backup failed with exit code $rc" ;;
esac

Troubleshooting

Access Denied

Symptom:

error: access denied

Fix:

• Open System Settings.
• Go to Privacy & Security > Photos.
• Enable access for your terminal app or launcher.
• Run photoscli albums again.

iCloud Assets Fail To Export

Try:

./bin/photoscli backup-all --out ./PhotosBackup --retry 3

If failures remain:

./bin/photoscli retry-failed --out ./PhotosBackup

Also check:

• Network connectivity.
• iCloud Photos status.
• Whether Photos.app can open/download the asset.

Destination Disk Is Full

Free space, then rerun the same command. The manifest should skip already-exported assets and continue with remaining work.

Duplicate Album Names

backup-all automatically appends the album ID to duplicate sibling album names. If you want exact control, export ambiguous albums individually using their PhotoKit local identifiers.

Too Slow

Try a moderate concurrency increase:

./bin/photoscli backup-all --out ./PhotosBackup --concurrency 8

If your library is heavily iCloud-backed, too much concurrency may increase transient failures. Use --retry 3 and reduce concurrency if needed.

Too Much Output Or Hard To Parse

Use JSON summary output:

./bin/photoscli backup-all --out ./PhotosBackup --json

Use structured logs:

./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log

Store SHA-256 checksums in the manifest when exporting:

./bin/photoscli backup-all --out ./PhotosBackup --checksum sha256

Checksums are opt-in and stored in JSONL or SQLite manifests. The default is --checksum none.

Deep-verify checksum-backed manifests:

./bin/photoscli verify --out ./PhotosBackup --deep

--deep recomputes SHA-256 only for manifest entries that already have checksum metadata.

Repair missing manifest metadata from files already present on disk:

./bin/photoscli manifest repair --out ./PhotosBackup --checksum sha256 --dry-run
./bin/photoscli manifest repair --out ./PhotosBackup --checksum sha256

Repair fills missing size/checksum metadata and skips missing, zero-byte, or unreadable files.

Preview and remove files not referenced by the manifest:

./bin/photoscli cleanup --out ./PhotosBackup --dry-run
./bin/photoscli cleanup --out ./PhotosBackup

Cleanup preserves manifest/log/failure files, .photoscli, and sidecars next to manifest-referenced media.

Safe Operating Practices

• Run --dry-run before the first large backup.
• Use --manifest sqlite for long-term backups.
• Use --log for unattended runs.
• Use --retry 3 for iCloud-heavy libraries.
• Run verify after large runs.
• Keep the destination on a reliable disk.
• Do not use --no-manifest for backups you expect to resume.
• Prefer album local identifiers when names are duplicated.

Limitations

• This is macOS-only.
• Real Photos access requires macOS privacy permission.
• --format heic|png is currently a validated CLI hint; true non-JPEG preview output still needs bridge support.
• --min-size and --max-size use estimated pixel count from dimensions, not encoded file size.
• Original export depends on asset resources exposed by PhotoKit.
• iCloud-backed assets may require network downloads and can fail for reasons outside the CLI's control.
• The manifest records exported asset IDs; it is not a cryptographic integrity database.