photocli/README.md

# photoscli

`photoscli` is a macOS-only command-line exporter for Apple Photos. It uses a small Objective-C PhotoKit bridge from Go to list albums, inspect assets, export optimized previews or originals, keep resumable manifests, and produce machine-readable logs for backup automation.

The tool is designed for repeatable, resumable Photos backups rather than one-off drag-and-drop exports.

For a practical step-by-step manual with recommended backup workflows, recovery steps, automation examples, and troubleshooting, see `USERGUIDE.md`.

## Highlights

- List albums, photos, and the Photos folder/album tree.
- Export one album or back up the whole Photos tree.
- Export JPEG previews with configurable size and quality, or original files.
- Skip already-exported assets through a JSONL or SQLite manifest.
- Resume interrupted runs safely.
- Parallel export with live worker progress and ETA.
- Optional structured logging to `export.log` or SQLite `logs` table.
- Dry-run mode for planning large backups.
- Filters for favorites, media type, date, estimated size, and excluded albums.
- Failure tracking with `failures.jsonl` and `retry-failed`.
- Deduplicated failure management with `failures list`, `failures clear`, and `retry-failed --clear-on-success`.
- Verification, reporting, and diff commands for backup integrity.
- Status command for quick backup summaries.
- Script-friendly exit codes and optional JSON summaries.
- 100% test coverage for the Go CLI and parsing layers.

## Requirements

- macOS with Apple Photos.
- Go 1.25 or newer.
- Xcode command-line tools.
- Photos privacy permission for the built binary or terminal app.

The production build uses cgo and links against Apple frameworks through the Objective-C bridge in `bridge/`.

## Build

```bash
make build
```

The binary is written to:

```bash
./bin/photoscli
```

Run the full local release pipeline:

```bash
make pipeline
```

The pipeline cleans artifacts, builds the test bridge, runs vet, runs race-enabled tests with coverage, builds the real PhotoKit bridge, builds the CLI, and verifies the embedded version.

## Quick Start

List albums:

```bash
./bin/photoscli albums
```

Inspect an album by title or PhotoKit local identifier:

```bash
./bin/photoscli photos --album-id "Vacation"
```

Export preview JPEGs from one album:

```bash
./bin/photoscli export --album-id "Vacation" --out ./export
```

Export higher-quality previews:

```bash
./bin/photoscli export --album-id "Vacation" --out ./export --size 2048 --quality 92
```

Export originals:

```bash
./bin/photoscli export --album-id "Vacation" --out ./originals --originals
```

Back up the entire Photos album tree:

```bash
./bin/photoscli backup-all --out ./photos-backup --manifest sqlite --log
```

Preview what would happen before writing anything:

```bash
./bin/photoscli backup-all --out ./photos-backup --dry-run --json
```

Verify a backup later:

```bash
./bin/photoscli verify --out ./photos-backup --manifest sqlite
```

## Commands

### `albums`

Lists user-created albums as tab-separated ID and title.

```bash
photoscli albums
```

Example output:

```text
5E9F.../L0/001	Vacation
8A1B.../L0/001	Work
```

### `photos`

Lists assets in an album. `--album-id` accepts a PhotoKit local identifier or an exact album title.

```bash
photoscli photos --album-id "Vacation"
```

Output includes ID, filename, cloud state, media type, dimensions, optional creation date, optional duration, and a `*` marker for favorites.

### `tree`

Prints the Photos folder and album hierarchy.

```bash
photoscli tree
```

### `export`

Exports assets from one album.

```bash
photoscli export --album-id <id-or-title> --out <dir> [flags]
```

Useful examples:

```bash
photoscli export --album-id "Family" --out ./family --size 2560 --quality 90
photoscli export --album-id "Favorites" --out ./favorites --only-favorites
photoscli export --album-id "Videos" --out ./videos --media videos --include-videos
photoscli export --album-id "Archive" --out ./archive --originals --retry 3
photoscli export --album-id "Vacation" --out ./vacation --date-template YYYY/MM/DD
```

### `backup-all`

Walks the Photos folder/album tree and exports every album to a matching folder structure.

```bash
photoscli backup-all --out <dir> [flags]
```

Duplicate sibling album names are disambiguated by adding the album ID to the generated folder name.

Useful examples:

```bash
photoscli backup-all --out ./backup --manifest sqlite --log
photoscli backup-all --out ./backup --exclude-album "Recently Deleted" --exclude-album "Temp*"
photoscli backup-all --out ./backup --since 2024-01-01 --sort newest
photoscli backup-all --out ./backup --concurrency 8 --retry 3
photoscli backup-all --out ./backup --dry-run --json
```

### `report`

Shows manifest and failure counts for an output directory.

```bash
photoscli report --out ./backup --manifest sqlite
```

Example output:

```text
entries	12345
failures	2
```

### `diff`

Compares an album against the manifest and prints assets missing from the manifest.

```bash
photoscli diff --album-id "Vacation" --out ./backup --manifest jsonl
```

Exit code is `2` when missing assets are found.

### `verify`

Checks that manifest entries have corresponding files on disk.

```bash
photoscli verify --out ./backup --manifest sqlite
```

Exit code is `2` when files are missing.

### `retry-failed`

Retries assets recorded in `failures.jsonl`.

```bash
photoscli retry-failed --out ./backup
```

This is useful after transient iCloud or network failures.

Use `--clear-on-success` to remove successfully retried assets from `failures.jsonl`.

### `failures`

Lists or clears deduplicated failure records.

```bash
photoscli failures list --out ./backup
photoscli failures clear --out ./backup
```

### `status`

Shows a quick backup summary.

```bash
photoscli status --out ./backup --manifest sqlite
photoscli status --out ./backup --manifest sqlite --json
```

## Export Flags

Common flags for `export` and `backup-all`:

- `--out <dir>`: destination directory.
- `--size <px>`: longest-side target for preview export, default `1024`.
- `--quality <1-100>`: JPEG preview quality, default `85`.
- `--originals`: export original files instead of previews.
- `--concurrency <N>`: parallel workers, default `3`, capped by bridge slot count.
- `--retry <N>`: retry failed exports with a small backoff.
- `--dry-run`: print planned exports without writing files, manifests, or logs.
- `--json`: print a machine-readable summary to stdout.
- `--verify`: run manifest/file verification after export.
- `--log`: enable structured export logging.
- `--manifest jsonl|sqlite`: choose manifest backend, default `jsonl`.
- `--no-manifest`: disable manifest reads/writes.
- `--sort oldest|newest`: asset sort order where supported, default `oldest`.
- `--since <date>`: only include assets on or after `YYYY-MM-DD` or RFC3339 date.
- `--only-favorites`: only export favorite assets.
- `--media photos|videos|all`: media filter, default `photos`.
- `--include-videos`: compatibility flag that includes videos/audio.
- `--min-size <n>`: filter by estimated pixel count.
- `--max-size <n>`: filter by estimated pixel count.
- `--format jpeg|heic|png`: preview format hint. Current bridge output is still the existing preview path; non-JPEG bridge output is future work.
- `--date-template <template>`: append date folders based on creation date, for example `YYYY/MM/DD`.

`backup-all` also supports:

- `--exclude-album <pattern>`: repeatable exact or glob-style album-name exclusion.

`export` also requires:

- `--album-id <id-or-title>`: PhotoKit local identifier or exact album title.

## Manifests

Manifests prevent re-exporting assets on subsequent runs.

JSONL backend:

```text
downloads.jsonl
```

SQLite backend:

```text
downloads.db
```

The manifest stores asset ID, filename, size, cloud state, and export timestamp. SQLite is useful for very large libraries and for keeping logs in the same database.

Use no manifest only when you intentionally want stateless export behavior:

```bash
photoscli export --album-id "Scratch" --out ./scratch --no-manifest
```

## Logging

Enable logs with:

```bash
photoscli backup-all --out ./backup --log
```

With JSONL/no-manifest mode, logs are written to:

```text
export.log
```

With SQLite manifest mode, logs are written to the `logs` table in `downloads.db`.

Logged events include session start/end, completed exports, skipped assets, and failures. Each event includes timestamp, level, event name, asset ID, album, filename, size, cloud state, duration, and message where available.

## Failure Tracking

Failed exports are deduplicated by asset ID and stored in:

```text
failures.jsonl
```

Retry them with:

```bash
photoscli retry-failed --out ./backup
```

Clear successful retries automatically:

```bash
photoscli retry-failed --out ./backup --clear-on-success
```

Use `--retry N` during normal exports to handle transient iCloud or network failures automatically.

## Configuration File

Defaults can be loaded from:

```text
~/.photoscli.toml
```

Or from a custom path:

```bash
PHOTOSCLI_CONFIG=/path/to/photoscli.toml photoscli backup-all --out ./backup
```

Supported keys mirror long flag names without the leading dashes:

```toml
size = 2048
quality = 90
concurrency = 8
manifest = "sqlite"
log = true
sort = "newest"
media = "photos"
retry = 3
```

Command-line flags override config-file values.

## Exit Codes

- `0`: success.
- `1`: command/config/runtime error.
- `2`: partial failure, such as some exports failed or diff/verify found missing items.
- `3`: Photos access denied.

## Permissions

On first use, macOS may prompt for Photos access. If access is denied, grant access in:

```text
System Settings > Privacy & Security > Photos
```

Depending on how you launch the binary, macOS may associate the permission with Terminal, iTerm, your shell, or the built executable.

## Development

Run tests with the stub bridge:

```bash
make test
```

Run all package coverage:

```bash
go test -tags=test -race -count=1 -coverprofile=coverage.out ./...
go tool cover -func=coverage.out
```

Run the release pipeline:

```bash
make pipeline
```

Create a release after committing and tagging:

```bash
make release
```

## Architecture

- `cmd/photoscli`: CLI, argument handling, filtering, manifests, progress, reporting, and command orchestration.
- `internal/photos`: Go interface and JSON parsing for PhotoKit responses.
- `internal/manifest`: JSONL/SQLite manifest backends and log writers.
- `bridge/`: Objective-C PhotoKit implementation plus a C stub used by tests.

Objective-C returns JSON to Go. Tests use the stub bridge and do not require real Photos access.

## Known Notes

- This project is macOS-only.
- Preview export currently follows the existing bridge preview implementation; `--format heic|png` is validated as an output-format hint but still needs bridge support for true non-JPEG previews.
- Album title lookup uses exact title matching. Use PhotoKit local identifiers when names are ambiguous.
- iCloud-backed assets may trigger downloads and can fail due to network or account state.
- `--min-size` and `--max-size` currently use estimated pixel count from dimensions, not encoded file size.