Compare commits

20 Commits

Author SHA1 Message Date
Ein Anderssono bed3ea7bd0 v0.9.1: add deep checksum verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:23:22 +02:00
Ein Anderssono 7555b561bd v0.9.0: add manifest checksums
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:19:47 +02:00
Ein Anderssono d909d30b87 v0.8.7: add JSON sidecars
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:11:02 +02:00
Ein Anderssono 5c40b1d3ba v0.8.6: add XMP keyword and rating controls
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:05:14 +02:00
Ein Anderssono 700d8ef05a v0.8.5: add XMP privacy controls
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:02:26 +02:00
Ein Anderssono fbc37b8d8d v0.8.4: add strict sidecar verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:58:38 +02:00
Ein Anderssono 32a5819c86 v0.8.3: add sidecar inspection
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 01:55:31 +02:00
Ein Anderssono a51db37fdb v0.8.2: add metadata-only sidecars
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:48:32 +02:00
Ein Anderssono 9cd702628d v0.8.1: improve XMP sidecars
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:36:04 +02:00
Ein Anderssono fffb30023b v0.8.0: enrich XMP metadata
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:21:49 +02:00
Ein Anderssono 4fe4c15adf v0.7.0: add XMP sidecars
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 00:57:13 +02:00
Ein Anderssono 36832060d0 docs: clarify Apple Silicon release target
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 00:41:49 +02:00
Ein Anderssono 05188e5451 v0.6.0: strengthen backup integrity
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 00:34:32 +02:00
Ein Anderssono 0a905758cc docs: add user guide and release zip packaging
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 00:06:27 +02:00
Ein Anderssono 2e73d01b40 v0.5.0: manifests, filters, logging, docs
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 00:00:06 +02:00
Ein Anderssono 3d3c4a4742 v0.4.0: scroll log, worker slots, disk skip, color gradient, parallel export 2026-06-12 14:03:18 +02:00
Ein Anderssono e888f7cad1 v0.2.5: Unicode progress bar with cloud download speed
- Unicode block progress bar (█▓░)
- Cloud downloads show ☁ with average speed (MB/s, KB/s, B/s)
- Truncated filenames with ellipsis for long names
- Error indicator (✗) in progress bar
- Simplified to serial export for clean cancel behavior
- Added IsCancelled() to Bridge interface
2026-06-11 21:59:42 +02:00
Ein Anderssono 479c284dfc v0.2.4: stop export loop on Ctrl+C instead of flooding failures
- Add IsCancelled() to Bridge interface
- Check bridge.IsCancelled() before each export in serial/parallel/backupTree
- Parallel workers mark remaining slots as 'cancelled' instead of exporting
- Add photos_request_is_cancelled to ObjC and C stub
2026-06-11 21:44:55 +02:00
Ein Anderssono 009c71e6bb v0.2.3: fix export write failures and Ctrl+C cancellation
- Add ensure_directory calls in both export functions (preview + original)
- Include NSError.localizedDescription in write failed messages
- Make semaphore_wait_with_timeout poll photos_cancelled every ~1s
- Add status messages: auth, loading tree, per-album progress
- Fix parallel export: slot-based progress shows results in order
2026-06-11 21:37:11 +02:00
Ein Anderssono b2d4c6188d fix: make Ctrl+C cancel ObjC semaphore waits within ~1s
semaphore_wait_with_timeout now polls photos_cancelled every second
instead of blocking for the full timeout duration
2026-06-11 21:24:03 +02:00
39 changed files with 12461 additions and 482 deletions
+34
View File
@@ -0,0 +1,34 @@
name: pipeline
on: [push, pull_request]
jobs:
test:
runs-on: macos-14
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.25"
- name: Build stub library
run: |
cc -c -o bridge/photokit_bridge_stub.o bridge/photokit_bridge_stub.c -I bridge
ar rcs bridge/libphotokit_bridge_stub.a bridge/photokit_bridge_stub.o
- name: Vet
run: go vet -tags=test ./...
- name: Test
run: go test -tags=test -race -coverprofile=coverage.out ./cmd/photoscli/ ./internal/photos/
- name: Coverage summary
run: go tool cover -func=coverage.out | tail -1
build:
runs-on: macos-14
needs: test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.25"
- name: Build
run: make build
- name: Verify version
run: ./bin/photoscli version
+20
View File
@@ -0,0 +1,20 @@
# AGENT.md
- Project: `photoscli`, macOS Apple Photos exporter, Go + cgo + Objective-C PhotoKit bridge.
- Release binary target: Apple Silicon only (`darwin/arm64`). Make this explicit in docs/assets.
- Module: `gitea.k3s.k0.nu/tools/photocli`.
- Run tests with `-tags=test`; tests use the C stub bridge, not real PhotoKit.
- Required before any release: `go test -tags=test -race -count=1 -coverprofile=coverage.out ./...` must show 100% for all packages, then `make pipeline`, then `make package`.
- Never release below 100% coverage.
- Release assets must include: binary, `USERGUIDE.md`, and `photoscli-<version>-macos-arm64.zip`.
- Release page must use clear notes from `RELEASE_NOTES.md`; `CHANGELOG.md` must be updated too.
- Keep README concise; put practical user workflows in `USERGUIDE.md`.
- Manifest backends: JSONL default, SQLite optional via `modernc.org/sqlite`.
- Preserve manifest compatibility and migration behavior.
- XMP sidecars are opt-in via `--sidecar xmp`; default must remain `none`.
- Reverse geocoding is opt-in via `--reverse-geocode`, uses MapKit on macOS 26+, and must fail safely on older macOS without failing export.
- Do not claim Photos people/animal/object labels are exported unless Vision/Core ML analysis is explicitly implemented.
- Roadmap: keep `0.8.x` focused on making XMP sidecars as rich, standard-compatible, and complete as possible from public PhotoKit metadata.
- Roadmap: use `0.9.0` through `0.9.5` for checksums, deep verification, manifest repair, cleanup, doctor, and backup-integrity hardening.
- Roadmap: start Vision/Core ML analysis only after `0.9.5`; keep it opt-in and label it as photoscli-generated analysis, not Apple Photos metadata.
- Do not commit generated artifacts from `bin/` or coverage files.
+221
View File
@@ -0,0 +1,221 @@
# Changelog
This changelog is maintained from git history plus the published Gitea release series. Future releases should update this file and publish matching release notes on the release page.
## v0.9.1
Deep checksum verification release.
- Add `verify --deep` to recompute SHA-256 checksums for manifest entries that have checksum metadata.
- Report `checksum-mismatch` when disk contents differ from the manifest checksum.
- Keep normal `verify` unchanged unless `--deep` is selected.
## v0.9.0
Manifest checksum release.
- Add opt-in `--checksum sha256` to store SHA-256 file checksums in JSONL and SQLite manifests.
- Add SQLite migration support for the manifest `checksum` column.
- Keep checksum collection disabled by default with `--checksum none`.
## v0.8.7
JSON sidecar release.
- Add `--sidecar json` for structured JSON metadata sidecars.
- Add `--sidecar xmp,json` to write both XMP and JSON sidecars from the same metadata.
- Keep `--sidecar none` as the default.
## v0.8.6
XMP keyword and rating controls release.
- Add `--xmp-keywords album-path|album|none` for generated sidecars.
- Add `--xmp-rating favorite|none` for generated sidecars.
- Keep existing keyword/rating behavior as defaults with `album-path` and `favorite`.
## v0.8.5
XMP privacy controls release.
- Add `--xmp-privacy keep|strip-location|strip-address` for generated sidecars.
- Keep existing XMP location/address behavior as the default with `keep`.
- Allow GPS coordinates to be kept while reverse-geocoded address fields are omitted with `strip-address`.
- Allow both GPS coordinates and address fields to be omitted with `strip-location`.
## v0.8.4
Strict XMP sidecar verification release.
- Add `verify --sidecar --strict` to require photoscli XMP schema metadata, sidecar generator metadata, and matching exported filename metadata.
- Keep existing `verify --sidecar` behavior unchanged for backup-wide existence, readability, and asset-ID checks.
## v0.8.3
XMP sidecar inspection release.
- Add `sidecar inspect <file.xmp>` to print key photoscli metadata from generated XMP sidecars.
- Add `sidecar inspect <file.xmp> --json` for scriptable inspection output.
## v0.8.2
Metadata-only XMP refresh release.
- Add `--metadata-only` for manifest-based XMP sidecar generation without re-exporting media files.
- Support metadata-only refresh for both `export` and `backup-all` when used with `--sidecar xmp`.
- Require a manifest for metadata-only mode so existing media paths are resolved safely.
- Keep media files untouched while overwriting/regenerating generated XMP sidecars.
- Support `--reverse-geocode` during metadata-only sidecar refresh.
## v0.8.1
XMP standards and sidecar verification release.
- Add XMP schema version metadata for generated sidecars.
- Add standard XMP fields for rating, metadata date, creation date, Photoshop date created, and EXIF GPS coordinates.
- Add `dc:subject` keywords from album/folder context.
- Add sidecar generator and generated timestamp metadata.
- Add `verify --sidecar` to check missing, zero-byte, unreadable, and asset-ID mismatched XMP sidecars.
## v0.8.0
Rich PhotoKit metadata and reverse-geocoded XMP release.
- Expand asset metadata from public PhotoKit fields: modification date, hidden state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment state, adjustment info, and richer resource metadata.
- Expand XMP sidecars with duration, hidden/adjustment state, modification date, structured media subtype/burst/resource lists, GPS coordinates, and adjustment metadata.
- Add opt-in `--reverse-geocode` using Apple reverse geocoding APIs to add address metadata for GPS assets.
- Cache reverse-geocode results under `.photoscli/geocode-cache.jsonl` so repeated exports do not repeatedly query Apple geocoding.
- Keep Vision/Core ML people, animal, object, and scene analysis out of this release; that remains future work.
## v0.7.0
XMP sidecar metadata release.
- Add `--sidecar none|xmp` with default `none`.
- Add config support for `sidecar = "xmp"`.
- Write XMP sidecars next to exported files using basename `.xmp` filenames.
- Include asset ID, filenames, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, photoscli version, exported timestamp, size, and creation date in XMP.
- Write XMP files atomically.
- Treat sidecar write failure as asset failure when XMP sidecars are explicitly requested.
## v0.6.0
Backup integrity and recovery release.
- Add manifest-relative paths for exported files so tree backups can be verified accurately.
- Add SQLite manifest migration for the new `path` column.
- Upgrade `verify` to report missing files, zero-byte files, and size mismatches by default.
- Add atomic staging export writes with per-asset temporary directories before final rename.
- Deduplicate `failures.jsonl` entries by asset ID and track attempts plus latest failure time.
- Add `failures list` and `failures clear` commands.
- Add `retry-failed --clear-on-success`.
- Add `status` command with text and JSON output.
- Update release workflow to publish release notes from `RELEASE_NOTES.md`.
- Rename release zip assets to include `macos-arm64` and document Apple Silicon-only releases.
- Add `AGENT.md` with the minimal project/release rules for future agents.
- Keep `CHANGELOG.md` as the canonical release history.
## v0.5.0
Manifest, filtering, logging, and documentation release.
- Add JSONL and SQLite manifests with resumable skip tracking.
- Add automatic conversion between JSONL and SQLite manifests.
- Add structured export logging to `export.log` or SQLite `logs` table.
- Add explicit exit codes: success, error, partial failure, and access denied.
- Add `--quality` for JPEG preview compression.
- Add `--concurrency` with progress slot cap.
- Add `--exclude-album` with exact/glob matching.
- Add `--since` date filtering.
- Add `--dry-run`.
- Add `--retry`.
- Add `--only-favorites`.
- Add `--media photos|videos|all`.
- Add `--json` command summaries.
- Add `--verify` integration.
- Add `--format jpeg|heic|png` validation/hint.
- Add `--min-size` and `--max-size` estimated pixel-count filters.
- Add `--date-template` folder layout support.
- Add `report`, `diff`, `verify`, and `retry-failed` commands.
- Add failure tracking in `failures.jsonl`.
- Add config-file defaults via `~/.photoscli.toml` or `PHOTOSCLI_CONFIG`.
- Add album name collision handling for duplicate sibling album names.
- Expand `README.md` and verbose CLI help.
- Add 100% test coverage for the CLI and manifest/photos packages.
## v0.4.1
Documentation and release asset follow-up after v0.5.0.
- Add `USERGUIDE.md` as a practical end-user manual.
- Link the user guide from `README.md`.
- Add release zip packaging with binary, README, USERGUIDE, and CHANGELOG.
- Attach `USERGUIDE.md` and the macOS zip package to the `v0.5.0` release.
## v0.4.0
Progress and backup-all usability release.
- Add scroll log for completed exports: copied, downloaded, skipped, and failed.
- Add worker status lines with live download progress bars for cloud files.
- Add color-gradient progress bars from red to yellow to green.
- Add disk skip for files already present during `backup-all`.
- Add parallel export with multiple workers for `backup-all`.
- Increase progress slot support in the bridge.
- Improve progress rendering for large export sessions.
## v0.3.x
Published on Gitea as `v0.3.0` through `v0.3.5`. These releases are not represented by local git tags in the current clone, so the exact per-release commit mapping is unavailable here.
Known purpose of this series:
- Iterative release packaging and Gitea release workflow improvements.
- Continued progress display and backup/export polish between `v0.2.5` and `v0.4.0`.
- Preparation for the larger `v0.4.0` progress/backup release.
## v0.2.6
Published on Gitea but not represented by a local git tag in the current clone.
- Post-`v0.2.5` maintenance release in the progress/export series.
## v0.2.5
- Add Unicode progress bar rendering.
- Add cloud download speed display.
## v0.2.4
- Stop export loop on Ctrl+C instead of flooding failures.
- Improve graceful cancellation behavior after interrupt.
## v0.2.3
- Fix export write failures.
- Fix Ctrl+C cancellation behavior.
## v0.2.2
No local tag is present for this version in the current clone.
## v0.2.1
- Add status messages during export.
- Fix parallel export progress display.
## v0.2.0
- Add semaphore timeouts.
- Add export error logging.
- Remove dead code.
- Add parallel exports.
## Pre-release
- Initial `applephotos` CLI with progress, cloud status, and per-asset export.
- Add `.gitignore` and remove build artifacts from tracking.
- Rename `applephotos` to `photoscli`.
- Update module path to `gitea.k3s.k0.nu/tools/photocli`.
- Add version flag.
- Add Gitea release targets and release pipeline.
+13 -8
View File
@@ -1,6 +1,8 @@
BINARY := ./bin/photoscli
MODULE := gitea.k3s.k0.nu/tools/photocli
VERSION := 0.2.1
VERSION := 0.9.1
RELEASE_ZIP := ./bin/photoscli-$(VERSION)-macos-arm64.zip
RELEASE_NOTES := RELEASE_NOTES.md
BRIDGE_DIR := bridge
LDFLAGS := -X main.version=$(VERSION)
OBJ := $(BRIDGE_DIR)/photokit_bridge.o
@@ -10,7 +12,7 @@ STUB_LIB := $(BRIDGE_DIR)/libphotokit_bridge_stub.a
GITEA_HOST := gitea-1.tail82444.ts.net
GITEA_REPO := tools/photocli
.PHONY: all build clean test coverage tag release pipeline
.PHONY: all build clean test coverage tag package release pipeline
all: build
@@ -18,7 +20,7 @@ $(LIB): $(OBJ)
ar rcs $@ $<
$(OBJ): $(BRIDGE_DIR)/photokit_bridge.m $(BRIDGE_DIR)/photokit_bridge.h
cc -c -x objective-c -fobjc-arc -framework Photos -framework Foundation -o $@ $<
cc -c -x objective-c -fobjc-arc -o $@ $<
$(STUB_LIB): $(STUB_OBJ)
ar rcs $@ $<
@@ -31,7 +33,7 @@ build: $(LIB)
test: $(STUB_LIB)
go vet -tags=test ./...
go test -tags=test -race -coverprofile=coverage.out -covermode=atomic -coverpkg=./... ./...
go test -tags=test -race -coverprofile=coverage.out ./cmd/photoscli/ ./internal/photos/
@grep -v 'main_main.go' coverage.out > coverage_filtered.out 2>/dev/null || true
@mv coverage_filtered.out coverage.out 2>/dev/null || true
@go tool cover -func=coverage.out | tail -1
@@ -43,17 +45,20 @@ coverage: $(STUB_LIB)
go tool cover -func=coverage.out
clean:
rm -f $(BINARY) $(OBJ) $(LIB) $(STUB_OBJ) $(STUB_LIB) coverage.out
rm -f $(BINARY) $(RELEASE_ZIP) $(OBJ) $(LIB) $(STUB_OBJ) $(STUB_LIB) coverage.out
package: build
zip -j $(RELEASE_ZIP) $(BINARY) README.md USERGUIDE.md CHANGELOG.md
tag:
git tag v$(VERSION)
git push origin v$(VERSION)
release:
tea releases create --repo $(GITEA_REPO) --tag v$(VERSION) --title "v$(VERSION)" --asset $(BINARY)
release: package
tea releases create --repo $(GITEA_REPO) --tag v$(VERSION) --title "v$(VERSION)" --note-file $(RELEASE_NOTES) --asset $(BINARY) --asset USERGUIDE.md --asset $(RELEASE_ZIP)
pipeline: clean test build
@echo "--- verifying version ---"
$(BINARY) version
@echo "--- all checks passed, ready to release ---"
@echo "run: make release"
@echo "run: make release"
+433 -131
View File
@@ -1,43 +1,53 @@
# photoscli
`photoscli` is a small macOS-only CLI written in Go that reads data from Apple Photos through a PhotoKit bridge.
`photoscli` is a macOS-only command-line exporter for Apple Photos. It uses a small Objective-C PhotoKit bridge from Go to list albums, inspect assets, export optimized previews or originals, keep resumable manifests, and produce machine-readable logs for backup automation.
It supports five tasks:
The tool is designed for repeatable, resumable Photos backups rather than one-off drag-and-drop exports.
- listing albums
- listing asset IDs, filenames, and cloud status in an album
- showing the folder and album tree
- backing up all albums into the Photos folder tree
- exporting resized JPEG previews or original files from an album
For a practical step-by-step manual with recommended backup workflows, recovery steps, automation examples, and troubleshooting, see `USERGUIDE.md`.
## What The Code Does
## Highlights
The executable lives in `cmd/photoscli` and calls into `internal/photos`, which wraps an Objective-C bridge in `bridge/`.
Current behavior:
- `albums` prints one line per album as `<album-id>\t<title>`
- `photos --album-id <id-or-title>` prints one asset per line as `<id>\t<filename>\t<cloud>`; accepts either a PhotoKit local identifier or an album title
- `tree` prints the human-readable folder and album hierarchy from Apple Photos
- `backup-all --out <dir> [--size <px>] [--originals]` exports every album into a matching folder tree, showing per-asset progress with filename, size, and cloud status
- `export --album-id <id-or-title> --out <dir> [--size <px>] [--originals]` exports either JPEG previews or original files with a progress bar showing filename, size, and cloud status; `--album-id` accepts either a PhotoKit local identifier or an album title
The bridge uses PhotoKit to:
- request access to the user's Photos library
- fetch album collections by local identifier or album title
- fetch album assets sorted by `creationDate` ascending
- render resized images with `PHImageManager`
- write JPEG files with compression `0.85`
- support graceful cancellation via a cancel flag checked between file exports
- List albums, photos, and the Photos folder/album tree.
- Export one album or back up the whole Photos tree.
- Export JPEG previews with configurable size and quality, or original files.
- Skip already-exported assets through a JSONL or SQLite manifest.
- Resume interrupted runs safely.
- Parallel export with live worker progress and ETA.
- Optional structured logging to `export.log` or SQLite `logs` table.
- Dry-run mode for planning large backups.
- Filters for favorites, media type, date, estimated size, and excluded albums.
- Failure tracking with `failures.jsonl` and `retry-failed`.
- Deduplicated failure management with `failures list`, `failures clear`, and `retry-failed --clear-on-success`.
- Verification, reporting, and diff commands for backup integrity.
- Optional XMP sidecar verification with `verify --sidecar`.
- Optional SHA-256 manifest checksums with `--checksum sha256`.
- Metadata-only XMP refresh for manifest-backed exports with `--metadata-only`.
- Status command for quick backup summaries.
- Opt-in rich XMP sidecar metadata with `--sidecar xmp`.
- Optional Apple MapKit reverse geocoding for GPS assets on macOS 26+ with `--reverse-geocode`.
- Script-friendly exit codes and optional JSON summaries.
- 100% test coverage for the Go CLI and parsing layers.
## Requirements
- macOS
- Go 1.22+
- Xcode command-line tools
- Apple Silicon Mac, M1/M2/M3/M4 or newer (`darwin/arm64`).
- macOS with Apple Photos.
- Go 1.25 or newer.
- Xcode command-line tools.
- Photos privacy permission for the built binary or terminal app.
The project builds with cgo and links against `Photos`, `Foundation`, and `AppKit`.
The provided release binary is Apple Silicon only. Intel Macs are not currently a supported release target. The production build uses cgo and links against Apple frameworks through the Objective-C bridge in `bridge/`.
## Release Assets
Release zip files are named with the supported architecture:
```text
photoscli-<version>-macos-arm64.zip
```
The zip contains the Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
## Build
@@ -45,41 +55,80 @@ The project builds with cgo and links against `Photos`, `Foundation`, and `AppKi
make build
```
Output binary:
The binary is written to:
```bash
./bin/photoscli
```
## Test
Run the full local release pipeline:
```bash
make test
make pipeline
```
Tests run against a stub bridge, so they do not require real Photos access.
The pipeline cleans artifacts, builds the test bridge, runs vet, runs race-enabled tests with coverage, builds the real PhotoKit bridge, builds the CLI, and verifies the embedded version.
## Usage
## Quick Start
List albums:
```bash
./bin/photoscli albums
./bin/photoscli photos --album-id "<album-local-identifier>"
./bin/photoscli photos --album-id "Vacation"
./bin/photoscli tree
./bin/photoscli backup-all --out ./backup
./bin/photoscli backup-all --out ./backup --originals
./bin/photoscli export --album-id "<album-local-identifier>" --out ./export
./bin/photoscli export --album-id "Vacation" --out ./export
./bin/photoscli export --album-id "<album-local-identifier>" --out ./export --size 2048
./bin/photoscli export --album-id "<album-local-identifier>" --out ./export --originals
```
### Commands
Inspect an album by title or PhotoKit local identifier:
`albums`
```bash
./bin/photoscli photos --album-id "Vacation"
```
- Requests Photos access
- Lists albums as tab-separated album ID and title
Export preview JPEGs from one album:
```bash
./bin/photoscli export --album-id "Vacation" --out ./export
```
Export higher-quality previews:
```bash
./bin/photoscli export --album-id "Vacation" --out ./export --size 2048 --quality 92
```
Export originals:
```bash
./bin/photoscli export --album-id "Vacation" --out ./originals --originals
```
Back up the entire Photos album tree:
```bash
./bin/photoscli backup-all --out ./photos-backup --manifest sqlite --log
```
Preview what would happen before writing anything:
```bash
./bin/photoscli backup-all --out ./photos-backup --dry-run --json
```
Verify a backup later:
```bash
./bin/photoscli verify --out ./photos-backup --manifest sqlite
./bin/photoscli verify --out ./photos-backup --deep
```
## Commands
### `albums`
Lists user-created albums as tab-separated ID and title.
```bash
photoscli albums
```
Example output:
@@ -88,122 +137,375 @@ Example output:
8A1B.../L0/001 Work
```
`photos --album-id <id-or-title>`
### `photos`
- Requests Photos access
- If the value looks like a PhotoKit local identifier, uses it directly
- Otherwise searches album titles for a match and resolves the identifier
- Lists asset local identifiers and cloud status for the given album
Lists assets in an album. `--album-id` accepts a PhotoKit local identifier or an exact album title.
```bash
photoscli photos --album-id "Vacation"
```
Output includes ID, filename, cloud state, media type, dimensions, optional creation date, optional duration, and a `*` marker for favorites.
### `tree`
Prints the Photos folder and album hierarchy.
```bash
photoscli tree
```
### `export`
Exports assets from one album.
```bash
photoscli export --album-id <id-or-title> --out <dir> [flags]
```
Useful examples:
```bash
photoscli export --album-id "Family" --out ./family --size 2560 --quality 90
photoscli export --album-id "Favorites" --out ./favorites --only-favorites
photoscli export --album-id "Videos" --out ./videos --media videos --include-videos
photoscli export --album-id "Archive" --out ./archive --originals --retry 3
photoscli export --album-id "Vacation" --out ./vacation --date-template YYYY/MM/DD
photoscli export --album-id "Vacation" --out ./vacation --sidecar xmp --reverse-geocode
```
### `backup-all`
Walks the Photos folder/album tree and exports every album to a matching folder structure.
```bash
photoscli backup-all --out <dir> [flags]
```
Duplicate sibling album names are disambiguated by adding the album ID to the generated folder name.
Useful examples:
```bash
photoscli backup-all --out ./backup --manifest sqlite --log
photoscli backup-all --out ./backup --checksum sha256
photoscli backup-all --out ./backup --exclude-album "Recently Deleted" --exclude-album "Temp*"
photoscli backup-all --out ./backup --since 2024-01-01 --sort newest
photoscli backup-all --out ./backup --concurrency 8 --retry 3
photoscli backup-all --out ./backup --dry-run --json
photoscli backup-all --out ./backup --sidecar xmp --reverse-geocode
```
### `report`
Shows manifest and failure counts for an output directory.
```bash
photoscli report --out ./backup --manifest sqlite
```
Example output:
```text
1F2A.../L0/001 IMG_0001.JPG local
9C4D.../L0/001 IMG_0002.JPG cloud
entries 12345
failures 2
```
`backup-all --out <dir> [--size <px>] [--originals]`
### `diff`
- Requests Photos access
- Walks the Photos folder and album hierarchy
- Creates directories as `out/folder/album/files`
- Exports previews by default, originals when `--originals` is present
- Shows per-asset progress bar with filename, file size, and cloud/local status
- Uses `--size` only for preview export
Compares an album against the manifest and prints assets missing from the manifest.
Example layout:
```bash
photoscli diff --album-id "Vacation" --out ./backup --manifest jsonl
```
Exit code is `2` when missing assets are found.
### `verify`
Checks that manifest entries have corresponding files on disk.
```bash
photoscli verify --out ./backup --manifest sqlite
```
Exit code is `2` when files are missing.
### `retry-failed`
Retries assets recorded in `failures.jsonl`.
```bash
photoscli retry-failed --out ./backup
```
This is useful after transient iCloud or network failures.
Use `--clear-on-success` to remove successfully retried assets from `failures.jsonl`.
### `failures`
Lists or clears deduplicated failure records.
```bash
photoscli failures list --out ./backup
photoscli failures clear --out ./backup
```
### `status`
Shows a quick backup summary.
```bash
photoscli status --out ./backup --manifest sqlite
photoscli status --out ./backup --manifest sqlite --json
```
## Export Flags
Common flags for `export` and `backup-all`:
- `--out <dir>`: destination directory.
- `--size <px>`: longest-side target for preview export, default `1024`.
- `--quality <1-100>`: JPEG preview quality, default `85`.
- `--originals`: export original files instead of previews.
- `--concurrency <N>`: parallel workers, default `3`, capped by bridge slot count.
- `--retry <N>`: retry failed exports with a small backoff.
- `--dry-run`: print planned exports without writing files, manifests, or logs.
- `--json`: print a machine-readable summary to stdout.
- `--verify`: run manifest/file verification after export.
- `--sidecar` with `verify`: also verify expected `.xmp` sidecars.
- `--log`: enable structured export logging.
- `--manifest jsonl|sqlite`: choose manifest backend, default `jsonl`.
- `--no-manifest`: disable manifest reads/writes.
- `--sort oldest|newest`: asset sort order where supported, default `oldest`.
- `--since <date>`: only include assets on or after `YYYY-MM-DD` or RFC3339 date.
- `--only-favorites`: only export favorite assets.
- `--media photos|videos|all`: media filter, default `photos`.
- `--include-videos`: compatibility flag that includes videos/audio.
- `--min-size <n>`: filter by estimated pixel count.
- `--max-size <n>`: filter by estimated pixel count.
- `--format jpeg|heic|png`: preview format hint. Current bridge output is still the existing preview path; non-JPEG bridge output is future work.
- `--sidecar none|xmp`: write opt-in XMP metadata sidecars next to exported files.
- `--metadata-only`: with `--sidecar xmp`, refresh XMP sidecars for manifest-backed files without exporting media.
- `--reverse-geocode`: with `--sidecar xmp`, add cached Apple MapKit address metadata for GPS assets on macOS 26+.
- `--date-template <template>`: append date folders based on creation date, for example `YYYY/MM/DD`.
`backup-all` also supports:
- `--exclude-album <pattern>`: repeatable exact or glob-style album-name exclusion.
`export` also requires:
- `--album-id <id-or-title>`: PhotoKit local identifier or exact album title.
## Manifests
Manifests prevent re-exporting assets on subsequent runs.
JSONL backend:
```text
backup/
Trips/
Italy 2024/
Venice/
0000_....jpg
Favorites/
0000_....jpg
downloads.jsonl
```
`export --album-id <id-or-title> --out <dir> [--size <px>] [--originals]`
- Requests Photos access
- Resolves `--album-id` by local identifier first, then by album title if not found
- Creates the output directory if needed
- Exports assets one at a time with a progress bar: `[=======---] 50% filename.jpg 1.2 MB cloud`
- Shows file size and cloud/local status for each exported asset
- Exports resized JPEG previews by default
- Exports original files when `--originals` is present
- Writes a summary like `exported 10 photos to ./export` or `exported 10 original files to ./export` to stderr
`--size` is the target bounding box passed to PhotoKit for preview export. Default: `1024`.
`--originals` switches export mode to original-file export. In that mode, `--size` is ignored.
`tree`
- Requests Photos access
- Prints folders and albums as an indented tree
- Omits internal album IDs for human-readable output
Example output:
SQLite backend:
```text
Trips
Italy 2024
Venice
Favorites
downloads.db
```
The manifest stores asset ID, filename, size, cloud state, and export timestamp. SQLite is useful for very large libraries and for keeping logs in the same database.
Use no manifest only when you intentionally want stateless export behavior:
```bash
photoscli export --album-id "Scratch" --out ./scratch --no-manifest
```
## Logging
Enable logs with:
```bash
photoscli backup-all --out ./backup --log
```
With JSONL/no-manifest mode, logs are written to:
```text
export.log
```
With SQLite manifest mode, logs are written to the `logs` table in `downloads.db`.
Logged events include session start/end, completed exports, skipped assets, and failures. Each event includes timestamp, level, event name, asset ID, album, filename, size, cloud state, duration, and message where available.
## XMP Sidecars
Write archival metadata sidecars with:
```bash
photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json
```
Sidecars are opt-in and use the exported file basename:
```text
IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json
```
The XMP contains photoscli metadata such as asset ID, filenames, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, version, exported timestamp, size, and creation date when available. If `--sidecar xmp` is explicitly selected and the sidecar cannot be written, that asset is treated as failed.
Sidecars also include richer public PhotoKit metadata where available: modification date, duration, hidden state, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP dates, EXIF GPS coordinates, favorite rating, and album/folder keywords. Add `--reverse-geocode` to include cached address fields from Apple MapKit for assets with GPS coordinates. Reverse geocoding requires macOS 26 or newer; on older macOS versions the export continues and XMP still includes GPS coordinates.
Control XMP location metadata with `--xmp-privacy keep|strip-location|strip-address`. The default is `keep`. Use `strip-address` to omit reverse-geocoded address fields while keeping GPS coordinates, or `strip-location` to omit both GPS and address fields.
Control generated XMP keywords and ratings with `--xmp-keywords album-path|album|none` and `--xmp-rating favorite|none`. Defaults preserve existing behavior: album/folder keywords and favorite assets mapped to `xmp:Rating="5"`.
Verify generated sidecars with:
```bash
photoscli verify --out ./backup --sidecar
```
For stricter checks against recent photoscli-generated XMP sidecars:
```bash
photoscli verify --out ./backup --sidecar --strict
```
Inspect one generated sidecar with:
```bash
photoscli sidecar inspect ./backup/IMG_0001.xmp
photoscli sidecar inspect ./backup/IMG_0001.xmp --json
```
Refresh sidecars for files already present in a manifest-backed export without rewriting media files:
```bash
photoscli backup-all --out ./backup --sidecar xmp --metadata-only
```
## Failure Tracking
Failed exports are deduplicated by asset ID and stored in:
```text
failures.jsonl
```
Retry them with:
```bash
photoscli retry-failed --out ./backup
```
Clear successful retries automatically:
```bash
photoscli retry-failed --out ./backup --clear-on-success
```
Use `--retry N` during normal exports to handle transient iCloud or network failures automatically.
## Configuration File
Defaults can be loaded from:
```text
~/.photoscli.toml
```
Or from a custom path:
```bash
PHOTOSCLI_CONFIG=/path/to/photoscli.toml photoscli backup-all --out ./backup
```
Supported keys mirror long flag names without the leading dashes:
```toml
size = 2048
quality = 90
concurrency = 8
manifest = "sqlite"
log = true
sort = "newest"
media = "photos"
retry = 3
```
Command-line flags override config-file values.
## Exit Codes
- `0`: success.
- `1`: command/config/runtime error.
- `2`: partial failure, such as some exports failed or diff/verify found missing items.
- `3`: Photos access denied.
## Permissions
On first use, macOS may prompt for Photos access.
On first use, macOS may prompt for Photos access. If access is denied, grant access in:
If access is denied, the CLI returns an error telling you to grant access in:
```text
System Settings > Privacy & Security > Photos
```
`System Settings > Privacy & Security`
Depending on how you launch the binary, macOS may associate the permission with Terminal, iTerm, your shell, or the built executable.
## Export Details
## Development
Exported files currently:
Run tests with the stub bridge:
- are JPEGs when exporting previews
- keep their original filenames when exporting originals when possible
- fall back to a sanitized asset identifier if an original filename is unavailable
- prefix duplicate original filenames with the asset index to avoid collisions
- name preview exports like `0000_<asset-local-identifier>.jpg`
- replace `/` and `\` in asset IDs with `_` for generated preview filenames
- replace `/` and `\` in folder and album names with `_` when creating backup directory names
- preserve ordering based on ascending asset creation date
```bash
make test
```
If some assets fail but at least one succeeds, the command still succeeds and reports the number exported.
Run all package coverage:
If all exports fail, the command returns an error.
```bash
go test -tags=test -race -count=1 -coverprofile=coverage.out ./...
go tool cover -func=coverage.out
```
## Signal Handling
Run the release pipeline:
Sending `SIGINT` (Ctrl+C) or `SIGTERM` during export or backup triggers a graceful shutdown:
```bash
make pipeline
```
1. The CLI prints `received signal, finishing current file...` to stderr
2. The current file export is allowed to complete
3. No further files are started
4. The process exits after the in-progress file finishes
Create a release after committing and tagging:
A second signal forces an immediate exit.
```bash
make release
```
## Architecture
- `cmd/photoscli`: CLI entrypoint, argument parsing, and album name resolution
- `internal/photos`: Go bridge interface, JSON parsing, and error mapping
- `bridge/`: Objective-C PhotoKit implementation plus a C test stub
- `cmd/photoscli`: CLI, argument handling, filtering, manifests, progress, reporting, and command orchestration.
- `internal/photos`: Go interface and JSON parsing for PhotoKit responses.
- `internal/manifest`: JSONL/SQLite manifest backends and log writers.
- `bridge/`: Objective-C PhotoKit implementation plus a C stub used by tests.
Data passed from Objective-C to Go is serialized as JSON and unmarshaled into Go structs.
Objective-C returns JSON to Go. Tests use the stub bridge and do not require real Photos access.
## Known Limitations
## Known Notes
- The tree view only shows user collections exposed through PhotoKit's top-level user collections API
- Album title resolution matches the first album with that title; if multiple albums share a title, use the local identifier instead
- `photos` only prints asset IDs and filenames, not dates or metadata
- Preview export uses PhotoKit preview rendering, not original file export
- Original export currently writes the first PhotoKit asset resource for each asset, which may not capture every related representation for complex assets
- iCloud-backed assets may require network download during export
- A second interrupt signal forces an immediate exit without waiting for the current file
- Partial export failures are not listed individually
- This project is macOS-only.
- Preview export currently follows the existing bridge preview implementation; `--format heic|png` is validated as an output-format hint but still needs bridge support for true non-JPEG previews.
- Album title lookup uses exact title matching. Use PhotoKit local identifiers when names are ambiguous.
- iCloud-backed assets may trigger downloads and can fail due to network or account state.
- `--min-size` and `--max-size` currently use estimated pixel count from dimensions, not encoded file size.
## Roadmap
- `0.8.x`: make XMP sidecars as rich, standard-compatible, and complete as possible from public PhotoKit metadata.
- `0.9.0` through `0.9.5`: checksums, deep verification, manifest repair, cleanup, doctor, and backup-integrity hardening.
- After `0.9.5`: opt-in Vision/Core ML analysis for photoscli-generated face/object/animal/scene metadata.
+18
View File
@@ -0,0 +1,18 @@
# v0.9.1
This release adds deep checksum verification for manifest-backed backups.
## Highlights
- Add `verify --deep` to recompute SHA-256 checksums when manifest entries include checksums.
- Report `checksum-mismatch` when file contents differ from the manifest checksum.
- Keep normal `verify` unchanged unless `--deep` is selected.
- `--deep` works with JSONL and SQLite manifests.
## Assets
- `photoscli`: Apple Silicon macOS binary (`darwin/arm64`).
- `photoscli-0.9.1-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
- `USERGUIDE.md`: standalone user guide.
Intel Macs are not currently a supported release target.
+860
View File
@@ -0,0 +1,860 @@
# photoscli User Guide
This guide explains how to use `photoscli` safely for Apple Photos exports and backups. It is written for day-to-day users who want reliable commands, clear recovery steps, and predictable backup behavior.
For a compact command reference, see `README.md` or run:
```bash
photoscli help
```
## What photoscli Is For
`photoscli` exports assets from Apple Photos on macOS.
It is especially useful when you want to:
- Back up the whole Photos album tree to normal folders.
- Export one album for sharing or archiving.
- Resume a large export without starting over.
- Keep a manifest of already-exported assets.
- Retry transient iCloud failures.
- Verify that files referenced by a manifest exist on disk.
- Detect missing, zero-byte, and size-mismatched manifest files.
- Store optional SHA-256 checksums in manifests.
- Inspect or clear deduplicated failure records.
- Write optional XMP sidecar metadata for archival workflows.
- Add optional reverse-geocoded address metadata to XMP sidecars for GPS assets.
- Script Photos exports with stable exit codes.
It is not intended to replace Apple Photos, iCloud Photos, or Time Machine. Think of it as an additional file-based export and backup tool.
## Before You Start
You need:
- An Apple Silicon Mac, M1/M2/M3/M4 or newer. The prebuilt release binary is `darwin/arm64` only.
- macOS.
- Apple Photos library available on the machine.
- Photos permission granted to the terminal app or binary.
- Enough disk space for the export.
- A built `photoscli` binary.
Build it from the repo:
```bash
make build
```
Run the binary directly:
```bash
./bin/photoscli version
```
If you install or copy it somewhere else, replace `./bin/photoscli` in examples with `photoscli`.
Intel Macs are not currently a supported release target. If Intel support is added later, release assets will include a separate architecture-specific package.
## First Run And Permissions
Start with a harmless command:
```bash
./bin/photoscli albums
```
macOS may ask for Photos access. Allow it.
If access is denied, open:
```text
System Settings > Privacy & Security > Photos
```
Then enable access for the terminal application you use, such as Terminal, iTerm, or another launcher. Depending on how you start the binary, macOS may associate the permission with the terminal app rather than the binary itself.
Verify access:
```bash
./bin/photoscli tree
```
## Recommended Backup Strategy
For most users, the safest full-library backup command is:
```bash
./bin/photoscli backup-all \
--out /Volumes/BackupDrive/PhotosExport \
--manifest sqlite \
--log \
--retry 3 \
--concurrency 4
```
This gives you:
- Full album-tree export.
- SQLite manifest for fast resumable backups.
- Structured logs in the same database.
- Automatic retries for transient failures.
- Moderate concurrency that is usually safe for large libraries.
Before running it for real, run a dry-run:
```bash
./bin/photoscli backup-all \
--out /Volumes/BackupDrive/PhotosExport \
--manifest sqlite \
--dry-run \
--json
```
## Preview Export vs Original Export
By default, `photoscli` exports optimized preview images.
Preview export:
- Produces smaller files.
- Uses `--size` and `--quality`.
- Is good for browsing, sharing, and lightweight archives.
- May not preserve original file bytes or all original metadata.
Original export:
- Uses `--originals`.
- Exports original asset resources where PhotoKit exposes them.
- Ignores `--size` and `--quality`.
- Uses more disk space.
- Is usually the better choice for archival backups.
Preview example:
```bash
./bin/photoscli export --album-id "Vacation" --out ./VacationPreview --size 2048 --quality 90
```
Original example:
```bash
./bin/photoscli export --album-id "Vacation" --out ./VacationOriginals --originals
```
## Export One Album
List albums first:
```bash
./bin/photoscli albums
```
Export by title:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation
```
Export by PhotoKit local identifier:
```bash
./bin/photoscli export --album-id "5E9F.../L0/001" --out ./Vacation
```
Use the local identifier when album names are duplicated or ambiguous.
Recommended one-album archival export:
```bash
./bin/photoscli export \
--album-id "Vacation" \
--out ./VacationOriginals \
--originals \
--manifest sqlite \
--log \
--retry 3
```
## Back Up The Whole Library
Use `backup-all` to export every album into a folder tree matching Photos folders and albums.
```bash
./bin/photoscli backup-all --out ./PhotosBackup
```
Recommended full backup:
```bash
./bin/photoscli backup-all \
--out ./PhotosBackup \
--manifest sqlite \
--log \
--retry 3 \
--concurrency 4
```
Recommended full original backup:
```bash
./bin/photoscli backup-all \
--out ./PhotosOriginals \
--originals \
--manifest sqlite \
--log \
--retry 3 \
--concurrency 4
```
If two sibling albums have the same name, `photoscli` adds the album ID to the generated folder name so the folders do not collide.
## Dry Runs
Dry-runs are strongly recommended before large exports.
```bash
./bin/photoscli backup-all --out ./PhotosBackup --dry-run
```
With JSON output:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --dry-run --json
```
Dry-run mode does not write exported files, manifests, logs, or failure files. It prints what would be exported.
Use dry-run when:
- You are testing filters.
- You are checking output paths.
- You are preparing a large first-time backup.
- You are validating an exclude list.
## Incremental Backups
Incremental backups are enabled by manifests.
Run the same backup command repeatedly:
```bash
./bin/photoscli backup-all \
--out ./PhotosBackup \
--manifest sqlite \
--log \
--retry 3
```
Already-exported assets are skipped based on the manifest.
For new assets since a date:
```bash
./bin/photoscli backup-all \
--out ./PhotosBackup \
--manifest sqlite \
--since 2024-01-01
```
The `--since` flag accepts:
- `YYYY-MM-DD`
- RFC3339, for example `2024-01-01T00:00:00Z`
## Manifests
Manifests track exported asset IDs and allow safe resume behavior.
JSONL manifest:
```text
downloads.jsonl
```
SQLite manifest:
```text
downloads.db
```
Use JSONL when:
- You want a simple append-friendly text file.
- Your library is small or medium-sized.
- You want easy inspection with normal text tools.
Use SQLite when:
- Your library is large.
- You want faster lookup behavior.
- You want logs stored in the same database.
- You plan to run repeated backups over time.
Recommended for serious backups:
```bash
--manifest sqlite
```
Disable manifests only for temporary exports:
```bash
./bin/photoscli export --album-id "Scratch" --out ./Scratch --no-manifest
```
Without a manifest, `photoscli` cannot use manifest-based resume behavior.
## Logging
Enable logs with:
```bash
--log
```
For JSONL or no-manifest exports, logs go to:
```text
export.log
```
For SQLite manifests, logs go to the `logs` table in:
```text
downloads.db
```
Logged events include:
- Session start.
- Session end.
- Export completed.
- Export skipped.
- Export failed.
Use logs when:
- You are running unattended backups.
- You need auditability.
- You want details about failed or skipped assets.
## Failed Exports And Retries
Failed exports are written to:
```text
failures.jsonl
```
Failure records are deduplicated by asset ID. Repeated failures update the existing record and increment the attempt count.
For transient failures, first use retries during normal export:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --retry 3
```
If failures remain, retry later:
```bash
./bin/photoscli retry-failed --out ./PhotosBackup
```
Clear successful retry records automatically:
```bash
./bin/photoscli retry-failed --out ./PhotosBackup --clear-on-success
```
List or clear failure records:
```bash
./bin/photoscli failures list --out ./PhotosBackup
./bin/photoscli failures clear --out ./PhotosBackup
```
Typical reasons for failures:
- iCloud asset not currently downloadable.
- Network interruption.
- Photos permission issue.
- Destination disk unavailable.
- Destination disk full.
- Invalid or changing Photos library state.
After retrying, run verification:
```bash
./bin/photoscli verify --out ./PhotosBackup --manifest sqlite
```
## Verification
Run verification after large backups:
```bash
./bin/photoscli verify --out ./PhotosBackup --manifest sqlite
```
Verification checks that manifest entries point to files on disk. It also reports zero-byte files and size mismatches when the manifest has a recorded size.
It exits with:
- `0` when all checked files exist.
- `2` when files are missing.
- `1` for argument or runtime errors.
You can also verify immediately after export:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --verify
```
## Reporting
Show manifest and failure counts:
```bash
./bin/photoscli report --out ./PhotosBackup --manifest sqlite
```
Example output:
```text
entries 12345
failures 2
```
Use this after long runs to quickly check whether work was recorded and whether failures occurred.
## Diffing An Album Against A Backup
Use `diff` to find album assets that are not in the manifest:
```bash
./bin/photoscli diff --album-id "Vacation" --out ./PhotosBackup --manifest sqlite
```
Missing assets are printed as:
```text
<asset-id> <filename>
```
Exit code is `2` if missing assets are found.
## Filtering
### Favorites Only
```bash
./bin/photoscli export --album-id "Favorites" --out ./Favorites --only-favorites
```
### Media Type
Photos only, the default:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --media photos
```
Videos only:
```bash
./bin/photoscli backup-all --out ./VideoBackup --media videos --include-videos
```
Everything:
```bash
./bin/photoscli backup-all --out ./FullBackup --media all --include-videos
```
### Exclude Albums
Exclude exact album names:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --exclude-album "Recently Deleted"
```
Exclude by glob:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --exclude-album "Temp*"
```
Use multiple exclusions:
```bash
./bin/photoscli backup-all \
--out ./PhotosBackup \
--exclude-album "Recently Deleted" \
--exclude-album "Temp*" \
--exclude-album "Screenshots"
```
### Date Filter
```bash
./bin/photoscli backup-all --out ./PhotosBackup --since 2024-01-01
```
### Estimated Size Filter
`--min-size` and `--max-size` currently filter by estimated pixel count from dimensions, not encoded file size.
```bash
./bin/photoscli export --album-id "Large" --out ./LargeOnly --min-size 12000000
```
## Date-Based Folder Layouts
Use `--date-template` to append date folders based on asset creation date.
Example:
```bash
./bin/photoscli export \
--album-id "Vacation" \
--out ./Vacation \
--date-template YYYY/MM/DD
```
Possible output:
```text
Vacation/
2024/
07/
15/
0000_....jpg
```
Supported tokens:
- `YYYY`
- `MM`
- `DD`
Assets without parseable creation dates stay in the base output path.
## XMP Sidecars
Use XMP sidecars when you want portable metadata next to exported files:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json
```
Sidecars are disabled by default. When enabled, they use the exported file basename:
```text
IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json
```
Use XMP for standards-oriented metadata workflows and JSON when you want structured photoscli metadata for scripts or audits.
The XMP includes photoscli archive metadata such as asset ID, original filename, exported filename, album, manifest path, media type, dimensions, favorite state, hidden state, cloud state, export mode, version, exported time, size, creation date, modification date, duration, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP date fields, EXIF GPS fields, favorite rating, and album/folder keywords when PhotoKit exposes them.
Control location privacy in generated sidecars:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-address
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-location
```
`keep` is the default. `strip-address` omits reverse-geocoded address fields while preserving GPS coordinates. `strip-location` omits both GPS coordinates and address fields.
Control generated keywords and ratings:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords album
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords none --xmp-rating none
```
`--xmp-keywords album-path` is the default and writes album/folder keywords. `album` writes only the album name. `none` omits generated `dc:subject` keywords. `--xmp-rating favorite` is the default and maps favorite assets to `xmp:Rating="5"`; `none` omits that generated rating.
For address metadata from GPS coordinates, opt in to Apple's reverse geocoder:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --reverse-geocode
```
Reverse geocoding uses Apple MapKit and requires macOS 26 or newer. It can require network access and may be rate-limited by Apple, so results are cached in `.photoscli/geocode-cache.jsonl` under the backup root. On older macOS versions, `--reverse-geocode` is treated as unavailable: the export continues, no address fields are added, and the XMP still contains GPS coordinates.
Verify sidecars after an export:
```bash
./bin/photoscli verify --out ./PhotosBackup --sidecar
```
This reports missing, zero-byte, unreadable, or asset-ID mismatched `.xmp` files.
Use strict verification for sidecars generated by recent photoscli versions:
```bash
./bin/photoscli verify --out ./PhotosBackup --sidecar --strict
```
Strict mode also checks photoscli schema metadata, generator metadata, and the exported filename recorded inside the sidecar.
Inspect one generated sidecar when troubleshooting or scripting:
```bash
./bin/photoscli sidecar inspect ./PhotosBackup/IMG_0001.xmp
./bin/photoscli sidecar inspect ./PhotosBackup/IMG_0001.xmp --json
```
Refresh metadata only for an existing manifest-backed backup:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --sidecar xmp --metadata-only
```
Metadata-only mode does not re-export media files. It uses manifest paths to find existing files and rewrites generated `.xmp` sidecars next to them. It requires `--sidecar xmp` and an enabled manifest.
If you explicitly request `--sidecar xmp` and the XMP file cannot be written, the asset is counted as failed.
## Configuration File
You can store default values in:
```text
~/.photoscli.toml
```
Example:
```toml
size = 2048
quality = 90
concurrency = 4
manifest = "sqlite"
sort = "newest"
media = "photos"
retry = 3
log = true
sidecar = "xmp"
reverse-geocode = true
```
Use a custom config path:
```bash
PHOTOSCLI_CONFIG=./photoscli.toml ./bin/photoscli backup-all --out ./PhotosBackup
```
Command-line flags override config-file defaults.
Recommended config for recurring backups:
```toml
manifest = "sqlite"
log = true
retry = 3
concurrency = 4
sort = "oldest"
media = "photos"
quality = 90
size = 2048
```
Then run:
```bash
./bin/photoscli backup-all --out ./PhotosBackup
```
## Automation Examples
### Weekly Incremental Backup
```bash
#!/bin/sh
set -eu
OUT="/Volumes/BackupDrive/PhotosExport"
BIN="$HOME/bin/photoscli"
"$BIN" backup-all \
--out "$OUT" \
--manifest sqlite \
--log \
--retry 3 \
--concurrency 4 \
--json
"$BIN" verify --out "$OUT" --manifest sqlite
```
### Export Favorites For Sharing
```bash
./bin/photoscli export \
--album-id "Favorites" \
--out ./FavoritesShare \
--only-favorites \
--size 2048 \
--quality 88 \
--no-manifest
```
### Archive Originals From One Album
```bash
./bin/photoscli export \
--album-id "Family Archive" \
--out ./FamilyArchiveOriginals \
--originals \
--manifest sqlite \
--log \
--retry 3
```
### Backup Everything Including Videos
```bash
./bin/photoscli backup-all \
--out ./FullPhotosBackup \
--media all \
--include-videos \
--manifest sqlite \
--log \
--retry 3
```
## Interpreting Exit Codes
`photoscli` uses stable exit codes for scripts:
- `0`: success.
- `1`: invalid arguments, runtime error, or all exports failed.
- `2`: partial failure, diff found missing manifest entries, or verify found missing files.
- `3`: Photos access denied.
Example shell handling:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log
rc=$?
case "$rc" in
0) echo "backup succeeded" ;;
2) echo "backup partially succeeded; inspect failures.jsonl" ;;
3) echo "Photos access denied; check macOS privacy settings" ;;
*) echo "backup failed with exit code $rc" ;;
esac
```
## Troubleshooting
### Access Denied
Symptom:
```text
error: access denied
```
Fix:
- Open System Settings.
- Go to Privacy & Security > Photos.
- Enable access for your terminal app or launcher.
- Run `photoscli albums` again.
### iCloud Assets Fail To Export
Try:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --retry 3
```
If failures remain:
```bash
./bin/photoscli retry-failed --out ./PhotosBackup
```
Also check:
- Network connectivity.
- iCloud Photos status.
- Whether Photos.app can open/download the asset.
### Destination Disk Is Full
Free space, then rerun the same command. The manifest should skip already-exported assets and continue with remaining work.
### Duplicate Album Names
`backup-all` automatically appends the album ID to duplicate sibling album names. If you want exact control, export ambiguous albums individually using their PhotoKit local identifiers.
### Too Slow
Try a moderate concurrency increase:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --concurrency 8
```
If your library is heavily iCloud-backed, too much concurrency may increase transient failures. Use `--retry 3` and reduce concurrency if needed.
### Too Much Output Or Hard To Parse
Use JSON summary output:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --json
```
Use structured logs:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log
```
Store SHA-256 checksums in the manifest when exporting:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --checksum sha256
```
Checksums are opt-in and stored in JSONL or SQLite manifests. The default is `--checksum none`.
Deep-verify checksum-backed manifests:
```bash
./bin/photoscli verify --out ./PhotosBackup --deep
```
`--deep` recomputes SHA-256 only for manifest entries that already have checksum metadata.
## Safe Operating Practices
- Run `--dry-run` before the first large backup.
- Use `--manifest sqlite` for long-term backups.
- Use `--log` for unattended runs.
- Use `--retry 3` for iCloud-heavy libraries.
- Run `verify` after large runs.
- Keep the destination on a reliable disk.
- Do not use `--no-manifest` for backups you expect to resume.
- Prefer album local identifiers when names are duplicated.
## Limitations
- This is macOS-only.
- Real Photos access requires macOS privacy permission.
- `--format heic|png` is currently a validated CLI hint; true non-JPEG preview output still needs bridge support.
- `--min-size` and `--max-size` use estimated pixel count from dimensions, not encoded file size.
- Original export depends on asset resources exposed by PhotoKit.
- iCloud-backed assets may require network downloads and can fail for reasons outside the CLI's control.
- The manifest records exported asset IDs; it is not a cryptographic integrity database.
+23 -2
View File
@@ -1,35 +1,56 @@
#ifndef PHOTOKIT_BRIDGE_H
#define PHOTOKIT_BRIDGE_H
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
typedef struct {
int active;
double progress;
int64_t bytes_done;
int64_t bytes_total;
} export_progress_t;
int photos_request_access(void);
char *photos_list_albums_json(void);
char *photos_list_assets_json(const char *album_id);
char *photos_reverse_geocode_json(double latitude, double longitude);
char *photos_export_preview_json(
const char *asset_id,
const char *output_dir,
int target_size,
int index
int quality,
int index,
int slot_index
);
char *photos_export_original_json(
const char *asset_id,
const char *output_dir,
int index
int index,
int slot_index
);
char *photos_list_tree_json(void);
void photos_request_cancel(void);
int photos_request_is_cancelled(void);
void photos_free_string(char *value);
export_progress_t *photos_get_progress_slots(void);
export_progress_t photos_get_progress_slot(export_progress_t *slots, int index);
int photos_get_progress_slot_count(void);
void photos_reset_progress_slots(void);
#ifdef __cplusplus
}
#endif
+498 -50
View File
@@ -1,17 +1,40 @@
#import <Foundation/Foundation.h>
#import <AppKit/AppKit.h>
#import <Photos/Photos.h>
#import <CoreLocation/CoreLocation.h>
#import <MapKit/MapKit.h>
#import <UniformTypeIdentifiers/UniformTypeIdentifiers.h>
#import <objc/message.h>
#import "photokit_bridge.h"
static volatile int photos_cancelled = 0;
#define PROGRESS_SLOT_COUNT 16
static export_progress_t progress_slots[PROGRESS_SLOT_COUNT];
static void reset_slot(int slot_index) {
if (slot_index >= 0 && slot_index < PROGRESS_SLOT_COUNT) {
progress_slots[slot_index].active = 0;
progress_slots[slot_index].progress = 0;
progress_slots[slot_index].bytes_done = 0;
progress_slots[slot_index].bytes_total = 0;
}
}
static NSDictionary *make_error_dict(NSString *message) {
return @{@"error": message};
}
static BOOL semaphore_wait_with_timeout(dispatch_semaphore_t sem, int64_t seconds) {
dispatch_time_t timeout = dispatch_time(DISPATCH_TIME_NOW, seconds * NSEC_PER_SEC);
return dispatch_semaphore_wait(sem, timeout) == 0;
int64_t deadline = (int64_t)[NSDate timeIntervalSinceReferenceDate] + seconds;
while (1) {
if (photos_cancelled) return NO;
int64_t remaining = deadline - (int64_t)[NSDate timeIntervalSinceReferenceDate];
if (remaining <= 0) return NO;
int64_t waitSecs = remaining < 1 ? remaining : 1;
dispatch_time_t timeout = dispatch_time(DISPATCH_TIME_NOW, waitSecs * NSEC_PER_SEC);
if (dispatch_semaphore_wait(sem, timeout) == 0) return YES;
}
}
static NSDictionary *collection_to_dict(PHCollection *collection) {
@@ -21,7 +44,7 @@ static NSDictionary *collection_to_dict(PHCollection *collection) {
if ([collection isKindOfClass:[PHCollectionList class]]) {
PHCollectionList *list = (PHCollectionList *)collection;
PHFetchResult<PHCollection *> *children = [PHCollectionList fetchCollectionsInCollectionList:list
options:nil];
options:nil];
NSMutableArray *childList = [NSMutableArray arrayWithCapacity:children.count];
for (NSUInteger i = 0; i < children.count; i++) {
NSDictionary *child = collection_to_dict(children[i]);
@@ -86,7 +109,7 @@ static BOOL ensure_directory(NSString *outputDir) {
NSFileManager *fm = [NSFileManager defaultManager];
NSError *dirErr = nil;
BOOL ok = [fm createDirectoryAtPath:outputDir withIntermediateDirectories:YES
attributes:nil error:&dirErr];
attributes:nil error:&dirErr];
if (!ok && dirErr) {
NSLog(@"ensure_directory failed: %@", dirErr);
}
@@ -130,7 +153,6 @@ static NSString *unique_path_for_filename(NSString *outputDir, NSString *filenam
return [outputDir stringByAppendingPathComponent:prefixed];
}
// Must stay in sync with sanitizePathComponent in cmd/photoscli/main.go
static NSString *sanitized_path_component(NSString *name) {
NSString *source = name ?: @"Untitled";
NSMutableString *safe = [[source stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]] mutableCopy];
@@ -149,6 +171,85 @@ static NSString *sanitized_path_component(NSString *name) {
return safe;
}
static BOOL resource_is_locally_available(PHAssetResource *res) {
@try {
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Warc-performSelector-leaks"
SEL sel = @selector(isLocallyAvailable);
if ([res respondsToSelector:sel]) {
return ((BOOL (*)(id, SEL))objc_msgSend)(res, sel);
}
#pragma clang diagnostic pop
} @catch (NSException *e) {}
return NO;
}
static NSArray<PHAssetResource *> *safe_asset_resources(PHAsset *asset) {
@try {
@autoreleasepool {
NSArray<PHAssetResource *> *resources = [PHAssetResource assetResourcesForAsset:asset];
return resources ?: @[];
}
} @catch (NSException *e) {
NSLog(@"photoscli: exception getting resources for asset %@: %@", asset.localIdentifier, e.reason);
return @[];
}
}
static NSString *asset_cloud_status_string_safe(PHAsset *asset) {
@try {
@autoreleasepool {
if (@available(macOS 11, *)) {
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Warc-performSelector-leaks"
SEL sel = @selector(isCloudShared);
if ([asset respondsToSelector:sel]) {
BOOL isShared = ((BOOL (*)(id, SEL))objc_msgSend)(asset, sel);
if (isShared) return @"cloud";
}
#pragma clang diagnostic pop
}
}
} @catch (NSException *exception) {
NSLog(@"photoscli: exception checking isCloudShared for %@: %@", asset.localIdentifier, exception.reason);
}
PHCloudIdentifier *cloudId = nil;
@try {
@autoreleasepool {
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Warc-performSelector-leaks"
cloudId = [asset performSelector:@selector(cloudIdentifier)];
#pragma clang diagnostic pop
}
} @catch (NSException *exception) {
cloudId = nil;
}
if (cloudId && ![cloudId isEqual:[NSNull null]]) {
return @"cloud";
}
BOOL isInCloud = NO;
@try {
@autoreleasepool {
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Warc-performSelector-leaks"
SEL sel = @selector(isInCloud);
if ([asset respondsToSelector:sel]) {
isInCloud = ((BOOL (*)(id, SEL))objc_msgSend)(asset, sel);
}
#pragma clang diagnostic pop
}
} @catch (NSException *exception) {
isInCloud = NO;
}
if (isInCloud) {
return @"cloud";
}
return @"local";
}
int photos_request_access(void) {
__block PHAuthorizationStatus status = [PHPhotoLibrary authorizationStatus];
if (status == PHAuthorizationStatusNotDetermined) {
@@ -185,28 +286,128 @@ char *photos_list_albums_json(void) {
return json_from_object(@{@"albums": list});
}
static NSString *asset_cloud_status_string(PHAsset *asset) {
@try {
id cloudId = [asset performSelector:@selector(cloudIdentifier)];
if (cloudId && ![cloudId isEqual:[NSNull null]]) {
return @"cloud";
}
} @catch (NSException *exception) {
static NSString *resource_type_string(NSInteger type) {
switch (type) {
case 1: return @"photo";
case 2: return @"video";
case 3: return @"audio";
case 4: return @"alternatePhoto";
case 5: return @"fullSizePhoto";
case 6: return @"fullSizeVideo";
case 7: return @"adjustmentData";
case 8: return @"adjustmentBasePhoto";
case 9: return @"pairedVideo";
case 10: return @"fullSizePairedVideo";
case 11: return @"adjustmentBasePairedVideo";
case 12: return @"adjustmentBaseVideo";
default: return [NSString stringWithFormat:@"other(%ld)", (long)type];
}
PHAssetResource *resource = [PHAssetResource assetResourcesForAsset:asset].firstObject;
if (resource) {
@try {
id locallyAvailable = [resource performSelector:@selector(isLocallyAvailable)];
if (locallyAvailable && [locallyAvailable boolValue]) {
return @"local";
}
return @"cloud";
} @catch (NSException *exception) {
}
}
return @"local";
}
static NSString *media_type_string(PHAssetMediaType type) {
switch (type) {
case PHAssetMediaTypeImage: return @"image";
case PHAssetMediaTypeVideo: return @"video";
case PHAssetMediaTypeAudio: return @"audio";
default: return @"unknown";
}
}
static NSString *source_type_string(PHAssetSourceType type) {
NSMutableArray *parts = [NSMutableArray array];
if (type & PHAssetSourceTypeUserLibrary) [parts addObject:@"userLibrary"];
if (type & PHAssetSourceTypeCloudShared) [parts addObject:@"cloudShared"];
if (type & PHAssetSourceTypeiTunesSynced) [parts addObject:@"iTunesSynced"];
return parts.count > 0 ? [parts componentsJoinedByString:@","] : @"unknown";
}
static NSString *playback_style_string(PHAssetPlaybackStyle style) {
switch (style) {
case PHAssetPlaybackStyleImage: return @"image";
case PHAssetPlaybackStyleImageAnimated: return @"imageAnimated";
case PHAssetPlaybackStyleLivePhoto: return @"livePhoto";
case PHAssetPlaybackStyleVideo: return @"video";
case PHAssetPlaybackStyleVideoLooping: return @"videoLooping";
default: return @"unknown";
}
}
static NSArray<NSString *> *media_subtype_strings(PHAssetMediaSubtype subtypes) {
NSMutableArray *parts = [NSMutableArray array];
if (subtypes & PHAssetMediaSubtypePhotoPanorama) [parts addObject:@"photoPanorama"];
if (subtypes & PHAssetMediaSubtypePhotoHDR) [parts addObject:@"photoHDR"];
if (subtypes & PHAssetMediaSubtypePhotoScreenshot) [parts addObject:@"photoScreenshot"];
if (subtypes & PHAssetMediaSubtypePhotoLive) [parts addObject:@"photoLive"];
if (subtypes & PHAssetMediaSubtypePhotoDepthEffect) [parts addObject:@"photoDepthEffect"];
if (subtypes & PHAssetMediaSubtypeVideoStreamed) [parts addObject:@"videoStreamed"];
if (subtypes & PHAssetMediaSubtypeVideoHighFrameRate) [parts addObject:@"videoHighFrameRate"];
if (subtypes & PHAssetMediaSubtypeVideoTimelapse) [parts addObject:@"videoTimelapse"];
return parts;
}
static NSArray<NSString *> *burst_selection_type_strings(PHAssetBurstSelectionType types) {
NSMutableArray *parts = [NSMutableArray array];
if (types & PHAssetBurstSelectionTypeAutoPick) [parts addObject:@"autoPick"];
if (types & PHAssetBurstSelectionTypeUserPick) [parts addObject:@"userPick"];
return parts;
}
static NSNumber *resource_file_size(PHAssetResource *res) {
@try {
id value = [res valueForKey:@"fileSize"];
if ([value respondsToSelector:@selector(longLongValue)]) {
return @([value longLongValue]);
}
} @catch (NSException *e) {}
return nil;
}
static NSDictionary *location_dict(CLLocation *location) {
if (!location) return nil;
NSMutableDictionary *dict = [NSMutableDictionary dictionary];
dict[@"latitude"] = @(location.coordinate.latitude);
dict[@"longitude"] = @(location.coordinate.longitude);
dict[@"altitude"] = @(location.altitude);
dict[@"horizontalAccuracy"] = @(location.horizontalAccuracy);
return dict;
}
static NSDictionary *editing_input_info(PHAsset *asset) {
PHContentEditingInputRequestOptions *opts = [[PHContentEditingInputRequestOptions alloc] init];
opts.networkAccessAllowed = NO;
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block NSMutableDictionary *dict = [NSMutableDictionary dictionary];
[asset requestContentEditingInputWithOptions:opts completionHandler:^(PHContentEditingInput *input, NSDictionary *info) {
if (input.adjustmentData) {
dict[@"formatIdentifier"] = input.adjustmentData.formatIdentifier ?: @"";
dict[@"formatVersion"] = input.adjustmentData.formatVersion ?: @"";
}
NSURL *url = input.fullSizeImageURL;
if (url.lastPathComponent.length > 0) {
dict[@"baseFilename"] = url.lastPathComponent;
}
dispatch_semaphore_signal(sem);
}];
if (!semaphore_wait_with_timeout(sem, 5)) {
return nil;
}
return dict.count > 0 ? dict : nil;
}
static NSString *iso8601_string(NSDate *date) {
if (!date) return nil;
static NSDateFormatter *fmt = nil;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
fmt = [[NSDateFormatter alloc] init];
fmt.locale = [NSLocale localeWithLocaleIdentifier:@"en_US_POSIX"];
fmt.dateFormat = @"yyyy-MM-dd'T'HH:mm:ssZ";
});
return [fmt stringFromDate:date];
}
#import <objc/message.h>
char *photos_list_assets_json(const char *album_id) {
if (!album_id) return json_from_object(make_error_dict(@"album_id is required"));
@@ -229,21 +430,107 @@ char *photos_list_assets_json(const char *album_id) {
PHAsset *asset = assets[i];
NSString *lid = asset.localIdentifier;
NSString *filename = nil;
NSArray<PHAssetResource *> *resources = [PHAssetResource assetResourcesForAsset:asset];
NSArray<PHAssetResource *> *resources = safe_asset_resources(asset);
if (resources.count > 0) {
filename = resources.firstObject.originalFilename;
}
NSString *cloudStatus = asset_cloud_status_string(asset);
[list addObject:@{
NSString *cloudStatus = asset_cloud_status_string_safe(asset);
NSString *mediaTypeStr = media_type_string(asset.mediaType);
NSString *creationDateStr = iso8601_string(asset.creationDate);
NSString *modificationDateStr = iso8601_string(asset.modificationDate);
NSMutableArray *resourcesList = [NSMutableArray arrayWithCapacity:resources.count];
for (PHAssetResource *res in resources) {
NSString *resTypeStr = resource_type_string(res.type);
NSString *uti = res.uniformTypeIdentifier ?: @"";
BOOL isLocal = resource_is_locally_available(res);
NSMutableDictionary *resourceDict = [NSMutableDictionary dictionaryWithDictionary:@{
@"type": resTypeStr,
@"filename": res.originalFilename ?: @"",
@"uti": uti,
@"local": @(isLocal)
}];
NSNumber *size = resource_file_size(res);
if (size) resourceDict[@"size"] = size;
[resourcesList addObject:resourceDict];
}
NSMutableDictionary *dict = [NSMutableDictionary dictionaryWithDictionary:@{
@"id": lid ?: @"",
@"filename": filename ?: @"",
@"cloud": cloudStatus
@"cloud": cloudStatus,
@"mediaType": mediaTypeStr,
@"mediaSubtypes": media_subtype_strings(asset.mediaSubtypes),
@"sourceType": source_type_string(asset.sourceType),
@"playbackStyle": playback_style_string(asset.playbackStyle),
@"pixelWidth": @(asset.pixelWidth),
@"pixelHeight": @(asset.pixelHeight),
@"duration": @(asset.duration),
@"isFavorite": @(asset.isFavorite),
@"isHidden": @(asset.isHidden),
@"representsBurst": @(asset.representsBurst),
@"burstSelectionTypes": burst_selection_type_strings(asset.burstSelectionTypes)
}];
if (creationDateStr) {
dict[@"creationDate"] = creationDateStr;
}
if (modificationDateStr) {
dict[@"modificationDate"] = modificationDateStr;
}
NSDictionary *loc = location_dict(asset.location);
if (loc) {
dict[@"location"] = loc;
}
if (asset.burstIdentifier.length > 0) {
dict[@"burstIdentifier"] = asset.burstIdentifier;
}
if (@available(macOS 12, *)) {
dict[@"hasAdjustments"] = @(asset.hasAdjustments);
}
NSDictionary *adjustmentInfo = editing_input_info(asset);
if (adjustmentInfo) {
dict[@"adjustmentInfo"] = adjustmentInfo;
}
if (resourcesList.count > 0) {
dict[@"resources"] = resourcesList;
}
[list addObject:dict];
}
return json_from_object(@{@"assets": list, @"total": @(assets.count)});
}
char *photos_reverse_geocode_json(double latitude, double longitude) {
if (@available(macOS 26.0, *)) {
CLLocation *location = [[CLLocation alloc] initWithLatitude:latitude longitude:longitude];
MKReverseGeocodingRequest *request = [[MKReverseGeocodingRequest alloc] initWithLocation:location];
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block MKMapItem *item = nil;
__block NSError *mapErr = nil;
[request getMapItemsWithCompletionHandler:^(NSArray<MKMapItem *> *mapItems, NSError *error) {
mapErr = error;
item = mapItems.firstObject;
dispatch_semaphore_signal(sem);
}];
if (!semaphore_wait_with_timeout(sem, 10)) {
[request cancel];
return json_from_object(@{@"error": @"timeout waiting for reverse geocode"});
}
if (mapErr || !item) {
NSString *msg = mapErr.localizedDescription ?: @"reverse geocode failed";
return json_from_object(@{@"error": msg});
}
NSMutableDictionary *dict = [NSMutableDictionary dictionary];
if (item.name) dict[@"name"] = item.name;
if (item.address.fullAddress) dict[@"formattedAddress"] = item.address.fullAddress;
if (item.address.shortAddress) dict[@"thoroughfare"] = item.address.shortAddress;
return json_from_object(@{@"placemark": dict});
}
return json_from_object(@{@"error": @"reverse geocoding requires macOS 26 or newer"});
}
char *photos_list_tree_json(void) {
PHFetchResult<PHCollection *> *collections = [PHCollectionList fetchTopLevelUserCollectionsWithOptions:nil];
@@ -259,19 +546,33 @@ char *photos_list_tree_json(void) {
}
char *photos_export_preview_json(const char *asset_id, const char *output_dir, int target_size, int index) {
if (!asset_id || !output_dir) return json_from_object(make_error_dict(@"asset_id and output_dir required"));
#define RETURN_PREVIEW(x) do { reset_slot(slot_index); return (x); } while(0)
char *photos_export_preview_json(const char *asset_id, const char *output_dir, int target_size, int quality, int index, int slot_index) {
if (!asset_id || !output_dir) RETURN_PREVIEW(json_from_object(make_error_dict(@"asset_id and output_dir required")));
if (target_size <= 0) target_size = 1024;
if (quality <= 0 || quality > 100) quality = 85;
NSString *nsAssetId = [NSString stringWithUTF8String:asset_id];
NSString *nsOutputDir = [NSString stringWithUTF8String:output_dir];
if (!nsAssetId || !nsOutputDir) return json_from_object(make_error_dict(@"invalid UTF-8 in arguments"));
if (!nsAssetId || !nsOutputDir) RETURN_PREVIEW(json_from_object(make_error_dict(@"invalid UTF-8 in arguments")));
if (!ensure_directory(nsOutputDir)) {
RETURN_PREVIEW(json_from_object(make_error_dict(@"failed to create output directory")));
}
PHFetchResult<PHAsset *> *fetch = [PHAsset fetchAssetsWithLocalIdentifiers:@[nsAssetId] options:nil];
if (fetch.count == 0) return json_from_object(make_error_dict(@"asset not found"));
if (fetch.count == 0) RETURN_PREVIEW(json_from_object(make_error_dict(@"asset not found")));
PHAsset *asset = fetch.firstObject;
if (slot_index >= 0 && slot_index < PROGRESS_SLOT_COUNT) {
progress_slots[slot_index].active = 1;
progress_slots[slot_index].progress = 0;
progress_slots[slot_index].bytes_done = 0;
progress_slots[slot_index].bytes_total = 0;
}
PHImageManager *im = [PHImageManager defaultManager];
CGFloat scale = (CGFloat)target_size;
CGSize targetCGSize = CGSizeMake(scale, scale);
@@ -280,6 +581,11 @@ char *photos_export_preview_json(const char *asset_id, const char *output_dir, i
imgOpts.resizeMode = PHImageRequestOptionsResizeModeExact;
imgOpts.networkAccessAllowed = YES;
imgOpts.synchronous = YES;
imgOpts.progressHandler = ^(double progress, NSError * _Nullable error, BOOL * _Nonnull stop, NSDictionary * _Nullable info) {
if (slot_index >= 0 && slot_index < PROGRESS_SLOT_COUNT) {
progress_slots[slot_index].progress = progress;
}
};
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block NSData *imageData = nil;
@@ -296,26 +602,42 @@ char *photos_export_preview_json(const char *asset_id, const char *output_dir, i
return;
}
if (result) {
imageData = nsimage_to_jpeg(result, 0.85);
CGFloat compression = (CGFloat)quality / 100.0;
imageData = nsimage_to_jpeg(result, compression);
}
dispatch_semaphore_signal(sem);
}];
if (!semaphore_wait_with_timeout(sem, 120)) {
return json_from_object(@{@"error": @"timeout waiting for image", @"cloud": asset_cloud_status_string(asset)});
RETURN_PREVIEW(json_from_object(@{@"error": @"timeout waiting for image", @"cloud": asset_cloud_status_string_safe(asset)}));
}
if (!imageData) {
return json_from_object(@{@"error": @"failed to render image", @"cloud": asset_cloud_status_string(asset)});
RETURN_PREVIEW(json_from_object(@{@"error": @"failed to render image", @"cloud": asset_cloud_status_string_safe(asset)}));
}
NSString *safe = sanitized_asset_identifier(asset.localIdentifier);
NSString *filename = [NSString stringWithFormat:@"%04lu_%@.jpg", (unsigned long)index, safe];
NSString *filepath = [nsOutputDir stringByAppendingPathComponent:filename];
NSFileManager *fm = [NSFileManager defaultManager];
NSDictionary *existingAttrs = [fm attributesOfItemAtPath:filepath error:nil];
if (existingAttrs) {
NSNumber *existingSize = existingAttrs[NSFileSize];
if (existingSize && existingSize.unsignedLongValue > 0) {
RETURN_PREVIEW(json_from_object(@{
@"filename": filename,
@"size": existingSize,
@"cloud": asset_cloud_status_string_safe(asset),
@"skipped": @YES
}));
}
}
NSError *writeErr = nil;
if (![imageData writeToFile:filepath options:NSDataWritingAtomic error:&writeErr]) {
return json_from_object(@{@"error": @"write failed", @"cloud": asset_cloud_status_string(asset)});
NSString *msg = writeErr ? writeErr.localizedDescription : @"unknown write error";
RETURN_PREVIEW(json_from_object(@{@"error": [NSString stringWithFormat:@"write failed: %@", msg], @"cloud": asset_cloud_status_string_safe(asset)}));
}
NSNumber *fileSize = nil;
@@ -324,28 +646,42 @@ char *photos_export_preview_json(const char *asset_id, const char *output_dir, i
fileSize = attrs[NSFileSize];
}
return json_from_object(@{
RETURN_PREVIEW(json_from_object(@{
@"filename": filename,
@"size": fileSize ?: @0,
@"cloud": asset_cloud_status_string(asset)
});
@"cloud": asset_cloud_status_string_safe(asset)
}));
}
#undef RETURN_PREVIEW
char *photos_export_original_json(const char *asset_id, const char *output_dir, int index) {
if (!asset_id || !output_dir) return json_from_object(make_error_dict(@"asset_id and output_dir required"));
#define RETURN_ORIGINAL(x) do { reset_slot(slot_index); return (x); } while(0)
char *photos_export_original_json(const char *asset_id, const char *output_dir, int index, int slot_index) {
if (!asset_id || !output_dir) RETURN_ORIGINAL(json_from_object(make_error_dict(@"asset_id and output_dir required")));
NSString *nsAssetId = [NSString stringWithUTF8String:asset_id];
NSString *nsOutputDir = [NSString stringWithUTF8String:output_dir];
if (!nsAssetId || !nsOutputDir) return json_from_object(make_error_dict(@"invalid UTF-8 in arguments"));
if (!nsAssetId || !nsOutputDir) RETURN_ORIGINAL(json_from_object(make_error_dict(@"invalid UTF-8 in arguments")));
if (!ensure_directory(nsOutputDir)) {
RETURN_ORIGINAL(json_from_object(make_error_dict(@"failed to create output directory")));
}
PHFetchResult<PHAsset *> *fetch = [PHAsset fetchAssetsWithLocalIdentifiers:@[nsAssetId] options:nil];
if (fetch.count == 0) return json_from_object(make_error_dict(@"asset not found"));
if (fetch.count == 0) RETURN_ORIGINAL(json_from_object(make_error_dict(@"asset not found")));
PHAsset *asset = fetch.firstObject;
NSArray<PHAssetResource *> *resources = [PHAssetResource assetResourcesForAsset:asset];
if (slot_index >= 0 && slot_index < PROGRESS_SLOT_COUNT) {
progress_slots[slot_index].active = 1;
progress_slots[slot_index].progress = 0;
progress_slots[slot_index].bytes_done = 0;
progress_slots[slot_index].bytes_total = 0;
}
NSArray<PHAssetResource *> *resources = safe_asset_resources(asset);
if (resources.count == 0) {
return json_from_object(@{@"error": @"no resources", @"cloud": asset_cloud_status_string(asset)});
RETURN_ORIGINAL(json_from_object(@{@"error": @"no resources", @"cloud": asset_cloud_status_string_safe(asset)}));
}
PHAssetResource *resource = resources.firstObject;
@@ -355,11 +691,38 @@ char *photos_export_original_json(const char *asset_id, const char *output_dir,
}
NSString *filepath = unique_path_for_filename(nsOutputDir, filename, (NSUInteger)index);
NSFileManager *fm = [NSFileManager defaultManager];
{
NSString *checkPath = [nsOutputDir stringByAppendingPathComponent:filename];
NSDictionary *existingAttrs = [fm attributesOfItemAtPath:checkPath error:nil];
if (existingAttrs) {
NSNumber *existingSize = existingAttrs[NSFileSize];
if (existingSize && existingSize.unsignedLongValue > 0) {
RETURN_ORIGINAL(json_from_object(@{
@"filename": filename,
@"size": existingSize,
@"cloud": asset_cloud_status_string_safe(asset),
@"skipped": @YES
}));
}
}
}
NSURL *fileURL = [NSURL fileURLWithPath:filepath];
if ([fm fileExistsAtPath:filepath]) {
[fm removeItemAtPath:filepath error:nil];
}
PHAssetResourceManager *rm = [PHAssetResourceManager defaultManager];
PHAssetResourceRequestOptions *opts = [[PHAssetResourceRequestOptions alloc] init];
opts.networkAccessAllowed = YES;
opts.progressHandler = ^(double progress) {
if (slot_index >= 0 && slot_index < PROGRESS_SLOT_COUNT) {
progress_slots[slot_index].progress = progress;
}
};
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block NSError *writeErr = nil;
@@ -371,11 +734,79 @@ char *photos_export_original_json(const char *asset_id, const char *output_dir,
dispatch_semaphore_signal(sem);
}];
if (!semaphore_wait_with_timeout(sem, 120)) {
return json_from_object(@{@"error": @"timeout waiting for resource", @"cloud": asset_cloud_status_string(asset)});
RETURN_ORIGINAL(json_from_object(@{@"error": @"timeout waiting for resource", @"cloud": asset_cloud_status_string_safe(asset)}));
}
if (writeErr) {
return json_from_object(@{@"error": @"write failed", @"cloud": asset_cloud_status_string(asset)});
PHImageRequestOptions *imgOpts = [[PHImageRequestOptions alloc] init];
imgOpts.networkAccessAllowed = YES;
imgOpts.synchronous = NO;
dispatch_semaphore_t sem2 = dispatch_semaphore_create(0);
__block NSData *fallbackData = nil;
__block NSError *fallbackErr = nil;
__block NSString *fallbackUTI = nil;
[[PHImageManager defaultManager] requestImageDataAndOrientationForAsset:asset
options:imgOpts
resultHandler:^(NSData *imageData, NSString *dataUTI, CGImagePropertyOrientation orientation, NSDictionary *info) {
if (info[PHImageErrorKey]) {
fallbackErr = info[PHImageErrorKey];
} else if (imageData) {
fallbackData = imageData;
fallbackUTI = dataUTI;
} else {
fallbackErr = [NSError errorWithDomain:@"photoscli" code:-1 userInfo:@{NSLocalizedDescriptionKey: @"no data"}];
}
dispatch_semaphore_signal(sem2);
}];
if (!semaphore_wait_with_timeout(sem2, 120)) {
RETURN_ORIGINAL(json_from_object(@{@"error": @"timeout waiting for image data", @"cloud": asset_cloud_status_string_safe(asset)}));
}
if (fallbackErr || !fallbackData) {
NSString *detail = writeErr.localizedDescription;
if (fallbackErr) {
detail = [NSString stringWithFormat:@"%@; fallback: %@", detail, fallbackErr.localizedDescription];
}
RETURN_ORIGINAL(json_from_object(@{@"error": [NSString stringWithFormat:@"write failed: %@", detail], @"cloud": asset_cloud_status_string_safe(asset)}));
}
NSString *ext = nil;
if (fallbackUTI) {
UTType *uti = [UTType typeWithIdentifier:fallbackUTI];
if (uti && uti.preferredFilenameExtension) {
ext = uti.preferredFilenameExtension;
}
}
if (ext.length == 0) {
ext = @"dng";
}
NSString *baseFilename = [filename stringByDeletingPathExtension];
if (baseFilename.length == 0) {
baseFilename = sanitized_asset_identifier(asset.localIdentifier);
}
NSString *fallbackFilename = [NSString stringWithFormat:@"%@.%@", baseFilename, ext];
NSString *fallbackPath = [nsOutputDir stringByAppendingPathComponent:fallbackFilename];
NSError *writeFallbackErr = nil;
if (![fallbackData writeToFile:fallbackPath options:NSDataWritingAtomic error:&writeFallbackErr]) {
RETURN_ORIGINAL(json_from_object(@{@"error": [NSString stringWithFormat:@"write failed: %@", writeFallbackErr.localizedDescription], @"cloud": asset_cloud_status_string_safe(asset)}));
}
NSNumber *fileSize = nil;
NSDictionary *attrs = [[NSFileManager defaultManager] attributesOfItemAtPath:fallbackPath error:nil];
if (attrs) {
fileSize = attrs[NSFileSize];
}
RETURN_ORIGINAL(json_from_object(@{
@"filename": fallbackFilename,
@"size": fileSize ?: @0,
@"cloud": asset_cloud_status_string_safe(asset)
}));
}
NSString *writtenFilename = [filepath lastPathComponent];
@@ -385,12 +816,13 @@ char *photos_export_original_json(const char *asset_id, const char *output_dir,
fileSize = attrs[NSFileSize];
}
return json_from_object(@{
RETURN_ORIGINAL(json_from_object(@{
@"filename": writtenFilename,
@"size": fileSize ?: @0,
@"cloud": asset_cloud_status_string(asset)
});
@"cloud": asset_cloud_status_string_safe(asset)
}));
}
#undef RETURN_ORIGINAL
void photos_free_string(char *value) {
if (value) free(value);
@@ -399,3 +831,19 @@ void photos_free_string(char *value) {
void photos_request_cancel(void) {
photos_cancelled = 1;
}
int photos_request_is_cancelled(void) {
return photos_cancelled;
}
export_progress_t *photos_get_progress_slots(void) {
return progress_slots;
}
int photos_get_progress_slot_count(void) {
return PROGRESS_SLOT_COUNT;
}
void photos_reset_progress_slots(void) {
memset(progress_slots, 0, sizeof(progress_slots));
}
+63 -3
View File
@@ -1,5 +1,10 @@
#include <stdlib.h>
#include <string.h>
#include "../bridge/photokit_bridge.h"
static export_progress_t stub_progress_slots[16];
static int stub_progress_slot_count = 16;
static int stub_progress_slots_null = 0;
static char *alloc_json(const char *s) {
size_t len = strlen(s);
@@ -13,9 +18,11 @@ static int stub_access_rc = 0;
static const char *stub_albums_json = "{\"albums\":[]}";
static const char *stub_assets_json = "{\"assets\":[]}";
static const char *stub_tree_json = "{\"collections\":[]}";
static const char *stub_geocode_json = "{\"placemark\":{}}";
static int stub_albums_null = 0;
static int stub_assets_null = 0;
static int stub_tree_null = 0;
static int stub_geocode_null = 0;
static int stub_cancelled = 0;
static const char *stub_export_preview_json = NULL;
static const char *stub_export_original_json = NULL;
@@ -29,6 +36,8 @@ void photos_test_set_access(int rc) { stub_access_rc = rc; }
void photos_test_set_albums(const char *json) { stub_albums_json = json; stub_albums_null = 0; }
void photos_test_set_assets(const char *json) { stub_assets_json = json; stub_assets_null = 0; }
void photos_test_set_tree(const char *json) { stub_tree_json = json; stub_tree_null = 0; }
void photos_test_set_geocode(const char *json) { stub_geocode_json = json; stub_geocode_null = 0; }
void photos_test_set_geocode_null(void) { stub_geocode_null = 1; }
void photos_test_set_albums_null(void) { stub_albums_null = 1; }
void photos_test_set_assets_null(void) { stub_assets_null = 1; }
void photos_test_set_tree_null(void) { stub_tree_null = 1; }
@@ -46,18 +55,28 @@ char *photos_list_assets_json(const char *album_id) {
return alloc_json(stub_assets_json);
}
char *photos_export_preview_json(const char *asset_id, const char *output_dir, int target_size, int index) {
char *photos_reverse_geocode_json(double latitude, double longitude) {
(void)latitude;
(void)longitude;
if (stub_geocode_null) return NULL;
return alloc_json(stub_geocode_json);
}
char *photos_export_preview_json(const char *asset_id, const char *output_dir, int target_size, int quality, int index, int slot_index) {
(void)asset_id;
(void)output_dir;
(void)target_size;
(void)quality;
(void)index;
(void)slot_index;
return maybe_alloc_json(stub_export_preview_json);
}
char *photos_export_original_json(const char *asset_id, const char *output_dir, int index) {
char *photos_export_original_json(const char *asset_id, const char *output_dir, int index, int slot_index) {
(void)asset_id;
(void)output_dir;
(void)index;
(void)slot_index;
return maybe_alloc_json(stub_export_original_json);
}
@@ -74,10 +93,51 @@ void photos_request_cancel(void) {
stub_cancelled = 1;
}
int photos_request_is_cancelled(void) {
return stub_cancelled;
}
void photos_test_set_export_preview_json(const char *json) {
stub_export_preview_json = json;
}
void photos_test_set_export_preview_json_null(void) {
stub_export_preview_json = NULL;
}
void photos_test_set_export_original_json(const char *json) {
stub_export_original_json = json;
}
}
void photos_test_set_export_original_json_null(void) {
stub_export_original_json = NULL;
}
export_progress_t *photos_get_progress_slots(void) {
if (stub_progress_slots_null) return NULL;
return stub_progress_slots;
}
export_progress_t photos_get_progress_slot(export_progress_t *slots, int index) {
export_progress_t result = {0, 0.0, 0, 0};
if (index >= 0 && index < 16) {
result = slots[index];
}
return result;
}
int photos_get_progress_slot_count(void) {
return stub_progress_slot_count;
}
void photos_reset_progress_slots(void) {
memset(stub_progress_slots, 0, sizeof(stub_progress_slots));
}
void photos_test_set_progress_slot_count(int count) {
stub_progress_slot_count = count;
}
void photos_test_set_progress_slots_null(int val) {
stub_progress_slots_null = val;
}
+2233 -143
View File
File diff suppressed because it is too large Load Diff
+3 -23
View File
@@ -1,10 +1,9 @@
//go:build !test
package main
import (
"os"
"os/signal"
"sync/atomic"
"syscall"
"gitea.k3s.k0.nu/tools/photocli/internal/photos"
)
@@ -12,24 +11,5 @@ import (
var version = "dev"
func main() {
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
done := make(chan struct{})
var rc atomic.Int32
go func() {
rc.Store(int32(run(os.Args[1:], os.Stdout, os.Stderr, photos.DefaultBridge)))
close(done)
}()
select {
case <-done:
case <-sigCh:
photos.DefaultBridge.Cancel()
os.Stderr.Write([]byte("\nreceived signal, finishing current file...\n"))
<-done
}
os.Exit(int(rc.Load()))
os.Exit(runMain(os.Args[1:], os.Stdout, os.Stderr, photos.DefaultBridge))
}
+5
View File
@@ -0,0 +1,5 @@
//go:build test
package main
var version = "dev"
+4595 -66
View File
File diff suppressed because it is too large Load Diff
+375
View File
@@ -0,0 +1,375 @@
package main
import (
"fmt"
"io"
"strings"
"sync"
"time"
)
type progressBar struct {
mu sync.Mutex
w io.Writer
width int
termH int
start time.Time
workers int
footerLines int
scrollSet bool
total barLine
album barLine
workerState []workerSlot
}
type barLine struct {
current int
total int
label string
detail string
}
type workerSlot struct {
filename string
size int64
cloud string
progress float64
bytesDone int64
bytesTotal int64
speed float64
status string
}
func newProgressBar(w io.Writer, workers int) *progressBar {
fl := workers + 2
return &progressBar{
w: w,
width: 80,
termH: 24,
start: time.Now(),
workers: workers,
footerLines: fl,
workerState: make([]workerSlot, workers),
}
}
func (p *progressBar) setTotal(current, total int, detail string) {
p.mu.Lock()
defer p.mu.Unlock()
p.total = barLine{current: current, total: total, label: "Total", detail: detail}
}
func (p *progressBar) setAlbum(name string, current, total int) {
p.mu.Lock()
defer p.mu.Unlock()
p.album = barLine{current: current, total: total, label: "Album", detail: name}
}
func (p *progressBar) setWorker(i int, filename string, size int64, cloud string, status string) {
p.mu.Lock()
defer p.mu.Unlock()
if i >= 0 && i < len(p.workerState) {
p.workerState[i].filename = filename
p.workerState[i].size = size
p.workerState[i].cloud = cloud
p.workerState[i].status = status
p.workerState[i].progress = 0
p.workerState[i].bytesDone = 0
p.workerState[i].bytesTotal = 0
}
}
func (p *progressBar) updateWorkerProgress(i int, progress float64, bytesDone, bytesTotal int64) {
p.mu.Lock()
defer p.mu.Unlock()
if i >= 0 && i < len(p.workerState) {
p.workerState[i].progress = progress
p.workerState[i].bytesDone = bytesDone
p.workerState[i].bytesTotal = bytesTotal
if bytesDone > 0 && bytesTotal > 0 {
p.workerState[i].size = bytesTotal
}
}
}
func (p *progressBar) logCompleted(line string) {
p.mu.Lock()
defer p.mu.Unlock()
p.ensureScrollRegion()
scrollTop := p.termH - p.footerLines
fmt.Fprintf(p.w, "\x1b[%d;1H\r\x1b[2K%s\n", scrollTop, line)
p.drawFooterLocked()
}
func (p *progressBar) draw() {
p.mu.Lock()
defer p.mu.Unlock()
p.ensureScrollRegion()
p.drawFooterLocked()
}
func (p *progressBar) ensureScrollRegion() {
w, h := termSize()
if w != p.width || h != p.termH || !p.scrollSet {
p.width = w
p.termH = h
scrollTop := p.termH - p.footerLines
if scrollTop < 1 {
scrollTop = 1
}
fmt.Fprintf(p.w, "\x1b[1;%dr", scrollTop)
p.scrollSet = true
}
}
func (p *progressBar) drawFooterLocked() {
scrollTop := p.termH - p.footerLines
if scrollTop < 1 {
scrollTop = 1
}
elapsed := time.Since(p.start)
fmt.Fprintf(p.w, "\x1b[?25l")
footerStart := scrollTop + 1
for i := 0; i < p.workers; i++ {
row := footerStart + i
fmt.Fprintf(p.w, "\x1b[%d;1H\r\x1b[2K", row)
if i < len(p.workerState) && p.workerState[i].filename != "" {
fmt.Fprintf(p.w, "%s", truncateOrPad(renderWorkerLine(p.workerState[i], p.width), p.width))
} else {
fmt.Fprintf(p.w, "%s", strings.Repeat(" ", p.width))
}
}
row := footerStart + p.workers
fmt.Fprintf(p.w, "\x1b[%d;1H\r\x1b[2K%s", row, renderLine(p.total, elapsed, p.width))
row++
fmt.Fprintf(p.w, "\x1b[%d;1H\r\x1b[2K%s", row, renderLine(p.album, elapsed, p.width))
fmt.Fprintf(p.w, "\x1b[%d;1H", scrollTop)
fmt.Fprintf(p.w, "\x1b[?25h")
}
func (p *progressBar) clear() {
p.mu.Lock()
defer p.mu.Unlock()
fmt.Fprintf(p.w, "\x1b[r")
fmt.Fprintf(p.w, "\x1b[?25h")
p.scrollSet = false
}
func renderWorkerLine(ws workerSlot, width int) string {
if width <= 0 {
width = 80
}
parts := []string{}
if ws.status == "FAIL" {
parts = append(parts, "\u274c")
parts = append(parts, ws.filename)
} else if ws.status == "skipped" {
parts = append(parts, "\u23ed")
parts = append(parts, ws.filename)
if s := formatSize(ws.size); s != "" {
parts = append(parts, s)
}
} else if ws.cloud == "cloud" && ws.progress > 0 && ws.progress < 1.0 {
parts = append(parts, "\u2601")
parts = append(parts, ws.filename)
barWidth := 20
bar := renderBar(int(ws.progress*100), barWidth)
pct := int(ws.progress * 100)
parts = append(parts, fmt.Sprintf("[%s] %d%%", bar, pct))
if ws.bytesTotal > 0 {
parts = append(parts, fmt.Sprintf("%s/%s", formatSize(ws.bytesDone), formatSize(ws.bytesTotal)))
}
if ws.speed > 0 {
parts = append(parts, formatSpeed(ws.speed))
}
} else if ws.cloud == "cloud" {
parts = append(parts, "\u2601")
parts = append(parts, ws.filename)
if s := formatSize(ws.size); s != "" {
parts = append(parts, s)
}
parts = append(parts, "downloaded")
if ws.speed > 0 {
parts = append(parts, formatSpeed(ws.speed))
}
} else {
parts = append(parts, "\u2705")
parts = append(parts, ws.filename)
if s := formatSize(ws.size); s != "" {
parts = append(parts, s)
}
parts = append(parts, "copied")
}
return strings.Join(parts, " ")
}
func renderLine(b barLine, elapsed time.Duration, width int) string {
if width <= 0 {
width = 80
}
pct := 0
if b.total > 0 {
pct = b.current * 100 / b.total
}
counter := ""
if b.total > 0 {
counter = fmt.Sprintf("%d/%d", b.current, b.total)
}
eta := ""
if pct > 0 && pct < 100 && elapsed > 500*time.Millisecond {
remaining := elapsed * time.Duration(100-pct) / time.Duration(pct)
if remaining > time.Second {
eta = formatDuration(remaining)
}
}
right := b.detail
if b.label == "Album" && counter != "" && b.detail != "" {
right = fmt.Sprintf("%s %s", b.detail, counter)
} else if counter != "" && right != "" {
right = fmt.Sprintf("%s %s", right, counter)
} else if counter != "" {
right = counter
}
if eta != "" {
right += " " + eta
}
labelWidth := 6
pctWidth := 4
gap := 2
rightWidth := runeWidth(right)
availableForBar := width - labelWidth - pctWidth - gap - rightWidth - gap
if availableForBar < 3 {
availableForBar = 3
}
if availableForBar > 40 {
availableForBar = 40
}
bar := renderBar(pct, availableForBar)
return fmt.Sprintf("%-6s [%s] %3d%% %s", b.label, bar, pct, right)
}
func renderBar(pct, barWidth int) string {
if barWidth <= 0 {
return ""
}
fraction := float64(pct) / 100.0
filled := fraction * float64(barWidth)
fullBlocks := int(filled)
partial := filled - float64(fullBlocks)
var r, g uint8
if pct <= 50 {
r = 255
g = uint8(float64(pct) * 5.1)
} else {
r = uint8(float64(100-pct) * 5.1)
g = 255
}
var sb strings.Builder
fmt.Fprintf(&sb, "\x1b[38;2;%d;%d;0m", r, g)
for i := 0; i < fullBlocks && i < barWidth; i++ {
sb.WriteString("\u2588")
}
if fullBlocks < barWidth {
fracs := []string{"", "\u258f", "\u258e", "\u258d", "\u258c", "\u258b", "\u258a", "\u2589"}
idx := int(partial * 8)
if idx > 0 {
sb.WriteString(fracs[idx])
fullBlocks++
}
}
sb.WriteString("\x1b[0m")
for i := fullBlocks; i < barWidth; i++ {
sb.WriteString("\u2591")
}
return sb.String()
}
func runeWidth(s string) int {
w := 0
for _, r := range s {
if r >= 0x1100 && (r <= 0x115f || r == 0x2329 || r == 0x232a ||
(r >= 0x2e80 && r <= 0xa4cf && r != 0x303f) ||
(r >= 0xac00 && r <= 0xd7a3) ||
(r >= 0xf900 && r <= 0xfaff) ||
(r >= 0xfe30 && r <= 0xfe6f) ||
(r >= 0xff01 && r <= 0xff60) ||
(r >= 0xffe0 && r <= 0xffe6) ||
(r >= 0x20000 && r <= 0x2fffd) ||
(r >= 0x30000 && r <= 0x3fffd)) {
w += 2
} else {
w += 1
}
}
return w
}
func truncateOrPad(s string, width int) string {
if width <= 0 {
width = 80
}
rw := runeWidth(s)
if rw > width {
runes := []rune(s)
for i := range runes {
if runeWidth(string(runes[:i+1])) > width-3 {
return string(runes[:i]) + "..."
}
}
}
return s + strings.Repeat(" ", width-rw)
}
func formatDuration(d time.Duration) string {
d = d.Round(time.Second)
if d < time.Minute {
return fmt.Sprintf("%ds", int(d.Seconds()))
}
m := int(d.Minutes())
s := int(d.Seconds()) % 60
return fmt.Sprintf("%dm%02ds", m, s)
}
func formatSpeed(bytesPerSec float64) string {
if bytesPerSec <= 0 {
return ""
}
const kb = 1024
const mb = kb * 1024
if bytesPerSec >= mb {
return fmt.Sprintf("%.1f MB/s", bytesPerSec/mb)
}
if bytesPerSec >= kb {
return fmt.Sprintf("%.1f KB/s", bytesPerSec/kb)
}
return fmt.Sprintf("%.0f B/s", bytesPerSec)
}
func formatSize(bytes int64) string {
if bytes <= 0 {
return ""
}
const kb = 1024
const mb = kb * 1024
if bytes >= mb {
return fmt.Sprintf("%.1f MB", float64(bytes)/float64(mb))
}
if bytes >= kb {
return fmt.Sprintf("%.1f KB", float64(bytes)/float64(kb))
}
return fmt.Sprintf("%d B", bytes)
}
+33
View File
@@ -0,0 +1,33 @@
package main
import (
"io"
"os"
"sync/atomic"
"gitea.k3s.k0.nu/tools/photocli/internal/photos"
)
func runMain(args []string, stdout, stderr *os.File, bridge photos.Bridge) int {
return runMainWithSignal(args, stdout, stderr, bridge, defaultSignalChan())
}
func runMainWithSignal(args []string, stdout *os.File, stderr io.Writer, bridge photos.Bridge, sigCh <-chan struct{}) int {
done := make(chan struct{})
var rc atomic.Int32
go func() {
rc.Store(int32(run(args, stdout, stderr, bridge)))
close(done)
}()
select {
case <-done:
case <-sigCh:
bridge.Cancel()
stderr.Write([]byte("\nreceived signal, finishing current file...\n"))
<-done
}
return int(rc.Load())
}
+20
View File
@@ -0,0 +1,20 @@
//go:build !test
package main
import (
"os"
"os/signal"
"syscall"
)
func defaultSignalChan() <-chan struct{} {
ch := make(chan struct{})
go func() {
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
<-sig
close(ch)
}()
return ch
}
+8
View File
@@ -0,0 +1,8 @@
//go:build test
package main
func defaultSignalChan() <-chan struct{} {
ch := make(chan struct{})
return ch
}
+23
View File
@@ -0,0 +1,23 @@
//go:build !test
package main
import (
"syscall"
"unsafe"
)
func termSize() (int, int) {
type winsize struct {
Rows uint16
Cols uint16
Xpixels uint16
Ypixels uint16
}
var ws winsize
_, _, errno := syscall.Syscall(syscall.SYS_IOCTL, uintptr(syscall.Stdout), syscall.TIOCGWINSZ, uintptr(unsafe.Pointer(&ws)))
if errno != 0 || ws.Cols == 0 || ws.Rows == 0 {
return 80, 24
}
return int(ws.Cols), int(ws.Rows)
}
+7
View File
@@ -0,0 +1,7 @@
//go:build test
package main
func termSize() (int, int) {
return 80, 24
}
+15 -1
View File
@@ -1,3 +1,17 @@
module gitea.k3s.k0.nu/tools/photocli
go 1.22
go 1.25.0
require modernc.org/sqlite v1.52.0
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/sys v0.42.0 // indirect
modernc.org/libc v1.72.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
+51
View File
@@ -0,0 +1,51 @@
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k=
golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0=
modernc.org/cc/v4 v4.28.2 h1:3tQ0lf2ADtoby2EtSP+J7IE2SHwEJdP8ioR59wx7XpY=
modernc.org/cc/v4 v4.28.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.34.0 h1:yRLPFZieg532OT4rp4JFNIVcquwalMX26G95WQDqwCQ=
modernc.org/ccgo/v4 v4.34.0/go.mod h1:AS5WYMyBakQ+fhsHhtP8mWB82KTGPkNNJDGfGQCe0/A=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo=
modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.72.3 h1:ZnDF4tXn4NBXFutMMQC4vtbTFSXhhKzR73fv0beZEAU=
modernc.org/libc v1.72.3/go.mod h1:dn0dZNnnn1clLyvRxLxYExxiKRZIRENOfqQ8XEeg4Qs=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.52.0 h1:p4dhYh2tXZCiyaqHwRVJDjIGKWyXayiQpThxgDzJaxo=
modernc.org/sqlite v1.52.0/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+56
View File
@@ -0,0 +1,56 @@
package manifest
import "testing"
func TestNewEntryPath(t *testing.T) {
e := newEntry("id1", "file.jpg", 123, "local")
if e.ID != "id1" || e.Filename != "file.jpg" || e.Path != "file.jpg" || e.Size != 123 || e.Cloud != "local" || e.Exported == 0 {
t.Fatalf("unexpected entry: %+v", e)
}
e = NewEntry("id2", "file2.jpg", "Album/file2.jpg", 456, "cloud")
if e.Path != "Album/file2.jpg" || e.Filename != "file2.jpg" || e.Size != 456 || e.Cloud != "cloud" {
t.Fatalf("unexpected entry with path: %+v", e)
}
e = NewEntryWithChecksum("id3", "file3.jpg", "Album/file3.jpg", 789, "local", "sha256:abc")
if e.Checksum != "sha256:abc" || e.Path != "Album/file3.jpg" {
t.Fatalf("unexpected checksum entry: %+v", e)
}
}
func TestAddEntryDefaultsPath(t *testing.T) {
dir := t.TempDir()
jm := LoadJSONL(dir)
if err := jm.OpenAppend(); err != nil {
t.Fatal(err)
}
jm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:abc"})
jm.Close()
loaded := LoadJSONL(dir).Entries()["x1"]
if got := loaded.Path; got != "file.jpg" {
t.Fatalf("jsonl path = %q", got)
}
if loaded.Checksum != "sha256:abc" {
t.Fatalf("jsonl checksum = %q", loaded.Checksum)
}
sdir := t.TempDir()
sm, err := LoadSQLite(sdir)
if err != nil {
t.Fatal(err)
}
if err := sm.OpenAppend(); err != nil {
t.Fatal(err)
}
sm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:def"})
if _, err := sm.db.Exec(`UPDATE downloads SET path = '' WHERE id = 'x1'`); err != nil {
t.Fatal(err)
}
sloaded := sm.Entries()["x1"]
if got := sloaded.Path; got != "file.jpg" {
t.Fatalf("sqlite path = %q", got)
}
if sloaded.Checksum != "sha256:def" {
t.Fatalf("sqlite checksum = %q", sloaded.Checksum)
}
sm.Close()
}
+39
View File
@@ -0,0 +1,39 @@
package manifest
import (
"encoding/json"
"os"
"sync"
)
type fileLogWriter struct {
mu sync.Mutex
f *os.File
}
func NewFileLogWriter(path string) (LogWriter, error) {
f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
return nil, err
}
return &fileLogWriter{f: f}, nil
}
func (w *fileLogWriter) Log(e LogEntry) {
data, _ := json.Marshal(e)
w.mu.Lock()
defer w.mu.Unlock()
if w.f != nil {
w.f.Write(data)
w.f.Write([]byte("\n"))
}
}
func (w *fileLogWriter) Close() {
w.mu.Lock()
defer w.mu.Unlock()
if w.f != nil {
w.f.Close()
w.f = nil
}
}
+155
View File
@@ -0,0 +1,155 @@
package manifest
import (
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
)
type jsonlManifest struct {
mu sync.Mutex
entries map[string]Entry
path string
file *os.File
syncFunc func() error
}
var jsonlSaveHook func() error
func SetJSONLSaveHook(fn func() error) func() error {
old := jsonlSaveHook
jsonlSaveHook = fn
return old
}
func JSONLPath(dir string) string {
return filepath.Join(dir, "downloads.jsonl")
}
func LoadJSONL(dir string) *jsonlManifest {
m := &jsonlManifest{
entries: make(map[string]Entry),
path: JSONLPath(dir),
}
data, err := os.ReadFile(m.path)
if err != nil {
return m
}
type entryWithID struct {
ID string `json:"id"`
Filename string `json:"filename"`
Path string `json:"path,omitempty"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"`
}
for _, line := range strings.Split(string(data), "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
var raw entryWithID
if json.Unmarshal([]byte(line), &raw) == nil && raw.ID != "" {
if raw.Path == "" {
raw.Path = raw.Filename
}
m.entries[raw.ID] = Entry{
ID: raw.ID,
Filename: raw.Filename,
Path: raw.Path,
Size: raw.Size,
Cloud: raw.Cloud,
Checksum: raw.Checksum,
Exported: raw.Exported,
}
}
}
return m
}
func (m *jsonlManifest) Has(id string) bool {
m.mu.Lock()
defer m.mu.Unlock()
_, ok := m.entries[id]
return ok
}
func (m *jsonlManifest) Add(id string, filename string, size int64, cloud string) {
m.AddEntry(newEntry(id, filename, size, cloud))
}
func (m *jsonlManifest) AddEntry(entry Entry) {
m.mu.Lock()
defer m.mu.Unlock()
if entry.Path == "" {
entry.Path = entry.Filename
}
m.entries[entry.ID] = entry
if m.file != nil {
data, _ := json.Marshal(struct {
ID string `json:"id"`
Filename string `json:"filename"`
Path string `json:"path,omitempty"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"`
}{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Checksum: entry.Checksum, Exported: entry.Exported})
m.file.Write(data)
m.file.Write([]byte("\n"))
}
}
func (m *jsonlManifest) Save() error {
m.mu.Lock()
defer m.mu.Unlock()
if m.syncFunc != nil {
return m.syncFunc()
}
if jsonlSaveHook != nil {
return jsonlSaveHook()
}
if m.file != nil {
return m.file.Sync()
}
return nil
}
func (m *jsonlManifest) Close() {
m.mu.Lock()
defer m.mu.Unlock()
if m.file != nil {
m.file.Close()
m.file = nil
}
}
func (m *jsonlManifest) OpenAppend() error {
m.mu.Lock()
defer m.mu.Unlock()
if m.file != nil {
return nil
}
if err := os.MkdirAll(filepath.Dir(m.path), 0755); err != nil {
return err
}
f, err := os.OpenFile(m.path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
return err
}
m.file = f
return nil
}
func (m *jsonlManifest) Entries() map[string]Entry {
m.mu.Lock()
defer m.mu.Unlock()
out := make(map[string]Entry, len(m.entries))
for k, v := range m.entries {
out[k] = v
}
return out
}
+30
View File
@@ -0,0 +1,30 @@
package manifest
type LogEntry struct {
Timestamp int64 `json:"ts"`
Level string `json:"level"`
Event string `json:"event"`
AssetID string `json:"asset_id,omitempty"`
Album string `json:"album,omitempty"`
Filename string `json:"filename,omitempty"`
Size int64 `json:"size,omitempty"`
Cloud string `json:"cloud,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
Message string `json:"message,omitempty"`
}
type LogWriter interface {
Log(entry LogEntry)
Close()
}
type noopLogWriter struct{}
func (noopLogWriter) Log(LogEntry) { _ = struct{}{} }
func (noopLogWriter) Close() { _ = struct{}{} }
var NoopLogWriter LogWriter = noopLogWriter{}
func LogPath(dir string) string {
return dir + "/export.log"
}
+211
View File
@@ -0,0 +1,211 @@
package manifest
import (
"database/sql"
"os"
"path/filepath"
"testing"
_ "modernc.org/sqlite"
)
func TestNoopLogWriter(t *testing.T) {
lw := NoopLogWriter
lw.Log(LogEntry{Event: "test"})
lw.Close()
noopLogWriter{}.Log(LogEntry{Event: "test"})
noopLogWriter{}.Close()
}
func TestNewSQLiteLogWriter(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "test.db")
db, err := sql.Open("sqlite", dbPath)
if err != nil {
t.Fatal(err)
}
defer db.Close()
lw, err := NewSQLiteLogWriter(db)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{
Timestamp: 1700000000,
Level: "info",
Event: "export_done",
AssetID: "asset-1",
Album: "Favorites",
Filename: "photo.jpg",
Size: 1024,
Cloud: "local",
DurationMs: 500,
Message: "",
})
lw.Log(LogEntry{
Timestamp: 1700000001,
Level: "error",
Event: "export_fail",
AssetID: "asset-2",
Filename: "bad.jpg",
Message: "timeout",
})
lw.Close()
var count int
err = db.QueryRow(`SELECT COUNT(*) FROM logs`).Scan(&count)
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected 2 log entries, got %d", count)
}
var event, level, assetID string
err = db.QueryRow(`SELECT event, level, asset_id FROM logs WHERE asset_id = 'asset-1'`).Scan(&event, &level, &assetID)
if err != nil {
t.Fatal(err)
}
if event != "export_done" || level != "info" || assetID != "asset-1" {
t.Errorf("unexpected row: event=%s level=%s asset_id=%s", event, level, assetID)
}
}
func TestSQLiteLogWriterNilDB(t *testing.T) {
w := &sqliteLogWriter{db: nil}
w.Log(LogEntry{Event: "test"})
w.Close()
}
func TestNewSQLiteLogWriterError(t *testing.T) {
db, err := sql.Open("sqlite", filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
if err := db.Close(); err != nil {
t.Fatal(err)
}
if _, err := NewSQLiteLogWriter(db); err == nil {
t.Error("expected error for closed db")
}
}
func TestSQLiteLogWriterCloseConcrete(t *testing.T) {
(&sqliteLogWriter{}).Close()
}
func TestNewFileLogWriter(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "export.log")
lw, err := NewFileLogWriter(path)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{
Timestamp: 1700000000,
Level: "info",
Event: "export_done",
Filename: "photo.jpg",
Size: 2048,
})
lw.Close()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if len(data) == 0 {
t.Error("expected log data")
}
if data[len(data)-1] != '\n' {
t.Error("expected trailing newline")
}
}
func TestFileLogWriterClosed(t *testing.T) {
w := &fileLogWriter{f: nil}
w.Log(LogEntry{Event: "test"})
w.Close()
}
func TestNewFileLogWriterError(t *testing.T) {
_, err := NewFileLogWriter("/nonexistent/dir/export.log")
if err == nil {
t.Error("expected error for bad path")
}
}
func TestFileLogWriterDoubleClose(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "export.log")
lw, err := NewFileLogWriter(path)
if err != nil {
t.Fatal(err)
}
lw.Close()
lw.Close()
}
func TestLogPath(t *testing.T) {
p := LogPath("/tmp/out")
if p != "/tmp/out/export.log" {
t.Errorf("expected /tmp/out/export.log, got %s", p)
}
}
func TestOpenLogWriterSQLite(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
lw, err := OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{Event: "test", Level: "info"})
lw.Close()
}
func TestOpenLogWriterJSONL(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
lw, err := OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{Event: "test", Level: "info"})
lw.Close()
if _, err := os.Stat(LogPath(dir)); os.IsNotExist(err) {
t.Error("expected export.log to exist")
}
}
func TestOpenLogWriterNilManifest(t *testing.T) {
dir := t.TempDir()
lw, err := OpenLogWriter(nil, dir)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{Event: "test", Level: "info"})
lw.Close()
if _, err := os.Stat(LogPath(dir)); os.IsNotExist(err) {
t.Error("expected export.log to exist for nil manifest")
}
}
+51
View File
@@ -0,0 +1,51 @@
package manifest
import "time"
type Entry struct {
ID string
Filename string
Path string
Size int64
Cloud string
Checksum string
Exported int64
}
type Manifest interface {
Has(id string) bool
Add(id string, filename string, size int64, cloud string)
AddEntry(entry Entry)
Save() error
Close()
OpenAppend() error
}
type EntryReader interface {
Entries() map[string]Entry
}
func newEntry(id, filename string, size int64, cloud string) Entry {
return Entry{
ID: id,
Filename: filename,
Path: filename,
Size: size,
Cloud: cloud,
Exported: time.Now().Unix(),
}
}
func NewEntry(id, filename, path string, size int64, cloud string) Entry {
e := newEntry(id, filename, size, cloud)
if path != "" {
e.Path = path
}
return e
}
func NewEntryWithChecksum(id, filename, path string, size int64, cloud, checksum string) Entry {
e := NewEntry(id, filename, path, size, cloud)
e.Checksum = checksum
return e
}
File diff suppressed because it is too large Load Diff
+108
View File
@@ -0,0 +1,108 @@
package manifest
import (
"fmt"
"os"
"strings"
)
type Format string
const (
FormatJSONL Format = "jsonl"
FormatSQLite Format = "sqlite"
)
func Open(dir string, format Format) (Manifest, error) {
jsonlPath := JSONLPath(dir)
sqlitePath := SQLitePath(dir)
jsonlExists := FileExists(jsonlPath)
sqliteExists := FileExists(sqlitePath)
switch {
case format == FormatJSONL && jsonlExists:
return LoadJSONL(dir), nil
case format == FormatSQLite && sqliteExists:
return LoadSQLite(dir)
case format == FormatJSONL && sqliteExists:
return ConvertFromSQLite(dir)
case format == FormatSQLite && jsonlExists:
return ConvertFromJSONL(dir)
default:
if format == FormatJSONL {
return LoadJSONL(dir), nil
}
return LoadSQLite(dir)
}
}
func ConvertFromJSONL(dir string) (Manifest, error) {
src := LoadJSONL(dir)
if err := src.OpenAppend(); err != nil {
return nil, fmt.Errorf("open jsonl for read: %w", err)
}
defer src.Close()
dst, _ := LoadSQLite(dir)
if err := dst.OpenAppend(); err != nil {
return nil, fmt.Errorf("open sqlite for write: %w", err)
}
for id, e := range src.Entries() {
e.ID = id
dst.AddEntry(e)
}
os.Remove(JSONLPath(dir))
return dst, nil
}
func ConvertFromSQLite(dir string) (Manifest, error) {
src, _ := LoadSQLite(dir)
if err := src.OpenAppend(); err != nil {
return nil, fmt.Errorf("open sqlite for read: %w", err)
}
defer src.Close()
dst := LoadJSONL(dir)
if err := dst.OpenAppend(); err != nil {
return nil, fmt.Errorf("open jsonl for write: %w", err)
}
for id, e := range src.Entries() {
e.ID = id
dst.AddEntry(e)
}
if err := dst.Save(); err != nil {
return nil, fmt.Errorf("save jsonl: %w", err)
}
os.Remove(SQLitePath(dir))
return dst, nil
}
func FileExists(path string) bool {
info, err := os.Stat(path)
if err != nil {
return false
}
return !info.IsDir()
}
func ParseFormat(s string) (Format, error) {
switch strings.ToLower(s) {
case "jsonl", "json":
return FormatJSONL, nil
case "sqlite", "db", "sqlite3":
return FormatSQLite, nil
default:
return "", fmt.Errorf("unknown manifest format %q (use jsonl or sqlite)", s)
}
}
func OpenLogWriter(m Manifest, dir string) (LogWriter, error) {
if sm, ok := m.(*sqliteManifest); ok && sm.DB() != nil {
return NewSQLiteLogWriter(sm.DB())
}
return NewFileLogWriter(LogPath(dir))
}
+147
View File
@@ -0,0 +1,147 @@
package manifest
import (
"database/sql"
"fmt"
"os"
"path/filepath"
_ "modernc.org/sqlite"
)
type sqliteManifest struct {
path string
db *sql.DB
open sqlOpenerFunc
execFunc func(query string, args ...any) (sql.Result, error)
}
type sqlOpenerFunc func(driverName, dataSourceName string) (*sql.DB, error)
var sqliteOpenFunc sqlOpenerFunc
func defaultSQLOpener() sqlOpenerFunc {
if sqliteOpenFunc != nil {
return sqliteOpenFunc
}
return sql.Open
}
func SQLitePath(dir string) string {
return filepath.Join(dir, "downloads.db")
}
func LoadSQLite(dir string) (*sqliteManifest, error) {
m := &sqliteManifest{
path: SQLitePath(dir),
open: defaultSQLOpener(),
}
return m, nil
}
func (m *sqliteManifest) OpenAppend() error {
if m.db != nil {
return nil
}
if err := os.MkdirAll(filepath.Dir(m.path), 0755); err != nil {
return err
}
opener := m.open
if opener == nil {
opener = sql.Open
}
db, err := opener("sqlite", m.path)
if err != nil {
return fmt.Errorf("open sqlite: %w", err)
}
execFn := m.execFunc
if execFn == nil {
execFn = db.Exec
}
_, err = execFn(`CREATE TABLE IF NOT EXISTS downloads (
id TEXT PRIMARY KEY,
filename TEXT NOT NULL DEFAULT '',
path TEXT NOT NULL DEFAULT '',
size INTEGER NOT NULL DEFAULT 0,
cloud TEXT NOT NULL DEFAULT '',
checksum TEXT NOT NULL DEFAULT '',
exported INTEGER NOT NULL DEFAULT 0
)`)
if err != nil {
db.Close()
return fmt.Errorf("create table: %w", err)
}
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN path TEXT NOT NULL DEFAULT ''`)
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN checksum TEXT NOT NULL DEFAULT ''`)
_, err = execFn(`CREATE INDEX IF NOT EXISTS idx_downloads_id ON downloads(id)`)
if err != nil {
db.Close()
return fmt.Errorf("create index: %w", err)
}
m.db = db
return nil
}
func (m *sqliteManifest) Has(id string) bool {
if m.db == nil {
return false
}
var count int
err := m.db.QueryRow(`SELECT COUNT(*) FROM downloads WHERE id = ?`, id).Scan(&count)
if err != nil {
return false
}
return count > 0
}
func (m *sqliteManifest) Add(id string, filename string, size int64, cloud string) {
m.AddEntry(newEntry(id, filename, size, cloud))
}
func (m *sqliteManifest) AddEntry(entry Entry) {
if m.db == nil {
return
}
if entry.Path == "" {
entry.Path = entry.Filename
}
m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, checksum, exported) VALUES (?, ?, ?, ?, ?, ?, ?)`,
entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Checksum, entry.Exported)
}
func (m *sqliteManifest) Save() error {
return nil
}
func (m *sqliteManifest) Close() {
if m.db != nil {
m.db.Close()
m.db = nil
}
}
func (m *sqliteManifest) DB() *sql.DB {
return m.db
}
func (m *sqliteManifest) Entries() map[string]Entry {
if m.db == nil {
return nil
}
out := make(map[string]Entry)
rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, checksum, exported FROM downloads`)
if err != nil {
return out
}
defer rows.Close()
for rows.Next() {
var e Entry
if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Checksum, &e.Exported); err == nil {
if e.Path == "" {
e.Path = e.Filename
}
out[e.ID] = e
}
}
return out
}
+402
View File
@@ -0,0 +1,402 @@
package manifest
import (
"database/sql"
"fmt"
"os"
"strings"
"testing"
_ "modernc.org/sqlite"
)
func TestSetJSONLSaveHook(t *testing.T) {
old := SetJSONLSaveHook(func() error { return fmt.Errorf("hook error") })
if old != nil {
t.Error("expected nil old hook")
}
restore := SetJSONLSaveHook(old)
if restore == nil {
t.Error("expected non-nil restore function")
}
}
func TestJSONLSaveHookError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
old := SetJSONLSaveHook(func() error { return fmt.Errorf("hook error") })
defer SetJSONLSaveHook(old)
if err := m.Save(); err == nil {
t.Error("expected hook error from Save")
}
m.Close()
}
func TestJSONLSyncFuncError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.syncFunc = func() error { return fmt.Errorf("sync func error") }
if err := m.Save(); err == nil {
t.Error("expected syncFunc error from Save")
}
m.Close()
}
func TestSQLiteOpenAppendSQLError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, fmt.Errorf("simulated open error")
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from sql.Open failure")
}
}
func TestSQLiteOpenAppendCreateTableError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from closed DB CREATE TABLE")
}
}
func TestSQLiteHasAfterClose(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
if m.Has("x1") {
t.Error("Has should return false after Close")
}
}
func TestSQLiteEntriesAfterClose(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
entries := m.Entries()
if entries != nil {
t.Errorf("Entries should return nil after Close, got %d entries", len(entries))
}
}
func TestSQLiteOpenAppendMkdirAllError(t *testing.T) {
m, err := LoadSQLite("/proc/cannot-write")
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error creating dir under /proc")
m.Close()
}
}
func TestSQLiteOpenAppendNilOpener(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
m.open = nil
if err := m.OpenAppend(); err != nil {
t.Errorf("expected nil opener to use sql.Open fallback, got err: %v", err)
}
m.Close()
}
func TestSQLiteOpenAppendCreateIndexError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from closed DB")
}
}
func TestSQLiteHasQueryError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
result := m.Has("x1")
if result {
t.Error("Has should return false with broken DB connection")
}
}
func TestSQLiteEntriesQueryError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
entries := m.Entries()
if entries == nil {
t.Error("Entries should return non-nil map with broken DB connection")
}
if len(entries) != 0 {
t.Errorf("Entries should be empty with broken DB connection, got %d", len(entries))
}
}
func TestSQLiteHasQueryErrorWithOpenDB(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
if m.Has("x1") {
t.Error("Has should return false with closed DB")
}
}
func TestSQLiteEntriesQueryErrorWithOpenDB(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
entries := m.Entries()
if entries != nil && len(entries) != 0 {
t.Errorf("Entries should be empty with closed DB, got %d", len(entries))
}
}
func TestSQLiteCreateIndexError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
callCount := 0
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
m.execFunc = func(query string, args ...any) (sql.Result, error) {
if strings.Contains(query, "CREATE INDEX") {
return nil, fmt.Errorf("injected CREATE INDEX error")
}
callCount++
return db.Exec(query, args...)
}
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from CREATE INDEX")
m.Close()
}
}
func TestConvertFromJSONLOpenAppendSQLError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
oldOpen := sqliteOpenFunc
sqliteOpenFunc = func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, fmt.Errorf("simulated sqlite open error")
}
defer func() { sqliteOpenFunc = oldOpen }()
_, err := ConvertFromJSONL(dir)
if err == nil {
t.Error("expected error from dst.OpenAppend during ConvertFromJSONL")
}
}
func TestConvertFromJSONLDstOpenAppendError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
realOpen := defaultSQLOpener()
oldOpen := sqliteOpenFunc
sqliteOpenFunc = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
defer func() { sqliteOpenFunc = oldOpen }()
_, err := ConvertFromJSONL(dir)
if err == nil {
t.Error("expected error from dst.OpenAppend during ConvertFromJSONL")
}
}
func TestConvertFromSQLiteSrcOpenAppendError(t *testing.T) {
dir := t.TempDir()
src, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := src.OpenAppend(); err != nil {
t.Fatal(err)
}
src.Add("x1", "photo.jpg", 1024, "local")
src.Close()
realOpen := defaultSQLOpener()
callCount := 0
oldOpen := sqliteOpenFunc
sqliteOpenFunc = func(driverName, dataSourceName string) (*sql.DB, error) {
callCount++
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
if callCount > 0 {
db.Close()
}
return db, nil
}
defer func() { sqliteOpenFunc = oldOpen }()
_, err = ConvertFromSQLite(dir)
if err == nil {
t.Error("expected error from src.OpenAppend during ConvertFromSQLite")
}
}
func TestConvertFromSQLiteDstOpenAppendError(t *testing.T) {
dir := t.TempDir()
src, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := src.OpenAppend(); err != nil {
t.Fatal(err)
}
src.Add("x1", "photo.jpg", 1024, "local")
src.Close()
jsonlPath := JSONLPath(dir)
f, err := os.Create(jsonlPath)
if err != nil {
t.Fatal(err)
}
f.Close()
os.Chmod(jsonlPath, 0444)
defer os.Chmod(jsonlPath, 0644)
_, err = ConvertFromSQLite(dir)
if err == nil {
t.Error("expected error from dst.OpenAppend during ConvertFromSQLite")
}
}
func TestJSONLSaveError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.file.Close()
if err := m.Save(); err == nil {
t.Error("expected Sync error on closed file")
}
m.Close()
}
func TestConvertFromSQLiteSaveError(t *testing.T) {
dir := t.TempDir()
src, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := src.OpenAppend(); err != nil {
t.Fatal(err)
}
src.Add("x1", "photo.jpg", 1024, "local")
src.Close()
oldHook := jsonlSaveHook
jsonlSaveHook = func() error { return fmt.Errorf("simulated sync error") }
defer func() { jsonlSaveHook = oldHook }()
_, err = ConvertFromSQLite(dir)
if err == nil {
t.Error("expected error from dst.Save during ConvertFromSQLite")
}
if err != nil && !strings.Contains(err.Error(), "save jsonl") {
t.Errorf("expected save jsonl error, got: %v", err)
}
}
+41
View File
@@ -0,0 +1,41 @@
package manifest
import (
"database/sql"
)
type sqliteLogWriter struct {
db *sql.DB
}
func NewSQLiteLogWriter(db *sql.DB) (LogWriter, error) {
_, err := db.Exec(`CREATE TABLE IF NOT EXISTS logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL,
level TEXT NOT NULL,
event TEXT NOT NULL,
asset_id TEXT,
album TEXT,
filename TEXT,
size INTEGER,
cloud TEXT,
duration_ms INTEGER,
message TEXT
)`)
if err != nil {
return nil, err
}
_, _ = db.Exec(`CREATE INDEX IF NOT EXISTS idx_logs_ts ON logs(ts)`)
_, _ = db.Exec(`CREATE INDEX IF NOT EXISTS idx_logs_event ON logs(event)`)
return &sqliteLogWriter{db: db}, nil
}
func (w *sqliteLogWriter) Log(e LogEntry) {
if w.db == nil {
return
}
w.db.Exec(`INSERT INTO logs (ts, level, event, asset_id, album, filename, size, cloud, duration_ms, message) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
e.Timestamp, e.Level, e.Event, e.AssetID, e.Album, e.Filename, e.Size, e.Cloud, e.DurationMs, e.Message)
}
func (w *sqliteLogWriter) Close() { _ = w }
+16 -1
View File
@@ -9,10 +9,14 @@ type Bridge interface {
RequestAccess() error
ListAlbums() ([]Album, error)
ListAssets(albumID string) ([]Asset, int, error)
ReverseGeocode(latitude, longitude float64) (Placemark, error)
ListTree() ([]CollectionNode, error)
ExportPreview(assetID, outputDir string, targetSize, index int) (ExportResult, error)
ExportPreview(assetID, outputDir string, targetSize, quality, index int) (ExportResult, error)
ExportOriginal(assetID, outputDir string, index int) (ExportResult, error)
ExportPreviewWithSlot(assetID, outputDir string, targetSize, quality, index, slotIndex int) (ExportResult, error)
ExportOriginalWithSlot(assetID, outputDir string, index, slotIndex int) (ExportResult, error)
Cancel()
IsCancelled() bool
}
func ParseAlbumsJSON(jsonStr string) ([]Album, error) {
@@ -36,6 +40,17 @@ func ParseAssetsJSON(jsonStr string) ([]Asset, int, error) {
return resp.Assets, resp.Total, nil
}
func ParsePlacemarkJSON(jsonStr string) (Placemark, error) {
var resp PlacemarkResponse
if err := json.Unmarshal([]byte(jsonStr), &resp); err != nil {
return Placemark{}, err
}
if resp.Error != "" {
return Placemark{}, fmt.Errorf("%s", resp.Error)
}
return resp.Placemark, nil
}
func ParseTreeJSON(jsonStr string) ([]CollectionNode, error) {
var errResp ErrorResponse
if err := json.Unmarshal([]byte(jsonStr), &errResp); err == nil && errResp.Error != "" {
+76 -13
View File
@@ -4,7 +4,7 @@ package photos
/*
#cgo CFLAGS: -I${SRCDIR}/../../bridge
#cgo LDFLAGS: -L${SRCDIR}/../../bridge -lphotokit_bridge -framework Photos -framework Foundation -framework AppKit
#cgo LDFLAGS: -L${SRCDIR}/../../bridge -lphotokit_bridge -framework Photos -framework Foundation -framework AppKit -framework UniformTypeIdentifiers -framework CoreLocation -framework MapKit
#include "photokit_bridge.h"
#include <stdlib.h>
*/
@@ -45,6 +45,15 @@ func (*CgoBridge) ListAssets(albumID string) ([]Asset, int, error) {
return ParseAssetsJSON(C.GoString(cs))
}
func (*CgoBridge) ReverseGeocode(latitude, longitude float64) (Placemark, error) {
cs := C.photos_reverse_geocode_json(C.double(latitude), C.double(longitude))
if cs == nil {
return Placemark{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParsePlacemarkJSON(C.GoString(cs))
}
func (*CgoBridge) ListTree() ([]CollectionNode, error) {
cs := C.photos_list_tree_json()
if cs == nil {
@@ -58,30 +67,84 @@ func (*CgoBridge) Cancel() {
C.photos_request_cancel()
}
func (*CgoBridge) ExportPreview(assetID, outputDir string, targetSize, index int) (ExportResult, error) {
cid := C.CString(assetID)
defer C.free(unsafe.Pointer(cid))
cdir := C.CString(outputDir)
defer C.free(unsafe.Pointer(cdir))
func (*CgoBridge) IsCancelled() bool {
return C.photos_request_is_cancelled() != 0
}
cs := C.photos_export_preview_json(cid, cdir, C.int(targetSize), C.int(index))
if cs == nil {
return ExportResult{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParseExportResultJSON(C.GoString(cs))
func (*CgoBridge) ExportPreview(assetID, outputDir string, targetSize, quality, index int) (ExportResult, error) {
return exportPreviewWithSlot(assetID, outputDir, targetSize, quality, index, -1)
}
func (*CgoBridge) ExportOriginal(assetID, outputDir string, index int) (ExportResult, error) {
return exportOriginalWithSlot(assetID, outputDir, index, -1)
}
func (*CgoBridge) ExportPreviewWithSlot(assetID, outputDir string, targetSize, quality, index, slotIndex int) (ExportResult, error) {
return exportPreviewWithSlot(assetID, outputDir, targetSize, quality, index, slotIndex)
}
func (*CgoBridge) ExportOriginalWithSlot(assetID, outputDir string, index, slotIndex int) (ExportResult, error) {
return exportOriginalWithSlot(assetID, outputDir, index, slotIndex)
}
func exportPreviewWithSlot(assetID, outputDir string, targetSize, quality, index, slotIndex int) (ExportResult, error) {
cid := C.CString(assetID)
defer C.free(unsafe.Pointer(cid))
cdir := C.CString(outputDir)
defer C.free(unsafe.Pointer(cdir))
cs := C.photos_export_original_json(cid, cdir, C.int(index))
cs := C.photos_export_preview_json(cid, cdir, C.int(targetSize), C.int(quality), C.int(index), C.int(slotIndex))
if cs == nil {
return ExportResult{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParseExportResultJSON(C.GoString(cs))
}
func exportOriginalWithSlot(assetID, outputDir string, index, slotIndex int) (ExportResult, error) {
cid := C.CString(assetID)
defer C.free(unsafe.Pointer(cid))
cdir := C.CString(outputDir)
defer C.free(unsafe.Pointer(cdir))
cs := C.photos_export_original_json(cid, cdir, C.int(index), C.int(slotIndex))
if cs == nil {
return ExportResult{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParseExportResultJSON(C.GoString(cs))
}
func GetProgressSlots() []ExportProgressSlot {
count := int(C.photos_get_progress_slot_count())
slots := C.photos_get_progress_slots()
if slots == nil || count == 0 {
return nil
}
result := make([]ExportProgressSlot, count)
for i := 0; i < count; i++ {
ptr := (*C.export_progress_t)(unsafe.Pointer(uintptr(unsafe.Pointer(slots)) + uintptr(i)*unsafe.Sizeof(C.export_progress_t{})))
result[i] = ExportProgressSlot{
Active: ptr.active != 0,
Progress: float64(ptr.progress),
BytesDone: int64(ptr.bytes_done),
BytesTotal: int64(ptr.bytes_total),
}
}
return result
}
func ResetProgressSlots() {
C.photos_reset_progress_slots()
}
func GetProgressSlotCount() int {
return int(C.photos_get_progress_slot_count())
}
type ExportProgressSlot struct {
Active bool
Progress float64
BytesDone int64
BytesTotal int64
}
+90 -12
View File
@@ -12,12 +12,18 @@ void photos_test_set_access(int rc);
void photos_test_set_albums(const char *json);
void photos_test_set_assets(const char *json);
void photos_test_set_tree(const char *json);
void photos_test_set_geocode(const char *json);
void photos_test_set_geocode_null(void);
void photos_test_set_albums_null(void);
void photos_test_set_assets_null(void);
void photos_test_set_tree_null(void);
void photos_request_cancel(void);
void photos_test_set_export_preview_json(const char *json);
void photos_test_set_export_preview_json_null(void);
void photos_test_set_export_original_json(const char *json);
void photos_test_set_export_original_json_null(void);
void photos_test_set_progress_slot_count(int count);
void photos_test_set_progress_slots_null(int val);
*/
import "C"
@@ -27,15 +33,21 @@ type CgoBridge struct{}
var DefaultBridge Bridge = &CgoBridge{}
func SetTestAccessRC(rc int) { C.photos_test_set_access(C.int(rc)) }
func SetTestAlbumsJSON(json string) { C.photos_test_set_albums(C.CString(json)) }
func SetTestAssetsJSON(json string) { C.photos_test_set_assets(C.CString(json)) }
func SetTestTreeJSON(json string) { C.photos_test_set_tree(C.CString(json)) }
func SetTestAlbumsNull() { C.photos_test_set_albums_null() }
func SetTestAssetsNull() { C.photos_test_set_assets_null() }
func SetTestTreeNull() { C.photos_test_set_tree_null() }
func SetTestExportPreviewJSON(json string) { C.photos_test_set_export_preview_json(C.CString(json)) }
func SetTestAccessRC(rc int) { C.photos_test_set_access(C.int(rc)) }
func SetTestAlbumsJSON(json string) { C.photos_test_set_albums(C.CString(json)) }
func SetTestAssetsJSON(json string) { C.photos_test_set_assets(C.CString(json)) }
func SetTestTreeJSON(json string) { C.photos_test_set_tree(C.CString(json)) }
func SetTestGeocodeJSON(json string) { C.photos_test_set_geocode(C.CString(json)) }
func SetTestGeocodeNull() { C.photos_test_set_geocode_null() }
func SetTestAlbumsNull() { C.photos_test_set_albums_null() }
func SetTestAssetsNull() { C.photos_test_set_assets_null() }
func SetTestTreeNull() { C.photos_test_set_tree_null() }
func SetTestExportPreviewJSON(json string) { C.photos_test_set_export_preview_json(C.CString(json)) }
func SetTestExportPreviewJSONNull() { C.photos_test_set_export_preview_json_null() }
func SetTestExportOriginalJSON(json string) { C.photos_test_set_export_original_json(C.CString(json)) }
func SetTestExportOriginalJSONNull() { C.photos_test_set_export_original_json_null() }
func SetTestProgressSlotCount(count int) { C.photos_test_set_progress_slot_count(C.int(count)) }
func SetTestProgressSlotsNull(val int) { C.photos_test_set_progress_slots_null(C.int(val)) }
func (*CgoBridge) RequestAccess() error {
rc := C.photos_request_access()
@@ -65,6 +77,15 @@ func (*CgoBridge) ListAssets(albumID string) ([]Asset, int, error) {
return ParseAssetsJSON(C.GoString(cs))
}
func (*CgoBridge) ReverseGeocode(latitude, longitude float64) (Placemark, error) {
cs := C.photos_reverse_geocode_json(C.double(latitude), C.double(longitude))
if cs == nil {
return Placemark{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParsePlacemarkJSON(C.GoString(cs))
}
func (*CgoBridge) ListTree() ([]CollectionNode, error) {
cs := C.photos_list_tree_json()
if cs == nil {
@@ -78,12 +99,32 @@ func (*CgoBridge) Cancel() {
C.photos_request_cancel()
}
func (*CgoBridge) ExportPreview(assetID, outputDir string, targetSize, index int) (ExportResult, error) {
func (*CgoBridge) IsCancelled() bool {
return C.photos_request_is_cancelled() != 0
}
func (*CgoBridge) ExportPreview(assetID, outputDir string, targetSize, quality, index int) (ExportResult, error) {
return exportPreviewWithSlotTest(assetID, outputDir, targetSize, quality, index, -1)
}
func (*CgoBridge) ExportOriginal(assetID, outputDir string, index int) (ExportResult, error) {
return exportOriginalWithSlotTest(assetID, outputDir, index, -1)
}
func (*CgoBridge) ExportPreviewWithSlot(assetID, outputDir string, targetSize, quality, index, slotIndex int) (ExportResult, error) {
return exportPreviewWithSlotTest(assetID, outputDir, targetSize, quality, index, slotIndex)
}
func (*CgoBridge) ExportOriginalWithSlot(assetID, outputDir string, index, slotIndex int) (ExportResult, error) {
return exportOriginalWithSlotTest(assetID, outputDir, index, slotIndex)
}
func exportPreviewWithSlotTest(assetID, outputDir string, targetSize, quality, index, slotIndex int) (ExportResult, error) {
cid := C.CString(assetID)
defer C.free(unsafe.Pointer(cid))
cdir := C.CString(outputDir)
defer C.free(unsafe.Pointer(cdir))
cs := C.photos_export_preview_json(cid, cdir, C.int(targetSize), C.int(index))
cs := C.photos_export_preview_json(cid, cdir, C.int(targetSize), C.int(quality), C.int(index), C.int(slotIndex))
if cs == nil {
return ExportResult{}, errBridgeNil
}
@@ -91,15 +132,52 @@ func (*CgoBridge) ExportPreview(assetID, outputDir string, targetSize, index int
return ParseExportResultJSON(C.GoString(cs))
}
func (*CgoBridge) ExportOriginal(assetID, outputDir string, index int) (ExportResult, error) {
func exportOriginalWithSlotTest(assetID, outputDir string, index, slotIndex int) (ExportResult, error) {
cid := C.CString(assetID)
defer C.free(unsafe.Pointer(cid))
cdir := C.CString(outputDir)
defer C.free(unsafe.Pointer(cdir))
cs := C.photos_export_original_json(cid, cdir, C.int(index))
cs := C.photos_export_original_json(cid, cdir, C.int(index), C.int(slotIndex))
if cs == nil {
return ExportResult{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParseExportResultJSON(C.GoString(cs))
}
type ExportProgressSlot struct {
Active bool
Progress float64
BytesDone int64
BytesTotal int64
}
func GetProgressSlots() []ExportProgressSlot {
count := int(C.photos_get_progress_slot_count())
if count <= 0 {
return nil
}
cSlots := C.photos_get_progress_slots()
if cSlots == nil {
return nil
}
slots := make([]ExportProgressSlot, count)
for i := 0; i < count; i++ {
s := C.photos_get_progress_slot(cSlots, C.int(i))
slots[i] = ExportProgressSlot{
Active: s.active != 0,
Progress: float64(s.progress),
BytesDone: int64(s.bytes_done),
BytesTotal: int64(s.bytes_total),
}
}
return slots
}
func ResetProgressSlots() {
C.photos_reset_progress_slots()
}
func GetProgressSlotCount() int {
return int(C.photos_get_progress_slot_count())
}
+1 -1
View File
@@ -5,4 +5,4 @@ package photos
import "fmt"
var errAccessDenied = fmt.Errorf("photos access denied: grant Full Disk Access or Photos permission in System Settings > Privacy & Security")
var errBridgeNil = fmt.Errorf("bridge returned nil")
var errBridgeNil = fmt.Errorf("bridge returned nil")
+374 -26
View File
@@ -35,9 +35,9 @@ func TestParseAlbumsJSON(t *testing.T) {
wantErr: true,
},
{
name: "missing albums key",
json: `{}`,
want: []Album{},
name: "missing albums key",
json: `{}`,
want: []Album{},
},
}
@@ -66,31 +66,60 @@ func TestParseAlbumsJSON(t *testing.T) {
func TestParseAssetsJSON(t *testing.T) {
tests := []struct {
name string
json string
want []Asset
name string
json string
want []Asset
wantTotal int
wantErr bool
errMsg string
wantErr bool
errMsg string
}{
{
name: "empty assets",
json: `{"assets":[],"total":0}`,
want: []Asset{},
name: "empty assets",
json: `{"assets":[],"total":0}`,
want: []Asset{},
wantTotal: 0,
},
{
name: "single asset",
json: `{"assets":[{"id":"asset1","filename":"IMG_0001.JPG"}],"total":1}`,
want: []Asset{{ID: "asset1", Filename: "IMG_0001.JPG"}},
name: "single asset",
json: `{"assets":[{"id":"asset1","filename":"IMG_0001.JPG"}],"total":1}`,
want: []Asset{{ID: "asset1", Filename: "IMG_0001.JPG"}},
wantTotal: 1,
},
{
name: "multiple assets",
json: `{"assets":[{"id":"a1","filename":"a.jpg"},{"id":"a2","filename":"b.jpg"},{"id":"a3","filename":"c.jpg"}],"total":3}`,
want: []Asset{{ID: "a1", Filename: "a.jpg"}, {ID: "a2", Filename: "b.jpg"}, {ID: "a3", Filename: "c.jpg"}},
name: "asset with metadata",
json: `{"assets":[{"id":"a1","filename":"IMG.JPG","cloud":"local","mediaType":"image","pixelWidth":4032,"pixelHeight":3024,"duration":0,"isFavorite":true,"hasAdjustments":false,"resources":[{"type":"photo","filename":"IMG.JPG","uti":"public.heic","local":true}]}],"total":1}`,
want: []Asset{{
ID: "a1", Filename: "IMG.JPG", Cloud: "local",
MediaType: "image", PixelWidth: 4032, PixelHeight: 3024,
IsFavorite: true, HasAdjustments: false,
Resources: []AssetResource{{Type: "photo", Filename: "IMG.JPG", UTI: "public.heic", Local: true}},
}},
wantTotal: 1,
},
{
name: "video asset with duration",
json: `{"assets":[{"id":"v1","filename":"clip.mov","cloud":"cloud","mediaType":"video","pixelWidth":1920,"pixelHeight":1080,"duration":12.5}],"total":1}`,
want: []Asset{{
ID: "v1", Filename: "clip.mov", Cloud: "cloud",
MediaType: "video", PixelWidth: 1920, PixelHeight: 1080, Duration: 12.5,
}},
wantTotal: 1,
},
{
name: "multiple assets",
json: `{"assets":[{"id":"a1","filename":"a.jpg"},{"id":"a2","filename":"b.jpg"},{"id":"a3","filename":"c.jpg"}],"total":3}`,
want: []Asset{{ID: "a1", Filename: "a.jpg"}, {ID: "a2", Filename: "b.jpg"}, {ID: "a3", Filename: "c.jpg"}},
wantTotal: 3,
},
{
name: "asset with creationDate",
json: `{"assets":[{"id":"d1","filename":"photo.jpg","creationDate":"2024-06-15T12:30:00+0200"}],"total":1}`,
want: func() []Asset {
d := "2024-06-15T12:30:00+0200"
return []Asset{{ID: "d1", Filename: "photo.jpg", CreationDate: &d}}
}(),
wantTotal: 1,
},
{
name: "error response",
json: `{"error":"album not found"}`,
@@ -103,9 +132,9 @@ func TestParseAssetsJSON(t *testing.T) {
wantErr: true,
},
{
name: "empty error is not an error",
json: `{"error":"","assets":[{"id":"a1","filename":"IMG.JPG"}],"total":1}`,
want: []Asset{{ID: "a1", Filename: "IMG.JPG"}},
name: "empty error is not an error",
json: `{"error":"","assets":[{"id":"a1","filename":"IMG.JPG"}],"total":1}`,
want: []Asset{{ID: "a1", Filename: "IMG.JPG"}},
wantTotal: 1,
},
}
@@ -131,7 +160,7 @@ func TestParseAssetsJSON(t *testing.T) {
return
}
for i := range got {
if got[i] != tt.want[i] {
if !equalAsset(got[i], tt.want[i]) {
t.Errorf("ParseAssetsJSON()[%d] = %+v, want %+v", i, got[i], tt.want[i])
}
}
@@ -139,6 +168,63 @@ func TestParseAssetsJSON(t *testing.T) {
}
}
func TestParseAssetsJSONExtendedMetadata(t *testing.T) {
created := "2024-01-01T00:00:00Z"
modified := "2024-01-02T00:00:00Z"
assets, total, err := ParseAssetsJSON(`{"assets":[{"id":"x1","filename":"IMG.HEIC","mediaType":"image","mediaSubtypes":["photoLive","photoHDR"],"sourceType":"userLibrary","playbackStyle":"livePhoto","pixelWidth":1,"pixelHeight":2,"creationDate":"2024-01-01T00:00:00Z","modificationDate":"2024-01-02T00:00:00Z","duration":3.5,"isFavorite":true,"isHidden":true,"hasAdjustments":true,"location":{"latitude":59.1,"longitude":18.2,"altitude":10,"horizontalAccuracy":5},"burstIdentifier":"burst","representsBurst":true,"burstSelectionTypes":["autoPick"],"adjustmentInfo":{"formatIdentifier":"fmt","formatVersion":"1","baseFilename":"base.heic"},"resources":[{"type":"adjustmentData","filename":"adj.plist","uti":"public.plist","local":true,"size":42}]}],"total":1}`)
if err != nil || total != 1 || len(assets) != 1 {
t.Fatalf("ParseAssetsJSON err=%v total=%d len=%d", err, total, len(assets))
}
a := assets[0]
if a.CreationDate == nil || *a.CreationDate != created || a.ModificationDate == nil || *a.ModificationDate != modified {
t.Fatalf("unexpected dates: %+v", a)
}
if a.Location == nil || a.Location.Latitude != 59.1 || a.Location.Longitude != 18.2 || !a.IsHidden || !a.HasAdjustments || !a.RepresentsBurst {
t.Fatalf("unexpected extended metadata: %+v", a)
}
if len(a.MediaSubtypes) != 2 || a.SourceType != "userLibrary" || a.PlaybackStyle != "livePhoto" || len(a.BurstSelectionTypes) != 1 {
t.Fatalf("unexpected type metadata: %+v", a)
}
if a.AdjustmentInfo == nil || a.AdjustmentInfo.FormatIdentifier != "fmt" || a.Resources[0].Size != 42 {
t.Fatalf("unexpected adjustment/resource metadata: %+v", a)
}
}
func TestParsePlacemarkJSON(t *testing.T) {
p, err := ParsePlacemarkJSON(`{"placemark":{"country":"Sweden","countryCode":"SE","locality":"Stockholm","formattedAddress":"Stockholm, Sweden","areasOfInterest":["Gamla stan"]}}`)
if err != nil {
t.Fatal(err)
}
if p.Country != "Sweden" || p.CountryCode != "SE" || p.Locality != "Stockholm" || len(p.AreasOfInterest) != 1 {
t.Fatalf("unexpected placemark: %+v", p)
}
if _, err := ParsePlacemarkJSON(`{"error":"geocode failed"}`); err == nil || err.Error() != "geocode failed" {
t.Fatalf("expected geocode error, got %v", err)
}
if _, err := ParsePlacemarkJSON(`bad`); err == nil {
t.Fatal("expected invalid JSON error")
}
}
func TestCgoBridgeReverseGeocode(t *testing.T) {
SetTestGeocodeJSON(`{"placemark":{"country":"Sweden","locality":"Stockholm"}}`)
p, err := (&CgoBridge{}).ReverseGeocode(59.3293, 18.0686)
if err != nil {
t.Fatal(err)
}
if p.Country != "Sweden" || p.Locality != "Stockholm" {
t.Fatalf("unexpected placemark: %+v", p)
}
SetTestGeocodeJSON(`{"error":"no network"}`)
if _, err := (&CgoBridge{}).ReverseGeocode(0, 0); err == nil || err.Error() != "no network" {
t.Fatalf("expected geocode error, got %v", err)
}
SetTestGeocodeNull()
if _, err := (&CgoBridge{}).ReverseGeocode(0, 0); err != errBridgeNil {
t.Fatalf("expected nil bridge error, got %v", err)
}
}
func TestParseTreeJSON(t *testing.T) {
tests := []struct {
name string
@@ -378,11 +464,11 @@ func TestErrBridgeNilMessage(t *testing.T) {
func TestParseExportResultJSON(t *testing.T) {
tests := []struct {
name string
json string
want ExportResult
wantErr bool
errMsg string
name string
json string
want ExportResult
wantErr bool
errMsg string
}{
{
name: "success",
@@ -438,3 +524,265 @@ func equalCollectionNode(a, b CollectionNode) bool {
}
return true
}
func equalAsset(a, b Asset) bool {
if a.ID != b.ID || a.Filename != b.Filename || a.Cloud != b.Cloud || a.MediaType != b.MediaType || a.PixelWidth != b.PixelWidth || a.PixelHeight != b.PixelHeight || a.Duration != b.Duration || a.IsFavorite != b.IsFavorite || a.HasAdjustments != b.HasAdjustments {
return false
}
if (a.CreationDate == nil) != (b.CreationDate == nil) {
return false
}
if a.CreationDate != nil && b.CreationDate != nil && *a.CreationDate != *b.CreationDate {
return false
}
if len(a.Resources) != len(b.Resources) {
return false
}
for i := range a.Resources {
if a.Resources[i] != b.Resources[i] {
return false
}
}
return true
}
func TestCgoBridgeExportPreviewViaStub(t *testing.T) {
SetTestExportPreviewJSON(`{"filename":"0001_img.jpg","size":2048,"cloud":"local"}`)
defer SetTestExportPreviewJSON(`{"filename":"0000_test.jpg","size":1024,"cloud":"local"}`)
bridge := &CgoBridge{}
result, err := bridge.ExportPreview("asset-1", "/tmp", 1024, 85, 0)
if err != nil {
t.Fatal(err)
}
if result.Filename != "0001_img.jpg" {
t.Errorf("got filename %q, want %q", result.Filename, "0001_img.jpg")
}
if result.Size != 2048 {
t.Errorf("got size %d, want %d", result.Size, 2048)
}
if result.Cloud != "local" {
t.Errorf("got cloud %q, want %q", result.Cloud, "local")
}
}
func TestCgoBridgeExportPreviewWithSlotViaStub(t *testing.T) {
SetTestExportPreviewJSON(`{"filename":"slot_img.jpg","size":4096,"cloud":"cloud"}`)
defer SetTestExportPreviewJSON(`{"filename":"0000_test.jpg","size":1024,"cloud":"local"}`)
bridge := &CgoBridge{}
result, err := bridge.ExportPreviewWithSlot("asset-1", "/tmp", 2048, 85, 0, 1)
if err != nil {
t.Fatal(err)
}
if result.Filename != "slot_img.jpg" {
t.Errorf("got filename %q", result.Filename)
}
if result.Cloud != "cloud" {
t.Errorf("got cloud %q, want %q", result.Cloud, "cloud")
}
}
func TestCgoBridgeExportOriginalViaStub(t *testing.T) {
SetTestExportOriginalJSON(`{"filename":"original.jpg","size":8192,"cloud":"local"}`)
defer SetTestExportOriginalJSON(`{"filename":"test.jpg","size":2048,"cloud":"cloud"}`)
bridge := &CgoBridge{}
result, err := bridge.ExportOriginal("asset-1", "/tmp", 0)
if err != nil {
t.Fatal(err)
}
if result.Filename != "original.jpg" {
t.Errorf("got filename %q, want %q", result.Filename, "original.jpg")
}
if result.Size != 8192 {
t.Errorf("got size %d, want %d", result.Size, 8192)
}
}
func TestCgoBridgeExportOriginalWithSlotViaStub(t *testing.T) {
SetTestExportOriginalJSON(`{"filename":"slot_orig.heic","size":16384,"cloud":"local"}`)
defer SetTestExportOriginalJSON(`{"filename":"test.jpg","size":2048,"cloud":"cloud"}`)
bridge := &CgoBridge{}
result, err := bridge.ExportOriginalWithSlot("asset-1", "/tmp", 0, 2)
if err != nil {
t.Fatal(err)
}
if result.Filename != "slot_orig.heic" {
t.Errorf("got filename %q", result.Filename)
}
}
func TestCgoBridgeExportPreviewErrorViaStub(t *testing.T) {
SetTestExportPreviewJSON(`{"error":"disk full","cloud":"local"}`)
defer SetTestExportPreviewJSON(`{"filename":"0000_test.jpg","size":1024,"cloud":"local"}`)
bridge := &CgoBridge{}
_, err := bridge.ExportPreview("asset-1", "/tmp", 1024, 85, 0)
if err == nil {
t.Fatal("expected error")
}
if err.Error() != "disk full" {
t.Errorf("got error %q, want %q", err.Error(), "disk full")
}
}
func TestCgoBridgeExportOriginalErrorViaStub(t *testing.T) {
SetTestExportOriginalJSON(`{"error":"write failed","cloud":"local"}`)
defer SetTestExportOriginalJSON(`{"filename":"test.jpg","size":2048,"cloud":"cloud"}`)
bridge := &CgoBridge{}
_, err := bridge.ExportOriginal("asset-1", "/tmp", 0)
if err == nil {
t.Fatal("expected error")
}
if err.Error() != "write failed" {
t.Errorf("got error %q, want %q", err.Error(), "write failed")
}
}
func TestCgoBridgeExportSkippedResult(t *testing.T) {
SetTestExportPreviewJSON(`{"filename":"skip.jpg","size":0,"cloud":"local","skipped":true}`)
defer SetTestExportPreviewJSON(`{"filename":"0000_test.jpg","size":1024,"cloud":"local"}`)
bridge := &CgoBridge{}
result, err := bridge.ExportPreview("asset-1", "/tmp", 1024, 85, 0)
if err != nil {
t.Fatal(err)
}
if !result.Skipped {
t.Error("expected Skipped to be true")
}
}
func TestCgoBridgeCancelAndIsCancelled(t *testing.T) {
bridge := &CgoBridge{}
if bridge.IsCancelled() {
t.Error("should not be cancelled initially")
}
bridge.Cancel()
if !bridge.IsCancelled() {
t.Error("should be cancelled after Cancel()")
}
}
func TestGetProgressSlotsReturnsSlots(t *testing.T) {
slots := GetProgressSlots()
if slots == nil {
t.Errorf("GetProgressSlots should return slots in test build")
}
if len(slots) != 16 {
t.Errorf("GetProgressSlots should return 16 slots, got %d", len(slots))
}
}
func TestResetProgressSlotsNoPanic(t *testing.T) {
ResetProgressSlots()
}
func TestGetProgressSlotCount(t *testing.T) {
count := GetProgressSlotCount()
if count != 16 {
t.Errorf("expected 16, got %d", count)
}
}
func TestCgoBridgeExportPreviewNilStub(t *testing.T) {
SetTestExportPreviewJSONNull()
defer SetTestExportPreviewJSON(`{"filename":"0000_test.jpg","size":1024,"cloud":"local"}`)
bridge := &CgoBridge{}
_, err := bridge.ExportPreview("asset-1", "/tmp", 1024, 85, 0)
if err == nil {
t.Fatal("expected errBridgeNil")
}
if err != errBridgeNil {
t.Errorf("got %v, want %v", err, errBridgeNil)
}
}
func TestCgoBridgeExportOriginalNilStub(t *testing.T) {
SetTestExportOriginalJSONNull()
defer SetTestExportOriginalJSON(`{"filename":"test.jpg","size":2048,"cloud":"cloud"}`)
bridge := &CgoBridge{}
_, err := bridge.ExportOriginal("asset-1", "/tmp", 0)
if err == nil {
t.Fatal("expected errBridgeNil")
}
if err != errBridgeNil {
t.Errorf("got %v, want %v", err, errBridgeNil)
}
}
func TestCgoBridgeExportPreviewWithSlotNilStub(t *testing.T) {
SetTestExportPreviewJSONNull()
defer SetTestExportPreviewJSON(`{"filename":"0000_test.jpg","size":1024,"cloud":"local"}`)
bridge := &CgoBridge{}
_, err := bridge.ExportPreviewWithSlot("asset-1", "/tmp", 1024, 85, 0, 0)
if err == nil {
t.Fatal("expected errBridgeNil")
}
if err != errBridgeNil {
t.Errorf("got %v, want %v", err, errBridgeNil)
}
}
func TestCgoBridgeExportOriginalWithSlotNilStub(t *testing.T) {
SetTestExportOriginalJSONNull()
defer SetTestExportOriginalJSON(`{"filename":"test.jpg","size":2048,"cloud":"cloud"}`)
bridge := &CgoBridge{}
_, err := bridge.ExportOriginalWithSlot("asset-1", "/tmp", 0, 0)
if err == nil {
t.Fatal("expected errBridgeNil")
}
if err != errBridgeNil {
t.Errorf("got %v, want %v", err, errBridgeNil)
}
}
func TestGetProgressSlotsWithActiveSlot(t *testing.T) {
ResetProgressSlots()
slots := GetProgressSlots()
if len(slots) != 16 {
t.Errorf("expected 16 slots, got %d", len(slots))
}
for i, s := range slots {
if s.Active {
t.Errorf("slot %d should not be active after reset", i)
}
}
}
func TestResetProgressSlotsClearsState(t *testing.T) {
ResetProgressSlots()
slots := GetProgressSlots()
if len(slots) > 0 && slots[0].Active {
t.Errorf("slot should be inactive after reset")
}
}
func TestGetProgressSlotsZeroCount(t *testing.T) {
SetTestProgressSlotCount(0)
defer SetTestProgressSlotCount(3)
slots := GetProgressSlots()
if slots != nil {
t.Errorf("expected nil with zero count, got %v", slots)
}
}
func TestGetProgressSlotsNullPointer(t *testing.T) {
SetTestProgressSlotsNull(1)
defer SetTestProgressSlotsNull(0)
slots := GetProgressSlots()
if slots != nil {
t.Errorf("expected nil with null pointer, got %v", slots)
}
}
func TestExportProgressSlotType(t *testing.T) {
slot := ExportProgressSlot{
Active: true,
Progress: 0.75,
BytesDone: 512,
BytesTotal: 1024,
}
if !slot.Active {
t.Error("expected Active to be true")
}
if slot.Progress != 0.75 {
t.Errorf("expected Progress 0.75, got %f", slot.Progress)
}
}
+64 -2
View File
@@ -6,15 +6,77 @@ type Album struct {
}
type Asset struct {
ID string `json:"id"`
ID string `json:"id"`
Filename string `json:"filename"`
Cloud string `json:"cloud"`
MediaType string `json:"mediaType"`
MediaSubtypes []string `json:"mediaSubtypes,omitempty"`
SourceType string `json:"sourceType,omitempty"`
PlaybackStyle string `json:"playbackStyle,omitempty"`
PixelWidth int `json:"pixelWidth"`
PixelHeight int `json:"pixelHeight"`
CreationDate *string `json:"creationDate,omitempty"`
ModificationDate *string `json:"modificationDate,omitempty"`
Duration float64 `json:"duration,omitempty"`
IsFavorite bool `json:"isFavorite,omitempty"`
IsHidden bool `json:"isHidden,omitempty"`
HasAdjustments bool `json:"hasAdjustments,omitempty"`
Location *AssetLocation `json:"location,omitempty"`
BurstIdentifier string `json:"burstIdentifier,omitempty"`
RepresentsBurst bool `json:"representsBurst,omitempty"`
BurstSelectionTypes []string `json:"burstSelectionTypes,omitempty"`
AdjustmentInfo *AdjustmentInfo `json:"adjustmentInfo,omitempty"`
Resources []AssetResource `json:"resources,omitempty"`
}
type AssetLocation struct {
Latitude float64 `json:"latitude"`
Longitude float64 `json:"longitude"`
Altitude float64 `json:"altitude,omitempty"`
HorizontalAccuracy float64 `json:"horizontalAccuracy,omitempty"`
}
type AdjustmentInfo struct {
FormatIdentifier string `json:"formatIdentifier,omitempty"`
FormatVersion string `json:"formatVersion,omitempty"`
BaseFilename string `json:"baseFilename,omitempty"`
}
type AssetResource struct {
Type string `json:"type"`
Filename string `json:"filename"`
Cloud string `json:"cloud"`
UTI string `json:"uti"`
Local bool `json:"local"`
Size int64 `json:"size,omitempty"`
}
type Placemark struct {
Name string `json:"name,omitempty"`
Country string `json:"country,omitempty"`
CountryCode string `json:"countryCode,omitempty"`
AdministrativeArea string `json:"administrativeArea,omitempty"`
SubAdministrativeArea string `json:"subAdministrativeArea,omitempty"`
Locality string `json:"locality,omitempty"`
SubLocality string `json:"subLocality,omitempty"`
Thoroughfare string `json:"thoroughfare,omitempty"`
SubThoroughfare string `json:"subThoroughfare,omitempty"`
PostalCode string `json:"postalCode,omitempty"`
FormattedAddress string `json:"formattedAddress,omitempty"`
InlandWater string `json:"inlandWater,omitempty"`
Ocean string `json:"ocean,omitempty"`
AreasOfInterest []string `json:"areasOfInterest,omitempty"`
}
type PlacemarkResponse struct {
Placemark Placemark `json:"placemark"`
Error string `json:"error,omitempty"`
}
type ExportResult struct {
Filename string `json:"filename"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Skipped bool `json:"skipped,omitempty"`
Error string `json:"error,omitempty"`
}