Compare commits

15 Commits

Author SHA1 Message Date
Ein Anderssono cf922fee03 gitignore: add ROADMAP.md
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 08:43:36 +02:00
Ein Anderssono c9ac014473 v0.10.0: ports and adapters refactor
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
- Extract shared manifest types into internal/manifest/types leaf package.
- Extract SQLite adapter into internal/manifest/sqlite.
- Extract JSONL adapter into internal/manifest/jsonl.
- Isolate modernc.org/sqlite import to sqlite/adapter.go.
- Add adapter-backed registry with manifest.Default.
- Adapter-agnostic ConvertManifest in types/.
- MemoryAdapter for in-memory manifest testing.
- CLI uses manifest.Default registry directly.
- SQLite LogWriter type assertion moved into SQLiteAdapter.
- Manifest interface includes Entries(); EntryReader removed.
- No behavior changes. 100% coverage across all 6 packages.
2026-06-15 08:27:38 +02:00
Ein Anderssono 9cd048d9f3 v0.9.4: add doctor diagnostics
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:47:44 +02:00
Ein Anderssono 98320c8235 v0.9.3: add cleanup command
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:40:12 +02:00
Ein Anderssono 84e70bda76 v0.9.2: add manifest repair
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:32:20 +02:00
Ein Anderssono bed3ea7bd0 v0.9.1: add deep checksum verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:23:22 +02:00
Ein Anderssono 7555b561bd v0.9.0: add manifest checksums
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:19:47 +02:00
Ein Anderssono d909d30b87 v0.8.7: add JSON sidecars
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:11:02 +02:00
Ein Anderssono 5c40b1d3ba v0.8.6: add XMP keyword and rating controls
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:05:14 +02:00
Ein Anderssono 700d8ef05a v0.8.5: add XMP privacy controls
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:02:26 +02:00
Ein Anderssono fbc37b8d8d v0.8.4: add strict sidecar verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:58:38 +02:00
Ein Anderssono 32a5819c86 v0.8.3: add sidecar inspection
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 01:55:31 +02:00
Ein Anderssono a51db37fdb v0.8.2: add metadata-only sidecars
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:48:32 +02:00
Ein Anderssono 9cd702628d v0.8.1: improve XMP sidecars
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:36:04 +02:00
Ein Anderssono fffb30023b v0.8.0: enrich XMP metadata
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:21:49 +02:00
40 changed files with 4563 additions and 1025 deletions
+2 -1
View File
@@ -1,4 +1,5 @@
bin/ bin/
bridge/*.o bridge/*.o
bridge/*.a bridge/*.a
coverage.out coverage.out
ROADMAP.md
+5
View File
@@ -12,4 +12,9 @@
- Manifest backends: JSONL default, SQLite optional via `modernc.org/sqlite`. - Manifest backends: JSONL default, SQLite optional via `modernc.org/sqlite`.
- Preserve manifest compatibility and migration behavior. - Preserve manifest compatibility and migration behavior.
- XMP sidecars are opt-in via `--sidecar xmp`; default must remain `none`. - XMP sidecars are opt-in via `--sidecar xmp`; default must remain `none`.
- Reverse geocoding is opt-in via `--reverse-geocode`, uses MapKit on macOS 26+, and must fail safely on older macOS without failing export.
- Do not claim Photos people/animal/object labels are exported unless Vision/Core ML analysis is explicitly implemented.
- Roadmap: keep `0.8.x` focused on making XMP sidecars as rich, standard-compatible, and complete as possible from public PhotoKit metadata.
- Roadmap: use `0.9.0` through `0.9.5` for checksums, deep verification, manifest repair, cleanup, doctor, and backup-integrity hardening.
- Roadmap: start Vision/Core ML analysis only after `0.9.5`; keep it opt-in and label it as photoscli-generated analysis, not Apple Photos metadata.
- Do not commit generated artifacts from `bin/` or coverage files. - Do not commit generated artifacts from `bin/` or coverage files.
+126
View File
@@ -2,6 +2,132 @@
This changelog is maintained from git history plus the published Gitea release series. Future releases should update this file and publish matching release notes on the release page. This changelog is maintained from git history plus the published Gitea release series. Future releases should update this file and publish matching release notes on the release page.
## v0.10.0
Ports and adapters refactor.
- Extract shared manifest types into `internal/manifest/types` leaf package.
- Extract SQLite adapter into `internal/manifest/sqlite`: Store, Adapter, LogWriter.
- Extract JSONL adapter into `internal/manifest/jsonl`: Store, Adapter.
- `modernc.org/sqlite` import isolated to `internal/manifest/sqlite/adapter.go`.
- Registry pattern: `manifest.Default` provides adapter-backed `ParseFormat`/`Open`/`OpenLogWriter`.
- Adapter-agnostic `ConvertManifest` in `types/` for JSONL↔SQLite conversion.
- `MemoryAdapter` for in-memory manifest testing.
- CLI uses `manifest.Default` registry directly; zero concrete adapter references.
- SQLite `LogWriter` type assertion moved from central code into `SQLiteAdapter`.
- `Manifest` interface now includes `Entries()`; `EntryReader` removed.
- No behavior changes, no new features.
- 100% statement coverage across all 6 packages.
## v0.9.4
Doctor release.
- Add `doctor` to check Photos access and optional backup health.
- Add `doctor --out <dir>` to report backup directory, manifest entries, and failure count.
- Add `doctor --json` for scriptable diagnostics.
## v0.9.3
Cleanup release.
- Add `cleanup --out <dir>` to remove files not referenced by the manifest.
- Add `cleanup --dry-run` to preview orphaned files before deletion.
- Preserve manifest/log/failure files, `.photoscli`, and sidecars for manifest-referenced media.
## v0.9.2
Manifest repair release.
- Add `manifest repair --out <dir>` to fill missing manifest size metadata from files on disk.
- Add `manifest repair --checksum sha256` to fill missing manifest checksums.
- Add `manifest repair --dry-run` to preview repairs without writing manifest updates.
## v0.9.1
Deep checksum verification release.
- Add `verify --deep` to recompute SHA-256 checksums for manifest entries that have checksum metadata.
- Report `checksum-mismatch` when disk contents differ from the manifest checksum.
- Keep normal `verify` unchanged unless `--deep` is selected.
## v0.9.0
Manifest checksum release.
- Add opt-in `--checksum sha256` to store SHA-256 file checksums in JSONL and SQLite manifests.
- Add SQLite migration support for the manifest `checksum` column.
- Keep checksum collection disabled by default with `--checksum none`.
## v0.8.7
JSON sidecar release.
- Add `--sidecar json` for structured JSON metadata sidecars.
- Add `--sidecar xmp,json` to write both XMP and JSON sidecars from the same metadata.
- Keep `--sidecar none` as the default.
## v0.8.6
XMP keyword and rating controls release.
- Add `--xmp-keywords album-path|album|none` for generated sidecars.
- Add `--xmp-rating favorite|none` for generated sidecars.
- Keep existing keyword/rating behavior as defaults with `album-path` and `favorite`.
## v0.8.5
XMP privacy controls release.
- Add `--xmp-privacy keep|strip-location|strip-address` for generated sidecars.
- Keep existing XMP location/address behavior as the default with `keep`.
- Allow GPS coordinates to be kept while reverse-geocoded address fields are omitted with `strip-address`.
- Allow both GPS coordinates and address fields to be omitted with `strip-location`.
## v0.8.4
Strict XMP sidecar verification release.
- Add `verify --sidecar --strict` to require photoscli XMP schema metadata, sidecar generator metadata, and matching exported filename metadata.
- Keep existing `verify --sidecar` behavior unchanged for backup-wide existence, readability, and asset-ID checks.
## v0.8.3
XMP sidecar inspection release.
- Add `sidecar inspect <file.xmp>` to print key photoscli metadata from generated XMP sidecars.
- Add `sidecar inspect <file.xmp> --json` for scriptable inspection output.
## v0.8.2
Metadata-only XMP refresh release.
- Add `--metadata-only` for manifest-based XMP sidecar generation without re-exporting media files.
- Support metadata-only refresh for both `export` and `backup-all` when used with `--sidecar xmp`.
- Require a manifest for metadata-only mode so existing media paths are resolved safely.
- Keep media files untouched while overwriting/regenerating generated XMP sidecars.
- Support `--reverse-geocode` during metadata-only sidecar refresh.
## v0.8.1
XMP standards and sidecar verification release.
- Add XMP schema version metadata for generated sidecars.
- Add standard XMP fields for rating, metadata date, creation date, Photoshop date created, and EXIF GPS coordinates.
- Add `dc:subject` keywords from album/folder context.
- Add sidecar generator and generated timestamp metadata.
- Add `verify --sidecar` to check missing, zero-byte, unreadable, and asset-ID mismatched XMP sidecars.
## v0.8.0
Rich PhotoKit metadata and reverse-geocoded XMP release.
- Expand asset metadata from public PhotoKit fields: modification date, hidden state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment state, adjustment info, and richer resource metadata.
- Expand XMP sidecars with duration, hidden/adjustment state, modification date, structured media subtype/burst/resource lists, GPS coordinates, and adjustment metadata.
- Add opt-in `--reverse-geocode` using Apple reverse geocoding APIs to add address metadata for GPS assets.
- Cache reverse-geocode results under `.photoscli/geocode-cache.jsonl` so repeated exports do not repeatedly query Apple geocoding.
- Keep Vision/Core ML people, animal, object, and scene analysis out of this release; that remains future work.
## v0.7.0 ## v0.7.0
XMP sidecar metadata release. XMP sidecar metadata release.
+2 -2
View File
@@ -1,6 +1,6 @@
BINARY := ./bin/photoscli BINARY := ./bin/photoscli
MODULE := gitea.k3s.k0.nu/tools/photocli MODULE := gitea.k3s.k0.nu/tools/photocli
VERSION := 0.7.0 VERSION := 0.10.0
RELEASE_ZIP := ./bin/photoscli-$(VERSION)-macos-arm64.zip RELEASE_ZIP := ./bin/photoscli-$(VERSION)-macos-arm64.zip
RELEASE_NOTES := RELEASE_NOTES.md RELEASE_NOTES := RELEASE_NOTES.md
BRIDGE_DIR := bridge BRIDGE_DIR := bridge
@@ -20,7 +20,7 @@ $(LIB): $(OBJ)
ar rcs $@ $< ar rcs $@ $<
$(OBJ): $(BRIDGE_DIR)/photokit_bridge.m $(BRIDGE_DIR)/photokit_bridge.h $(OBJ): $(BRIDGE_DIR)/photokit_bridge.m $(BRIDGE_DIR)/photokit_bridge.h
cc -c -x objective-c -fobjc-arc -framework Photos -framework Foundation -o $@ $< cc -c -x objective-c -fobjc-arc -o $@ $<
$(STUB_LIB): $(STUB_OBJ) $(STUB_LIB): $(STUB_OBJ)
ar rcs $@ $< ar rcs $@ $<
+55 -1
View File
@@ -20,8 +20,12 @@ For a practical step-by-step manual with recommended backup workflows, recovery
- Failure tracking with `failures.jsonl` and `retry-failed`. - Failure tracking with `failures.jsonl` and `retry-failed`.
- Deduplicated failure management with `failures list`, `failures clear`, and `retry-failed --clear-on-success`. - Deduplicated failure management with `failures list`, `failures clear`, and `retry-failed --clear-on-success`.
- Verification, reporting, and diff commands for backup integrity. - Verification, reporting, and diff commands for backup integrity.
- Optional XMP sidecar verification with `verify --sidecar`.
- Optional SHA-256 manifest checksums with `--checksum sha256`.
- Metadata-only XMP refresh for manifest-backed exports with `--metadata-only`.
- Status command for quick backup summaries. - Status command for quick backup summaries.
- Opt-in XMP sidecar metadata with `--sidecar xmp`. - Opt-in rich XMP sidecar metadata with `--sidecar xmp`.
- Optional Apple MapKit reverse geocoding for GPS assets on macOS 26+ with `--reverse-geocode`.
- Script-friendly exit codes and optional JSON summaries. - Script-friendly exit codes and optional JSON summaries.
- 100% test coverage for the Go CLI and parsing layers. - 100% test coverage for the Go CLI and parsing layers.
@@ -113,6 +117,10 @@ Verify a backup later:
```bash ```bash
./bin/photoscli verify --out ./photos-backup --manifest sqlite ./bin/photoscli verify --out ./photos-backup --manifest sqlite
./bin/photoscli verify --out ./photos-backup --deep
./bin/photoscli manifest repair --out ./photos-backup --checksum sha256 --dry-run
./bin/photoscli cleanup --out ./photos-backup --dry-run
./bin/photoscli doctor --out ./photos-backup
``` ```
## Commands ## Commands
@@ -166,6 +174,7 @@ photoscli export --album-id "Favorites" --out ./favorites --only-favorites
photoscli export --album-id "Videos" --out ./videos --media videos --include-videos photoscli export --album-id "Videos" --out ./videos --media videos --include-videos
photoscli export --album-id "Archive" --out ./archive --originals --retry 3 photoscli export --album-id "Archive" --out ./archive --originals --retry 3
photoscli export --album-id "Vacation" --out ./vacation --date-template YYYY/MM/DD photoscli export --album-id "Vacation" --out ./vacation --date-template YYYY/MM/DD
photoscli export --album-id "Vacation" --out ./vacation --sidecar xmp --reverse-geocode
``` ```
### `backup-all` ### `backup-all`
@@ -182,10 +191,12 @@ Useful examples:
```bash ```bash
photoscli backup-all --out ./backup --manifest sqlite --log photoscli backup-all --out ./backup --manifest sqlite --log
photoscli backup-all --out ./backup --checksum sha256
photoscli backup-all --out ./backup --exclude-album "Recently Deleted" --exclude-album "Temp*" photoscli backup-all --out ./backup --exclude-album "Recently Deleted" --exclude-album "Temp*"
photoscli backup-all --out ./backup --since 2024-01-01 --sort newest photoscli backup-all --out ./backup --since 2024-01-01 --sort newest
photoscli backup-all --out ./backup --concurrency 8 --retry 3 photoscli backup-all --out ./backup --concurrency 8 --retry 3
photoscli backup-all --out ./backup --dry-run --json photoscli backup-all --out ./backup --dry-run --json
photoscli backup-all --out ./backup --sidecar xmp --reverse-geocode
``` ```
### `report` ### `report`
@@ -266,6 +277,7 @@ Common flags for `export` and `backup-all`:
- `--dry-run`: print planned exports without writing files, manifests, or logs. - `--dry-run`: print planned exports without writing files, manifests, or logs.
- `--json`: print a machine-readable summary to stdout. - `--json`: print a machine-readable summary to stdout.
- `--verify`: run manifest/file verification after export. - `--verify`: run manifest/file verification after export.
- `--sidecar` with `verify`: also verify expected `.xmp` sidecars.
- `--log`: enable structured export logging. - `--log`: enable structured export logging.
- `--manifest jsonl|sqlite`: choose manifest backend, default `jsonl`. - `--manifest jsonl|sqlite`: choose manifest backend, default `jsonl`.
- `--no-manifest`: disable manifest reads/writes. - `--no-manifest`: disable manifest reads/writes.
@@ -278,6 +290,8 @@ Common flags for `export` and `backup-all`:
- `--max-size <n>`: filter by estimated pixel count. - `--max-size <n>`: filter by estimated pixel count.
- `--format jpeg|heic|png`: preview format hint. Current bridge output is still the existing preview path; non-JPEG bridge output is future work. - `--format jpeg|heic|png`: preview format hint. Current bridge output is still the existing preview path; non-JPEG bridge output is future work.
- `--sidecar none|xmp`: write opt-in XMP metadata sidecars next to exported files. - `--sidecar none|xmp`: write opt-in XMP metadata sidecars next to exported files.
- `--metadata-only`: with `--sidecar xmp`, refresh XMP sidecars for manifest-backed files without exporting media.
- `--reverse-geocode`: with `--sidecar xmp`, add cached Apple MapKit address metadata for GPS assets on macOS 26+.
- `--date-template <template>`: append date folders based on creation date, for example `YYYY/MM/DD`. - `--date-template <template>`: append date folders based on creation date, for example `YYYY/MM/DD`.
`backup-all` also supports: `backup-all` also supports:
@@ -336,6 +350,8 @@ Write archival metadata sidecars with:
```bash ```bash
photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json
``` ```
Sidecars are opt-in and use the exported file basename: Sidecars are opt-in and use the exported file basename:
@@ -343,10 +359,42 @@ Sidecars are opt-in and use the exported file basename:
```text ```text
IMG_0001.jpg -> IMG_0001.xmp IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json
``` ```
The XMP contains photoscli metadata such as asset ID, filenames, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, version, exported timestamp, size, and creation date when available. If `--sidecar xmp` is explicitly selected and the sidecar cannot be written, that asset is treated as failed. The XMP contains photoscli metadata such as asset ID, filenames, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, version, exported timestamp, size, and creation date when available. If `--sidecar xmp` is explicitly selected and the sidecar cannot be written, that asset is treated as failed.
Sidecars also include richer public PhotoKit metadata where available: modification date, duration, hidden state, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP dates, EXIF GPS coordinates, favorite rating, and album/folder keywords. Add `--reverse-geocode` to include cached address fields from Apple MapKit for assets with GPS coordinates. Reverse geocoding requires macOS 26 or newer; on older macOS versions the export continues and XMP still includes GPS coordinates.
Control XMP location metadata with `--xmp-privacy keep|strip-location|strip-address`. The default is `keep`. Use `strip-address` to omit reverse-geocoded address fields while keeping GPS coordinates, or `strip-location` to omit both GPS and address fields.
Control generated XMP keywords and ratings with `--xmp-keywords album-path|album|none` and `--xmp-rating favorite|none`. Defaults preserve existing behavior: album/folder keywords and favorite assets mapped to `xmp:Rating="5"`.
Verify generated sidecars with:
```bash
photoscli verify --out ./backup --sidecar
```
For stricter checks against recent photoscli-generated XMP sidecars:
```bash
photoscli verify --out ./backup --sidecar --strict
```
Inspect one generated sidecar with:
```bash
photoscli sidecar inspect ./backup/IMG_0001.xmp
photoscli sidecar inspect ./backup/IMG_0001.xmp --json
```
Refresh sidecars for files already present in a manifest-backed export without rewriting media files:
```bash
photoscli backup-all --out ./backup --sidecar xmp --metadata-only
```
## Failure Tracking ## Failure Tracking
Failed exports are deduplicated by asset ID and stored in: Failed exports are deduplicated by asset ID and stored in:
@@ -458,3 +506,9 @@ Objective-C returns JSON to Go. Tests use the stub bridge and do not require rea
- Album title lookup uses exact title matching. Use PhotoKit local identifiers when names are ambiguous. - Album title lookup uses exact title matching. Use PhotoKit local identifiers when names are ambiguous.
- iCloud-backed assets may trigger downloads and can fail due to network or account state. - iCloud-backed assets may trigger downloads and can fail due to network or account state.
- `--min-size` and `--max-size` currently use estimated pixel count from dimensions, not encoded file size. - `--min-size` and `--max-size` currently use estimated pixel count from dimensions, not encoded file size.
## Roadmap
- `0.8.x`: make XMP sidecars as rich, standard-compatible, and complete as possible from public PhotoKit metadata.
- `0.9.0` through `0.9.5`: checksums, deep verification, manifest repair, cleanup, doctor, and backup-integrity hardening.
- After `0.9.5`: opt-in Vision/Core ML analysis for photoscli-generated face/object/animal/scene metadata.
+14 -9
View File
@@ -1,20 +1,25 @@
# v0.7.0 # v0.10.0
This release adds opt-in XMP sidecar metadata for archival exports. Ports and adapters refactor. No user-visible changes.
## Highlights ## Highlights
- Add `--sidecar none|xmp` with default `none`. - Extract manifest types, registry, and conversion logic into `internal/manifest/types`.
- Write XMP sidecars next to exported files when `--sidecar xmp` is selected. - Extract SQLite adapter into `internal/manifest/sqlite` with its own store, adapter, and log writer.
- XMP files use the exported file basename, for example `IMG_0001.jpg` -> `IMG_0001.xmp`. - Extract JSONL adapter into `internal/manifest/jsonl` with its own store and adapter.
- Sidecars include photoscli metadata such as asset ID, filenames, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, version, exported timestamp, size, and creation date when available. - Isolate `modernc.org/sqlite` import to the SQLite adapter package only.
- XMP writes are atomic and fail the asset when explicitly requested sidecar output cannot be written. - Replace central `Open`/`ParseFormat`/`OpenLogWriter` with adapter-backed registry.
- Config files can set `sidecar = "xmp"`. - Adapter-agnostic manifest conversion via `ConvertManifest`.
- SQLite log writer selection moved from central code into `SQLiteAdapter`.
- CLI uses `manifest.Default` registry; no direct references to concrete JSONL or SQLite types.
- `MemoryAdapter` available for in-memory manifest tests.
- 100% statement coverage across all 6 packages.
- No behavior changes.
## Assets ## Assets
- `photoscli`: Apple Silicon macOS binary (`darwin/arm64`). - `photoscli`: Apple Silicon macOS binary (`darwin/arm64`).
- `photoscli-0.7.0-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG. - `photoscli-0.10.0-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
- `USERGUIDE.md`: standalone user guide. - `USERGUIDE.md`: standalone user guide.
Intel Macs are not currently a supported release target. Intel Macs are not currently a supported release target.
+110 -1
View File
@@ -21,8 +21,10 @@ It is especially useful when you want to:
- Retry transient iCloud failures. - Retry transient iCloud failures.
- Verify that files referenced by a manifest exist on disk. - Verify that files referenced by a manifest exist on disk.
- Detect missing, zero-byte, and size-mismatched manifest files. - Detect missing, zero-byte, and size-mismatched manifest files.
- Store optional SHA-256 checksums in manifests.
- Inspect or clear deduplicated failure records. - Inspect or clear deduplicated failure records.
- Write optional XMP sidecar metadata for archival workflows. - Write optional XMP sidecar metadata for archival workflows.
- Add optional reverse-geocoded address metadata to XMP sidecars for GPS assets.
- Script Photos exports with stable exit codes. - Script Photos exports with stable exit codes.
It is not intended to replace Apple Photos, iCloud Photos, or Time Machine. Think of it as an additional file-based export and backup tool. It is not intended to replace Apple Photos, iCloud Photos, or Time Machine. Think of it as an additional file-based export and backup tool.
@@ -545,6 +547,8 @@ Use XMP sidecars when you want portable metadata next to exported files:
```bash ```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp ./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json
``` ```
Sidecars are disabled by default. When enabled, they use the exported file basename: Sidecars are disabled by default. When enabled, they use the exported file basename:
@@ -552,9 +556,69 @@ Sidecars are disabled by default. When enabled, they use the exported file basen
```text ```text
IMG_0001.jpg -> IMG_0001.xmp IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json
``` ```
The XMP includes photoscli archive metadata such as asset ID, original filename, exported filename, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, version, exported time, size, and creation date when available. Use XMP for standards-oriented metadata workflows and JSON when you want structured photoscli metadata for scripts or audits.
The XMP includes photoscli archive metadata such as asset ID, original filename, exported filename, album, manifest path, media type, dimensions, favorite state, hidden state, cloud state, export mode, version, exported time, size, creation date, modification date, duration, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP date fields, EXIF GPS fields, favorite rating, and album/folder keywords when PhotoKit exposes them.
Control location privacy in generated sidecars:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-address
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-location
```
`keep` is the default. `strip-address` omits reverse-geocoded address fields while preserving GPS coordinates. `strip-location` omits both GPS coordinates and address fields.
Control generated keywords and ratings:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords album
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords none --xmp-rating none
```
`--xmp-keywords album-path` is the default and writes album/folder keywords. `album` writes only the album name. `none` omits generated `dc:subject` keywords. `--xmp-rating favorite` is the default and maps favorite assets to `xmp:Rating="5"`; `none` omits that generated rating.
For address metadata from GPS coordinates, opt in to Apple's reverse geocoder:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --reverse-geocode
```
Reverse geocoding uses Apple MapKit and requires macOS 26 or newer. It can require network access and may be rate-limited by Apple, so results are cached in `.photoscli/geocode-cache.jsonl` under the backup root. On older macOS versions, `--reverse-geocode` is treated as unavailable: the export continues, no address fields are added, and the XMP still contains GPS coordinates.
Verify sidecars after an export:
```bash
./bin/photoscli verify --out ./PhotosBackup --sidecar
```
This reports missing, zero-byte, unreadable, or asset-ID mismatched `.xmp` files.
Use strict verification for sidecars generated by recent photoscli versions:
```bash
./bin/photoscli verify --out ./PhotosBackup --sidecar --strict
```
Strict mode also checks photoscli schema metadata, generator metadata, and the exported filename recorded inside the sidecar.
Inspect one generated sidecar when troubleshooting or scripting:
```bash
./bin/photoscli sidecar inspect ./PhotosBackup/IMG_0001.xmp
./bin/photoscli sidecar inspect ./PhotosBackup/IMG_0001.xmp --json
```
Refresh metadata only for an existing manifest-backed backup:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --sidecar xmp --metadata-only
```
Metadata-only mode does not re-export media files. It uses manifest paths to find existing files and rewrites generated `.xmp` sidecars next to them. It requires `--sidecar xmp` and an enabled manifest.
If you explicitly request `--sidecar xmp` and the XMP file cannot be written, the asset is counted as failed. If you explicitly request `--sidecar xmp` and the XMP file cannot be written, the asset is counted as failed.
@@ -578,6 +642,7 @@ media = "photos"
retry = 3 retry = 3
log = true log = true
sidecar = "xmp" sidecar = "xmp"
reverse-geocode = true
``` ```
Use a custom config path: Use a custom config path:
@@ -757,6 +822,50 @@ Use structured logs:
./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log ./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log
``` ```
Store SHA-256 checksums in the manifest when exporting:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --checksum sha256
```
Checksums are opt-in and stored in JSONL or SQLite manifests. The default is `--checksum none`.
Deep-verify checksum-backed manifests:
```bash
./bin/photoscli verify --out ./PhotosBackup --deep
```
`--deep` recomputes SHA-256 only for manifest entries that already have checksum metadata.
Repair missing manifest metadata from files already present on disk:
```bash
./bin/photoscli manifest repair --out ./PhotosBackup --checksum sha256 --dry-run
./bin/photoscli manifest repair --out ./PhotosBackup --checksum sha256
```
Repair fills missing size/checksum metadata and skips missing, zero-byte, or unreadable files.
Preview and remove files not referenced by the manifest:
```bash
./bin/photoscli cleanup --out ./PhotosBackup --dry-run
./bin/photoscli cleanup --out ./PhotosBackup
```
Cleanup preserves manifest/log/failure files, `.photoscli`, and sidecars next to manifest-referenced media.
Run read-only diagnostics:
```bash
./bin/photoscli doctor
./bin/photoscli doctor --out ./PhotosBackup
./bin/photoscli doctor --out ./PhotosBackup --json
```
Doctor checks Photos access and, when `--out` is provided, backup directory status, manifest entries, and failure count.
## Safe Operating Practices ## Safe Operating Practices
- Run `--dry-run` before the first large backup. - Run `--dry-run` before the first large backup.
+2
View File
@@ -20,6 +20,8 @@ char *photos_list_albums_json(void);
char *photos_list_assets_json(const char *album_id); char *photos_list_assets_json(const char *album_id);
char *photos_reverse_geocode_json(double latitude, double longitude);
char *photos_export_preview_json( char *photos_export_preview_json(
const char *asset_id, const char *asset_id,
const char *output_dir, const char *output_dir,
+140 -3
View File
@@ -1,6 +1,8 @@
#import <Foundation/Foundation.h> #import <Foundation/Foundation.h>
#import <AppKit/AppKit.h> #import <AppKit/AppKit.h>
#import <Photos/Photos.h> #import <Photos/Photos.h>
#import <CoreLocation/CoreLocation.h>
#import <MapKit/MapKit.h>
#import <UniformTypeIdentifiers/UniformTypeIdentifiers.h> #import <UniformTypeIdentifiers/UniformTypeIdentifiers.h>
#import <objc/message.h> #import <objc/message.h>
#import "photokit_bridge.h" #import "photokit_bridge.h"
@@ -311,6 +313,87 @@ static NSString *media_type_string(PHAssetMediaType type) {
} }
} }
static NSString *source_type_string(PHAssetSourceType type) {
NSMutableArray *parts = [NSMutableArray array];
if (type & PHAssetSourceTypeUserLibrary) [parts addObject:@"userLibrary"];
if (type & PHAssetSourceTypeCloudShared) [parts addObject:@"cloudShared"];
if (type & PHAssetSourceTypeiTunesSynced) [parts addObject:@"iTunesSynced"];
return parts.count > 0 ? [parts componentsJoinedByString:@","] : @"unknown";
}
static NSString *playback_style_string(PHAssetPlaybackStyle style) {
switch (style) {
case PHAssetPlaybackStyleImage: return @"image";
case PHAssetPlaybackStyleImageAnimated: return @"imageAnimated";
case PHAssetPlaybackStyleLivePhoto: return @"livePhoto";
case PHAssetPlaybackStyleVideo: return @"video";
case PHAssetPlaybackStyleVideoLooping: return @"videoLooping";
default: return @"unknown";
}
}
static NSArray<NSString *> *media_subtype_strings(PHAssetMediaSubtype subtypes) {
NSMutableArray *parts = [NSMutableArray array];
if (subtypes & PHAssetMediaSubtypePhotoPanorama) [parts addObject:@"photoPanorama"];
if (subtypes & PHAssetMediaSubtypePhotoHDR) [parts addObject:@"photoHDR"];
if (subtypes & PHAssetMediaSubtypePhotoScreenshot) [parts addObject:@"photoScreenshot"];
if (subtypes & PHAssetMediaSubtypePhotoLive) [parts addObject:@"photoLive"];
if (subtypes & PHAssetMediaSubtypePhotoDepthEffect) [parts addObject:@"photoDepthEffect"];
if (subtypes & PHAssetMediaSubtypeVideoStreamed) [parts addObject:@"videoStreamed"];
if (subtypes & PHAssetMediaSubtypeVideoHighFrameRate) [parts addObject:@"videoHighFrameRate"];
if (subtypes & PHAssetMediaSubtypeVideoTimelapse) [parts addObject:@"videoTimelapse"];
return parts;
}
static NSArray<NSString *> *burst_selection_type_strings(PHAssetBurstSelectionType types) {
NSMutableArray *parts = [NSMutableArray array];
if (types & PHAssetBurstSelectionTypeAutoPick) [parts addObject:@"autoPick"];
if (types & PHAssetBurstSelectionTypeUserPick) [parts addObject:@"userPick"];
return parts;
}
static NSNumber *resource_file_size(PHAssetResource *res) {
@try {
id value = [res valueForKey:@"fileSize"];
if ([value respondsToSelector:@selector(longLongValue)]) {
return @([value longLongValue]);
}
} @catch (NSException *e) {}
return nil;
}
static NSDictionary *location_dict(CLLocation *location) {
if (!location) return nil;
NSMutableDictionary *dict = [NSMutableDictionary dictionary];
dict[@"latitude"] = @(location.coordinate.latitude);
dict[@"longitude"] = @(location.coordinate.longitude);
dict[@"altitude"] = @(location.altitude);
dict[@"horizontalAccuracy"] = @(location.horizontalAccuracy);
return dict;
}
static NSDictionary *editing_input_info(PHAsset *asset) {
PHContentEditingInputRequestOptions *opts = [[PHContentEditingInputRequestOptions alloc] init];
opts.networkAccessAllowed = NO;
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block NSMutableDictionary *dict = [NSMutableDictionary dictionary];
[asset requestContentEditingInputWithOptions:opts completionHandler:^(PHContentEditingInput *input, NSDictionary *info) {
if (input.adjustmentData) {
dict[@"formatIdentifier"] = input.adjustmentData.formatIdentifier ?: @"";
dict[@"formatVersion"] = input.adjustmentData.formatVersion ?: @"";
}
NSURL *url = input.fullSizeImageURL;
if (url.lastPathComponent.length > 0) {
dict[@"baseFilename"] = url.lastPathComponent;
}
dispatch_semaphore_signal(sem);
}];
if (!semaphore_wait_with_timeout(sem, 5)) {
return nil;
}
return dict.count > 0 ? dict : nil;
}
static NSString *iso8601_string(NSDate *date) { static NSString *iso8601_string(NSDate *date) {
if (!date) return nil; if (!date) return nil;
static NSDateFormatter *fmt = nil; static NSDateFormatter *fmt = nil;
@@ -355,18 +438,22 @@ char *photos_list_assets_json(const char *album_id) {
NSString *mediaTypeStr = media_type_string(asset.mediaType); NSString *mediaTypeStr = media_type_string(asset.mediaType);
NSString *creationDateStr = iso8601_string(asset.creationDate); NSString *creationDateStr = iso8601_string(asset.creationDate);
NSString *modificationDateStr = iso8601_string(asset.modificationDate);
NSMutableArray *resourcesList = [NSMutableArray arrayWithCapacity:resources.count]; NSMutableArray *resourcesList = [NSMutableArray arrayWithCapacity:resources.count];
for (PHAssetResource *res in resources) { for (PHAssetResource *res in resources) {
NSString *resTypeStr = resource_type_string(res.type); NSString *resTypeStr = resource_type_string(res.type);
NSString *uti = res.uniformTypeIdentifier ?: @""; NSString *uti = res.uniformTypeIdentifier ?: @"";
BOOL isLocal = resource_is_locally_available(res); BOOL isLocal = resource_is_locally_available(res);
[resourcesList addObject:@{ NSMutableDictionary *resourceDict = [NSMutableDictionary dictionaryWithDictionary:@{
@"type": resTypeStr, @"type": resTypeStr,
@"filename": res.originalFilename ?: @"", @"filename": res.originalFilename ?: @"",
@"uti": uti, @"uti": uti,
@"local": @(isLocal) @"local": @(isLocal)
}]; }];
NSNumber *size = resource_file_size(res);
if (size) resourceDict[@"size"] = size;
[resourcesList addObject:resourceDict];
} }
NSMutableDictionary *dict = [NSMutableDictionary dictionaryWithDictionary:@{ NSMutableDictionary *dict = [NSMutableDictionary dictionaryWithDictionary:@{
@@ -374,17 +461,37 @@ char *photos_list_assets_json(const char *album_id) {
@"filename": filename ?: @"", @"filename": filename ?: @"",
@"cloud": cloudStatus, @"cloud": cloudStatus,
@"mediaType": mediaTypeStr, @"mediaType": mediaTypeStr,
@"mediaSubtypes": media_subtype_strings(asset.mediaSubtypes),
@"sourceType": source_type_string(asset.sourceType),
@"playbackStyle": playback_style_string(asset.playbackStyle),
@"pixelWidth": @(asset.pixelWidth), @"pixelWidth": @(asset.pixelWidth),
@"pixelHeight": @(asset.pixelHeight), @"pixelHeight": @(asset.pixelHeight),
@"duration": @(asset.duration), @"duration": @(asset.duration),
@"isFavorite": @(asset.isFavorite) @"isFavorite": @(asset.isFavorite),
@"isHidden": @(asset.isHidden),
@"representsBurst": @(asset.representsBurst),
@"burstSelectionTypes": burst_selection_type_strings(asset.burstSelectionTypes)
}]; }];
if (creationDateStr) { if (creationDateStr) {
dict[@"creationDate"] = creationDateStr; dict[@"creationDate"] = creationDateStr;
} }
if (modificationDateStr) {
dict[@"modificationDate"] = modificationDateStr;
}
NSDictionary *loc = location_dict(asset.location);
if (loc) {
dict[@"location"] = loc;
}
if (asset.burstIdentifier.length > 0) {
dict[@"burstIdentifier"] = asset.burstIdentifier;
}
if (@available(macOS 12, *)) { if (@available(macOS 12, *)) {
dict[@"hasAdjustments"] = @(asset.hasAdjustments); dict[@"hasAdjustments"] = @(asset.hasAdjustments);
} }
NSDictionary *adjustmentInfo = editing_input_info(asset);
if (adjustmentInfo) {
dict[@"adjustmentInfo"] = adjustmentInfo;
}
if (resourcesList.count > 0) { if (resourcesList.count > 0) {
dict[@"resources"] = resourcesList; dict[@"resources"] = resourcesList;
} }
@@ -394,6 +501,36 @@ char *photos_list_assets_json(const char *album_id) {
return json_from_object(@{@"assets": list, @"total": @(assets.count)}); return json_from_object(@{@"assets": list, @"total": @(assets.count)});
} }
char *photos_reverse_geocode_json(double latitude, double longitude) {
if (@available(macOS 26.0, *)) {
CLLocation *location = [[CLLocation alloc] initWithLatitude:latitude longitude:longitude];
MKReverseGeocodingRequest *request = [[MKReverseGeocodingRequest alloc] initWithLocation:location];
dispatch_semaphore_t sem = dispatch_semaphore_create(0);
__block MKMapItem *item = nil;
__block NSError *mapErr = nil;
[request getMapItemsWithCompletionHandler:^(NSArray<MKMapItem *> *mapItems, NSError *error) {
mapErr = error;
item = mapItems.firstObject;
dispatch_semaphore_signal(sem);
}];
if (!semaphore_wait_with_timeout(sem, 10)) {
[request cancel];
return json_from_object(@{@"error": @"timeout waiting for reverse geocode"});
}
if (mapErr || !item) {
NSString *msg = mapErr.localizedDescription ?: @"reverse geocode failed";
return json_from_object(@{@"error": msg});
}
NSMutableDictionary *dict = [NSMutableDictionary dictionary];
if (item.name) dict[@"name"] = item.name;
if (item.address.fullAddress) dict[@"formattedAddress"] = item.address.fullAddress;
if (item.address.shortAddress) dict[@"thoroughfare"] = item.address.shortAddress;
return json_from_object(@{@"placemark": dict});
}
return json_from_object(@{@"error": @"reverse geocoding requires macOS 26 or newer"});
}
char *photos_list_tree_json(void) { char *photos_list_tree_json(void) {
PHFetchResult<PHCollection *> *collections = [PHCollectionList fetchTopLevelUserCollectionsWithOptions:nil]; PHFetchResult<PHCollection *> *collections = [PHCollectionList fetchTopLevelUserCollectionsWithOptions:nil];
@@ -709,4 +846,4 @@ int photos_get_progress_slot_count(void) {
void photos_reset_progress_slots(void) { void photos_reset_progress_slots(void) {
memset(progress_slots, 0, sizeof(progress_slots)); memset(progress_slots, 0, sizeof(progress_slots));
} }
+11
View File
@@ -18,9 +18,11 @@ static int stub_access_rc = 0;
static const char *stub_albums_json = "{\"albums\":[]}"; static const char *stub_albums_json = "{\"albums\":[]}";
static const char *stub_assets_json = "{\"assets\":[]}"; static const char *stub_assets_json = "{\"assets\":[]}";
static const char *stub_tree_json = "{\"collections\":[]}"; static const char *stub_tree_json = "{\"collections\":[]}";
static const char *stub_geocode_json = "{\"placemark\":{}}";
static int stub_albums_null = 0; static int stub_albums_null = 0;
static int stub_assets_null = 0; static int stub_assets_null = 0;
static int stub_tree_null = 0; static int stub_tree_null = 0;
static int stub_geocode_null = 0;
static int stub_cancelled = 0; static int stub_cancelled = 0;
static const char *stub_export_preview_json = NULL; static const char *stub_export_preview_json = NULL;
static const char *stub_export_original_json = NULL; static const char *stub_export_original_json = NULL;
@@ -34,6 +36,8 @@ void photos_test_set_access(int rc) { stub_access_rc = rc; }
void photos_test_set_albums(const char *json) { stub_albums_json = json; stub_albums_null = 0; } void photos_test_set_albums(const char *json) { stub_albums_json = json; stub_albums_null = 0; }
void photos_test_set_assets(const char *json) { stub_assets_json = json; stub_assets_null = 0; } void photos_test_set_assets(const char *json) { stub_assets_json = json; stub_assets_null = 0; }
void photos_test_set_tree(const char *json) { stub_tree_json = json; stub_tree_null = 0; } void photos_test_set_tree(const char *json) { stub_tree_json = json; stub_tree_null = 0; }
void photos_test_set_geocode(const char *json) { stub_geocode_json = json; stub_geocode_null = 0; }
void photos_test_set_geocode_null(void) { stub_geocode_null = 1; }
void photos_test_set_albums_null(void) { stub_albums_null = 1; } void photos_test_set_albums_null(void) { stub_albums_null = 1; }
void photos_test_set_assets_null(void) { stub_assets_null = 1; } void photos_test_set_assets_null(void) { stub_assets_null = 1; }
void photos_test_set_tree_null(void) { stub_tree_null = 1; } void photos_test_set_tree_null(void) { stub_tree_null = 1; }
@@ -51,6 +55,13 @@ char *photos_list_assets_json(const char *album_id) {
return alloc_json(stub_assets_json); return alloc_json(stub_assets_json);
} }
char *photos_reverse_geocode_json(double latitude, double longitude) {
(void)latitude;
(void)longitude;
if (stub_geocode_null) return NULL;
return alloc_json(stub_geocode_json);
}
char *photos_export_preview_json(const char *asset_id, const char *output_dir, int target_size, int quality, int index, int slot_index) { char *photos_export_preview_json(const char *asset_id, const char *output_dir, int target_size, int quality, int index, int slot_index) {
(void)asset_id; (void)asset_id;
(void)output_dir; (void)output_dir;
+953 -92
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-44
View File
@@ -1,44 +0,0 @@
package manifest
import "testing"
func TestNewEntryPath(t *testing.T) {
e := newEntry("id1", "file.jpg", 123, "local")
if e.ID != "id1" || e.Filename != "file.jpg" || e.Path != "file.jpg" || e.Size != 123 || e.Cloud != "local" || e.Exported == 0 {
t.Fatalf("unexpected entry: %+v", e)
}
e = NewEntry("id2", "file2.jpg", "Album/file2.jpg", 456, "cloud")
if e.Path != "Album/file2.jpg" || e.Filename != "file2.jpg" || e.Size != 456 || e.Cloud != "cloud" {
t.Fatalf("unexpected entry with path: %+v", e)
}
}
func TestAddEntryDefaultsPath(t *testing.T) {
dir := t.TempDir()
jm := LoadJSONL(dir)
if err := jm.OpenAppend(); err != nil {
t.Fatal(err)
}
jm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local"})
jm.Close()
if got := LoadJSONL(dir).Entries()["x1"].Path; got != "file.jpg" {
t.Fatalf("jsonl path = %q", got)
}
sdir := t.TempDir()
sm, err := LoadSQLite(sdir)
if err != nil {
t.Fatal(err)
}
if err := sm.OpenAppend(); err != nil {
t.Fatal(err)
}
sm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local"})
if _, err := sm.db.Exec(`UPDATE downloads SET path = '' WHERE id = 'x1'`); err != nil {
t.Fatal(err)
}
if got := sm.Entries()["x1"].Path; got != "file.jpg" {
t.Fatalf("sqlite path = %q", got)
}
sm.Close()
}
+17
View File
@@ -0,0 +1,17 @@
package jsonl
import (
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
type Adapter struct{}
func (Adapter) Format() types.Format { return types.FormatJSONL }
func (Adapter) Aliases() []string { return []string{"json"} }
func (Adapter) Path(dir string) string { return Path(dir) }
func (Adapter) Exists(dir string) bool { return FileExists(Path(dir)) }
func (Adapter) Open(dir string) (types.Manifest, error) { return Load(dir), nil }
func (Adapter) OpenLogWriter(_ types.Manifest, dir string) (types.LogWriter, error) {
return types.NewFileLogWriter(types.LogPath(dir))
}
+209
View File
@@ -0,0 +1,209 @@
package jsonl
import (
"os"
"testing"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
func TestAdapter(t *testing.T) {
a := Adapter{}
if a.Format() != types.FormatJSONL {
t.Fatal("expected JSONL format")
}
if len(a.Aliases()) != 1 || a.Aliases()[0] != "json" {
t.Fatal("expected json alias")
}
dir := t.TempDir()
if a.Path(dir) != Path(dir) {
t.Fatal("expected path match")
}
if a.Exists(dir) {
t.Fatal("expected not to exist in empty dir")
}
m, err := a.Open(dir)
if err != nil {
t.Fatal(err)
}
m.Close()
w, err := a.OpenLogWriter(nil, dir)
if err != nil {
t.Fatal(err)
}
w.Close()
}
func TestStoreLoadEmpty(t *testing.T) {
m := Load(t.TempDir())
if m == nil {
t.Fatal("expected non-nil store")
}
}
func TestStoreLoadNonexistent(t *testing.T) {
m := Load("/nonexistent/path")
if m == nil {
t.Fatal("expected non-nil store")
}
}
func TestStoreAddAndHas(t *testing.T) {
m := Load(t.TempDir())
if m.Has("x") {
t.Fatal("expected Has to return false")
}
m.Add("x", "photo.jpg", 42, "s3")
if !m.Has("x") {
t.Fatal("expected Has to return true")
}
}
func TestStoreSaveAndReload(t *testing.T) {
dir := t.TempDir()
m := Load(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("id1", "file1.jpg", 10, "aws")
m.Add("id2", "file2.jpg", 20, "gcs")
if err := m.Save(); err != nil {
t.Fatal(err)
}
m.Close()
m2 := Load(dir)
if !m2.Has("id1") {
t.Fatal("expected id1 after reload")
}
if !m2.Has("id2") {
t.Fatal("expected id2 after reload")
}
}
func TestStoreOpenAppendCreatesDirs(t *testing.T) {
dir := t.TempDir()
subdir := dir + "/a/b"
m := Load(subdir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
}
func TestStoreCloseIdempotent(t *testing.T) {
m := Load(t.TempDir())
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
m.Close()
}
func TestStoreOpenAppendIdempotent(t *testing.T) {
m := Load(t.TempDir())
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
}
func TestStoreEntries(t *testing.T) {
m := Load(t.TempDir())
m.Add("e1", "f1.jpg", 1, "c1")
m.Add("e2", "f2.jpg", 2, "c2")
entries := m.Entries()
if len(entries) != 2 {
t.Fatalf("expected 2 entries, got %d", len(entries))
}
}
func TestStoreManifestFormat(t *testing.T) {
m := Load(t.TempDir())
if m.ManifestFormat() != types.FormatJSONL {
t.Fatal("expected JSONL format")
}
}
func TestStoreOpenAppendMkdirError(t *testing.T) {
m := Load("/proc/cannot-create-dir-here")
if err := m.OpenAppend(); err == nil {
t.Fatal("expected error")
}
}
func TestStoreSaveWithNoFile(t *testing.T) {
m := Load(t.TempDir())
if err := m.Save(); err != nil {
t.Fatal(err)
}
}
func TestStoreLoadFromExistingFile(t *testing.T) {
dir := t.TempDir()
m := Load(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("abc", "a.jpg", 100, "gcs")
m.Close()
m2 := Load(dir)
if !m2.Has("abc") {
t.Fatal("expected abc after reload")
}
}
func TestFileExists(t *testing.T) {
dir := t.TempDir()
if FileExists(dir) {
t.Fatal("expected false for directory")
}
if FileExists("/nonexistent/file") {
t.Fatal("expected false for nonexistent")
}
}
func TestLoadWithPathFallback(t *testing.T) {
dir := t.TempDir()
path := Path(dir)
osWriteFile(path, []byte(`{"id":"abc","filename":"a.jpg","size":100,"cloud":"gcs","exported":1234}`+"\n"), 0644)
m := Load(dir)
if !m.Has("abc") {
t.Fatal("expected abc")
}
if e := m.Entries()["abc"]; e.Path != "a.jpg" {
t.Fatalf("expected path fallback to filename, got %q", e.Path)
}
}
func TestOpenAppendAlreadyOpen(t *testing.T) {
m := Load(t.TempDir())
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
}
func TestOpenAppendOpenFileError(t *testing.T) {
dir := t.TempDir()
os.MkdirAll(Path(dir), 0755)
m := Load(dir)
if err := m.OpenAppend(); err == nil {
t.Fatal("expected error when path is a directory")
}
}
func TestOpenAppendMkdirError(t *testing.T) {
m := Load("/proc/cannot-create-dir-here")
if err := m.OpenAppend(); err == nil {
t.Fatal("expected error")
}
}
var osWriteFile = os.WriteFile
@@ -1,4 +1,4 @@
package manifest package jsonl
import ( import (
"encoding/json" "encoding/json"
@@ -6,32 +6,34 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"sync" "sync"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
) )
type jsonlManifest struct { type Store struct {
mu sync.Mutex mu sync.Mutex
entries map[string]Entry entries map[string]types.Entry
path string path string
file *os.File file *os.File
syncFunc func() error SyncFunc func() error
} }
var jsonlSaveHook func() error var saveHook func() error
func SetJSONLSaveHook(fn func() error) func() error { func SetSaveHook(fn func() error) func() error {
old := jsonlSaveHook old := saveHook
jsonlSaveHook = fn saveHook = fn
return old return old
} }
func JSONLPath(dir string) string { func Path(dir string) string {
return filepath.Join(dir, "downloads.jsonl") return filepath.Join(dir, "downloads.jsonl")
} }
func LoadJSONL(dir string) *jsonlManifest { func Load(dir string) *Store {
m := &jsonlManifest{ m := &Store{
entries: make(map[string]Entry), entries: make(map[string]types.Entry),
path: JSONLPath(dir), path: Path(dir),
} }
data, err := os.ReadFile(m.path) data, err := os.ReadFile(m.path)
if err != nil { if err != nil {
@@ -43,6 +45,7 @@ func LoadJSONL(dir string) *jsonlManifest {
Path string `json:"path,omitempty"` Path string `json:"path,omitempty"`
Size int64 `json:"size"` Size int64 `json:"size"`
Cloud string `json:"cloud"` Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"` Exported int64 `json:"exported"`
} }
for _, line := range strings.Split(string(data), "\n") { for _, line := range strings.Split(string(data), "\n") {
@@ -55,12 +58,13 @@ func LoadJSONL(dir string) *jsonlManifest {
if raw.Path == "" { if raw.Path == "" {
raw.Path = raw.Filename raw.Path = raw.Filename
} }
m.entries[raw.ID] = Entry{ m.entries[raw.ID] = types.Entry{
ID: raw.ID, ID: raw.ID,
Filename: raw.Filename, Filename: raw.Filename,
Path: raw.Path, Path: raw.Path,
Size: raw.Size, Size: raw.Size,
Cloud: raw.Cloud, Cloud: raw.Cloud,
Checksum: raw.Checksum,
Exported: raw.Exported, Exported: raw.Exported,
} }
} }
@@ -68,18 +72,22 @@ func LoadJSONL(dir string) *jsonlManifest {
return m return m
} }
func (m *jsonlManifest) Has(id string) bool { func (m *Store) Has(id string) bool {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
_, ok := m.entries[id] _, ok := m.entries[id]
return ok return ok
} }
func (m *jsonlManifest) Add(id string, filename string, size int64, cloud string) { func (m *Store) ManifestFormat() types.Format {
m.AddEntry(newEntry(id, filename, size, cloud)) return types.FormatJSONL
} }
func (m *jsonlManifest) AddEntry(entry Entry) { func (m *Store) Add(id string, filename string, size int64, cloud string) {
m.AddEntry(types.NewEntry(id, filename, filename, size, cloud))
}
func (m *Store) AddEntry(entry types.Entry) {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
if entry.Path == "" { if entry.Path == "" {
@@ -93,21 +101,22 @@ func (m *jsonlManifest) AddEntry(entry Entry) {
Path string `json:"path,omitempty"` Path string `json:"path,omitempty"`
Size int64 `json:"size"` Size int64 `json:"size"`
Cloud string `json:"cloud"` Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"` Exported int64 `json:"exported"`
}{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Exported: entry.Exported}) }{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Checksum: entry.Checksum, Exported: entry.Exported})
m.file.Write(data) m.file.Write(data)
m.file.Write([]byte("\n")) m.file.Write([]byte("\n"))
} }
} }
func (m *jsonlManifest) Save() error { func (m *Store) Save() error {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
if m.syncFunc != nil { if m.SyncFunc != nil {
return m.syncFunc() return m.SyncFunc()
} }
if jsonlSaveHook != nil { if saveHook != nil {
return jsonlSaveHook() return saveHook()
} }
if m.file != nil { if m.file != nil {
return m.file.Sync() return m.file.Sync()
@@ -115,7 +124,7 @@ func (m *jsonlManifest) Save() error {
return nil return nil
} }
func (m *jsonlManifest) Close() { func (m *Store) Close() {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
if m.file != nil { if m.file != nil {
@@ -124,7 +133,7 @@ func (m *jsonlManifest) Close() {
} }
} }
func (m *jsonlManifest) OpenAppend() error { func (m *Store) OpenAppend() error {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
if m.file != nil { if m.file != nil {
@@ -141,12 +150,20 @@ func (m *jsonlManifest) OpenAppend() error {
return nil return nil
} }
func (m *jsonlManifest) Entries() map[string]Entry { func (m *Store) Entries() map[string]types.Entry {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
out := make(map[string]Entry, len(m.entries)) out := make(map[string]types.Entry, len(m.entries))
for k, v := range m.entries { for k, v := range m.entries {
out[k] = v out[k] = v
} }
return out return out
} }
func FileExists(path string) bool {
info, err := os.Stat(path)
if err != nil {
return false
}
return !info.IsDir()
}
+79
View File
@@ -0,0 +1,79 @@
package jsonl
import (
"fmt"
"testing"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
func TestSetSaveHook(t *testing.T) {
old := SetSaveHook(func() error { return fmt.Errorf("hook error") })
if old != nil {
t.Error("expected nil old hook")
}
restore := SetSaveHook(old)
if restore == nil {
t.Error("expected non-nil restore function")
}
}
func TestSaveHookError(t *testing.T) {
dir := t.TempDir()
m := Load(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
old := SetSaveHook(func() error { return fmt.Errorf("hook error") })
defer SetSaveHook(old)
if err := m.Save(); err == nil {
t.Error("expected hook error from Save")
}
m.Close()
}
func TestSyncFuncError(t *testing.T) {
dir := t.TempDir()
m := Load(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.SyncFunc = func() error { return fmt.Errorf("sync func error") }
if err := m.Save(); err == nil {
t.Error("expected syncFunc error from Save")
}
m.Close()
}
func TestSaveError(t *testing.T) {
dir := t.TempDir()
m := Load(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.file.Close()
if err := m.Save(); err == nil {
t.Error("expected Sync error on closed file")
}
m.Close()
}
func TestAddEntryDefaultsPath(t *testing.T) {
dir := t.TempDir()
m := Load(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(types.Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:abc"})
m.Close()
loaded := Load(dir).Entries()["x1"]
if got := loaded.Path; got != "file.jpg" {
t.Fatalf("jsonl path = %q", got)
}
if loaded.Checksum != "sha256:abc" {
t.Fatalf("jsonl checksum = %q", loaded.Checksum)
}
}
-30
View File
@@ -1,30 +0,0 @@
package manifest
type LogEntry struct {
Timestamp int64 `json:"ts"`
Level string `json:"level"`
Event string `json:"event"`
AssetID string `json:"asset_id,omitempty"`
Album string `json:"album,omitempty"`
Filename string `json:"filename,omitempty"`
Size int64 `json:"size,omitempty"`
Cloud string `json:"cloud,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
Message string `json:"message,omitempty"`
}
type LogWriter interface {
Log(entry LogEntry)
Close()
}
type noopLogWriter struct{}
func (noopLogWriter) Log(LogEntry) { _ = struct{}{} }
func (noopLogWriter) Close() { _ = struct{}{} }
var NoopLogWriter LogWriter = noopLogWriter{}
func LogPath(dir string) string {
return dir + "/export.log"
}
-211
View File
@@ -1,211 +0,0 @@
package manifest
import (
"database/sql"
"os"
"path/filepath"
"testing"
_ "modernc.org/sqlite"
)
func TestNoopLogWriter(t *testing.T) {
lw := NoopLogWriter
lw.Log(LogEntry{Event: "test"})
lw.Close()
noopLogWriter{}.Log(LogEntry{Event: "test"})
noopLogWriter{}.Close()
}
func TestNewSQLiteLogWriter(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "test.db")
db, err := sql.Open("sqlite", dbPath)
if err != nil {
t.Fatal(err)
}
defer db.Close()
lw, err := NewSQLiteLogWriter(db)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{
Timestamp: 1700000000,
Level: "info",
Event: "export_done",
AssetID: "asset-1",
Album: "Favorites",
Filename: "photo.jpg",
Size: 1024,
Cloud: "local",
DurationMs: 500,
Message: "",
})
lw.Log(LogEntry{
Timestamp: 1700000001,
Level: "error",
Event: "export_fail",
AssetID: "asset-2",
Filename: "bad.jpg",
Message: "timeout",
})
lw.Close()
var count int
err = db.QueryRow(`SELECT COUNT(*) FROM logs`).Scan(&count)
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected 2 log entries, got %d", count)
}
var event, level, assetID string
err = db.QueryRow(`SELECT event, level, asset_id FROM logs WHERE asset_id = 'asset-1'`).Scan(&event, &level, &assetID)
if err != nil {
t.Fatal(err)
}
if event != "export_done" || level != "info" || assetID != "asset-1" {
t.Errorf("unexpected row: event=%s level=%s asset_id=%s", event, level, assetID)
}
}
func TestSQLiteLogWriterNilDB(t *testing.T) {
w := &sqliteLogWriter{db: nil}
w.Log(LogEntry{Event: "test"})
w.Close()
}
func TestNewSQLiteLogWriterError(t *testing.T) {
db, err := sql.Open("sqlite", filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
if err := db.Close(); err != nil {
t.Fatal(err)
}
if _, err := NewSQLiteLogWriter(db); err == nil {
t.Error("expected error for closed db")
}
}
func TestSQLiteLogWriterCloseConcrete(t *testing.T) {
(&sqliteLogWriter{}).Close()
}
func TestNewFileLogWriter(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "export.log")
lw, err := NewFileLogWriter(path)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{
Timestamp: 1700000000,
Level: "info",
Event: "export_done",
Filename: "photo.jpg",
Size: 2048,
})
lw.Close()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if len(data) == 0 {
t.Error("expected log data")
}
if data[len(data)-1] != '\n' {
t.Error("expected trailing newline")
}
}
func TestFileLogWriterClosed(t *testing.T) {
w := &fileLogWriter{f: nil}
w.Log(LogEntry{Event: "test"})
w.Close()
}
func TestNewFileLogWriterError(t *testing.T) {
_, err := NewFileLogWriter("/nonexistent/dir/export.log")
if err == nil {
t.Error("expected error for bad path")
}
}
func TestFileLogWriterDoubleClose(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "export.log")
lw, err := NewFileLogWriter(path)
if err != nil {
t.Fatal(err)
}
lw.Close()
lw.Close()
}
func TestLogPath(t *testing.T) {
p := LogPath("/tmp/out")
if p != "/tmp/out/export.log" {
t.Errorf("expected /tmp/out/export.log, got %s", p)
}
}
func TestOpenLogWriterSQLite(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
lw, err := OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{Event: "test", Level: "info"})
lw.Close()
}
func TestOpenLogWriterJSONL(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
lw, err := OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{Event: "test", Level: "info"})
lw.Close()
if _, err := os.Stat(LogPath(dir)); os.IsNotExist(err) {
t.Error("expected export.log to exist")
}
}
func TestOpenLogWriterNilManifest(t *testing.T) {
dir := t.TempDir()
lw, err := OpenLogWriter(nil, dir)
if err != nil {
t.Fatal(err)
}
lw.Log(LogEntry{Event: "test", Level: "info"})
lw.Close()
if _, err := os.Stat(LogPath(dir)); os.IsNotExist(err) {
t.Error("expected export.log to exist for nil manifest")
}
}
+30 -37
View File
@@ -1,44 +1,37 @@
package manifest package manifest
import "time" import "gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
type Entry struct { type Entry = types.Entry
ID string
Filename string
Path string
Size int64
Cloud string
Exported int64
}
type Manifest interface { type Manifest = types.Manifest
Has(id string) bool
Add(id string, filename string, size int64, cloud string)
AddEntry(entry Entry)
Save() error
Close()
OpenAppend() error
}
type EntryReader interface { type EntryReader = types.EntryReader
Entries() map[string]Entry
}
func newEntry(id, filename string, size int64, cloud string) Entry { type Format = types.Format
return Entry{
ID: id,
Filename: filename,
Path: filename,
Size: size,
Cloud: cloud,
Exported: time.Now().Unix(),
}
}
func NewEntry(id, filename, path string, size int64, cloud string) Entry { const (
e := newEntry(id, filename, size, cloud) FormatJSONL = types.FormatJSONL
if path != "" { FormatSQLite = types.FormatSQLite
e.Path = path )
}
return e type LogEntry = types.LogEntry
}
type LogWriter = types.LogWriter
var NoopLogWriter = types.NoopLogWriter
type Adapter = types.Adapter
type Registry = types.Registry
type FormatReporter = types.FormatReporter
var (
NewRegistry = types.NewRegistry
NewEntry = types.NewEntry
NewEntryWithChecksum = types.NewEntryWithChecksum
LogPath = types.LogPath
ConvertManifest = types.ConvertManifest
NewFileLogWriter = types.NewFileLogWriter
)
+52
View File
@@ -0,0 +1,52 @@
package manifest
import "gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
type memoryStore struct {
entries map[string]types.Entry
}
func newMemoryStore() *memoryStore {
return &memoryStore{entries: make(map[string]types.Entry)}
}
func (m *memoryStore) Has(id string) bool {
_, ok := m.entries[id]
return ok
}
func (m *memoryStore) Add(id string, filename string, size int64, cloud string) {
m.AddEntry(types.NewEntry(id, filename, filename, size, cloud))
}
func (m *memoryStore) AddEntry(entry types.Entry) {
if entry.Path == "" {
entry.Path = entry.Filename
}
m.entries[entry.ID] = entry
}
func (m *memoryStore) Save() error { return nil }
func (m *memoryStore) Close() { _ = m }
func (m *memoryStore) OpenAppend() error { return nil }
func (m *memoryStore) Entries() map[string]types.Entry {
out := make(map[string]types.Entry, len(m.entries))
for k, v := range m.entries {
out[k] = v
}
return out
}
type MemoryAdapter struct{}
func (MemoryAdapter) Format() types.Format { return types.FormatJSONL }
func (MemoryAdapter) Aliases() []string { return nil }
func (MemoryAdapter) Path(string) string { return "" }
func (MemoryAdapter) Exists(string) bool { return false }
func (MemoryAdapter) Open(string) (types.Manifest, error) { return newMemoryStore(), nil }
func (MemoryAdapter) OpenLogWriter(types.Manifest, string) (types.LogWriter, error) {
return types.NoopLogWriter, nil
}
+17 -78
View File
@@ -1,84 +1,21 @@
package manifest package manifest
import ( import (
"fmt"
"os" "os"
"strings"
)
type Format string "gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
const (
FormatJSONL Format = "jsonl"
FormatSQLite Format = "sqlite"
) )
func Open(dir string, format Format) (Manifest, error) { func Open(dir string, format Format) (Manifest, error) {
jsonlPath := JSONLPath(dir) return Default.Open(dir, format)
sqlitePath := SQLitePath(dir)
jsonlExists := FileExists(jsonlPath)
sqliteExists := FileExists(sqlitePath)
switch {
case format == FormatJSONL && jsonlExists:
return LoadJSONL(dir), nil
case format == FormatSQLite && sqliteExists:
return LoadSQLite(dir)
case format == FormatJSONL && sqliteExists:
return ConvertFromSQLite(dir)
case format == FormatSQLite && jsonlExists:
return ConvertFromJSONL(dir)
default:
if format == FormatJSONL {
return LoadJSONL(dir), nil
}
return LoadSQLite(dir)
}
} }
func ConvertFromJSONL(dir string) (Manifest, error) { func ConvertFromJSONL(dir string) (Manifest, error) {
src := LoadJSONL(dir) return types.ConvertManifest(dir, JSONLAdapter, SQLiteAdapter)
if err := src.OpenAppend(); err != nil {
return nil, fmt.Errorf("open jsonl for read: %w", err)
}
defer src.Close()
dst, _ := LoadSQLite(dir)
if err := dst.OpenAppend(); err != nil {
return nil, fmt.Errorf("open sqlite for write: %w", err)
}
for id, e := range src.Entries() {
e.ID = id
dst.AddEntry(e)
}
os.Remove(JSONLPath(dir))
return dst, nil
} }
func ConvertFromSQLite(dir string) (Manifest, error) { func ConvertFromSQLite(dir string) (Manifest, error) {
src, _ := LoadSQLite(dir) return types.ConvertManifest(dir, SQLiteAdapter, JSONLAdapter)
if err := src.OpenAppend(); err != nil {
return nil, fmt.Errorf("open sqlite for read: %w", err)
}
defer src.Close()
dst := LoadJSONL(dir)
if err := dst.OpenAppend(); err != nil {
return nil, fmt.Errorf("open jsonl for write: %w", err)
}
for id, e := range src.Entries() {
e.ID = id
dst.AddEntry(e)
}
if err := dst.Save(); err != nil {
return nil, fmt.Errorf("save jsonl: %w", err)
}
os.Remove(SQLitePath(dir))
return dst, nil
} }
func FileExists(path string) bool { func FileExists(path string) bool {
@@ -90,19 +27,21 @@ func FileExists(path string) bool {
} }
func ParseFormat(s string) (Format, error) { func ParseFormat(s string) (Format, error) {
switch strings.ToLower(s) { return Default.ParseFormat(s)
case "jsonl", "json":
return FormatJSONL, nil
case "sqlite", "db", "sqlite3":
return FormatSQLite, nil
default:
return "", fmt.Errorf("unknown manifest format %q (use jsonl or sqlite)", s)
}
} }
func OpenLogWriter(m Manifest, dir string) (LogWriter, error) { func OpenLogWriter(m Manifest, dir string) (LogWriter, error) {
if sm, ok := m.(*sqliteManifest); ok && sm.DB() != nil { format := FormatJSONL
return NewSQLiteLogWriter(sm.DB()) if reporter, ok := m.(types.FormatReporter); ok {
format = reporter.ManifestFormat()
} }
return NewFileLogWriter(LogPath(dir)) return Default.OpenLogWriter(m, dir, format)
}
func osRemove(path string) error {
return os.Remove(path)
}
func init() {
types.SetRemoveFunc(osRemove)
} }
+34
View File
@@ -0,0 +1,34 @@
package manifest
import (
jsonladapter "gitea.k3s.k0.nu/tools/photocli/internal/manifest/jsonl"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/sqlite"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
var (
JSONLAdapter = jsonladapter.Adapter{}
SQLiteAdapter = sqlite.Adapter{}
)
var Default = types.NewRegistry(JSONLAdapter, SQLiteAdapter)
func LoadJSONL(dir string) *jsonladapter.Store {
return jsonladapter.Load(dir)
}
func JSONLPath(dir string) string {
return jsonladapter.Path(dir)
}
func SetJSONLSaveHook(fn func() error) func() error {
return jsonladapter.SetSaveHook(fn)
}
func LoadSQLite(dir string) (Manifest, error) {
return sqlite.Load(dir)
}
func SQLitePath(dir string) string {
return sqlite.Path(dir)
}
+279
View File
@@ -0,0 +1,279 @@
package manifest
import (
"fmt"
"testing"
jsonladapter "gitea.k3s.k0.nu/tools/photocli/internal/manifest/jsonl"
sqliteadapter "gitea.k3s.k0.nu/tools/photocli/internal/manifest/sqlite"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
func TestRegistryParseFormatAliases(t *testing.T) {
r := Default
cases := map[string]Format{
"jsonl": FormatJSONL,
"json": FormatJSONL,
"sqlite": FormatSQLite,
"db": FormatSQLite,
"sqlite3": FormatSQLite,
}
for input, want := range cases {
got, err := r.ParseFormat(input)
if err != nil {
t.Fatalf("ParseFormat(%q): %v", input, err)
}
if got != want {
t.Fatalf("ParseFormat(%q) = %q, want %q", input, got, want)
}
}
}
func TestRegistryParseFormatUnknown(t *testing.T) {
if _, err := Default.ParseFormat("bad"); err == nil {
t.Fatal("expected unknown format error")
}
}
func TestRegistryOpenConvertsExistingManifest(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 12, "local")
if err := m.Save(); err != nil {
t.Fatal(err)
}
m.Close()
converted, err := Default.Open(dir, FormatSQLite)
if err != nil {
t.Fatal(err)
}
defer converted.Close()
if !converted.Has("x1") {
t.Fatal("expected converted sqlite manifest to contain entry")
}
if FileExists(JSONLPath(dir)) {
t.Fatal("expected source jsonl manifest to be removed")
}
if !FileExists(SQLitePath(dir)) {
t.Fatal("expected sqlite manifest to exist")
}
}
func TestMemoryAdapterStore(t *testing.T) {
s := newMemoryStore()
if s.Has("x") {
t.Fatal("expected empty")
}
s.Add("x", "photo.jpg", 42, "local")
if !s.Has("x") {
t.Fatal("expected has after add")
}
s.AddEntry(Entry{ID: "y", Filename: "file.jpg"})
if e := s.Entries()["y"]; e.Path != "file.jpg" {
t.Fatal("expected default path")
}
if err := s.Save(); err != nil {
t.Fatal(err)
}
s.Close()
if err := s.OpenAppend(); err != nil {
t.Fatal(err)
}
adapter := MemoryAdapter{}
if adapter.Format() != FormatJSONL {
t.Fatal("expected JSONL format")
}
if len(adapter.Aliases()) != 0 {
t.Fatal("expected no aliases")
}
if adapter.Path("") != "" {
t.Fatal("expected empty path")
}
if adapter.Exists("") {
t.Fatal("expected not to exist")
}
m, err := adapter.Open(t.TempDir())
if err != nil {
t.Fatal(err)
}
m.Close()
w, err := adapter.OpenLogWriter(nil, t.TempDir())
if err != nil {
t.Fatal(err)
}
w.Close()
}
func TestRegistryOpenLogWriterUsesAdapter(t *testing.T) {
dir := t.TempDir()
m, err := Default.Open(dir, FormatJSONL)
if err != nil {
t.Fatal(err)
}
defer m.Close()
w, err := Default.OpenLogWriter(m, dir, FormatJSONL)
if err != nil {
t.Fatal(err)
}
w.Close()
if !FileExists(LogPath(dir)) {
t.Fatal("expected jsonl adapter to create file log")
}
}
func TestRegistryOpenCreatesRequestedAdapter(t *testing.T) {
dir := t.TempDir()
m, err := Default.Open(dir, FormatJSONL)
if err != nil {
t.Fatal(err)
}
defer m.Close()
if _, ok := m.(*jsonladapter.Store); !ok {
t.Fatalf("expected jsonl store, got %T", m)
}
}
func TestRegistryUnknownAdapterErrors(t *testing.T) {
r := NewRegistry(JSONLAdapter)
if _, err := r.Open(t.TempDir(), FormatSQLite); err == nil {
t.Fatal("expected open error for unregistered format")
}
if _, err := r.OpenLogWriter(LoadJSONL(t.TempDir()), t.TempDir(), FormatSQLite); err == nil {
t.Fatal("expected log writer error for unregistered format")
}
}
func TestSQLiteAdapterOpenLogWriterUsesSQLite(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
w, err := SQLiteAdapter.OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
defer w.Close()
if _, ok := w.(*sqliteadapter.LogWriter); !ok {
t.Fatalf("expected sqlite log writer, got %T", w)
}
}
func TestOpenLogWriterUsesManifestFormat(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
w, err := OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
defer w.Close()
if _, ok := w.(*sqliteadapter.LogWriter); !ok {
t.Fatalf("expected sqlite log writer, got %T", w)
}
}
func TestSQLiteAdapterOpenLogWriterFallsBackToFile(t *testing.T) {
dir := t.TempDir()
w, err := SQLiteAdapter.OpenLogWriter(LoadJSONL(dir), dir)
if err != nil {
t.Fatal(err)
}
w.Close()
if !FileExists(LogPath(dir)) {
t.Fatal("expected sqlite adapter fallback to create file log")
}
}
func TestSetJSONLSaveHookWrapper(t *testing.T) {
old := SetJSONLSaveHook(func() error { return nil })
SetJSONLSaveHook(old)
}
func TestConvertManifestErrorBranches(t *testing.T) {
dir := t.TempDir()
if _, err := ConvertManifest(dir, failingAdapter{format: FormatJSONL, openErr: fmt.Errorf("boom")}, JSONLAdapter); err == nil {
t.Fatal("expected source open error")
}
if _, err := ConvertManifest(dir, staticAdapter{format: FormatJSONL, store: badOpenManifest{}}, JSONLAdapter); err == nil {
t.Fatal("expected source OpenAppend error")
}
if _, err := ConvertManifest(dir, staticAdapter{format: FormatJSONL, store: noReadManifest{}}, JSONLAdapter); err == nil {
t.Fatal("expected entry reader error")
}
if _, err := ConvertManifest(dir, staticAdapter{format: FormatJSONL, store: readableManifest{}}, failingAdapter{format: FormatSQLite, openErr: fmt.Errorf("boom")}); err == nil {
t.Fatal("expected destination open error")
}
if _, err := ConvertManifest(dir, staticAdapter{format: FormatJSONL, store: readableManifest{}}, staticAdapter{format: FormatSQLite, store: badOpenManifest{}}); err == nil {
t.Fatal("expected destination OpenAppend error")
}
oldRemove := types.RemoveFunc()
types.SetRemoveFunc(func(string) error { return fmt.Errorf("remove failed") })
defer func() { types.SetRemoveFunc(oldRemove) }()
if _, err := ConvertManifest(dir, staticAdapter{format: FormatJSONL, store: readableManifest{}}, staticAdapter{format: FormatSQLite, store: readableManifest{}}); err == nil {
t.Fatal("expected remove error")
}
}
type staticAdapter struct {
format Format
store Manifest
}
func (a staticAdapter) Format() Format { return a.format }
func (a staticAdapter) Aliases() []string { return nil }
func (a staticAdapter) Path(string) string { return "manifest.file" }
func (a staticAdapter) Exists(string) bool { return true }
func (a staticAdapter) Open(string) (Manifest, error) { return a.store, nil }
func (a staticAdapter) OpenLogWriter(Manifest, string) (LogWriter, error) { return NoopLogWriter, nil }
type failingAdapter struct {
format Format
openErr error
}
func (a failingAdapter) Format() Format { return a.format }
func (a failingAdapter) Aliases() []string { return nil }
func (a failingAdapter) Path(string) string { return "manifest.file" }
func (a failingAdapter) Exists(string) bool { return true }
func (a failingAdapter) Open(string) (Manifest, error) { return nil, a.openErr }
func (a failingAdapter) OpenLogWriter(Manifest, string) (LogWriter, error) { return nil, a.openErr }
type readableManifest struct{}
func (readableManifest) Has(string) bool { return false }
func (readableManifest) Add(string, string, int64, string) {}
func (readableManifest) AddEntry(Entry) {}
func (readableManifest) Save() error { return nil }
func (readableManifest) Close() {}
func (readableManifest) OpenAppend() error { return nil }
func (readableManifest) Entries() map[string]Entry { return map[string]Entry{"x": {Filename: "x.jpg"}} }
type noReadManifest struct{}
func (noReadManifest) Has(string) bool { return false }
func (noReadManifest) Add(string, string, int64, string) {}
func (noReadManifest) AddEntry(Entry) {}
func (noReadManifest) Save() error { return nil }
func (noReadManifest) Close() {}
func (noReadManifest) OpenAppend() error { return nil }
func (noReadManifest) Entries() map[string]Entry { return nil }
type badOpenManifest struct{ readableManifest }
func (badOpenManifest) OpenAppend() error { return fmt.Errorf("open failed") }
+21
View File
@@ -0,0 +1,21 @@
package sqlite
import (
_ "modernc.org/sqlite"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
type Adapter struct{}
func (Adapter) Format() types.Format { return types.FormatSQLite }
func (Adapter) Aliases() []string { return []string{"db", "sqlite3"} }
func (Adapter) Path(dir string) string { return Path(dir) }
func (Adapter) Exists(dir string) bool { return FileExists(Path(dir)) }
func (Adapter) Open(dir string) (types.Manifest, error) { return Load(dir) }
func (Adapter) OpenLogWriter(m types.Manifest, dir string) (types.LogWriter, error) {
if sm, ok := m.(*Store); ok && sm.DB() != nil {
return NewLogWriter(sm.DB())
}
return types.NewFileLogWriter(types.LogPath(dir))
}
+290
View File
@@ -0,0 +1,290 @@
package sqlite
import (
"database/sql"
"fmt"
"path/filepath"
"testing"
_ "modernc.org/sqlite"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
func TestAdapter(t *testing.T) {
a := Adapter{}
if a.Format() != types.FormatSQLite {
t.Fatal("expected SQLite format")
}
if len(a.Aliases()) != 2 {
t.Fatal("expected 2 aliases")
}
dir := t.TempDir()
if a.Path(dir) != Path(dir) {
t.Fatal("expected path match")
}
if a.Exists(dir) {
t.Fatal("expected not to exist in empty dir")
}
m, err := a.Open(dir)
if err != nil {
t.Fatal(err)
}
m.Close()
w, err := a.OpenLogWriter(nil, dir)
if err != nil {
t.Fatal(err)
}
w.Close()
}
func TestAdapterOpenLogWriterSQLite(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
w, err := Adapter{}.OpenLogWriter(m, dir)
if err != nil {
t.Fatal(err)
}
defer w.Close()
if _, ok := w.(*LogWriter); !ok {
t.Fatalf("expected sqlite log writer, got %T", w)
}
}
func TestStoreLoadEmpty(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if m == nil {
t.Fatal("expected non-nil store")
}
m.Close()
}
func TestStoreLoadNonexistent(t *testing.T) {
m, err := Load("/nonexistent/path")
if err != nil {
t.Fatal(err)
}
if m == nil {
t.Fatal("expected non-nil store")
}
m.Close()
}
func TestStoreAddHasSaveClose(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("sid1", "sfile.jpg", 99, "azure")
if !m.Has("sid1") {
t.Fatal("expected Has to return true")
}
if m.Has("nope") {
t.Fatal("expected Has to return false")
}
if err := m.Save(); err != nil {
t.Fatal(err)
}
m.Close()
}
func TestStoreRoundTrip(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("rid1", "rfile1.jpg", 10, "aws")
m.Add("rid2", "rfile2.jpg", 20, "gcs")
if err := m.Save(); err != nil {
t.Fatal(err)
}
m.Close()
m2, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m2.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m2.Close()
if !m2.Has("rid1") {
t.Fatal("expected rid1 after reload")
}
if !m2.Has("rid2") {
t.Fatal("expected rid2 after reload")
}
}
func TestStoreEntries(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
defer m.Close()
m.Add("se1", "sf1.jpg", 1, "sc1")
m.Add("se2", "sf2.jpg", 2, "sc2")
entries := m.Entries()
if len(entries) != 2 {
t.Fatalf("expected 2 entries, got %d", len(entries))
}
}
func TestStoreCloseIdempotent(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
m.Close()
}
func TestStoreOpenAppendIdempotent(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
}
func TestStoreManifestFormat(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if m.ManifestFormat() != types.FormatSQLite {
t.Fatal("expected SQLite format")
}
}
func TestStoreSave(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if err := m.Save(); err != nil {
t.Fatal(err)
}
}
func TestStoreOpenAppendCreateDir(t *testing.T) {
dir := t.TempDir()
subdir := filepath.Join(dir, "subdir")
m, err := Load(subdir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Close()
}
func TestStoreCloseWithoutOpen(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
m.Close()
}
func TestStoreEntriesWithoutOpen(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
entries := m.Entries()
if entries != nil {
t.Errorf("Entries without open should return nil, got %v", entries)
}
}
func TestStoreHasWithoutOpen(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
if m.Has("x") {
t.Fatal("Has should return false without open")
}
}
func TestStoreAddEntryWithoutOpen(t *testing.T) {
m, err := Load(t.TempDir())
if err != nil {
t.Fatal(err)
}
m.AddEntry(types.Entry{ID: "x", Filename: "f.jpg"})
}
func TestSetOpener(t *testing.T) {
old := SetOpener(func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, fmt.Errorf("test")
})
if old != nil {
t.Fatal("expected nil old opener")
}
restore := SetOpener(old)
if restore == nil {
t.Fatal("expected non-nil restore")
}
SetOpener(nil)
}
func TestFileExists(t *testing.T) {
dir := t.TempDir()
if FileExists(dir) {
t.Fatal("expected false for directory")
}
if FileExists("/nonexistent/file") {
t.Fatal("expected false for nonexistent")
}
}
func TestLogWriterClose(t *testing.T) {
w := &LogWriter{}
w.Close()
}
func TestOpenerWithOverride(t *testing.T) {
SetOpener(func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, nil
})
if opener() == nil {
t.Fatal("expected override")
}
SetOpener(nil)
}
func TestLogWriterCloseDirect(t *testing.T) {
(&LogWriter{}).Close()
}
@@ -1,14 +1,16 @@
package manifest package sqlite
import ( import (
"database/sql" "database/sql"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
) )
type sqliteLogWriter struct { type LogWriter struct {
db *sql.DB db *sql.DB
} }
func NewSQLiteLogWriter(db *sql.DB) (LogWriter, error) { func NewLogWriter(db *sql.DB) (types.LogWriter, error) {
_, err := db.Exec(`CREATE TABLE IF NOT EXISTS logs ( _, err := db.Exec(`CREATE TABLE IF NOT EXISTS logs (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL, ts INTEGER NOT NULL,
@@ -27,10 +29,10 @@ func NewSQLiteLogWriter(db *sql.DB) (LogWriter, error) {
} }
_, _ = db.Exec(`CREATE INDEX IF NOT EXISTS idx_logs_ts ON logs(ts)`) _, _ = db.Exec(`CREATE INDEX IF NOT EXISTS idx_logs_ts ON logs(ts)`)
_, _ = db.Exec(`CREATE INDEX IF NOT EXISTS idx_logs_event ON logs(event)`) _, _ = db.Exec(`CREATE INDEX IF NOT EXISTS idx_logs_event ON logs(event)`)
return &sqliteLogWriter{db: db}, nil return &LogWriter{db: db}, nil
} }
func (w *sqliteLogWriter) Log(e LogEntry) { func (w *LogWriter) Log(e types.LogEntry) {
if w.db == nil { if w.db == nil {
return return
} }
@@ -38,4 +40,4 @@ func (w *sqliteLogWriter) Log(e LogEntry) {
e.Timestamp, e.Level, e.Event, e.AssetID, e.Album, e.Filename, e.Size, e.Cloud, e.DurationMs, e.Message) e.Timestamp, e.Level, e.Event, e.AssetID, e.Album, e.Filename, e.Size, e.Cloud, e.DurationMs, e.Message)
} }
func (w *sqliteLogWriter) Close() { _ = w } func (w *LogWriter) Close() { _ = w }
@@ -1,4 +1,4 @@
package manifest package sqlite
import ( import (
"database/sql" "database/sql"
@@ -6,40 +6,46 @@ import (
"os" "os"
"path/filepath" "path/filepath"
_ "modernc.org/sqlite" "gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
) )
type sqliteManifest struct { type OpenerFunc func(driverName, dataSourceName string) (*sql.DB, error)
var openerOverride OpenerFunc
func SetOpener(fn OpenerFunc) (old OpenerFunc) {
old = openerOverride
openerOverride = fn
return old
}
type Store struct {
path string path string
db *sql.DB db *sql.DB
open sqlOpenerFunc open OpenerFunc
execFunc func(query string, args ...any) (sql.Result, error) execFunc func(query string, args ...any) (sql.Result, error)
} }
type sqlOpenerFunc func(driverName, dataSourceName string) (*sql.DB, error) func Path(dir string) string {
var sqliteOpenFunc sqlOpenerFunc
func defaultSQLOpener() sqlOpenerFunc {
if sqliteOpenFunc != nil {
return sqliteOpenFunc
}
return sql.Open
}
func SQLitePath(dir string) string {
return filepath.Join(dir, "downloads.db") return filepath.Join(dir, "downloads.db")
} }
func LoadSQLite(dir string) (*sqliteManifest, error) { func Load(dir string) (*Store, error) {
m := &sqliteManifest{ m := &Store{
path: SQLitePath(dir), path: Path(dir),
open: defaultSQLOpener(), open: opener(),
} }
return m, nil return m, nil
} }
func (m *sqliteManifest) OpenAppend() error { func opener() OpenerFunc {
if openerOverride != nil {
return openerOverride
}
return sql.Open
}
func (m *Store) OpenAppend() error {
if m.db != nil { if m.db != nil {
return nil return nil
} }
@@ -64,6 +70,7 @@ func (m *sqliteManifest) OpenAppend() error {
path TEXT NOT NULL DEFAULT '', path TEXT NOT NULL DEFAULT '',
size INTEGER NOT NULL DEFAULT 0, size INTEGER NOT NULL DEFAULT 0,
cloud TEXT NOT NULL DEFAULT '', cloud TEXT NOT NULL DEFAULT '',
checksum TEXT NOT NULL DEFAULT '',
exported INTEGER NOT NULL DEFAULT 0 exported INTEGER NOT NULL DEFAULT 0
)`) )`)
if err != nil { if err != nil {
@@ -71,6 +78,7 @@ func (m *sqliteManifest) OpenAppend() error {
return fmt.Errorf("create table: %w", err) return fmt.Errorf("create table: %w", err)
} }
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN path TEXT NOT NULL DEFAULT ''`) _, _ = execFn(`ALTER TABLE downloads ADD COLUMN path TEXT NOT NULL DEFAULT ''`)
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN checksum TEXT NOT NULL DEFAULT ''`)
_, err = execFn(`CREATE INDEX IF NOT EXISTS idx_downloads_id ON downloads(id)`) _, err = execFn(`CREATE INDEX IF NOT EXISTS idx_downloads_id ON downloads(id)`)
if err != nil { if err != nil {
db.Close() db.Close()
@@ -80,7 +88,13 @@ func (m *sqliteManifest) OpenAppend() error {
return nil return nil
} }
func (m *sqliteManifest) Has(id string) bool { func (m *Store) DB() *sql.DB { return m.db }
func (m *Store) SetOpen(fn OpenerFunc) { m.open = fn }
func (m *Store) SetExecFunc(fn func(query string, args ...any) (sql.Result, error)) {
m.execFunc = fn
}
func (m *Store) Has(id string) bool {
if m.db == nil { if m.db == nil {
return false return false
} }
@@ -92,49 +106,47 @@ func (m *sqliteManifest) Has(id string) bool {
return count > 0 return count > 0
} }
func (m *sqliteManifest) Add(id string, filename string, size int64, cloud string) { func (m *Store) ManifestFormat() types.Format {
m.AddEntry(newEntry(id, filename, size, cloud)) return types.FormatSQLite
} }
func (m *sqliteManifest) AddEntry(entry Entry) { func (m *Store) Add(id string, filename string, size int64, cloud string) {
m.AddEntry(types.NewEntry(id, filename, filename, size, cloud))
}
func (m *Store) AddEntry(entry types.Entry) {
if m.db == nil { if m.db == nil {
return return
} }
if entry.Path == "" { if entry.Path == "" {
entry.Path = entry.Filename entry.Path = entry.Filename
} }
m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, exported) VALUES (?, ?, ?, ?, ?, ?)`, m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, checksum, exported) VALUES (?, ?, ?, ?, ?, ?, ?)`,
entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Exported) entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Checksum, entry.Exported)
} }
func (m *sqliteManifest) Save() error { func (m *Store) Save() error { return nil }
return nil
}
func (m *sqliteManifest) Close() { func (m *Store) Close() {
if m.db != nil { if m.db != nil {
m.db.Close() m.db.Close()
m.db = nil m.db = nil
} }
} }
func (m *sqliteManifest) DB() *sql.DB { func (m *Store) Entries() map[string]types.Entry {
return m.db
}
func (m *sqliteManifest) Entries() map[string]Entry {
if m.db == nil { if m.db == nil {
return nil return nil
} }
out := make(map[string]Entry) out := make(map[string]types.Entry)
rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, exported FROM downloads`) rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, checksum, exported FROM downloads`)
if err != nil { if err != nil {
return out return out
} }
defer rows.Close() defer rows.Close()
for rows.Next() { for rows.Next() {
var e Entry var e types.Entry
if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Exported); err == nil { if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Checksum, &e.Exported); err == nil {
if e.Path == "" { if e.Path == "" {
e.Path = e.Filename e.Path = e.Filename
} }
@@ -143,3 +155,11 @@ func (m *sqliteManifest) Entries() map[string]Entry {
} }
return out return out
} }
func FileExists(path string) bool {
info, err := os.Stat(path)
if err != nil {
return false
}
return !info.IsDir()
}
+323
View File
@@ -0,0 +1,323 @@
package sqlite
import (
"database/sql"
"fmt"
"path/filepath"
"strings"
"testing"
_ "modernc.org/sqlite"
"gitea.k3s.k0.nu/tools/photocli/internal/manifest/types"
)
func TestStoreOpenAppendSQLError(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
m.SetOpen(func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, fmt.Errorf("simulated open error")
})
if err := m.OpenAppend(); err == nil {
t.Error("expected error from sql.Open failure")
}
}
func TestStoreOpenAppendCreateTableError(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.SetOpen(func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
})
if err := m.OpenAppend(); err == nil {
t.Error("expected error from closed DB CREATE TABLE")
}
}
func TestStoreHasAfterClose(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
if m.Has("x1") {
t.Error("Has should return false after Close")
}
}
func TestStoreEntriesAfterClose(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
entries := m.Entries()
if entries != nil {
t.Errorf("Entries should return nil after Close, got %d entries", len(entries))
}
}
func TestStoreOpenAppendMkdirAllError(t *testing.T) {
m, err := Load("/proc/cannot-write")
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error creating dir under /proc")
m.Close()
}
}
func TestStoreOpenAppendNilOpener(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
m.open = nil
if err := m.OpenAppend(); err != nil {
t.Errorf("expected nil opener to use sql.Open fallback, got err: %v", err)
}
m.Close()
}
func TestStoreOpenAppendCreateIndexError(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from closed DB")
}
}
func TestStoreHasQueryError(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.DB()
closedDB.Close()
if m.Has("x1") {
t.Error("Has should return false with broken DB connection")
}
}
func TestStoreEntriesQueryError(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.DB()
closedDB.Close()
entries := m.Entries()
if entries == nil {
t.Error("Entries should return non-nil map with broken DB connection")
}
if len(entries) != 0 {
t.Errorf("Entries should be empty with broken DB connection, got %d", len(entries))
}
}
func TestStoreHasQueryErrorWithOpenDB(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.DB()
closedDB.Close()
if m.Has("x1") {
t.Error("Has should return false with closed DB")
}
}
func TestStoreEntriesQueryErrorWithOpenDB(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.DB()
closedDB.Close()
entries := m.Entries()
if entries != nil && len(entries) != 0 {
t.Errorf("Entries should be empty with closed DB, got %d", len(entries))
}
}
func TestStoreCreateIndexError(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
m.SetExecFunc(func(query string, args ...any) (sql.Result, error) {
if strings.Contains(query, "CREATE INDEX") {
return nil, fmt.Errorf("injected CREATE INDEX error")
}
return db.Exec(query, args...)
})
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from CREATE INDEX")
m.Close()
}
}
func TestAddEntryDefaultsPath(t *testing.T) {
dir := t.TempDir()
m, err := Load(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(types.Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:def"})
if _, err := m.DB().Exec(`UPDATE downloads SET path = '' WHERE id = 'x1'`); err != nil {
t.Fatal(err)
}
sloaded := m.Entries()["x1"]
if got := sloaded.Path; got != "file.jpg" {
t.Fatalf("sqlite path = %q", got)
}
if sloaded.Checksum != "sha256:def" {
t.Fatalf("sqlite checksum = %q", sloaded.Checksum)
}
m.Close()
}
func TestNewLogWriter(t *testing.T) {
dir := t.TempDir()
dbPath := filepath.Join(dir, "test.db")
db, err := sql.Open("sqlite", dbPath)
if err != nil {
t.Fatal(err)
}
defer db.Close()
lw, err := NewLogWriter(db)
if err != nil {
t.Fatal(err)
}
lw.Log(types.LogEntry{
Timestamp: 1700000000,
Level: "info",
Event: "export_done",
AssetID: "asset-1",
Album: "Favorites",
Filename: "photo.jpg",
Size: 1024,
Cloud: "local",
DurationMs: 500,
})
lw.Log(types.LogEntry{
Timestamp: 1700000001,
Level: "error",
Event: "export_fail",
AssetID: "asset-2",
Filename: "bad.jpg",
Message: "timeout",
})
lw.Close()
var count int
err = db.QueryRow(`SELECT COUNT(*) FROM logs`).Scan(&count)
if err != nil {
t.Fatal(err)
}
if count != 2 {
t.Errorf("expected 2 log entries, got %d", count)
}
var event, level, assetID string
err = db.QueryRow(`SELECT event, level, asset_id FROM logs WHERE asset_id = 'asset-1'`).Scan(&event, &level, &assetID)
if err != nil {
t.Fatal(err)
}
if event != "export_done" || level != "info" || assetID != "asset-1" {
t.Errorf("unexpected row: event=%s level=%s asset_id=%s", event, level, assetID)
}
}
func TestLogWriterNilDB(t *testing.T) {
w := &LogWriter{db: nil}
w.Log(types.LogEntry{Event: "test"})
w.Close()
}
func TestNewLogWriterError(t *testing.T) {
db, err := sql.Open("sqlite", filepath.Join(t.TempDir(), "test.db"))
if err != nil {
t.Fatal(err)
}
if err := db.Close(); err != nil {
t.Fatal(err)
}
if _, err := NewLogWriter(db); err == nil {
t.Error("expected error for closed db")
}
}
func TestLogWriterCloseConcrete(t *testing.T) {
(&LogWriter{}).Close()
}
-402
View File
@@ -1,402 +0,0 @@
package manifest
import (
"database/sql"
"fmt"
"os"
"strings"
"testing"
_ "modernc.org/sqlite"
)
func TestSetJSONLSaveHook(t *testing.T) {
old := SetJSONLSaveHook(func() error { return fmt.Errorf("hook error") })
if old != nil {
t.Error("expected nil old hook")
}
restore := SetJSONLSaveHook(old)
if restore == nil {
t.Error("expected non-nil restore function")
}
}
func TestJSONLSaveHookError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
old := SetJSONLSaveHook(func() error { return fmt.Errorf("hook error") })
defer SetJSONLSaveHook(old)
if err := m.Save(); err == nil {
t.Error("expected hook error from Save")
}
m.Close()
}
func TestJSONLSyncFuncError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.syncFunc = func() error { return fmt.Errorf("sync func error") }
if err := m.Save(); err == nil {
t.Error("expected syncFunc error from Save")
}
m.Close()
}
func TestSQLiteOpenAppendSQLError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, fmt.Errorf("simulated open error")
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from sql.Open failure")
}
}
func TestSQLiteOpenAppendCreateTableError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from closed DB CREATE TABLE")
}
}
func TestSQLiteHasAfterClose(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
if m.Has("x1") {
t.Error("Has should return false after Close")
}
}
func TestSQLiteEntriesAfterClose(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
entries := m.Entries()
if entries != nil {
t.Errorf("Entries should return nil after Close, got %d entries", len(entries))
}
}
func TestSQLiteOpenAppendMkdirAllError(t *testing.T) {
m, err := LoadSQLite("/proc/cannot-write")
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error creating dir under /proc")
m.Close()
}
}
func TestSQLiteOpenAppendNilOpener(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
m.open = nil
if err := m.OpenAppend(); err != nil {
t.Errorf("expected nil opener to use sql.Open fallback, got err: %v", err)
}
m.Close()
}
func TestSQLiteOpenAppendCreateIndexError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from closed DB")
}
}
func TestSQLiteHasQueryError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
result := m.Has("x1")
if result {
t.Error("Has should return false with broken DB connection")
}
}
func TestSQLiteEntriesQueryError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
entries := m.Entries()
if entries == nil {
t.Error("Entries should return non-nil map with broken DB connection")
}
if len(entries) != 0 {
t.Errorf("Entries should be empty with broken DB connection, got %d", len(entries))
}
}
func TestSQLiteHasQueryErrorWithOpenDB(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
if m.Has("x1") {
t.Error("Has should return false with closed DB")
}
}
func TestSQLiteEntriesQueryErrorWithOpenDB(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
closedDB := m.db
closedDB.Close()
entries := m.Entries()
if entries != nil && len(entries) != 0 {
t.Errorf("Entries should be empty with closed DB, got %d", len(entries))
}
}
func TestSQLiteCreateIndexError(t *testing.T) {
dir := t.TempDir()
m, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
realOpen := m.open
callCount := 0
m.open = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
m.execFunc = func(query string, args ...any) (sql.Result, error) {
if strings.Contains(query, "CREATE INDEX") {
return nil, fmt.Errorf("injected CREATE INDEX error")
}
callCount++
return db.Exec(query, args...)
}
return db, nil
}
if err := m.OpenAppend(); err == nil {
t.Error("expected error from CREATE INDEX")
m.Close()
}
}
func TestConvertFromJSONLOpenAppendSQLError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
oldOpen := sqliteOpenFunc
sqliteOpenFunc = func(driverName, dataSourceName string) (*sql.DB, error) {
return nil, fmt.Errorf("simulated sqlite open error")
}
defer func() { sqliteOpenFunc = oldOpen }()
_, err := ConvertFromJSONL(dir)
if err == nil {
t.Error("expected error from dst.OpenAppend during ConvertFromJSONL")
}
}
func TestConvertFromJSONLDstOpenAppendError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.Close()
realOpen := defaultSQLOpener()
oldOpen := sqliteOpenFunc
sqliteOpenFunc = func(driverName, dataSourceName string) (*sql.DB, error) {
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
db.Close()
return db, nil
}
defer func() { sqliteOpenFunc = oldOpen }()
_, err := ConvertFromJSONL(dir)
if err == nil {
t.Error("expected error from dst.OpenAppend during ConvertFromJSONL")
}
}
func TestConvertFromSQLiteSrcOpenAppendError(t *testing.T) {
dir := t.TempDir()
src, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := src.OpenAppend(); err != nil {
t.Fatal(err)
}
src.Add("x1", "photo.jpg", 1024, "local")
src.Close()
realOpen := defaultSQLOpener()
callCount := 0
oldOpen := sqliteOpenFunc
sqliteOpenFunc = func(driverName, dataSourceName string) (*sql.DB, error) {
callCount++
db, err := realOpen(driverName, dataSourceName)
if err != nil {
return nil, err
}
if callCount > 0 {
db.Close()
}
return db, nil
}
defer func() { sqliteOpenFunc = oldOpen }()
_, err = ConvertFromSQLite(dir)
if err == nil {
t.Error("expected error from src.OpenAppend during ConvertFromSQLite")
}
}
func TestConvertFromSQLiteDstOpenAppendError(t *testing.T) {
dir := t.TempDir()
src, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := src.OpenAppend(); err != nil {
t.Fatal(err)
}
src.Add("x1", "photo.jpg", 1024, "local")
src.Close()
jsonlPath := JSONLPath(dir)
f, err := os.Create(jsonlPath)
if err != nil {
t.Fatal(err)
}
f.Close()
os.Chmod(jsonlPath, 0444)
defer os.Chmod(jsonlPath, 0644)
_, err = ConvertFromSQLite(dir)
if err == nil {
t.Error("expected error from dst.OpenAppend during ConvertFromSQLite")
}
}
func TestJSONLSaveError(t *testing.T) {
dir := t.TempDir()
m := LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.Add("x1", "photo.jpg", 1024, "local")
m.file.Close()
if err := m.Save(); err == nil {
t.Error("expected Sync error on closed file")
}
m.Close()
}
func TestConvertFromSQLiteSaveError(t *testing.T) {
dir := t.TempDir()
src, err := LoadSQLite(dir)
if err != nil {
t.Fatal(err)
}
if err := src.OpenAppend(); err != nil {
t.Fatal(err)
}
src.Add("x1", "photo.jpg", 1024, "local")
src.Close()
oldHook := jsonlSaveHook
jsonlSaveHook = func() error { return fmt.Errorf("simulated sync error") }
defer func() { jsonlSaveHook = oldHook }()
_, err = ConvertFromSQLite(dir)
if err == nil {
t.Error("expected error from dst.Save during ConvertFromSQLite")
}
if err != nil && !strings.Contains(err.Error(), "save jsonl") {
t.Errorf("expected save jsonl error, got: %v", err)
}
}
@@ -1,4 +1,4 @@
package manifest package types
import ( import (
"encoding/json" "encoding/json"
+117
View File
@@ -0,0 +1,117 @@
package types
import (
"fmt"
"strings"
)
type Adapter interface {
Format() Format
Aliases() []string
Path(dir string) string
Exists(dir string) bool
Open(dir string) (Manifest, error)
OpenLogWriter(m Manifest, dir string) (LogWriter, error)
}
type Registry struct {
adapters []Adapter
}
func NewRegistry(adapters ...Adapter) Registry {
return Registry{adapters: adapters}
}
func (r Registry) ParseFormat(s string) (Format, error) {
want := strings.ToLower(s)
for _, adapter := range r.adapters {
if want == string(adapter.Format()) {
return adapter.Format(), nil
}
for _, alias := range adapter.Aliases() {
if want == strings.ToLower(alias) {
return adapter.Format(), nil
}
}
}
return "", fmt.Errorf("unknown manifest format %q (use jsonl or sqlite)", s)
}
func (r Registry) Open(dir string, format Format) (Manifest, error) {
dst, err := r.adapter(format)
if err != nil {
return nil, err
}
if dst.Exists(dir) {
return dst.Open(dir)
}
for _, src := range r.adapters {
if src.Format() != dst.Format() && src.Exists(dir) {
return ConvertManifest(dir, src, dst)
}
}
return dst.Open(dir)
}
func (r Registry) OpenLogWriter(m Manifest, dir string, format Format) (LogWriter, error) {
adapter, err := r.adapter(format)
if err != nil {
return nil, err
}
return adapter.OpenLogWriter(m, dir)
}
func (r Registry) adapter(format Format) (Adapter, error) {
for _, adapter := range r.adapters {
if adapter.Format() == format {
return adapter, nil
}
}
return nil, fmt.Errorf("unknown manifest format %q (use jsonl or sqlite)", format)
}
func ConvertManifest(dir string, src Adapter, dst Adapter) (Manifest, error) {
source, err := src.Open(dir)
if err != nil {
return nil, fmt.Errorf("open %s for read: %w", src.Format(), err)
}
if err := source.OpenAppend(); err != nil {
source.Close()
return nil, fmt.Errorf("open %s for read: %w", src.Format(), err)
}
defer source.Close()
target, err := dst.Open(dir)
if err != nil {
return nil, fmt.Errorf("open %s for write: %w", dst.Format(), err)
}
if err := target.OpenAppend(); err != nil {
target.Close()
return nil, fmt.Errorf("open %s for write: %w", dst.Format(), err)
}
for id, entry := range source.Entries() {
entry.ID = id
target.AddEntry(entry)
}
if err := target.Save(); err != nil {
target.Close()
return nil, fmt.Errorf("save %s: %w", dst.Format(), err)
}
if err := removeFile(src.Path(dir)); err != nil {
target.Close()
return nil, err
}
return target, nil
}
var removeFile func(string) error
func SetRemoveFunc(fn func(string) error) {
removeFile = fn
}
func RemoveFunc() func(string) error {
return removeFile
}
+266
View File
@@ -0,0 +1,266 @@
package types
import (
"fmt"
"testing"
)
func TestNewRegistry(t *testing.T) {
r := NewRegistry()
if _, err := r.ParseFormat("jsonl"); err == nil {
t.Fatal("expected error from empty registry")
}
}
func TestRegistryParseFormatAliases(t *testing.T) {
r := NewRegistry(testAdapter{format: FormatJSONL, aliases: []string{"json"}}, testAdapter{format: FormatSQLite, aliases: []string{"db", "sqlite3"}})
cases := map[string]Format{
"jsonl": FormatJSONL,
"json": FormatJSONL,
"sqlite": FormatSQLite,
"db": FormatSQLite,
"sqlite3": FormatSQLite,
}
for input, want := range cases {
got, err := r.ParseFormat(input)
if err != nil {
t.Fatalf("ParseFormat(%q): %v", input, err)
}
if got != want {
t.Fatalf("ParseFormat(%q) = %q, want %q", input, got, want)
}
}
}
func TestRegistryParseFormatUnknown(t *testing.T) {
r := NewRegistry()
if _, err := r.ParseFormat("bad"); err == nil {
t.Fatal("expected unknown format error")
}
}
func TestRegistryOpenDefaultsToOpen(t *testing.T) {
r := NewRegistry(testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}, exists: false})
m, err := r.Open(t.TempDir(), FormatJSONL)
if err != nil {
t.Fatal(err)
}
m.Close()
}
func TestRegistryOpenUnknownFormat(t *testing.T) {
r := NewRegistry()
if _, err := r.Open(t.TempDir(), FormatSQLite); err == nil {
t.Fatal("expected error")
}
}
func TestRegistryOpenNoConversionFallback(t *testing.T) {
r := NewRegistry(
testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}, exists: false},
testAdapter{format: FormatSQLite, store: &testStore{entries: map[string]Entry{}}, exists: false},
)
m, err := r.Open(t.TempDir(), FormatJSONL)
if err != nil {
t.Fatal(err)
}
m.Close()
}
func TestRegistryOpenSameFormatAdapterInLoop(t *testing.T) {
r := NewRegistry(
testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}, exists: false},
testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}, exists: false},
)
m, err := r.Open(t.TempDir(), FormatJSONL)
if err != nil {
t.Fatal(err)
}
m.Close()
}
func TestRegistryOpenOtherFormatNotExists(t *testing.T) {
r := NewRegistry(
testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}, exists: false},
testAdapter{format: FormatSQLite, store: &testStore{entries: map[string]Entry{}}, exists: false},
)
m, err := r.Open(t.TempDir(), FormatJSONL)
if err != nil {
t.Fatal(err)
}
m.Close()
}
func TestRegistryOpenExistingManifest(t *testing.T) {
r := NewRegistry(
testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}, exists: true},
)
m, err := r.Open(t.TempDir(), FormatJSONL)
if err != nil {
t.Fatal(err)
}
m.Close()
}
func TestRegistryOpenConverts(t *testing.T) {
dir := t.TempDir()
src := &testStore{entries: map[string]Entry{"x1": {ID: "x1", Filename: "photo.jpg", Path: "photo.jpg"}}}
dst := &testStore{entries: map[string]Entry{}}
r := NewRegistry(
testAdapter{format: FormatJSONL, store: src, exists: true},
testAdapter{format: FormatSQLite, store: dst, exists: false},
)
SetRemoveFunc(func(string) error { return nil })
m, err := r.Open(dir, FormatSQLite)
if err != nil {
t.Fatal(err)
}
m.Close()
if _, ok := dst.entries["x1"]; !ok {
t.Fatal("expected converted entry")
}
}
func TestRegistryOpenLogWriter(t *testing.T) {
r := NewRegistry(testAdapter{format: FormatJSONL, logWriter: NoopLogWriter})
w, err := r.OpenLogWriter(nil, t.TempDir(), FormatJSONL)
if err != nil {
t.Fatal(err)
}
w.Close()
}
func TestRegistryOpenLogWriterUnknown(t *testing.T) {
r := NewRegistry()
if _, err := r.OpenLogWriter(nil, t.TempDir(), FormatSQLite); err == nil {
t.Fatal("expected error")
}
}
func TestConvertManifest(t *testing.T) {
dir := t.TempDir()
src := &testStore{entries: map[string]Entry{"x1": {ID: "x1", Filename: "photo.jpg", Path: "photo.jpg"}}}
dst := &testStore{entries: map[string]Entry{}}
SetRemoveFunc(func(string) error { return nil })
m, err := ConvertManifest(dir, testAdapter{format: FormatJSONL, store: src, exists: true}, testAdapter{format: FormatSQLite, store: dst, exists: false})
if err != nil {
t.Fatal(err)
}
m.Close()
if _, ok := dst.entries["x1"]; !ok {
t.Fatal("expected converted entry")
}
}
func TestConvertManifestSourceOpenError(t *testing.T) {
if _, err := ConvertManifest(t.TempDir(), testAdapter{format: FormatJSONL, openErr: fmt.Errorf("boom")}, testAdapter{format: FormatSQLite, store: &testStore{}}); err == nil {
t.Fatal("expected error")
}
}
func TestConvertManifestSourceOpenAppendError(t *testing.T) {
if _, err := ConvertManifest(t.TempDir(), testAdapter{format: FormatJSONL, store: &testStore{openAppendErr: fmt.Errorf("boom")}}, testAdapter{format: FormatSQLite, store: &testStore{}}); err == nil {
t.Fatal("expected error")
}
}
func TestConvertManifestDstOpenError(t *testing.T) {
if _, err := ConvertManifest(t.TempDir(), testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}}, testAdapter{format: FormatSQLite, openErr: fmt.Errorf("boom")}); err == nil {
t.Fatal("expected error")
}
}
func TestConvertManifestDstOpenAppendError(t *testing.T) {
if _, err := ConvertManifest(t.TempDir(), testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}}, testAdapter{format: FormatSQLite, store: &testStore{openAppendErr: fmt.Errorf("boom")}}); err == nil {
t.Fatal("expected error")
}
}
func TestConvertManifestDstSaveError(t *testing.T) {
if _, err := ConvertManifest(t.TempDir(), testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}}, testAdapter{format: FormatSQLite, store: &testStore{saveErr: fmt.Errorf("boom")}}); err == nil {
t.Fatal("expected error")
}
}
func TestConvertManifestRemoveError(t *testing.T) {
SetRemoveFunc(func(string) error { return fmt.Errorf("remove failed") })
defer SetRemoveFunc(nil)
if _, err := ConvertManifest(t.TempDir(), testAdapter{format: FormatJSONL, store: &testStore{entries: map[string]Entry{}}}, testAdapter{format: FormatSQLite, store: &testStore{}}); err == nil {
t.Fatal("expected error")
}
}
func TestSetRemoveFunc(t *testing.T) {
SetRemoveFunc(func(string) error { return nil })
if RemoveFunc() == nil {
t.Fatal("expected non-nil remove func")
}
SetRemoveFunc(nil)
if RemoveFunc() != nil {
t.Fatal("expected nil remove func")
}
}
func TestNoopLogWriterMethods(t *testing.T) {
NoopLogWriter.Log(LogEntry{})
NoopLogWriter.Close()
var n noopLogWriter
n.Log(LogEntry{})
n.Close()
}
type testAdapter struct {
format Format
aliases []string
store Manifest
exists bool
openErr error
logWriter LogWriter
}
func (a testAdapter) Format() Format { return a.format }
func (a testAdapter) Aliases() []string { return a.aliases }
func (a testAdapter) Path(string) string { return "test" }
func (a testAdapter) Exists(string) bool { return a.exists }
func (a testAdapter) Open(string) (Manifest, error) {
if a.openErr != nil {
return nil, a.openErr
}
return a.store, nil
}
func (a testAdapter) OpenLogWriter(Manifest, string) (LogWriter, error) {
if a.logWriter != nil {
return a.logWriter, nil
}
return NoopLogWriter, nil
}
type testStore struct {
entries map[string]Entry
openAppendErr error
saveErr error
}
func (s *testStore) Has(id string) bool {
_, ok := s.entries[id]
return ok
}
func (s *testStore) Add(id string, filename string, size int64, cloud string) {
s.entries[id] = NewEntry(id, filename, filename, size, cloud)
}
func (s *testStore) AddEntry(e Entry) {
if e.Path == "" {
e.Path = e.Filename
}
s.entries[e.ID] = e
}
func (s *testStore) Save() error { return s.saveErr }
func (s *testStore) Close() {}
func (s *testStore) OpenAppend() error { return s.openAppendErr }
func (s *testStore) Entries() map[string]Entry {
out := make(map[string]Entry, len(s.entries))
for k, v := range s.entries {
out[k] = v
}
return out
}
+86
View File
@@ -0,0 +1,86 @@
package types
import "time"
type Entry struct {
ID string
Filename string
Path string
Size int64
Cloud string
Checksum string
Exported int64
}
type Manifest interface {
Has(id string) bool
Add(id string, filename string, size int64, cloud string)
AddEntry(entry Entry)
Save() error
Close()
OpenAppend() error
Entries() map[string]Entry
}
type EntryReader = Manifest
type Format string
const (
FormatJSONL Format = "jsonl"
FormatSQLite Format = "sqlite"
)
type LogEntry struct {
Timestamp int64 `json:"ts"`
Level string `json:"level"`
Event string `json:"event"`
AssetID string `json:"asset_id,omitempty"`
Album string `json:"album,omitempty"`
Filename string `json:"filename,omitempty"`
Size int64 `json:"size,omitempty"`
Cloud string `json:"cloud,omitempty"`
DurationMs int64 `json:"duration_ms,omitempty"`
Message string `json:"message,omitempty"`
}
type LogWriter interface {
Log(entry LogEntry)
Close()
}
type noopLogWriter struct{}
func (noopLogWriter) Log(LogEntry) { _ = struct{}{} }
func (noopLogWriter) Close() { _ = struct{}{} }
var NoopLogWriter LogWriter = noopLogWriter{}
type FormatReporter interface {
ManifestFormat() Format
}
func LogPath(dir string) string {
return dir + "/export.log"
}
func NewEntry(id, filename, path string, size int64, cloud string) Entry {
e := Entry{
ID: id,
Filename: filename,
Path: filename,
Size: size,
Cloud: cloud,
Exported: time.Now().Unix(),
}
if path != "" {
e.Path = path
}
return e
}
func NewEntryWithChecksum(id, filename, path string, size int64, cloud, checksum string) Entry {
e := NewEntry(id, filename, path, size, cloud)
e.Checksum = checksum
return e
}
+77
View File
@@ -0,0 +1,77 @@
package types
import (
"os"
"testing"
)
func TestNewEntryPath(t *testing.T) {
e := NewEntry("id1", "file.jpg", "", 123, "local")
if e.ID != "id1" || e.Filename != "file.jpg" || e.Path != "file.jpg" || e.Size != 123 || e.Cloud != "local" || e.Exported == 0 {
t.Fatalf("unexpected entry: %+v", e)
}
e = NewEntry("id2", "file2.jpg", "Album/file2.jpg", 456, "cloud")
if e.Path != "Album/file2.jpg" || e.Filename != "file2.jpg" || e.Size != 456 || e.Cloud != "cloud" {
t.Fatalf("unexpected entry with path: %+v", e)
}
e = NewEntryWithChecksum("id3", "file3.jpg", "Album/file3.jpg", 789, "local", "sha256:abc")
if e.Checksum != "sha256:abc" || e.Path != "Album/file3.jpg" {
t.Fatalf("unexpected checksum entry: %+v", e)
}
}
func TestNoopLogWriter(t *testing.T) {
NoopLogWriter.Log(LogEntry{Event: "test"})
NoopLogWriter.Close()
}
func TestNewFileLogWriter(t *testing.T) {
dir := t.TempDir()
path := LogPath(dir)
w, err := NewFileLogWriter(path)
if err != nil {
t.Fatal(err)
}
w.Log(LogEntry{Timestamp: 1700000000, Level: "info", Event: "export_done", Filename: "photo.jpg", Size: 2048})
w.Close()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if len(data) == 0 {
t.Error("expected log data")
}
if data[len(data)-1] != '\n' {
t.Error("expected trailing newline")
}
}
func TestFileLogWriterClosed(t *testing.T) {
w, err := NewFileLogWriter(LogPath(t.TempDir()))
if err != nil {
t.Fatal(err)
}
w.Close()
w.Log(LogEntry{Event: "after-close"})
}
func TestNewFileLogWriterError(t *testing.T) {
if _, err := NewFileLogWriter("/nonexistent/dir/export.log"); err == nil {
t.Error("expected error for bad path")
}
}
func TestFileLogWriterDoubleClose(t *testing.T) {
w, err := NewFileLogWriter(LogPath(t.TempDir()))
if err != nil {
t.Fatal(err)
}
w.Close()
w.Close()
}
func TestLogPath(t *testing.T) {
if got := LogPath("/tmp/out"); got != "/tmp/out/export.log" {
t.Fatalf("LogPath = %q", got)
}
}
+12
View File
@@ -9,6 +9,7 @@ type Bridge interface {
RequestAccess() error RequestAccess() error
ListAlbums() ([]Album, error) ListAlbums() ([]Album, error)
ListAssets(albumID string) ([]Asset, int, error) ListAssets(albumID string) ([]Asset, int, error)
ReverseGeocode(latitude, longitude float64) (Placemark, error)
ListTree() ([]CollectionNode, error) ListTree() ([]CollectionNode, error)
ExportPreview(assetID, outputDir string, targetSize, quality, index int) (ExportResult, error) ExportPreview(assetID, outputDir string, targetSize, quality, index int) (ExportResult, error)
ExportOriginal(assetID, outputDir string, index int) (ExportResult, error) ExportOriginal(assetID, outputDir string, index int) (ExportResult, error)
@@ -39,6 +40,17 @@ func ParseAssetsJSON(jsonStr string) ([]Asset, int, error) {
return resp.Assets, resp.Total, nil return resp.Assets, resp.Total, nil
} }
func ParsePlacemarkJSON(jsonStr string) (Placemark, error) {
var resp PlacemarkResponse
if err := json.Unmarshal([]byte(jsonStr), &resp); err != nil {
return Placemark{}, err
}
if resp.Error != "" {
return Placemark{}, fmt.Errorf("%s", resp.Error)
}
return resp.Placemark, nil
}
func ParseTreeJSON(jsonStr string) ([]CollectionNode, error) { func ParseTreeJSON(jsonStr string) ([]CollectionNode, error) {
var errResp ErrorResponse var errResp ErrorResponse
if err := json.Unmarshal([]byte(jsonStr), &errResp); err == nil && errResp.Error != "" { if err := json.Unmarshal([]byte(jsonStr), &errResp); err == nil && errResp.Error != "" {
+10 -1
View File
@@ -4,7 +4,7 @@ package photos
/* /*
#cgo CFLAGS: -I${SRCDIR}/../../bridge #cgo CFLAGS: -I${SRCDIR}/../../bridge
#cgo LDFLAGS: -L${SRCDIR}/../../bridge -lphotokit_bridge -framework Photos -framework Foundation -framework AppKit -framework UniformTypeIdentifiers #cgo LDFLAGS: -L${SRCDIR}/../../bridge -lphotokit_bridge -framework Photos -framework Foundation -framework AppKit -framework UniformTypeIdentifiers -framework CoreLocation -framework MapKit
#include "photokit_bridge.h" #include "photokit_bridge.h"
#include <stdlib.h> #include <stdlib.h>
*/ */
@@ -45,6 +45,15 @@ func (*CgoBridge) ListAssets(albumID string) ([]Asset, int, error) {
return ParseAssetsJSON(C.GoString(cs)) return ParseAssetsJSON(C.GoString(cs))
} }
func (*CgoBridge) ReverseGeocode(latitude, longitude float64) (Placemark, error) {
cs := C.photos_reverse_geocode_json(C.double(latitude), C.double(longitude))
if cs == nil {
return Placemark{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParsePlacemarkJSON(C.GoString(cs))
}
func (*CgoBridge) ListTree() ([]CollectionNode, error) { func (*CgoBridge) ListTree() ([]CollectionNode, error) {
cs := C.photos_list_tree_json() cs := C.photos_list_tree_json()
if cs == nil { if cs == nil {
+13
View File
@@ -12,6 +12,8 @@ void photos_test_set_access(int rc);
void photos_test_set_albums(const char *json); void photos_test_set_albums(const char *json);
void photos_test_set_assets(const char *json); void photos_test_set_assets(const char *json);
void photos_test_set_tree(const char *json); void photos_test_set_tree(const char *json);
void photos_test_set_geocode(const char *json);
void photos_test_set_geocode_null(void);
void photos_test_set_albums_null(void); void photos_test_set_albums_null(void);
void photos_test_set_assets_null(void); void photos_test_set_assets_null(void);
void photos_test_set_tree_null(void); void photos_test_set_tree_null(void);
@@ -35,6 +37,8 @@ func SetTestAccessRC(rc int) { C.photos_test_set_access(C.int(rc)
func SetTestAlbumsJSON(json string) { C.photos_test_set_albums(C.CString(json)) } func SetTestAlbumsJSON(json string) { C.photos_test_set_albums(C.CString(json)) }
func SetTestAssetsJSON(json string) { C.photos_test_set_assets(C.CString(json)) } func SetTestAssetsJSON(json string) { C.photos_test_set_assets(C.CString(json)) }
func SetTestTreeJSON(json string) { C.photos_test_set_tree(C.CString(json)) } func SetTestTreeJSON(json string) { C.photos_test_set_tree(C.CString(json)) }
func SetTestGeocodeJSON(json string) { C.photos_test_set_geocode(C.CString(json)) }
func SetTestGeocodeNull() { C.photos_test_set_geocode_null() }
func SetTestAlbumsNull() { C.photos_test_set_albums_null() } func SetTestAlbumsNull() { C.photos_test_set_albums_null() }
func SetTestAssetsNull() { C.photos_test_set_assets_null() } func SetTestAssetsNull() { C.photos_test_set_assets_null() }
func SetTestTreeNull() { C.photos_test_set_tree_null() } func SetTestTreeNull() { C.photos_test_set_tree_null() }
@@ -73,6 +77,15 @@ func (*CgoBridge) ListAssets(albumID string) ([]Asset, int, error) {
return ParseAssetsJSON(C.GoString(cs)) return ParseAssetsJSON(C.GoString(cs))
} }
func (*CgoBridge) ReverseGeocode(latitude, longitude float64) (Placemark, error) {
cs := C.photos_reverse_geocode_json(C.double(latitude), C.double(longitude))
if cs == nil {
return Placemark{}, errBridgeNil
}
defer C.photos_free_string(cs)
return ParsePlacemarkJSON(C.GoString(cs))
}
func (*CgoBridge) ListTree() ([]CollectionNode, error) { func (*CgoBridge) ListTree() ([]CollectionNode, error) {
cs := C.photos_list_tree_json() cs := C.photos_list_tree_json()
if cs == nil { if cs == nil {
+57
View File
@@ -168,6 +168,63 @@ func TestParseAssetsJSON(t *testing.T) {
} }
} }
func TestParseAssetsJSONExtendedMetadata(t *testing.T) {
created := "2024-01-01T00:00:00Z"
modified := "2024-01-02T00:00:00Z"
assets, total, err := ParseAssetsJSON(`{"assets":[{"id":"x1","filename":"IMG.HEIC","mediaType":"image","mediaSubtypes":["photoLive","photoHDR"],"sourceType":"userLibrary","playbackStyle":"livePhoto","pixelWidth":1,"pixelHeight":2,"creationDate":"2024-01-01T00:00:00Z","modificationDate":"2024-01-02T00:00:00Z","duration":3.5,"isFavorite":true,"isHidden":true,"hasAdjustments":true,"location":{"latitude":59.1,"longitude":18.2,"altitude":10,"horizontalAccuracy":5},"burstIdentifier":"burst","representsBurst":true,"burstSelectionTypes":["autoPick"],"adjustmentInfo":{"formatIdentifier":"fmt","formatVersion":"1","baseFilename":"base.heic"},"resources":[{"type":"adjustmentData","filename":"adj.plist","uti":"public.plist","local":true,"size":42}]}],"total":1}`)
if err != nil || total != 1 || len(assets) != 1 {
t.Fatalf("ParseAssetsJSON err=%v total=%d len=%d", err, total, len(assets))
}
a := assets[0]
if a.CreationDate == nil || *a.CreationDate != created || a.ModificationDate == nil || *a.ModificationDate != modified {
t.Fatalf("unexpected dates: %+v", a)
}
if a.Location == nil || a.Location.Latitude != 59.1 || a.Location.Longitude != 18.2 || !a.IsHidden || !a.HasAdjustments || !a.RepresentsBurst {
t.Fatalf("unexpected extended metadata: %+v", a)
}
if len(a.MediaSubtypes) != 2 || a.SourceType != "userLibrary" || a.PlaybackStyle != "livePhoto" || len(a.BurstSelectionTypes) != 1 {
t.Fatalf("unexpected type metadata: %+v", a)
}
if a.AdjustmentInfo == nil || a.AdjustmentInfo.FormatIdentifier != "fmt" || a.Resources[0].Size != 42 {
t.Fatalf("unexpected adjustment/resource metadata: %+v", a)
}
}
func TestParsePlacemarkJSON(t *testing.T) {
p, err := ParsePlacemarkJSON(`{"placemark":{"country":"Sweden","countryCode":"SE","locality":"Stockholm","formattedAddress":"Stockholm, Sweden","areasOfInterest":["Gamla stan"]}}`)
if err != nil {
t.Fatal(err)
}
if p.Country != "Sweden" || p.CountryCode != "SE" || p.Locality != "Stockholm" || len(p.AreasOfInterest) != 1 {
t.Fatalf("unexpected placemark: %+v", p)
}
if _, err := ParsePlacemarkJSON(`{"error":"geocode failed"}`); err == nil || err.Error() != "geocode failed" {
t.Fatalf("expected geocode error, got %v", err)
}
if _, err := ParsePlacemarkJSON(`bad`); err == nil {
t.Fatal("expected invalid JSON error")
}
}
func TestCgoBridgeReverseGeocode(t *testing.T) {
SetTestGeocodeJSON(`{"placemark":{"country":"Sweden","locality":"Stockholm"}}`)
p, err := (&CgoBridge{}).ReverseGeocode(59.3293, 18.0686)
if err != nil {
t.Fatal(err)
}
if p.Country != "Sweden" || p.Locality != "Stockholm" {
t.Fatalf("unexpected placemark: %+v", p)
}
SetTestGeocodeJSON(`{"error":"no network"}`)
if _, err := (&CgoBridge{}).ReverseGeocode(0, 0); err == nil || err.Error() != "no network" {
t.Fatalf("expected geocode error, got %v", err)
}
SetTestGeocodeNull()
if _, err := (&CgoBridge{}).ReverseGeocode(0, 0); err != errBridgeNil {
t.Fatalf("expected nil bridge error, got %v", err)
}
}
func TestParseTreeJSON(t *testing.T) { func TestParseTreeJSON(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
+57 -11
View File
@@ -6,17 +6,40 @@ type Album struct {
} }
type Asset struct { type Asset struct {
ID string `json:"id"` ID string `json:"id"`
Filename string `json:"filename"` Filename string `json:"filename"`
Cloud string `json:"cloud"` Cloud string `json:"cloud"`
MediaType string `json:"mediaType"` MediaType string `json:"mediaType"`
PixelWidth int `json:"pixelWidth"` MediaSubtypes []string `json:"mediaSubtypes,omitempty"`
PixelHeight int `json:"pixelHeight"` SourceType string `json:"sourceType,omitempty"`
CreationDate *string `json:"creationDate,omitempty"` PlaybackStyle string `json:"playbackStyle,omitempty"`
Duration float64 `json:"duration,omitempty"` PixelWidth int `json:"pixelWidth"`
IsFavorite bool `json:"isFavorite,omitempty"` PixelHeight int `json:"pixelHeight"`
HasAdjustments bool `json:"hasAdjustments,omitempty"` CreationDate *string `json:"creationDate,omitempty"`
Resources []AssetResource `json:"resources,omitempty"` ModificationDate *string `json:"modificationDate,omitempty"`
Duration float64 `json:"duration,omitempty"`
IsFavorite bool `json:"isFavorite,omitempty"`
IsHidden bool `json:"isHidden,omitempty"`
HasAdjustments bool `json:"hasAdjustments,omitempty"`
Location *AssetLocation `json:"location,omitempty"`
BurstIdentifier string `json:"burstIdentifier,omitempty"`
RepresentsBurst bool `json:"representsBurst,omitempty"`
BurstSelectionTypes []string `json:"burstSelectionTypes,omitempty"`
AdjustmentInfo *AdjustmentInfo `json:"adjustmentInfo,omitempty"`
Resources []AssetResource `json:"resources,omitempty"`
}
type AssetLocation struct {
Latitude float64 `json:"latitude"`
Longitude float64 `json:"longitude"`
Altitude float64 `json:"altitude,omitempty"`
HorizontalAccuracy float64 `json:"horizontalAccuracy,omitempty"`
}
type AdjustmentInfo struct {
FormatIdentifier string `json:"formatIdentifier,omitempty"`
FormatVersion string `json:"formatVersion,omitempty"`
BaseFilename string `json:"baseFilename,omitempty"`
} }
type AssetResource struct { type AssetResource struct {
@@ -24,6 +47,29 @@ type AssetResource struct {
Filename string `json:"filename"` Filename string `json:"filename"`
UTI string `json:"uti"` UTI string `json:"uti"`
Local bool `json:"local"` Local bool `json:"local"`
Size int64 `json:"size,omitempty"`
}
type Placemark struct {
Name string `json:"name,omitempty"`
Country string `json:"country,omitempty"`
CountryCode string `json:"countryCode,omitempty"`
AdministrativeArea string `json:"administrativeArea,omitempty"`
SubAdministrativeArea string `json:"subAdministrativeArea,omitempty"`
Locality string `json:"locality,omitempty"`
SubLocality string `json:"subLocality,omitempty"`
Thoroughfare string `json:"thoroughfare,omitempty"`
SubThoroughfare string `json:"subThoroughfare,omitempty"`
PostalCode string `json:"postalCode,omitempty"`
FormattedAddress string `json:"formattedAddress,omitempty"`
InlandWater string `json:"inlandWater,omitempty"`
Ocean string `json:"ocean,omitempty"`
AreasOfInterest []string `json:"areasOfInterest,omitempty"`
}
type PlacemarkResponse struct {
Placemark Placemark `json:"placemark"`
Error string `json:"error,omitempty"`
} }
type ExportResult struct { type ExportResult struct {