Compare commits

6 Commits

Author SHA1 Message Date
Ein Anderssono bed3ea7bd0 v0.9.1: add deep checksum verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:23:22 +02:00
Ein Anderssono 7555b561bd v0.9.0: add manifest checksums
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:19:47 +02:00
Ein Anderssono d909d30b87 v0.8.7: add JSON sidecars
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:11:02 +02:00
Ein Anderssono 5c40b1d3ba v0.8.6: add XMP keyword and rating controls
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:05:14 +02:00
Ein Anderssono 700d8ef05a v0.8.5: add XMP privacy controls
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:02:26 +02:00
Ein Anderssono fbc37b8d8d v0.8.4: add strict sidecar verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 01:58:38 +02:00
11 changed files with 672 additions and 45 deletions
+48
View File
@@ -2,6 +2,54 @@
This changelog is maintained from git history plus the published Gitea release series. Future releases should update this file and publish matching release notes on the release page.
## v0.9.1
Deep checksum verification release.
- Add `verify --deep` to recompute SHA-256 checksums for manifest entries that have checksum metadata.
- Report `checksum-mismatch` when disk contents differ from the manifest checksum.
- Keep normal `verify` unchanged unless `--deep` is selected.
## v0.9.0
Manifest checksum release.
- Add opt-in `--checksum sha256` to store SHA-256 file checksums in JSONL and SQLite manifests.
- Add SQLite migration support for the manifest `checksum` column.
- Keep checksum collection disabled by default with `--checksum none`.
## v0.8.7
JSON sidecar release.
- Add `--sidecar json` for structured JSON metadata sidecars.
- Add `--sidecar xmp,json` to write both XMP and JSON sidecars from the same metadata.
- Keep `--sidecar none` as the default.
## v0.8.6
XMP keyword and rating controls release.
- Add `--xmp-keywords album-path|album|none` for generated sidecars.
- Add `--xmp-rating favorite|none` for generated sidecars.
- Keep existing keyword/rating behavior as defaults with `album-path` and `favorite`.
## v0.8.5
XMP privacy controls release.
- Add `--xmp-privacy keep|strip-location|strip-address` for generated sidecars.
- Keep existing XMP location/address behavior as the default with `keep`.
- Allow GPS coordinates to be kept while reverse-geocoded address fields are omitted with `strip-address`.
- Allow both GPS coordinates and address fields to be omitted with `strip-location`.
## v0.8.4
Strict XMP sidecar verification release.
- Add `verify --sidecar --strict` to require photoscli XMP schema metadata, sidecar generator metadata, and matching exported filename metadata.
- Keep existing `verify --sidecar` behavior unchanged for backup-wide existence, readability, and asset-ID checks.
## v0.8.3
XMP sidecar inspection release.
+1 -1
View File
@@ -1,6 +1,6 @@
BINARY := ./bin/photoscli
MODULE := gitea.k3s.k0.nu/tools/photocli
VERSION := 0.8.3
VERSION := 0.9.1
RELEASE_ZIP := ./bin/photoscli-$(VERSION)-macos-arm64.zip
RELEASE_NOTES := RELEASE_NOTES.md
BRIDGE_DIR := bridge
+16
View File
@@ -21,6 +21,7 @@ For a practical step-by-step manual with recommended backup workflows, recovery
- Deduplicated failure management with `failures list`, `failures clear`, and `retry-failed --clear-on-success`.
- Verification, reporting, and diff commands for backup integrity.
- Optional XMP sidecar verification with `verify --sidecar`.
- Optional SHA-256 manifest checksums with `--checksum sha256`.
- Metadata-only XMP refresh for manifest-backed exports with `--metadata-only`.
- Status command for quick backup summaries.
- Opt-in rich XMP sidecar metadata with `--sidecar xmp`.
@@ -116,6 +117,7 @@ Verify a backup later:
```bash
./bin/photoscli verify --out ./photos-backup --manifest sqlite
./bin/photoscli verify --out ./photos-backup --deep
```
## Commands
@@ -186,6 +188,7 @@ Useful examples:
```bash
photoscli backup-all --out ./backup --manifest sqlite --log
photoscli backup-all --out ./backup --checksum sha256
photoscli backup-all --out ./backup --exclude-album "Recently Deleted" --exclude-album "Temp*"
photoscli backup-all --out ./backup --since 2024-01-01 --sort newest
photoscli backup-all --out ./backup --concurrency 8 --retry 3
@@ -344,6 +347,8 @@ Write archival metadata sidecars with:
```bash
photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json
```
Sidecars are opt-in and use the exported file basename:
@@ -351,18 +356,29 @@ Sidecars are opt-in and use the exported file basename:
```text
IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json
```
The XMP contains photoscli metadata such as asset ID, filenames, album, manifest path, media type, dimensions, favorite state, cloud state, export mode, version, exported timestamp, size, and creation date when available. If `--sidecar xmp` is explicitly selected and the sidecar cannot be written, that asset is treated as failed.
Sidecars also include richer public PhotoKit metadata where available: modification date, duration, hidden state, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP dates, EXIF GPS coordinates, favorite rating, and album/folder keywords. Add `--reverse-geocode` to include cached address fields from Apple MapKit for assets with GPS coordinates. Reverse geocoding requires macOS 26 or newer; on older macOS versions the export continues and XMP still includes GPS coordinates.
Control XMP location metadata with `--xmp-privacy keep|strip-location|strip-address`. The default is `keep`. Use `strip-address` to omit reverse-geocoded address fields while keeping GPS coordinates, or `strip-location` to omit both GPS and address fields.
Control generated XMP keywords and ratings with `--xmp-keywords album-path|album|none` and `--xmp-rating favorite|none`. Defaults preserve existing behavior: album/folder keywords and favorite assets mapped to `xmp:Rating="5"`.
Verify generated sidecars with:
```bash
photoscli verify --out ./backup --sidecar
```
For stricter checks against recent photoscli-generated XMP sidecars:
```bash
photoscli verify --out ./backup --sidecar --strict
```
Inspect one generated sidecar with:
```bash
+7 -6
View File
@@ -1,17 +1,18 @@
# v0.8.3
# v0.9.1
This release adds XMP sidecar inspection for quick validation and scripting.
This release adds deep checksum verification for manifest-backed backups.
## Highlights
- Add `sidecar inspect <file.xmp>` to print photoscli metadata embedded in a generated sidecar.
- Add `sidecar inspect <file.xmp> --json` for scriptable output.
- Keep `verify --sidecar` for backup-wide checks and use `sidecar inspect` for one-file investigation.
- Add `verify --deep` to recompute SHA-256 checksums when manifest entries include checksums.
- Report `checksum-mismatch` when file contents differ from the manifest checksum.
- Keep normal `verify` unchanged unless `--deep` is selected.
- `--deep` works with JSONL and SQLite manifests.
## Assets
- `photoscli`: Apple Silicon macOS binary (`darwin/arm64`).
- `photoscli-0.8.3-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
- `photoscli-0.9.1-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
- `USERGUIDE.md`: standalone user guide.
Intel Macs are not currently a supported release target.
+48
View File
@@ -21,6 +21,7 @@ It is especially useful when you want to:
- Retry transient iCloud failures.
- Verify that files referenced by a manifest exist on disk.
- Detect missing, zero-byte, and size-mismatched manifest files.
- Store optional SHA-256 checksums in manifests.
- Inspect or clear deduplicated failure records.
- Write optional XMP sidecar metadata for archival workflows.
- Add optional reverse-geocoded address metadata to XMP sidecars for GPS assets.
@@ -546,6 +547,8 @@ Use XMP sidecars when you want portable metadata next to exported files:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar json
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp,json
```
Sidecars are disabled by default. When enabled, they use the exported file basename:
@@ -553,10 +556,31 @@ Sidecars are disabled by default. When enabled, they use the exported file basen
```text
IMG_0001.jpg -> IMG_0001.xmp
IMG_0001.HEIC -> IMG_0001.xmp
IMG_0001.jpg -> IMG_0001.json
```
Use XMP for standards-oriented metadata workflows and JSON when you want structured photoscli metadata for scripts or audits.
The XMP includes photoscli archive metadata such as asset ID, original filename, exported filename, album, manifest path, media type, dimensions, favorite state, hidden state, cloud state, export mode, version, exported time, size, creation date, modification date, duration, adjustment state, media subtypes, source type, playback style, burst data, GPS coordinates, adjustment info, structured asset resources, standard XMP date fields, EXIF GPS fields, favorite rating, and album/folder keywords when PhotoKit exposes them.
Control location privacy in generated sidecars:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-address
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-privacy strip-location
```
`keep` is the default. `strip-address` omits reverse-geocoded address fields while preserving GPS coordinates. `strip-location` omits both GPS coordinates and address fields.
Control generated keywords and ratings:
```bash
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords album
./bin/photoscli export --album-id "Vacation" --out ./Vacation --sidecar xmp --xmp-keywords none --xmp-rating none
```
`--xmp-keywords album-path` is the default and writes album/folder keywords. `album` writes only the album name. `none` omits generated `dc:subject` keywords. `--xmp-rating favorite` is the default and maps favorite assets to `xmp:Rating="5"`; `none` omits that generated rating.
For address metadata from GPS coordinates, opt in to Apple's reverse geocoder:
```bash
@@ -573,6 +597,14 @@ Verify sidecars after an export:
This reports missing, zero-byte, unreadable, or asset-ID mismatched `.xmp` files.
Use strict verification for sidecars generated by recent photoscli versions:
```bash
./bin/photoscli verify --out ./PhotosBackup --sidecar --strict
```
Strict mode also checks photoscli schema metadata, generator metadata, and the exported filename recorded inside the sidecar.
Inspect one generated sidecar when troubleshooting or scripting:
```bash
@@ -790,6 +822,22 @@ Use structured logs:
./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log
```
Store SHA-256 checksums in the manifest when exporting:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --checksum sha256
```
Checksums are opt-in and stored in JSONL or SQLite manifests. The default is `--checksum none`.
Deep-verify checksum-backed manifests:
```bash
./bin/photoscli verify --out ./PhotosBackup --deep
```
`--deep` recomputes SHA-256 only for manifest entries that already have checksum metadata.
## Safe Operating Practices
- Run `--dry-run` before the first large backup.
+212 -23
View File
@@ -2,6 +2,8 @@ package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"encoding/xml"
"fmt"
@@ -42,6 +44,10 @@ type exportOptions struct {
verify bool
format string
sidecar string
checksum string
xmpPrivacy string
xmpKeywords string
xmpRating string
metadataOnly bool
reverseGeocode bool
minSize int64
@@ -172,7 +178,9 @@ COMMANDS
verify --out <dir> [--manifest jsonl|sqlite]
Verify that manifest entries point to files that exist on disk. Missing
files are printed as <asset-id><TAB><filename>. Exits 2 on missing files.
Add --sidecar to verify expected XMP sidecars too.
Add --sidecar to verify expected XMP sidecars too. Add --strict with
--sidecar to require photoscli schema/generator and exported filename metadata.
Add --deep to verify manifest SHA-256 checksums when present.
retry-failed --out <dir>
Retry assets previously written to failures.jsonl.
@@ -224,9 +232,21 @@ COMMON EXPORT FLAGS
--verify
Run manifest/file verification after export or backup-all.
--sidecar none|xmp
Write opt-in XMP sidecar metadata next to each exported file. Default:
none. If XMP writing fails, the asset is counted as failed.
--checksum none|sha256
Store optional file checksum metadata in the manifest. Default: none.
--sidecar none|xmp|json|xmp,json
Write opt-in metadata sidecars next to each exported file. Default: none.
If sidecar writing fails, the asset is counted as failed.
--xmp-privacy keep|strip-location|strip-address
Control location/address metadata in generated XMP sidecars. Default: keep.
--xmp-keywords album-path|album|none
Control dc:subject keywords in generated XMP sidecars. Default: album-path.
--xmp-rating favorite|none
Control favorite-to-rating mapping in generated XMP sidecars. Default: favorite.
--metadata-only
With --sidecar xmp, write or refresh XMP sidecars for files already in
@@ -776,7 +796,7 @@ func logEntry(event, level, assetID, album, filename, cloud string, size int64,
}
}
func addManifestEntry(m manifest.Manifest, pa pendingAsset, result photos.ExportResult) {
func addManifestEntry(m manifest.Manifest, pa pendingAsset, result photos.ExportResult, checksum string) {
if m == nil {
return
}
@@ -789,7 +809,33 @@ func addManifestEntry(m manifest.Manifest, pa pendingAsset, result photos.Export
if err != nil || strings.HasPrefix(relPath, "..") {
relPath = result.Filename
}
m.AddEntry(manifest.NewEntry(pa.asset.ID, result.Filename, relPath, result.Size, result.Cloud))
m.AddEntry(manifest.NewEntryWithChecksum(pa.asset.ID, result.Filename, relPath, result.Size, result.Cloud, checksum))
}
func addManifestEntryForResult(m manifest.Manifest, pa pendingAsset, result photos.ExportResult, opts exportOptions) error {
checksum := ""
if opts.checksum == "sha256" && !result.Skipped {
var err error
checksum, err = fileSHA256(filepath.Join(pa.path, result.Filename))
if err != nil {
return err
}
}
addManifestEntry(m, pa, result, checksum)
return nil
}
func fileSHA256(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
type xmpSidecarData struct {
@@ -888,6 +934,20 @@ func sidecarPath(exportedPath string) string {
return strings.TrimSuffix(exportedPath, ext) + ".xmp"
}
func jsonSidecarPath(exportedPath string) string {
ext := filepath.Ext(exportedPath)
return strings.TrimSuffix(exportedPath, ext) + ".json"
}
func sidecarEnabled(sidecar, format string) bool {
for _, part := range strings.Split(sidecar, ",") {
if strings.TrimSpace(part) == format {
return true
}
}
return false
}
func renderXMP(d xmpSidecarData) []byte {
attrs := []struct{ key, val string }{
{"photoscli:xmpSchemaVersion", "2"},
@@ -1061,8 +1121,31 @@ func writeXMPSidecar(path string, data xmpSidecarData) error {
return nil
}
func writeJSONSidecar(path string, data xmpSidecarData) error {
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
return err
}
f, err := createTempFunc(filepath.Dir(path), ".*.json.tmp")
if err != nil {
return err
}
tmp := f.Name()
_ = f.Close()
payload, _ := json.MarshalIndent(data, "", " ")
payload = append(payload, '\n')
if err := writeFileFunc(tmp, payload, 0644); err != nil {
os.Remove(tmp)
return err
}
if err := renameFunc(tmp, path); err != nil {
os.Remove(tmp)
return err
}
return nil
}
func writeSidecarIfNeeded(pa pendingAsset, result photos.ExportResult, originals bool, opts exportOptions, cache *geocodeCache, bridge photos.Bridge) error {
if opts.sidecar != "xmp" {
if opts.sidecar == "none" || opts.sidecar == "" {
return nil
}
mode := "preview"
@@ -1090,17 +1173,48 @@ func writeSidecarIfNeeded(pa pendingAsset, result photos.ExportResult, originals
if pa.asset.ModificationDate != nil {
modifyDate = *pa.asset.ModificationDate
}
location := pa.asset.Location
xmpPrivacy := opts.xmpPrivacy
if xmpPrivacy == "" {
xmpPrivacy = "keep"
}
xmpKeywords := opts.xmpKeywords
if xmpKeywords == "" {
xmpKeywords = "album-path"
}
xmpRating := opts.xmpRating
if xmpRating == "" {
xmpRating = "favorite"
}
var placemark *photos.Placemark
if opts.reverseGeocode && pa.asset.Location != nil && cache != nil {
if opts.reverseGeocode && location != nil && cache != nil && xmpPrivacy == "keep" {
placemark = cache.lookup(pa.asset.Location.Latitude, pa.asset.Location.Longitude, bridge)
}
return writeXMPSidecar(sidecarPath(fullPath), xmpSidecarData{
if xmpPrivacy == "strip-location" {
location = nil
placemark = nil
}
if xmpPrivacy == "strip-address" {
placemark = nil
}
keywords := keywordsFromAlbumPath(pa.album, relDir)
if xmpKeywords == "album" {
keywords = keywordsFromAlbumPath(pa.album, "")
}
if xmpKeywords == "none" {
keywords = nil
}
isFavorite := pa.asset.IsFavorite
if xmpRating == "none" {
isFavorite = false
}
data := xmpSidecarData{
AssetID: pa.asset.ID,
OriginalFilename: pa.asset.Filename,
ExportedFilename: result.Filename,
Album: pa.album,
AlbumPath: pa.path,
Keywords: keywordsFromAlbumPath(pa.album, relDir),
Keywords: keywords,
ManifestPath: relPath,
MediaType: pa.asset.MediaType,
MediaSubtypes: pa.asset.MediaSubtypes,
@@ -1109,7 +1223,7 @@ func writeSidecarIfNeeded(pa pendingAsset, result photos.ExportResult, originals
PixelWidth: pa.asset.PixelWidth,
PixelHeight: pa.asset.PixelHeight,
Duration: pa.asset.Duration,
IsFavorite: pa.asset.IsFavorite,
IsFavorite: isFavorite,
IsHidden: pa.asset.IsHidden,
HasAdjustments: pa.asset.HasAdjustments,
Cloud: result.Cloud,
@@ -1119,14 +1233,25 @@ func writeSidecarIfNeeded(pa pendingAsset, result photos.ExportResult, originals
Size: result.Size,
CreateDate: createDate,
ModifyDate: modifyDate,
Location: pa.asset.Location,
Location: location,
Placemark: placemark,
BurstIdentifier: pa.asset.BurstIdentifier,
RepresentsBurst: pa.asset.RepresentsBurst,
BurstSelectionTypes: pa.asset.BurstSelectionTypes,
AdjustmentInfo: pa.asset.AdjustmentInfo,
Resources: pa.asset.Resources,
})
}
if sidecarEnabled(opts.sidecar, "xmp") {
if err := writeXMPSidecar(sidecarPath(fullPath), data); err != nil {
return err
}
}
if sidecarEnabled(opts.sidecar, "json") {
if err := writeJSONSidecar(jsonSidecarPath(fullPath), data); err != nil {
return err
}
}
return nil
}
func writeMetadataOnlySidecar(pa pendingAsset, entry manifest.Entry, originals bool, opts exportOptions, cache *geocodeCache, bridge photos.Bridge) error {
@@ -1376,16 +1501,20 @@ func exportPendingSerial(pending []pendingAsset, targetSize, quality int, origin
failed++
appendFailure(pa.path, pa, exportErr)
} else if isSkipped {
addManifestEntry(m, pa, result)
_ = addManifestEntryForResult(m, pa, result, opts)
} else {
if sidecarErr := writeSidecarIfNeeded(pa, result, originals, opts, geo, bridge); sidecarErr != nil {
failed++
exportErr = sidecarErr
isErr = true
appendFailure(pa.path, pa, sidecarErr)
} else if checksumErr := addManifestEntryForResult(m, pa, result, opts); checksumErr != nil {
failed++
exportErr = checksumErr
isErr = true
appendFailure(pa.path, pa, checksumErr)
} else {
done++
addManifestEntry(m, pa, result)
}
}
avgSpeed := float64(0)
@@ -1516,16 +1645,20 @@ func exportPendingParallel(pending []pendingAsset, targetSize, quality int, orig
failed++
appendFailure(entry.pa.path, entry.pa, entry.err)
} else if isSkipped {
addManifestEntry(m, entry.pa, entry.result)
_ = addManifestEntryForResult(m, entry.pa, entry.result, opts)
} else {
if sidecarErr := writeSidecarIfNeeded(entry.pa, entry.result, originals, opts, geo, bridge); sidecarErr != nil {
failed++
entry.err = sidecarErr
isErr = true
appendFailure(entry.pa.path, entry.pa, sidecarErr)
} else if checksumErr := addManifestEntryForResult(m, entry.pa, entry.result, opts); checksumErr != nil {
failed++
entry.err = checksumErr
isErr = true
appendFailure(entry.pa.path, entry.pa, checksumErr)
} else {
done++
addManifestEntry(m, entry.pa, entry.result)
}
}
avgSpeed := float64(0)
@@ -1874,6 +2007,10 @@ func parseExportOptions(args []string, stderr io.Writer) (exportOptions, bool) {
verify: hasFlag(args, "--verify"),
format: flagValWithDefault(args, "--format", "jpeg"),
sidecar: flagValWithDefault(args, "--sidecar", "none"),
checksum: flagValWithDefault(args, "--checksum", "none"),
xmpPrivacy: flagValWithDefault(args, "--xmp-privacy", "keep"),
xmpKeywords: flagValWithDefault(args, "--xmp-keywords", "album-path"),
xmpRating: flagValWithDefault(args, "--xmp-rating", "favorite"),
metadataOnly: hasFlag(args, "--metadata-only"),
reverseGeocode: hasFlag(args, "--reverse-geocode"),
dateTemplate: flagVal(args, "--date-template"),
@@ -1886,12 +2023,37 @@ func parseExportOptions(args []string, stderr io.Writer) (exportOptions, bool) {
fmt.Fprintf(stderr, "error: --format must be jpeg, heic, or png, got %q\n", opts.format)
return opts, false
}
if opts.sidecar != "none" && opts.sidecar != "xmp" {
fmt.Fprintf(stderr, "error: --sidecar must be none or xmp, got %q\n", opts.sidecar)
if opts.sidecar != "none" && !sidecarEnabled(opts.sidecar, "xmp") && !sidecarEnabled(opts.sidecar, "json") {
fmt.Fprintf(stderr, "error: --sidecar must be none, xmp, json, or xmp,json, got %q\n", opts.sidecar)
return opts, false
}
if opts.metadataOnly && opts.sidecar != "xmp" {
fmt.Fprintln(stderr, "error: --metadata-only requires --sidecar xmp")
if opts.sidecar != "none" {
for _, part := range strings.Split(opts.sidecar, ",") {
part = strings.TrimSpace(part)
if part != "xmp" && part != "json" {
fmt.Fprintf(stderr, "error: --sidecar must be none, xmp, json, or xmp,json, got %q\n", opts.sidecar)
return opts, false
}
}
}
if opts.checksum != "none" && opts.checksum != "sha256" {
fmt.Fprintf(stderr, "error: --checksum must be none or sha256, got %q\n", opts.checksum)
return opts, false
}
if opts.xmpPrivacy != "keep" && opts.xmpPrivacy != "strip-location" && opts.xmpPrivacy != "strip-address" {
fmt.Fprintf(stderr, "error: --xmp-privacy must be keep, strip-location, or strip-address, got %q\n", opts.xmpPrivacy)
return opts, false
}
if opts.xmpKeywords != "album-path" && opts.xmpKeywords != "album" && opts.xmpKeywords != "none" {
fmt.Fprintf(stderr, "error: --xmp-keywords must be album-path, album, or none, got %q\n", opts.xmpKeywords)
return opts, false
}
if opts.xmpRating != "favorite" && opts.xmpRating != "none" {
fmt.Fprintf(stderr, "error: --xmp-rating must be favorite or none, got %q\n", opts.xmpRating)
return opts, false
}
if opts.metadataOnly && opts.sidecar == "none" {
fmt.Fprintln(stderr, "error: --metadata-only requires --sidecar xmp, json, or xmp,json")
return opts, false
}
if v := flagVal(args, "--retry"); v != "" {
@@ -2117,6 +2279,8 @@ func cmdDiff(args []string, stdout, stderr io.Writer, bridge photos.Bridge) int
func cmdVerify(args []string, stdout, stderr io.Writer) int {
outDir := flagVal(args, "--out")
checkSidecar := hasFlag(args, "--sidecar")
strictSidecar := hasFlag(args, "--strict")
deep := hasFlag(args, "--deep")
if outDir == "" {
fmt.Fprintln(stderr, "error: --out is required")
return exitErr
@@ -2155,8 +2319,18 @@ func cmdVerify(args []string, stdout, stderr io.Writer) int {
bad++
fmt.Fprintf(stdout, "%s\t%s\tsize-mismatch\tmanifest=%d\tdisk=%d\n", id, checkPath, e.Size, info.Size())
}
if deep && e.Checksum != "" {
checksum, err := fileSHA256(filepath.Join(outDir, checkPath))
if err != nil {
bad++
fmt.Fprintf(stdout, "%s\t%s\tchecksum-unreadable\n", id, checkPath)
} else if checksum != e.Checksum {
bad++
fmt.Fprintf(stdout, "%s\t%s\tchecksum-mismatch\tmanifest=%s\tdisk=%s\n", id, checkPath, e.Checksum, checksum)
}
}
if checkSidecar {
bad += verifySidecar(stdout, outDir, id, checkPath)
bad += verifySidecar(stdout, outDir, id, checkPath, strictSidecar)
}
}
if bad > 0 {
@@ -2166,7 +2340,7 @@ func cmdVerify(args []string, stdout, stderr io.Writer) int {
return exitOK
}
func verifySidecar(stdout io.Writer, outDir, id, checkPath string) int {
func verifySidecar(stdout io.Writer, outDir, id, checkPath string, strict bool) int {
xmpPath := sidecarPath(filepath.Join(outDir, checkPath))
rel, err := filepath.Rel(outDir, xmpPath)
if err != nil || strings.HasPrefix(rel, "..") {
@@ -2191,6 +2365,21 @@ func verifySidecar(stdout io.Writer, outDir, id, checkPath string) int {
fmt.Fprintf(stdout, "%s\t%s\tsidecar-asset-mismatch\n", id, rel)
return 1
}
if strict {
meta := inspectXMP(data)
if meta["xmpSchemaVersion"] != "2" {
fmt.Fprintf(stdout, "%s\t%s\tsidecar-schema-missing\n", id, rel)
return 1
}
if meta["sidecarGenerator"] == "" {
fmt.Fprintf(stdout, "%s\t%s\tsidecar-generator-missing\n", id, rel)
return 1
}
if meta["exportedFilename"] != filepath.Base(checkPath) {
fmt.Fprintf(stdout, "%s\t%s\tsidecar-filename-mismatch\n", id, rel)
return 1
}
}
return 0
}
+307 -6
View File
@@ -2662,6 +2662,23 @@ func TestExportPendingParallelManifestAdd(t *testing.T) {
}
}
func TestExportPendingParallelChecksumError(t *testing.T) {
dir := t.TempDir()
mf := manifest.LoadJSONL(dir)
if err := mf.OpenAppend(); err != nil {
t.Fatal(err)
}
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
return photos.ExportResult{Filename: "missing.jpg", Size: 3}, nil
}
bar := newProgressBar(io.Discard, 1)
done, failed := exportPendingParallel([]pendingAsset{{asset: b.assets[0], path: dir}}, 1024, 85, false, 1, bar, b, 1, mf, manifest.NoopLogWriter, exportOptions{checksum: "sha256"})
if done != 0 || failed != 1 {
t.Fatalf("done=%d failed=%d", done, failed)
}
}
func TestExportPendingParallelCancel(t *testing.T) {
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
b.Cancel()
@@ -2804,6 +2821,58 @@ func TestExportAssetsManifestWrite(t *testing.T) {
}
}
func TestExportAssetsChecksum(t *testing.T) {
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
dir := t.TempDir()
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
if err := os.WriteFile(filepath.Join(out, "img.jpg"), []byte("abc"), 0644); err != nil {
return photos.ExportResult{}, err
}
return photos.ExportResult{Filename: "img.jpg", Size: 3, Cloud: "local"}, nil
}
done, failed := exportAssets(b.assets, dir, 1024, 85, 1, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{checksum: "sha256"})
if done != 1 || failed != 0 {
t.Fatalf("done=%d failed=%d", done, failed)
}
entry := manifest.LoadJSONL(dir).Entries()["x1"]
if entry.Checksum != "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" {
t.Fatalf("checksum=%q", entry.Checksum)
}
got, err := fileSHA256(filepath.Join(dir, "img.jpg"))
if err != nil || got != entry.Checksum {
t.Fatalf("fileSHA256 got=%q err=%v", got, err)
}
if _, err := fileSHA256(filepath.Join(dir, "missing.jpg")); err == nil {
t.Fatal("expected missing checksum error")
}
if _, err := fileSHA256(dir); err == nil {
t.Fatal("expected directory checksum error")
}
if err := addManifestEntryForResult(nil, pendingAsset{}, photos.ExportResult{}, exportOptions{}); err != nil {
t.Fatalf("nil manifest add error: %v", err)
}
if err := addManifestEntryForResult(manifest.LoadJSONL(dir), pendingAsset{asset: photos.Asset{ID: "skip"}, path: dir}, photos.ExportResult{Filename: "missing.jpg", Skipped: true}, exportOptions{checksum: "sha256"}); err != nil {
t.Fatalf("skipped checksum add error: %v", err)
}
}
func TestExportAssetsChecksumErrors(t *testing.T) {
dir := t.TempDir()
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
return photos.ExportResult{Filename: "missing.jpg", Size: 3}, nil
}
done, failed := exportAssets(b.assets, dir, 1024, 85, 1, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{checksum: "sha256"})
if done != 0 || failed != 1 {
t.Fatalf("serial checksum error done=%d failed=%d", done, failed)
}
dir = t.TempDir()
done, failed = exportAssets(b.assets, dir, 1024, 85, 2, false, 2, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{checksum: "sha256"})
if done != 0 || failed != 1 {
t.Fatalf("parallel checksum error done=%d failed=%d", done, failed)
}
}
func TestBackupTreeManifestOpenErr(t *testing.T) {
b := &mockBridge{
tree: []photos.CollectionNode{
@@ -4204,6 +4273,39 @@ func TestMoreIntegrityBranches(t *testing.T) {
}
}
func TestVerifyDeepChecksums(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "good.jpg"), []byte("abc"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "bad.jpg"), []byte("bad"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "nocheck.jpg"), []byte("data"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(filepath.Join(dir, "adir"), 0755); err != nil {
t.Fatal(err)
}
m := manifest.LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(manifest.Entry{ID: "good", Filename: "good.jpg", Path: "good.jpg", Size: 3, Cloud: "local", Checksum: "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "bad", Filename: "bad.jpg", Path: "bad.jpg", Size: 3, Cloud: "local", Checksum: "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "nocheck", Filename: "nocheck.jpg", Path: "nocheck.jpg", Size: 4, Cloud: "local", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "unreadable", Filename: "adir", Path: "adir", Cloud: "local", Checksum: "abc", Exported: time.Now().Unix()})
m.Close()
out, stderr, rc := runWith([]string{"verify", "--out", dir, "--deep"}, &mockBridge{})
if rc != exitPartial || stderr != "" || !strings.Contains(out, "checksum-mismatch") || !strings.Contains(out, "checksum-unreadable") || strings.Contains(out, "good.jpg") || strings.Contains(out, "nocheck") {
t.Fatalf("deep verify rc=%d out=%q stderr=%q", rc, out, stderr)
}
out, stderr, rc = runWith([]string{"verify", "--out", dir}, &mockBridge{})
if rc != exitOK || stderr != "" || strings.Contains(out, "checksum") {
t.Fatalf("plain verify rc=%d out=%q stderr=%q", rc, out, stderr)
}
}
func TestVerifySidecarBranches(t *testing.T) {
dir := t.TempDir()
subdir := filepath.Join(dir, "sub")
@@ -4217,12 +4319,48 @@ func TestVerifySidecarBranches(t *testing.T) {
oldRead := readFileFunc
readFileFunc = func(string) ([]byte, error) { return nil, fmt.Errorf("read") }
var out bytes.Buffer
if got := verifySidecar(&out, subdir, "x1", "../asset.jpg"); got != 1 || !strings.Contains(out.String(), "sidecar-unreadable") {
if got := verifySidecar(&out, subdir, "x1", "../asset.jpg", false); got != 1 || !strings.Contains(out.String(), "sidecar-unreadable") {
t.Fatalf("expected unreadable with rel fallback, got=%d out=%q", got, out.String())
}
readFileFunc = oldRead
}
func TestVerifySidecarStrict(t *testing.T) {
dir := t.TempDir()
media := filepath.Join(dir, "photo.jpg")
if err := os.WriteFile(media, []byte("data"), 0644); err != nil {
t.Fatal(err)
}
if err := writeXMPSidecar(sidecarPath(media), xmpSidecarData{AssetID: "x1", ExportedFilename: "photo.jpg"}); err != nil {
t.Fatal(err)
}
var out bytes.Buffer
if got := verifySidecar(&out, dir, "x1", "photo.jpg", true); got != 0 || out.Len() != 0 {
t.Fatalf("strict valid got=%d out=%q", got, out.String())
}
if err := os.WriteFile(sidecarPath(media), []byte(`photoscli:assetID="x1"`), 0644); err != nil {
t.Fatal(err)
}
out.Reset()
if got := verifySidecar(&out, dir, "x1", "photo.jpg", true); got != 1 || !strings.Contains(out.String(), "sidecar-schema-missing") {
t.Fatalf("strict schema got=%d out=%q", got, out.String())
}
if err := os.WriteFile(sidecarPath(media), []byte(`<?xpacket begin=""?><x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description xmlns:photoscli="https://gitea.k3s.k0.nu/tools/photocli/ns/1.0/" photoscli:assetID="x1" photoscli:xmpSchemaVersion="2" photoscli:exportedFilename="photo.jpg" /></rdf:RDF></x:xmpmeta>`), 0644); err != nil {
t.Fatal(err)
}
out.Reset()
if got := verifySidecar(&out, dir, "x1", "photo.jpg", true); got != 1 || !strings.Contains(out.String(), "sidecar-generator-missing") {
t.Fatalf("strict generator got=%d out=%q", got, out.String())
}
if err := os.WriteFile(sidecarPath(media), []byte(`<?xpacket begin=""?><x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description xmlns:photoscli="https://gitea.k3s.k0.nu/tools/photocli/ns/1.0/" photoscli:assetID="x1" photoscli:xmpSchemaVersion="2" photoscli:sidecarGenerator="photoscli" photoscli:exportedFilename="other.jpg" /></rdf:RDF></x:xmpmeta>`), 0644); err != nil {
t.Fatal(err)
}
out.Reset()
if got := verifySidecar(&out, dir, "x1", "photo.jpg", true); got != 1 || !strings.Contains(out.String(), "sidecar-filename-mismatch") {
t.Fatalf("strict filename got=%d out=%q", got, out.String())
}
}
func TestSidecarInspect(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "photo.xmp")
@@ -4349,6 +4487,52 @@ func TestWriteXMPSidecar(t *testing.T) {
}
}
func TestWriteJSONSidecar(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "photo.json")
if got := jsonSidecarPath(filepath.Join(dir, "photo.jpg")); got != path {
t.Fatalf("json sidecar path=%q", got)
}
if !sidecarEnabled("xmp,json", "json") || sidecarEnabled("xmp", "json") {
t.Fatal("sidecarEnabled mismatch")
}
if err := writeJSONSidecar(path, xmpSidecarData{AssetID: "x1", ExportedFilename: "photo.jpg"}); err != nil {
t.Fatal(err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(data), `"AssetID": "x1"`) {
t.Fatalf("unexpected json sidecar: %s", string(data))
}
badParent := filepath.Join(t.TempDir(), "file")
if err := os.WriteFile(badParent, []byte("x"), 0644); err != nil {
t.Fatal(err)
}
if err := writeJSONSidecar(filepath.Join(badParent, "bad.json"), xmpSidecarData{}); err == nil {
t.Fatal("expected mkdir error")
}
oldCreate := createTempFunc
createTempFunc = func(string, string) (*os.File, error) { return nil, fmt.Errorf("create") }
if err := writeJSONSidecar(path, xmpSidecarData{}); err == nil {
t.Fatal("expected create temp error")
}
createTempFunc = oldCreate
oldWrite := writeFileFunc
writeFileFunc = func(string, []byte, os.FileMode) error { return fmt.Errorf("write") }
if err := writeJSONSidecar(path, xmpSidecarData{}); err == nil {
t.Fatal("expected write error")
}
writeFileFunc = oldWrite
oldRename := renameFunc
renameFunc = func(string, string) error { return fmt.Errorf("rename") }
if err := writeJSONSidecar(path, xmpSidecarData{}); err == nil {
t.Fatal("expected rename error")
}
renameFunc = oldRename
}
func TestSidecarExportIntegration(t *testing.T) {
dir := t.TempDir()
date := "2024-01-02T03:04:05Z"
@@ -4359,7 +4543,7 @@ func TestSidecarExportIntegration(t *testing.T) {
}
return photos.ExportResult{Filename: "photo.jpg", Size: 4, Cloud: "local"}, nil
}
exported, failed := exportAssets(b.assets, dir, 1024, 85, 3, false, 1, io.Discard, b, "Album", false, manifest.FormatJSONL, false, exportOptions{sidecar: "xmp"})
exported, failed := exportAssets(b.assets, dir, 1024, 85, 3, false, 1, io.Discard, b, "Album", false, manifest.FormatJSONL, false, exportOptions{sidecar: "xmp,json"})
if exported != 1 || failed != 0 {
t.Fatalf("exported=%d failed=%d", exported, failed)
}
@@ -4376,6 +4560,13 @@ func TestSidecarExportIntegration(t *testing.T) {
if _, err := os.Stat(filepath.Join(dir, "photo.jpg.xmp")); !os.IsNotExist(err) {
t.Fatal("sidecar should use basename, not double extension")
}
jsonData, err := os.ReadFile(filepath.Join(dir, "photo.json"))
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(jsonData), `"AssetID": "x1"`) {
t.Fatalf("json sidecar missing asset ID: %s", string(jsonData))
}
}
func TestSidecarReverseGeocodeCache(t *testing.T) {
@@ -4548,6 +4739,30 @@ func TestSidecarConfigAndErrors(t *testing.T) {
if _, ok := parseExportOptions([]string{"--sidecar", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--sidecar") {
t.Fatalf("expected sidecar validation error, stderr=%q", stderr.String())
}
stderr.Reset()
if opts, ok := parseExportOptions([]string{"--sidecar", "json"}, &stderr); !ok || opts.sidecar != "json" || stderr.Len() != 0 {
t.Fatalf("expected json sidecar option, opts=%+v ok=%v stderr=%q", opts, ok, stderr.String())
}
stderr.Reset()
if _, ok := parseExportOptions([]string{"--sidecar", "xmp,bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--sidecar") {
t.Fatalf("expected mixed sidecar validation error, stderr=%q", stderr.String())
}
stderr.Reset()
if _, ok := parseExportOptions([]string{"--xmp-privacy", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--xmp-privacy") {
t.Fatalf("expected xmp privacy validation error, stderr=%q", stderr.String())
}
stderr.Reset()
if _, ok := parseExportOptions([]string{"--xmp-keywords", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--xmp-keywords") {
t.Fatalf("expected xmp keywords validation error, stderr=%q", stderr.String())
}
stderr.Reset()
if _, ok := parseExportOptions([]string{"--xmp-rating", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--xmp-rating") {
t.Fatalf("expected xmp rating validation error, stderr=%q", stderr.String())
}
stderr.Reset()
if _, ok := parseExportOptions([]string{"--checksum", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--checksum") {
t.Fatalf("expected checksum validation error, stderr=%q", stderr.String())
}
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "photo.jpg"}}}
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
@@ -4555,10 +4770,96 @@ func TestSidecarConfigAndErrors(t *testing.T) {
}
oldRename := renameFunc
renameFunc = func(string, string) error { return fmt.Errorf("sidecar rename") }
exported, failed := exportAssets(b.assets, dir, 1024, 85, 3, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{sidecar: "xmp"})
exported, failed := exportAssets(b.assets, dir, 1024, 85, 3, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{sidecar: "json"})
if exported != 0 || failed != 1 {
t.Fatalf("expected json sidecar failure, exported=%d failed=%d", exported, failed)
}
exported, failed = exportAssets(b.assets, dir, 1024, 85, 3, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{sidecar: "xmp"})
renameFunc = oldRename
if exported != 0 || failed != 1 {
t.Fatalf("expected sidecar failure, exported=%d failed=%d", exported, failed)
t.Fatalf("expected xmp sidecar failure, exported=%d failed=%d", exported, failed)
}
}
func TestXMPSidecarPrivacy(t *testing.T) {
dir := t.TempDir()
asset := photos.Asset{ID: "x1", Filename: "geo.jpg", Location: &photos.AssetLocation{Latitude: 59.3293, Longitude: 18.0686}}
bridge := &mockBridge{}
bridge.reverseGeocodeFn = func(float64, float64) (photos.Placemark, error) {
return photos.Placemark{Country: "Sweden", Locality: "Stockholm"}, nil
}
pa := pendingAsset{asset: asset, root: dir, path: dir, album: "Album"}
for _, tc := range []struct {
privacy string
wantGPS bool
wantAddress bool
}{
{privacy: "keep", wantGPS: true, wantAddress: true},
{privacy: "strip-address", wantGPS: true, wantAddress: false},
{privacy: "strip-location", wantGPS: false, wantAddress: false},
} {
path := filepath.Join(dir, tc.privacy+".jpg")
if err := os.WriteFile(path, []byte("data"), 0644); err != nil {
t.Fatal(err)
}
if err := writeSidecarIfNeeded(pa, photos.ExportResult{Filename: filepath.Base(path), Size: 4}, false, exportOptions{sidecar: "xmp", reverseGeocode: true, xmpPrivacy: tc.privacy}, newGeocodeCache(dir), bridge); err != nil {
t.Fatalf("%s write sidecar: %v", tc.privacy, err)
}
data, err := os.ReadFile(sidecarPath(path))
if err != nil {
t.Fatal(err)
}
content := string(data)
if strings.Contains(content, "photoscli:latitude") != tc.wantGPS {
t.Fatalf("%s GPS presence mismatch in %s", tc.privacy, content)
}
if strings.Contains(content, "photoscli:addressCountry") != tc.wantAddress {
t.Fatalf("%s address presence mismatch in %s", tc.privacy, content)
}
}
}
func TestXMPSidecarKeywordAndRatingOptions(t *testing.T) {
dir := t.TempDir()
asset := photos.Asset{ID: "x1", Filename: "photo.jpg", IsFavorite: true}
pa := pendingAsset{asset: asset, root: dir, path: filepath.Join(dir, "Trips", "Beach"), album: "Beach"}
if err := os.MkdirAll(pa.path, 0755); err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name string
keywords string
rating string
wantTrip bool
wantBeach bool
wantRate bool
}{
{name: "default", wantTrip: true, wantBeach: true, wantRate: true},
{name: "album", keywords: "album", wantTrip: false, wantBeach: true, wantRate: true},
{name: "none", keywords: "none", rating: "none", wantTrip: false, wantBeach: false, wantRate: false},
} {
filename := tc.name + ".jpg"
path := filepath.Join(pa.path, filename)
if err := os.WriteFile(path, []byte("data"), 0644); err != nil {
t.Fatal(err)
}
if err := writeSidecarIfNeeded(pa, photos.ExportResult{Filename: filename, Size: 4}, false, exportOptions{sidecar: "xmp", xmpKeywords: tc.keywords, xmpRating: tc.rating}, nil, &mockBridge{}); err != nil {
t.Fatalf("%s write sidecar: %v", tc.name, err)
}
data, err := os.ReadFile(sidecarPath(path))
if err != nil {
t.Fatal(err)
}
content := string(data)
if strings.Contains(content, "<rdf:li>Trips</rdf:li>") != tc.wantTrip {
t.Fatalf("%s Trips keyword mismatch in %s", tc.name, content)
}
if strings.Contains(content, "<rdf:li>Beach</rdf:li>") != tc.wantBeach {
t.Fatalf("%s Beach keyword mismatch in %s", tc.name, content)
}
if strings.Contains(content, "xmp:Rating=\"5\"") != tc.wantRate {
t.Fatalf("%s rating mismatch in %s", tc.name, content)
}
}
}
@@ -4595,7 +4896,7 @@ func TestMetadataOnlyExportErrors(t *testing.T) {
dir := t.TempDir()
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "photo.jpg"}}}
_, stderr, rc := runWith([]string{"export", "--album-id", "x", "--out", dir, "--metadata-only"}, b)
if rc != exitErr || !strings.Contains(stderr, "--metadata-only requires --sidecar xmp") {
if rc != exitErr || !strings.Contains(stderr, "--metadata-only requires --sidecar") {
t.Fatalf("expected sidecar requirement rc=%d stderr=%q", rc, stderr)
}
_, stderr, rc = runWith([]string{"export", "--album-id", "x", "--out", dir, "--sidecar", "xmp", "--metadata-only", "--no-manifest"}, b)
@@ -4823,7 +5124,7 @@ func TestInjectedErrorBranchesForCoverage(t *testing.T) {
removeFunc = oldRemove
mf := &mockManifest{}
addManifestEntry(mf, pendingAsset{asset: photos.Asset{ID: "x"}, root: filepath.Join(dir, "other"), path: dir}, photos.ExportResult{Filename: "file.jpg", Size: 1})
addManifestEntry(mf, pendingAsset{asset: photos.Asset{ID: "x"}, root: filepath.Join(dir, "other"), path: dir}, photos.ExportResult{Filename: "file.jpg", Size: 1}, "")
if mf.last.Path != "file.jpg" {
t.Fatalf("expected fallback rel path, got %+v", mf.last)
}
+16 -4
View File
@@ -11,6 +11,10 @@ func TestNewEntryPath(t *testing.T) {
if e.Path != "Album/file2.jpg" || e.Filename != "file2.jpg" || e.Size != 456 || e.Cloud != "cloud" {
t.Fatalf("unexpected entry with path: %+v", e)
}
e = NewEntryWithChecksum("id3", "file3.jpg", "Album/file3.jpg", 789, "local", "sha256:abc")
if e.Checksum != "sha256:abc" || e.Path != "Album/file3.jpg" {
t.Fatalf("unexpected checksum entry: %+v", e)
}
}
func TestAddEntryDefaultsPath(t *testing.T) {
@@ -19,11 +23,15 @@ func TestAddEntryDefaultsPath(t *testing.T) {
if err := jm.OpenAppend(); err != nil {
t.Fatal(err)
}
jm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local"})
jm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:abc"})
jm.Close()
if got := LoadJSONL(dir).Entries()["x1"].Path; got != "file.jpg" {
loaded := LoadJSONL(dir).Entries()["x1"]
if got := loaded.Path; got != "file.jpg" {
t.Fatalf("jsonl path = %q", got)
}
if loaded.Checksum != "sha256:abc" {
t.Fatalf("jsonl checksum = %q", loaded.Checksum)
}
sdir := t.TempDir()
sm, err := LoadSQLite(sdir)
@@ -33,12 +41,16 @@ func TestAddEntryDefaultsPath(t *testing.T) {
if err := sm.OpenAppend(); err != nil {
t.Fatal(err)
}
sm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local"})
sm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:def"})
if _, err := sm.db.Exec(`UPDATE downloads SET path = '' WHERE id = 'x1'`); err != nil {
t.Fatal(err)
}
if got := sm.Entries()["x1"].Path; got != "file.jpg" {
sloaded := sm.Entries()["x1"]
if got := sloaded.Path; got != "file.jpg" {
t.Fatalf("sqlite path = %q", got)
}
if sloaded.Checksum != "sha256:def" {
t.Fatalf("sqlite checksum = %q", sloaded.Checksum)
}
sm.Close()
}
+4 -1
View File
@@ -43,6 +43,7 @@ func LoadJSONL(dir string) *jsonlManifest {
Path string `json:"path,omitempty"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"`
}
for _, line := range strings.Split(string(data), "\n") {
@@ -61,6 +62,7 @@ func LoadJSONL(dir string) *jsonlManifest {
Path: raw.Path,
Size: raw.Size,
Cloud: raw.Cloud,
Checksum: raw.Checksum,
Exported: raw.Exported,
}
}
@@ -93,8 +95,9 @@ func (m *jsonlManifest) AddEntry(entry Entry) {
Path string `json:"path,omitempty"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"`
}{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Exported: entry.Exported})
}{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Checksum: entry.Checksum, Exported: entry.Exported})
m.file.Write(data)
m.file.Write([]byte("\n"))
}
+7
View File
@@ -8,6 +8,7 @@ type Entry struct {
Path string
Size int64
Cloud string
Checksum string
Exported int64
}
@@ -42,3 +43,9 @@ func NewEntry(id, filename, path string, size int64, cloud string) Entry {
}
return e
}
func NewEntryWithChecksum(id, filename, path string, size int64, cloud, checksum string) Entry {
e := NewEntry(id, filename, path, size, cloud)
e.Checksum = checksum
return e
}
+6 -4
View File
@@ -64,6 +64,7 @@ func (m *sqliteManifest) OpenAppend() error {
path TEXT NOT NULL DEFAULT '',
size INTEGER NOT NULL DEFAULT 0,
cloud TEXT NOT NULL DEFAULT '',
checksum TEXT NOT NULL DEFAULT '',
exported INTEGER NOT NULL DEFAULT 0
)`)
if err != nil {
@@ -71,6 +72,7 @@ func (m *sqliteManifest) OpenAppend() error {
return fmt.Errorf("create table: %w", err)
}
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN path TEXT NOT NULL DEFAULT ''`)
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN checksum TEXT NOT NULL DEFAULT ''`)
_, err = execFn(`CREATE INDEX IF NOT EXISTS idx_downloads_id ON downloads(id)`)
if err != nil {
db.Close()
@@ -103,8 +105,8 @@ func (m *sqliteManifest) AddEntry(entry Entry) {
if entry.Path == "" {
entry.Path = entry.Filename
}
m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, exported) VALUES (?, ?, ?, ?, ?, ?)`,
entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Exported)
m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, checksum, exported) VALUES (?, ?, ?, ?, ?, ?, ?)`,
entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Checksum, entry.Exported)
}
func (m *sqliteManifest) Save() error {
@@ -127,14 +129,14 @@ func (m *sqliteManifest) Entries() map[string]Entry {
return nil
}
out := make(map[string]Entry)
rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, exported FROM downloads`)
rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, checksum, exported FROM downloads`)
if err != nil {
return out
}
defer rows.Close()
for rows.Next() {
var e Entry
if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Exported); err == nil {
if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Checksum, &e.Exported); err == nil {
if e.Path == "" {
e.Path = e.Filename
}