Compare commits

5 Commits

Author SHA1 Message Date
Ein Anderssono 9cd048d9f3 v0.9.4: add doctor diagnostics
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:47:44 +02:00
Ein Anderssono 98320c8235 v0.9.3: add cleanup command
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:40:12 +02:00
Ein Anderssono 84e70bda76 v0.9.2: add manifest repair
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:32:20 +02:00
Ein Anderssono bed3ea7bd0 v0.9.1: add deep checksum verification
pipeline / test (push) Has been cancelled
pipeline / build (push) Has been cancelled
2026-06-15 02:23:22 +02:00
Ein Anderssono 7555b561bd v0.9.0: add manifest checksums
pipeline / build (push) Has been cancelled
pipeline / test (push) Has been cancelled
2026-06-15 02:19:47 +02:00
11 changed files with 780 additions and 24 deletions
+40
View File
@@ -2,6 +2,46 @@
This changelog is maintained from git history plus the published Gitea release series. Future releases should update this file and publish matching release notes on the release page.
## v0.9.4
Doctor release.
- Add `doctor` to check Photos access and optional backup health.
- Add `doctor --out <dir>` to report backup directory, manifest entries, and failure count.
- Add `doctor --json` for scriptable diagnostics.
## v0.9.3
Cleanup release.
- Add `cleanup --out <dir>` to remove files not referenced by the manifest.
- Add `cleanup --dry-run` to preview orphaned files before deletion.
- Preserve manifest/log/failure files, `.photoscli`, and sidecars for manifest-referenced media.
## v0.9.2
Manifest repair release.
- Add `manifest repair --out <dir>` to fill missing manifest size metadata from files on disk.
- Add `manifest repair --checksum sha256` to fill missing manifest checksums.
- Add `manifest repair --dry-run` to preview repairs without writing manifest updates.
## v0.9.1
Deep checksum verification release.
- Add `verify --deep` to recompute SHA-256 checksums for manifest entries that have checksum metadata.
- Report `checksum-mismatch` when disk contents differ from the manifest checksum.
- Keep normal `verify` unchanged unless `--deep` is selected.
## v0.9.0
Manifest checksum release.
- Add opt-in `--checksum sha256` to store SHA-256 file checksums in JSONL and SQLite manifests.
- Add SQLite migration support for the manifest `checksum` column.
- Keep checksum collection disabled by default with `--checksum none`.
## v0.8.7
JSON sidecar release.
+1 -1
View File
@@ -1,6 +1,6 @@
BINARY := ./bin/photoscli
MODULE := gitea.k3s.k0.nu/tools/photocli
VERSION := 0.8.7
VERSION := 0.9.4
RELEASE_ZIP := ./bin/photoscli-$(VERSION)-macos-arm64.zip
RELEASE_NOTES := RELEASE_NOTES.md
BRIDGE_DIR := bridge
+6
View File
@@ -21,6 +21,7 @@ For a practical step-by-step manual with recommended backup workflows, recovery
- Deduplicated failure management with `failures list`, `failures clear`, and `retry-failed --clear-on-success`.
- Verification, reporting, and diff commands for backup integrity.
- Optional XMP sidecar verification with `verify --sidecar`.
- Optional SHA-256 manifest checksums with `--checksum sha256`.
- Metadata-only XMP refresh for manifest-backed exports with `--metadata-only`.
- Status command for quick backup summaries.
- Opt-in rich XMP sidecar metadata with `--sidecar xmp`.
@@ -116,6 +117,10 @@ Verify a backup later:
```bash
./bin/photoscli verify --out ./photos-backup --manifest sqlite
./bin/photoscli verify --out ./photos-backup --deep
./bin/photoscli manifest repair --out ./photos-backup --checksum sha256 --dry-run
./bin/photoscli cleanup --out ./photos-backup --dry-run
./bin/photoscli doctor --out ./photos-backup
```
## Commands
@@ -186,6 +191,7 @@ Useful examples:
```bash
photoscli backup-all --out ./backup --manifest sqlite --log
photoscli backup-all --out ./backup --checksum sha256
photoscli backup-all --out ./backup --exclude-album "Recently Deleted" --exclude-album "Temp*"
photoscli backup-all --out ./backup --since 2024-01-01 --sort newest
photoscli backup-all --out ./backup --concurrency 8 --retry 3
+7 -7
View File
@@ -1,18 +1,18 @@
# v0.8.7
# v0.9.4
This release adds JSON sidecars alongside XMP sidecars.
This release adds `doctor` diagnostics for Photos access and backups.
## Highlights
- Add `--sidecar json` for structured JSON metadata sidecars.
- Add `--sidecar xmp,json` to write both XMP and JSON sidecars.
- Keep `--sidecar none` as the default.
- JSON sidecars use the same metadata as XMP sidecars and the exported file basename.
- Add `doctor` to check Photos access.
- Add `doctor --out <dir>` to report backup directory, manifest entries, and failure count.
- Add `doctor --json` for scriptable diagnostics.
- `doctor` is read-only and does not modify backup data.
## Assets
- `photoscli`: Apple Silicon macOS binary (`darwin/arm64`).
- `photoscli-0.8.7-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
- `photoscli-0.9.4-macos-arm64.zip`: Apple Silicon binary plus README, USERGUIDE, and CHANGELOG.
- `USERGUIDE.md`: standalone user guide.
Intel Macs are not currently a supported release target.
+45
View File
@@ -21,6 +21,7 @@ It is especially useful when you want to:
- Retry transient iCloud failures.
- Verify that files referenced by a manifest exist on disk.
- Detect missing, zero-byte, and size-mismatched manifest files.
- Store optional SHA-256 checksums in manifests.
- Inspect or clear deduplicated failure records.
- Write optional XMP sidecar metadata for archival workflows.
- Add optional reverse-geocoded address metadata to XMP sidecars for GPS assets.
@@ -821,6 +822,50 @@ Use structured logs:
./bin/photoscli backup-all --out ./PhotosBackup --manifest sqlite --log
```
Store SHA-256 checksums in the manifest when exporting:
```bash
./bin/photoscli backup-all --out ./PhotosBackup --checksum sha256
```
Checksums are opt-in and stored in JSONL or SQLite manifests. The default is `--checksum none`.
Deep-verify checksum-backed manifests:
```bash
./bin/photoscli verify --out ./PhotosBackup --deep
```
`--deep` recomputes SHA-256 only for manifest entries that already have checksum metadata.
Repair missing manifest metadata from files already present on disk:
```bash
./bin/photoscli manifest repair --out ./PhotosBackup --checksum sha256 --dry-run
./bin/photoscli manifest repair --out ./PhotosBackup --checksum sha256
```
Repair fills missing size/checksum metadata and skips missing, zero-byte, or unreadable files.
Preview and remove files not referenced by the manifest:
```bash
./bin/photoscli cleanup --out ./PhotosBackup --dry-run
./bin/photoscli cleanup --out ./PhotosBackup
```
Cleanup preserves manifest/log/failure files, `.photoscli`, and sidecars next to manifest-referenced media.
Run read-only diagnostics:
```bash
./bin/photoscli doctor
./bin/photoscli doctor --out ./PhotosBackup
./bin/photoscli doctor --out ./PhotosBackup --json
```
Doctor checks Photos access and, when `--out` is provided, backup directory status, manifest entries, and failure count.
## Safe Operating Practices
- Run `--dry-run` before the first large backup.
+294 -6
View File
@@ -2,10 +2,13 @@ package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"encoding/xml"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"sort"
@@ -42,6 +45,7 @@ type exportOptions struct {
verify bool
format string
sidecar string
checksum string
xmpPrivacy string
xmpKeywords string
xmpRating string
@@ -89,6 +93,12 @@ func run(args []string, stdout, stderr io.Writer, bridge photos.Bridge) int {
return cmdDiff(args[1:], stdout, stderr, bridge)
case "verify":
return cmdVerify(args[1:], stdout, stderr)
case "manifest":
return cmdManifest(args[1:], stdout, stderr)
case "cleanup":
return cmdCleanup(args[1:], stdout, stderr)
case "doctor":
return cmdDoctor(args[1:], stdout, stderr, bridge)
case "retry-failed":
return cmdRetryFailed(args[1:], stdout, stderr, bridge)
case "failures":
@@ -134,6 +144,9 @@ USAGE
photoscli report --out <dir> [--manifest jsonl|sqlite]
photoscli diff --album-id <id-or-title> --out <dir> [--manifest jsonl|sqlite]
photoscli verify --out <dir> [--manifest jsonl|sqlite]
photoscli manifest repair --out <dir> [--manifest jsonl|sqlite] [--checksum sha256] [--dry-run]
photoscli cleanup --out <dir> [--manifest jsonl|sqlite] [--dry-run]
photoscli doctor [--out <dir>] [--manifest jsonl|sqlite] [--json]
photoscli retry-failed --out <dir>
photoscli retry-failed --out <dir> --clear-on-success
photoscli failures list --out <dir>
@@ -177,6 +190,16 @@ COMMANDS
files are printed as <asset-id><TAB><filename>. Exits 2 on missing files.
Add --sidecar to verify expected XMP sidecars too. Add --strict with
--sidecar to require photoscli schema/generator and exported filename metadata.
Add --deep to verify manifest SHA-256 checksums when present.
manifest repair --out <dir> [--manifest jsonl|sqlite] [--checksum sha256] [--dry-run]
Fill missing manifest size/checksum metadata from files that exist on disk.
cleanup --out <dir> [--manifest jsonl|sqlite] [--dry-run]
Remove files not referenced by the manifest. Use --dry-run first.
doctor [--out <dir>] [--manifest jsonl|sqlite] [--json]
Check Photos access and optional backup/manifest health without changing data.
retry-failed --out <dir>
Retry assets previously written to failures.jsonl.
@@ -228,6 +251,9 @@ COMMON EXPORT FLAGS
--verify
Run manifest/file verification after export or backup-all.
--checksum none|sha256
Store optional file checksum metadata in the manifest. Default: none.
--sidecar none|xmp|json|xmp,json
Write opt-in metadata sidecars next to each exported file. Default: none.
If sidecar writing fails, the asset is counted as failed.
@@ -789,7 +815,7 @@ func logEntry(event, level, assetID, album, filename, cloud string, size int64,
}
}
func addManifestEntry(m manifest.Manifest, pa pendingAsset, result photos.ExportResult) {
func addManifestEntry(m manifest.Manifest, pa pendingAsset, result photos.ExportResult, checksum string) {
if m == nil {
return
}
@@ -802,7 +828,33 @@ func addManifestEntry(m manifest.Manifest, pa pendingAsset, result photos.Export
if err != nil || strings.HasPrefix(relPath, "..") {
relPath = result.Filename
}
m.AddEntry(manifest.NewEntry(pa.asset.ID, result.Filename, relPath, result.Size, result.Cloud))
m.AddEntry(manifest.NewEntryWithChecksum(pa.asset.ID, result.Filename, relPath, result.Size, result.Cloud, checksum))
}
func addManifestEntryForResult(m manifest.Manifest, pa pendingAsset, result photos.ExportResult, opts exportOptions) error {
checksum := ""
if opts.checksum == "sha256" && !result.Skipped {
var err error
checksum, err = fileSHA256(filepath.Join(pa.path, result.Filename))
if err != nil {
return err
}
}
addManifestEntry(m, pa, result, checksum)
return nil
}
func fileSHA256(path string) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return "", err
}
return hex.EncodeToString(h.Sum(nil)), nil
}
type xmpSidecarData struct {
@@ -1468,16 +1520,20 @@ func exportPendingSerial(pending []pendingAsset, targetSize, quality int, origin
failed++
appendFailure(pa.path, pa, exportErr)
} else if isSkipped {
addManifestEntry(m, pa, result)
_ = addManifestEntryForResult(m, pa, result, opts)
} else {
if sidecarErr := writeSidecarIfNeeded(pa, result, originals, opts, geo, bridge); sidecarErr != nil {
failed++
exportErr = sidecarErr
isErr = true
appendFailure(pa.path, pa, sidecarErr)
} else if checksumErr := addManifestEntryForResult(m, pa, result, opts); checksumErr != nil {
failed++
exportErr = checksumErr
isErr = true
appendFailure(pa.path, pa, checksumErr)
} else {
done++
addManifestEntry(m, pa, result)
}
}
avgSpeed := float64(0)
@@ -1608,16 +1664,20 @@ func exportPendingParallel(pending []pendingAsset, targetSize, quality int, orig
failed++
appendFailure(entry.pa.path, entry.pa, entry.err)
} else if isSkipped {
addManifestEntry(m, entry.pa, entry.result)
_ = addManifestEntryForResult(m, entry.pa, entry.result, opts)
} else {
if sidecarErr := writeSidecarIfNeeded(entry.pa, entry.result, originals, opts, geo, bridge); sidecarErr != nil {
failed++
entry.err = sidecarErr
isErr = true
appendFailure(entry.pa.path, entry.pa, sidecarErr)
} else if checksumErr := addManifestEntryForResult(m, entry.pa, entry.result, opts); checksumErr != nil {
failed++
entry.err = checksumErr
isErr = true
appendFailure(entry.pa.path, entry.pa, checksumErr)
} else {
done++
addManifestEntry(m, entry.pa, entry.result)
}
}
avgSpeed := float64(0)
@@ -1966,6 +2026,7 @@ func parseExportOptions(args []string, stderr io.Writer) (exportOptions, bool) {
verify: hasFlag(args, "--verify"),
format: flagValWithDefault(args, "--format", "jpeg"),
sidecar: flagValWithDefault(args, "--sidecar", "none"),
checksum: flagValWithDefault(args, "--checksum", "none"),
xmpPrivacy: flagValWithDefault(args, "--xmp-privacy", "keep"),
xmpKeywords: flagValWithDefault(args, "--xmp-keywords", "album-path"),
xmpRating: flagValWithDefault(args, "--xmp-rating", "favorite"),
@@ -1994,6 +2055,10 @@ func parseExportOptions(args []string, stderr io.Writer) (exportOptions, bool) {
}
}
}
if opts.checksum != "none" && opts.checksum != "sha256" {
fmt.Fprintf(stderr, "error: --checksum must be none or sha256, got %q\n", opts.checksum)
return opts, false
}
if opts.xmpPrivacy != "keep" && opts.xmpPrivacy != "strip-location" && opts.xmpPrivacy != "strip-address" {
fmt.Fprintf(stderr, "error: --xmp-privacy must be keep, strip-location, or strip-address, got %q\n", opts.xmpPrivacy)
return opts, false
@@ -2234,6 +2299,7 @@ func cmdVerify(args []string, stdout, stderr io.Writer) int {
outDir := flagVal(args, "--out")
checkSidecar := hasFlag(args, "--sidecar")
strictSidecar := hasFlag(args, "--strict")
deep := hasFlag(args, "--deep")
if outDir == "" {
fmt.Fprintln(stderr, "error: --out is required")
return exitErr
@@ -2272,6 +2338,16 @@ func cmdVerify(args []string, stdout, stderr io.Writer) int {
bad++
fmt.Fprintf(stdout, "%s\t%s\tsize-mismatch\tmanifest=%d\tdisk=%d\n", id, checkPath, e.Size, info.Size())
}
if deep && e.Checksum != "" {
checksum, err := fileSHA256(filepath.Join(outDir, checkPath))
if err != nil {
bad++
fmt.Fprintf(stdout, "%s\t%s\tchecksum-unreadable\n", id, checkPath)
} else if checksum != e.Checksum {
bad++
fmt.Fprintf(stdout, "%s\t%s\tchecksum-mismatch\tmanifest=%s\tdisk=%s\n", id, checkPath, e.Checksum, checksum)
}
}
if checkSidecar {
bad += verifySidecar(stdout, outDir, id, checkPath, strictSidecar)
}
@@ -2326,6 +2402,218 @@ func verifySidecar(stdout io.Writer, outDir, id, checkPath string, strict bool)
return 0
}
func cmdDoctor(args []string, stdout, stderr io.Writer, bridge photos.Bridge) int {
result := map[string]any{"photos_access": "ok"}
problems := 0
if rc := mustAuth(stderr, bridge); rc != exitOK {
result["photos_access"] = "denied"
problems++
}
outDir := flagVal(args, "--out")
if outDir != "" {
result["out"] = outDir
if info, err := statFunc(outDir); err != nil || !info.IsDir() {
result["backup_dir"] = "missing"
problems++
} else {
result["backup_dir"] = "ok"
mf, err := manifest.ParseFormat(flagValWithDefault(args, "--manifest", "jsonl"))
if err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
entries, err := loadManifestEntries(outDir, mf)
if err != nil {
result["manifest"] = "error"
problems++
} else {
result["manifest"] = string(mf)
result["entries"] = len(entries)
}
failures := loadFailures(outDir)
result["failures"] = len(failures)
if len(failures) > 0 {
problems++
}
}
}
if hasFlag(args, "--json") {
result["problems"] = problems
if err := json.NewEncoder(stdout).Encode(result); err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
} else {
keys := make([]string, 0, len(result))
for k := range result {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
fmt.Fprintf(stdout, "%s\t%v\n", k, result[k])
}
fmt.Fprintf(stdout, "problems\t%d\n", problems)
}
if problems > 0 {
return exitPartial
}
return exitOK
}
func cmdCleanup(args []string, stdout, stderr io.Writer) int {
outDir := flagVal(args, "--out")
if outDir == "" {
fmt.Fprintln(stderr, "error: --out is required")
return exitErr
}
mf, err := manifest.ParseFormat(flagValWithDefault(args, "--manifest", "jsonl"))
if err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
entries, err := loadManifestEntries(outDir, mf)
if err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
keep := map[string]bool{
"downloads.jsonl": true,
"downloads.db": true,
"export.log": true,
"failures.jsonl": true,
}
for _, e := range entries {
checkPath := e.Path
if checkPath == "" {
checkPath = e.Filename
}
if checkPath == "" {
continue
}
clean := filepath.Clean(checkPath)
if clean == "." || strings.HasPrefix(clean, "..") || filepath.IsAbs(clean) {
continue
}
keep[clean] = true
keep[filepath.Clean(sidecarPath(clean))] = true
keep[filepath.Clean(jsonSidecarPath(clean))] = true
}
dryRun := hasFlag(args, "--dry-run")
removed := 0
skipped := 0
_ = filepath.WalkDir(outDir, func(path string, d fs.DirEntry, _ error) error {
if path == outDir {
return nil
}
rel, _ := filepath.Rel(outDir, path)
rel = filepath.Clean(rel)
if d.IsDir() {
if rel == ".photoscli" || strings.HasPrefix(rel, ".photoscli"+string(os.PathSeparator)) {
return filepath.SkipDir
}
return nil
}
if keep[rel] {
return nil
}
fmt.Fprintf(stdout, "%s\torphan\n", rel)
removed++
if !dryRun {
if err := removeFunc(path); err != nil {
skipped++
}
}
return nil
})
fmt.Fprintf(stdout, "removed\t%d\nskipped\t%d\n", removed, skipped)
return exitOK
}
func cmdManifest(args []string, stdout, stderr io.Writer) int {
if len(args) < 1 || args[0] != "repair" {
fmt.Fprintln(stderr, "error: expected manifest repair")
return exitErr
}
outDir := flagVal(args, "--out")
if outDir == "" {
fmt.Fprintln(stderr, "error: --out is required")
return exitErr
}
checksumMode := flagValWithDefault(args, "--checksum", "none")
if checksumMode != "none" && checksumMode != "sha256" {
fmt.Fprintf(stderr, "error: --checksum must be none or sha256, got %q\n", checksumMode)
return exitErr
}
mf, err := manifest.ParseFormat(flagValWithDefault(args, "--manifest", "jsonl"))
if err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
m, err := manifest.Open(outDir, mf)
if err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
defer m.Close()
reader := m.(manifest.EntryReader)
if !hasFlag(args, "--dry-run") {
if err := m.OpenAppend(); err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
}
repaired := 0
skipped := 0
for id, entry := range reader.Entries() {
checkPath := entry.Path
if checkPath == "" {
checkPath = entry.Filename
}
if checkPath == "" {
skipped++
continue
}
fullPath := filepath.Join(outDir, checkPath)
info, err := statFunc(fullPath)
if err != nil || info.IsDir() || info.Size() == 0 {
skipped++
continue
}
updated := entry
updated.ID = id
changed := false
if updated.Size != info.Size() {
updated.Size = info.Size()
changed = true
}
if checksumMode == "sha256" && updated.Checksum == "" {
checksum, err := fileSHA256(fullPath)
if err != nil {
skipped++
continue
}
updated.Checksum = checksum
changed = true
}
if !changed {
continue
}
repaired++
fmt.Fprintf(stdout, "%s\t%s\trepaired\n", id, checkPath)
if !hasFlag(args, "--dry-run") {
m.AddEntry(updated)
}
}
if !hasFlag(args, "--dry-run") {
if err := m.Save(); err != nil {
fmt.Fprintf(stderr, "error: %v\n", err)
return exitErr
}
}
fmt.Fprintf(stdout, "repaired\t%d\nskipped\t%d\n", repaired, skipped)
return exitOK
}
func cmdSidecar(args []string, stdout, stderr io.Writer) int {
if len(args) < 1 || args[0] != "inspect" {
fmt.Fprintln(stderr, "error: expected sidecar inspect <file.xmp>")
+354 -1
View File
@@ -39,6 +39,15 @@ type errWriter struct{}
func (errWriter) Write([]byte) (int, error) { return 0, fmt.Errorf("write") }
type testFileInfo struct{ size int64 }
func (t testFileInfo) Name() string { return "test" }
func (t testFileInfo) Size() int64 { return t.size }
func (t testFileInfo) Mode() os.FileMode { return 0644 }
func (t testFileInfo) ModTime() time.Time { return time.Time{} }
func (t testFileInfo) IsDir() bool { return false }
func (t testFileInfo) Sys() any { return nil }
type noEntryManifest struct{}
func (noEntryManifest) Has(string) bool { return false }
@@ -2662,6 +2671,23 @@ func TestExportPendingParallelManifestAdd(t *testing.T) {
}
}
func TestExportPendingParallelChecksumError(t *testing.T) {
dir := t.TempDir()
mf := manifest.LoadJSONL(dir)
if err := mf.OpenAppend(); err != nil {
t.Fatal(err)
}
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
return photos.ExportResult{Filename: "missing.jpg", Size: 3}, nil
}
bar := newProgressBar(io.Discard, 1)
done, failed := exportPendingParallel([]pendingAsset{{asset: b.assets[0], path: dir}}, 1024, 85, false, 1, bar, b, 1, mf, manifest.NoopLogWriter, exportOptions{checksum: "sha256"})
if done != 0 || failed != 1 {
t.Fatalf("done=%d failed=%d", done, failed)
}
}
func TestExportPendingParallelCancel(t *testing.T) {
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
b.Cancel()
@@ -2804,6 +2830,58 @@ func TestExportAssetsManifestWrite(t *testing.T) {
}
}
func TestExportAssetsChecksum(t *testing.T) {
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
dir := t.TempDir()
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
if err := os.WriteFile(filepath.Join(out, "img.jpg"), []byte("abc"), 0644); err != nil {
return photos.ExportResult{}, err
}
return photos.ExportResult{Filename: "img.jpg", Size: 3, Cloud: "local"}, nil
}
done, failed := exportAssets(b.assets, dir, 1024, 85, 1, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{checksum: "sha256"})
if done != 1 || failed != 0 {
t.Fatalf("done=%d failed=%d", done, failed)
}
entry := manifest.LoadJSONL(dir).Entries()["x1"]
if entry.Checksum != "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" {
t.Fatalf("checksum=%q", entry.Checksum)
}
got, err := fileSHA256(filepath.Join(dir, "img.jpg"))
if err != nil || got != entry.Checksum {
t.Fatalf("fileSHA256 got=%q err=%v", got, err)
}
if _, err := fileSHA256(filepath.Join(dir, "missing.jpg")); err == nil {
t.Fatal("expected missing checksum error")
}
if _, err := fileSHA256(dir); err == nil {
t.Fatal("expected directory checksum error")
}
if err := addManifestEntryForResult(nil, pendingAsset{}, photos.ExportResult{}, exportOptions{}); err != nil {
t.Fatalf("nil manifest add error: %v", err)
}
if err := addManifestEntryForResult(manifest.LoadJSONL(dir), pendingAsset{asset: photos.Asset{ID: "skip"}, path: dir}, photos.ExportResult{Filename: "missing.jpg", Skipped: true}, exportOptions{checksum: "sha256"}); err != nil {
t.Fatalf("skipped checksum add error: %v", err)
}
}
func TestExportAssetsChecksumErrors(t *testing.T) {
dir := t.TempDir()
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "img.jpg"}}}
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
return photos.ExportResult{Filename: "missing.jpg", Size: 3}, nil
}
done, failed := exportAssets(b.assets, dir, 1024, 85, 1, false, 1, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{checksum: "sha256"})
if done != 0 || failed != 1 {
t.Fatalf("serial checksum error done=%d failed=%d", done, failed)
}
dir = t.TempDir()
done, failed = exportAssets(b.assets, dir, 1024, 85, 2, false, 2, io.Discard, b, "", false, manifest.FormatJSONL, false, exportOptions{checksum: "sha256"})
if done != 0 || failed != 1 {
t.Fatalf("parallel checksum error done=%d failed=%d", done, failed)
}
}
func TestBackupTreeManifestOpenErr(t *testing.T) {
b := &mockBridge{
tree: []photos.CollectionNode{
@@ -4204,6 +4282,277 @@ func TestMoreIntegrityBranches(t *testing.T) {
}
}
func TestVerifyDeepChecksums(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "good.jpg"), []byte("abc"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "bad.jpg"), []byte("bad"), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "nocheck.jpg"), []byte("data"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(filepath.Join(dir, "adir"), 0755); err != nil {
t.Fatal(err)
}
m := manifest.LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(manifest.Entry{ID: "good", Filename: "good.jpg", Path: "good.jpg", Size: 3, Cloud: "local", Checksum: "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "bad", Filename: "bad.jpg", Path: "bad.jpg", Size: 3, Cloud: "local", Checksum: "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "nocheck", Filename: "nocheck.jpg", Path: "nocheck.jpg", Size: 4, Cloud: "local", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "unreadable", Filename: "adir", Path: "adir", Cloud: "local", Checksum: "abc", Exported: time.Now().Unix()})
m.Close()
out, stderr, rc := runWith([]string{"verify", "--out", dir, "--deep"}, &mockBridge{})
if rc != exitPartial || stderr != "" || !strings.Contains(out, "checksum-mismatch") || !strings.Contains(out, "checksum-unreadable") || strings.Contains(out, "good.jpg") || strings.Contains(out, "nocheck") {
t.Fatalf("deep verify rc=%d out=%q stderr=%q", rc, out, stderr)
}
out, stderr, rc = runWith([]string{"verify", "--out", dir}, &mockBridge{})
if rc != exitOK || stderr != "" || strings.Contains(out, "checksum") {
t.Fatalf("plain verify rc=%d out=%q stderr=%q", rc, out, stderr)
}
}
func TestManifestRepair(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "photo.jpg"), []byte("abc"), 0644); err != nil {
t.Fatal(err)
}
m := manifest.LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(manifest.Entry{ID: "x1", Filename: "photo.jpg", Path: "photo.jpg", Size: 0, Cloud: "local", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "missing", Filename: "missing.jpg", Path: "missing.jpg", Size: 0, Cloud: "local", Exported: time.Now().Unix()})
m.Close()
out, stderr, rc := runWith([]string{"manifest", "repair", "--out", dir, "--checksum", "sha256", "--dry-run"}, &mockBridge{})
if rc != exitOK || stderr != "" || !strings.Contains(out, "x1\tphoto.jpg\trepaired") || !strings.Contains(out, "skipped\t1") {
t.Fatalf("dry repair rc=%d out=%q stderr=%q", rc, out, stderr)
}
if got := manifest.LoadJSONL(dir).Entries()["x1"].Checksum; got != "" {
t.Fatalf("dry run wrote checksum %q", got)
}
out, stderr, rc = runWith([]string{"manifest", "repair", "--out", dir, "--checksum", "sha256"}, &mockBridge{})
if rc != exitOK || stderr != "" || !strings.Contains(out, "repaired\t1") {
t.Fatalf("repair rc=%d out=%q stderr=%q", rc, out, stderr)
}
entry := manifest.LoadJSONL(dir).Entries()["x1"]
if entry.Size != 3 || entry.Checksum != "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad" {
t.Fatalf("entry not repaired: %+v", entry)
}
out, stderr, rc = runWith([]string{"manifest", "repair", "--out", dir, "--checksum", "sha256"}, &mockBridge{})
if rc != exitOK || stderr != "" || !strings.Contains(out, "repaired\t0") {
t.Fatalf("second repair rc=%d out=%q stderr=%q", rc, out, stderr)
}
}
func TestManifestRepairErrors(t *testing.T) {
dir := t.TempDir()
_, stderr, rc := runWith([]string{"manifest"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "expected manifest repair") {
t.Fatalf("manifest missing subcommand rc=%d stderr=%q", rc, stderr)
}
_, stderr, rc = runWith([]string{"manifest", "repair"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "--out") {
t.Fatalf("manifest repair missing out rc=%d stderr=%q", rc, stderr)
}
_, stderr, rc = runWith([]string{"manifest", "repair", "--out", dir, "--checksum", "bad"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "--checksum") {
t.Fatalf("manifest repair bad checksum rc=%d stderr=%q", rc, stderr)
}
_, stderr, rc = runWith([]string{"manifest", "repair", "--out", dir, "--manifest", "bad"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "manifest") {
t.Fatalf("manifest repair bad manifest rc=%d stderr=%q", rc, stderr)
}
fileOut := filepath.Join(dir, "file-out")
if err := os.WriteFile(fileOut, []byte("x"), 0644); err != nil {
t.Fatal(err)
}
_, stderr, rc = runWith([]string{"manifest", "repair", "--out", fileOut}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "error:") {
t.Fatalf("manifest repair open append rc=%d stderr=%q", rc, stderr)
}
badDBDir := t.TempDir()
if err := os.WriteFile(manifest.SQLitePath(badDBDir), []byte("not sqlite"), 0644); err != nil {
t.Fatal(err)
}
_, stderr, rc = runWith([]string{"manifest", "repair", "--out", badDBDir}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "error:") {
t.Fatalf("manifest repair open rc=%d stderr=%q", rc, stderr)
}
saveDir := t.TempDir()
sm := manifest.LoadJSONL(saveDir)
if err := sm.OpenAppend(); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(saveDir, "photo.jpg"), []byte("abc"), 0644); err != nil {
t.Fatal(err)
}
sm.AddEntry(manifest.Entry{ID: "x1", Filename: "photo.jpg", Path: "photo.jpg", Exported: time.Now().Unix()})
sm.Close()
oldHook := manifest.SetJSONLSaveHook(func() error { return fmt.Errorf("save") })
_, stderr, rc = runWith([]string{"manifest", "repair", "--out", saveDir}, &mockBridge{})
manifest.SetJSONLSaveHook(oldHook)
if rc != exitErr || !strings.Contains(stderr, "save") {
t.Fatalf("manifest repair save rc=%d stderr=%q", rc, stderr)
}
}
func TestManifestRepairBranches(t *testing.T) {
dir := t.TempDir()
m := manifest.LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(manifest.Entry{ID: "empty"})
m.AddEntry(manifest.Entry{ID: "stat-only", Filename: "missing.jpg", Path: "missing.jpg", Checksum: "", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "hash-fail", Filename: "hash.jpg", Path: "hash.jpg", Checksum: "", Exported: time.Now().Unix()})
m.Close()
oldStat := statFunc
statFunc = func(path string) (os.FileInfo, error) {
if strings.HasSuffix(path, "hash.jpg") {
return testFileInfo{size: 3}, nil
}
return oldStat(path)
}
out, stderr, rc := runWith([]string{"manifest", "repair", "--out", dir, "--checksum", "sha256", "--dry-run"}, &mockBridge{})
statFunc = oldStat
if rc != exitOK || stderr != "" || !strings.Contains(out, "skipped\t3") {
t.Fatalf("manifest repair branches rc=%d out=%q stderr=%q", rc, out, stderr)
}
}
func TestCleanup(t *testing.T) {
dir := t.TempDir()
for _, path := range []string{"keep.jpg", "keep.xmp", "keep.json", "orphan.jpg", "export.log", "failures.jsonl", filepath.Join(".photoscli", "geocode-cache.jsonl")} {
full := filepath.Join(dir, path)
if err := os.MkdirAll(filepath.Dir(full), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte("data"), 0644); err != nil {
t.Fatal(err)
}
}
m := manifest.LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(manifest.Entry{ID: "x1", Filename: "keep.jpg", Path: "keep.jpg", Size: 4, Cloud: "local", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "fallback", Filename: "fallback.jpg", Size: 4, Cloud: "local", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "empty", Exported: time.Now().Unix()})
m.AddEntry(manifest.Entry{ID: "badrel", Filename: "../bad.jpg", Path: "../bad.jpg", Exported: time.Now().Unix()})
m.Close()
if err := os.WriteFile(filepath.Join(dir, "fallback.jpg"), []byte("data"), 0644); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(filepath.Join(dir, "subdir"), 0755); err != nil {
t.Fatal(err)
}
out, stderr, rc := runWith([]string{"cleanup", "--out", dir, "--dry-run"}, &mockBridge{})
if rc != exitOK || stderr != "" || !strings.Contains(out, "orphan.jpg\torphan") || !strings.Contains(out, "removed\t1") {
t.Fatalf("cleanup dry rc=%d out=%q stderr=%q", rc, out, stderr)
}
if _, err := os.Stat(filepath.Join(dir, "orphan.jpg")); err != nil {
t.Fatalf("dry-run removed orphan: %v", err)
}
out, stderr, rc = runWith([]string{"cleanup", "--out", dir}, &mockBridge{})
if rc != exitOK || stderr != "" || !strings.Contains(out, "removed\t1") {
t.Fatalf("cleanup rc=%d out=%q stderr=%q", rc, out, stderr)
}
if _, err := os.Stat(filepath.Join(dir, "orphan.jpg")); !os.IsNotExist(err) {
t.Fatalf("orphan still exists or bad error: %v", err)
}
for _, path := range []string{"keep.jpg", "keep.xmp", "keep.json", "fallback.jpg", "downloads.jsonl", "export.log", "failures.jsonl", filepath.Join(".photoscli", "geocode-cache.jsonl")} {
if _, err := os.Stat(filepath.Join(dir, path)); err != nil {
t.Fatalf("kept file missing %s: %v", path, err)
}
}
}
func TestCleanupErrors(t *testing.T) {
dir := t.TempDir()
_, stderr, rc := runWith([]string{"cleanup"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "--out") {
t.Fatalf("cleanup missing out rc=%d stderr=%q", rc, stderr)
}
_, stderr, rc = runWith([]string{"cleanup", "--out", dir, "--manifest", "bad"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "manifest") {
t.Fatalf("cleanup bad manifest rc=%d stderr=%q", rc, stderr)
}
badDBDir := t.TempDir()
if err := os.WriteFile(manifest.SQLitePath(badDBDir), []byte("not sqlite"), 0644); err != nil {
t.Fatal(err)
}
_, stderr, rc = runWith([]string{"cleanup", "--out", badDBDir, "--manifest", "sqlite"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "error:") {
t.Fatalf("cleanup load error rc=%d stderr=%q", rc, stderr)
}
_, stderr, rc = runWith([]string{"cleanup", "--out", filepath.Join(dir, "missing")}, &mockBridge{})
if rc != exitOK || stderr != "" {
t.Fatalf("cleanup missing root rc=%d stderr=%q", rc, stderr)
}
dir = t.TempDir()
oldRemove := removeFunc
removeFunc = func(string) error { return fmt.Errorf("remove") }
if err := os.WriteFile(filepath.Join(dir, "orphan.jpg"), []byte("data"), 0644); err != nil {
t.Fatal(err)
}
out, stderr, rc := runWith([]string{"cleanup", "--out", dir}, &mockBridge{})
removeFunc = oldRemove
if rc != exitOK || stderr != "" || !strings.Contains(out, "skipped\t1") {
t.Fatalf("cleanup remove error rc=%d out=%q stderr=%q", rc, out, stderr)
}
}
func TestDoctor(t *testing.T) {
dir := t.TempDir()
m := manifest.LoadJSONL(dir)
if err := m.OpenAppend(); err != nil {
t.Fatal(err)
}
m.AddEntry(manifest.Entry{ID: "x1", Filename: "photo.jpg", Path: "photo.jpg", Size: 4, Cloud: "local", Exported: time.Now().Unix()})
m.Close()
out, stderr, rc := runWith([]string{"doctor", "--out", dir}, &mockBridge{})
if rc != exitOK || !strings.Contains(stderr, "access granted") || !strings.Contains(out, "backup_dir\tok") || !strings.Contains(out, "entries\t1") || !strings.Contains(out, "problems\t0") {
t.Fatalf("doctor rc=%d out=%q stderr=%q", rc, out, stderr)
}
out, stderr, rc = runWith([]string{"doctor", "--out", dir, "--json"}, &mockBridge{})
if rc != exitOK || !strings.Contains(out, `"entries":1`) || !strings.Contains(out, `"problems":0`) {
t.Fatalf("doctor json rc=%d out=%q stderr=%q", rc, out, stderr)
}
}
func TestDoctorProblems(t *testing.T) {
dir := t.TempDir()
appendFailure(dir, pendingAsset{asset: photos.Asset{ID: "x1", Filename: "photo.jpg"}}, fmt.Errorf("failed"))
out, stderr, rc := runWith([]string{"doctor", "--out", dir}, &mockBridge{accessErr: fmt.Errorf("denied")})
if rc != exitPartial || !strings.Contains(stderr, "denied") || !strings.Contains(out, "photos_access\tdenied") || !strings.Contains(out, "failures\t1") {
t.Fatalf("doctor problems rc=%d out=%q stderr=%q", rc, out, stderr)
}
_, stderr, rc = runWith([]string{"doctor", "--out", dir, "--manifest", "bad"}, &mockBridge{})
if rc != exitErr || !strings.Contains(stderr, "manifest") {
t.Fatalf("doctor bad manifest rc=%d stderr=%q", rc, stderr)
}
out, stderr, rc = runWith([]string{"doctor", "--out", filepath.Join(dir, "missing")}, &mockBridge{})
if rc != exitPartial || !strings.Contains(out, "backup_dir\tmissing") {
t.Fatalf("doctor missing dir rc=%d out=%q stderr=%q", rc, out, stderr)
}
badDBDir := t.TempDir()
if err := os.WriteFile(manifest.SQLitePath(badDBDir), []byte("not sqlite"), 0644); err != nil {
t.Fatal(err)
}
out, stderr, rc = runWith([]string{"doctor", "--out", badDBDir, "--manifest", "sqlite"}, &mockBridge{})
if rc != exitPartial || !strings.Contains(out, "manifest\terror") {
t.Fatalf("doctor manifest error rc=%d out=%q stderr=%q", rc, out, stderr)
}
stderrBuf := &bytes.Buffer{}
if rc := cmdDoctor([]string{"--json"}, errWriter{}, stderrBuf, &mockBridge{}); rc != exitErr || !strings.Contains(stderrBuf.String(), "error:") {
t.Fatalf("doctor json writer error rc=%d stderr=%q", rc, stderrBuf.String())
}
}
func TestVerifySidecarBranches(t *testing.T) {
dir := t.TempDir()
subdir := filepath.Join(dir, "sub")
@@ -4657,6 +5006,10 @@ func TestSidecarConfigAndErrors(t *testing.T) {
if _, ok := parseExportOptions([]string{"--xmp-rating", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--xmp-rating") {
t.Fatalf("expected xmp rating validation error, stderr=%q", stderr.String())
}
stderr.Reset()
if _, ok := parseExportOptions([]string{"--checksum", "bad"}, &stderr); ok || !strings.Contains(stderr.String(), "--checksum") {
t.Fatalf("expected checksum validation error, stderr=%q", stderr.String())
}
b := &mockBridge{assets: []photos.Asset{{ID: "x1", Filename: "photo.jpg"}}}
b.exportPreviewFn = func(assetID, out string, targetSize, quality, index int) (photos.ExportResult, error) {
@@ -5018,7 +5371,7 @@ func TestInjectedErrorBranchesForCoverage(t *testing.T) {
removeFunc = oldRemove
mf := &mockManifest{}
addManifestEntry(mf, pendingAsset{asset: photos.Asset{ID: "x"}, root: filepath.Join(dir, "other"), path: dir}, photos.ExportResult{Filename: "file.jpg", Size: 1})
addManifestEntry(mf, pendingAsset{asset: photos.Asset{ID: "x"}, root: filepath.Join(dir, "other"), path: dir}, photos.ExportResult{Filename: "file.jpg", Size: 1}, "")
if mf.last.Path != "file.jpg" {
t.Fatalf("expected fallback rel path, got %+v", mf.last)
}
+16 -4
View File
@@ -11,6 +11,10 @@ func TestNewEntryPath(t *testing.T) {
if e.Path != "Album/file2.jpg" || e.Filename != "file2.jpg" || e.Size != 456 || e.Cloud != "cloud" {
t.Fatalf("unexpected entry with path: %+v", e)
}
e = NewEntryWithChecksum("id3", "file3.jpg", "Album/file3.jpg", 789, "local", "sha256:abc")
if e.Checksum != "sha256:abc" || e.Path != "Album/file3.jpg" {
t.Fatalf("unexpected checksum entry: %+v", e)
}
}
func TestAddEntryDefaultsPath(t *testing.T) {
@@ -19,11 +23,15 @@ func TestAddEntryDefaultsPath(t *testing.T) {
if err := jm.OpenAppend(); err != nil {
t.Fatal(err)
}
jm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local"})
jm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:abc"})
jm.Close()
if got := LoadJSONL(dir).Entries()["x1"].Path; got != "file.jpg" {
loaded := LoadJSONL(dir).Entries()["x1"]
if got := loaded.Path; got != "file.jpg" {
t.Fatalf("jsonl path = %q", got)
}
if loaded.Checksum != "sha256:abc" {
t.Fatalf("jsonl checksum = %q", loaded.Checksum)
}
sdir := t.TempDir()
sm, err := LoadSQLite(sdir)
@@ -33,12 +41,16 @@ func TestAddEntryDefaultsPath(t *testing.T) {
if err := sm.OpenAppend(); err != nil {
t.Fatal(err)
}
sm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local"})
sm.AddEntry(Entry{ID: "x1", Filename: "file.jpg", Size: 3, Cloud: "local", Checksum: "sha256:def"})
if _, err := sm.db.Exec(`UPDATE downloads SET path = '' WHERE id = 'x1'`); err != nil {
t.Fatal(err)
}
if got := sm.Entries()["x1"].Path; got != "file.jpg" {
sloaded := sm.Entries()["x1"]
if got := sloaded.Path; got != "file.jpg" {
t.Fatalf("sqlite path = %q", got)
}
if sloaded.Checksum != "sha256:def" {
t.Fatalf("sqlite checksum = %q", sloaded.Checksum)
}
sm.Close()
}
+4 -1
View File
@@ -43,6 +43,7 @@ func LoadJSONL(dir string) *jsonlManifest {
Path string `json:"path,omitempty"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"`
}
for _, line := range strings.Split(string(data), "\n") {
@@ -61,6 +62,7 @@ func LoadJSONL(dir string) *jsonlManifest {
Path: raw.Path,
Size: raw.Size,
Cloud: raw.Cloud,
Checksum: raw.Checksum,
Exported: raw.Exported,
}
}
@@ -93,8 +95,9 @@ func (m *jsonlManifest) AddEntry(entry Entry) {
Path string `json:"path,omitempty"`
Size int64 `json:"size"`
Cloud string `json:"cloud"`
Checksum string `json:"checksum,omitempty"`
Exported int64 `json:"exported"`
}{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Exported: entry.Exported})
}{ID: entry.ID, Filename: entry.Filename, Path: entry.Path, Size: entry.Size, Cloud: entry.Cloud, Checksum: entry.Checksum, Exported: entry.Exported})
m.file.Write(data)
m.file.Write([]byte("\n"))
}
+7
View File
@@ -8,6 +8,7 @@ type Entry struct {
Path string
Size int64
Cloud string
Checksum string
Exported int64
}
@@ -42,3 +43,9 @@ func NewEntry(id, filename, path string, size int64, cloud string) Entry {
}
return e
}
func NewEntryWithChecksum(id, filename, path string, size int64, cloud, checksum string) Entry {
e := NewEntry(id, filename, path, size, cloud)
e.Checksum = checksum
return e
}
+6 -4
View File
@@ -64,6 +64,7 @@ func (m *sqliteManifest) OpenAppend() error {
path TEXT NOT NULL DEFAULT '',
size INTEGER NOT NULL DEFAULT 0,
cloud TEXT NOT NULL DEFAULT '',
checksum TEXT NOT NULL DEFAULT '',
exported INTEGER NOT NULL DEFAULT 0
)`)
if err != nil {
@@ -71,6 +72,7 @@ func (m *sqliteManifest) OpenAppend() error {
return fmt.Errorf("create table: %w", err)
}
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN path TEXT NOT NULL DEFAULT ''`)
_, _ = execFn(`ALTER TABLE downloads ADD COLUMN checksum TEXT NOT NULL DEFAULT ''`)
_, err = execFn(`CREATE INDEX IF NOT EXISTS idx_downloads_id ON downloads(id)`)
if err != nil {
db.Close()
@@ -103,8 +105,8 @@ func (m *sqliteManifest) AddEntry(entry Entry) {
if entry.Path == "" {
entry.Path = entry.Filename
}
m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, exported) VALUES (?, ?, ?, ?, ?, ?)`,
entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Exported)
m.db.Exec(`INSERT OR REPLACE INTO downloads (id, filename, path, size, cloud, checksum, exported) VALUES (?, ?, ?, ?, ?, ?, ?)`,
entry.ID, entry.Filename, entry.Path, entry.Size, entry.Cloud, entry.Checksum, entry.Exported)
}
func (m *sqliteManifest) Save() error {
@@ -127,14 +129,14 @@ func (m *sqliteManifest) Entries() map[string]Entry {
return nil
}
out := make(map[string]Entry)
rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, exported FROM downloads`)
rows, err := m.db.Query(`SELECT id, filename, path, size, cloud, checksum, exported FROM downloads`)
if err != nil {
return out
}
defer rows.Close()
for rows.Next() {
var e Entry
if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Exported); err == nil {
if err := rows.Scan(&e.ID, &e.Filename, &e.Path, &e.Size, &e.Cloud, &e.Checksum, &e.Exported); err == nil {
if e.Path == "" {
e.Path = e.Filename
}