rsync_dryrun.out

nohup: ignoring input

============================================================
  Pre-flight Checks
============================================================

✅ Source directory exists: /mnt/backupdrive/restsdk/data/files/
ℹ️    Files: 3,583,981 | Size: 1.76 TB
✅ Destination directory exists: /mnt/nfs-media/
ℹ️    Files: 1,638,692 | Size: 1.92 TB
ℹ️    Free space: 2.90 TB
✅ Database found: /mnt/backupdrive/restsdk/data/db/index.db
ℹ️    Total files in DB: 1,655,973
ℹ️    Files to recover: 1,655,973
✅ Symlink farm exists: /tmp/restore-farm
ℹ️    Symlinks: 27
✅ rsync found: /usr/bin/rsync
ℹ️  Memory: 19.8% used (8.97 GB available)
ℹ️  Load average: 1.37 1.29 1.26

============================================================
  Symlink Farm
============================================================

ℹ️  Force rebuild requested - purging existing farm...
ℹ️  Creating symlink farm (streaming from database)...
  Progress: 5% (82,799/1,655,973)
  Progress: 10% (165,598/1,655,973)
  Progress: 15% (248,396/1,655,973)
  Progress: 20% (331,195/1,655,973)
  Progress: 25% (413,994/1,655,973)
  Progress: 30% (496,792/1,655,973)
  Progress: 35% (579,591/1,655,973)
  Progress: 40% (662,390/1,655,973)
  Progress: 45% (745,188/1,655,973)
  Progress: 50% (827,987/1,655,973)
  Progress: 55% (910,786/1,655,973)
  Progress: 60% (993,584/1,655,973)
  Progress: 65% (1,076,383/1,655,973)
  Progress: 70% (1,159,182/1,655,973)
  Progress: 75% (1,241,980/1,655,973)
  Progress: 80% (1,324,779/1,655,973)
  Progress: 85% (1,407,578/1,655,973)
  Progress: 90% (1,490,376/1,655,973)
  Progress: 95% (1,573,175/1,655,973)
  Progress: 100% (1,655,973/1,655,973)
✅ Created 27 symlinks
ℹ️  Skipped 1 (no source or duplicate)
⚠️  Errors: 1,655,945

============================================================
  Starting rsync
============================================================

[2026-02-16 10:11:31] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 17.3% | Load: 1.16 1.18 1.21
ℹ️  Running: rsync -avL --progress --checksum --dry-run /tmp/restore-farm/ /mnt/nfs-media/

⚠️  rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]

============================================================
  Summary
============================================================


  Source:        3583981 files | 1.76 TB
  Destination:   1638692 files | 1.92 TB

  Started:       2026-02-16 09:17:48
  Finished:      2026-02-16 10:56:59
  Duration:      53m 57s

  Files transferred: 0
  Data transferred:  312.79 KB
  Average speed:     98.00 B/s

⚠️    Errors:            1

  Error details:
    - rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]

ℹ️  Full log written to: rsync_restore.log


nohup: ignoring input

============================================================
  Cleanup Wizard
============================================================

ℹ️  Loading canonical paths from database...
✅ Loaded 1,655,972 canonical paths
ℹ️  Scanning destination: /mnt/nfs-media/
  Scanned 10,000 files...
  Scanned 20,000 files...
  Scanned 30,000 files...
  Scanned 40,000 files...
  Scanned 50,000 files...
  Scanned 60,000 files...
  Scanned 70,000 files...
  Scanned 80,000 files...
  Scanned 90,000 files...
  Scanned 100,000 files...
  Scanned 110,000 files...
  Scanned 120,000 files...
  Scanned 130,000 files...
  Scanned 140,000 files...
  Scanned 150,000 files...
  Scanned 160,000 files...
  Scanned 170,000 files...
  Scanned 180,000 files...
  Scanned 190,000 files...
  Scanned 200,000 files...
  Scanned 210,000 files...
  Scanned 220,000 files...
  Scanned 230,000 files...
  Scanned 240,000 files...
  Scanned 250,000 files...
  Scanned 260,000 files...
  Scanned 270,000 files...
  Scanned 280,000 files...
  Scanned 290,000 files...
  Scanned 300,000 files...
  Scanned 310,000 files...
  Scanned 320,000 files...
  Scanned 330,000 files...
  Scanned 340,000 files...
  Scanned 350,000 files...
  Scanned 360,000 files...
  Scanned 370,000 files...
  Scanned 380,000 files...
  Scanned 390,000 files...
  Scanned 400,000 files...
  Scanned 410,000 files...
  Scanned 420,000 files...
  Scanned 430,000 files...
  Scanned 440,000 files...
  Scanned 450,000 files...
  Scanned 460,000 files...
  Scanned 470,000 files...
  Scanned 480,000 files...
  Scanned 490,000 files...
  Scanned 500,000 files...
  Scanned 510,000 files...
  Scanned 520,000 files...
  Scanned 530,000 files...
  Scanned 540,000 files...
  Scanned 550,000 files...
  Scanned 560,000 files...
  Scanned 570,000 files...
  Scanned 580,000 files...
  Scanned 590,000 files...
  Scanned 600,000 files...
  Scanned 610,000 files...
  Scanned 620,000 files...
  Scanned 630,000 files...
  Scanned 640,000 files...
  Scanned 650,000 files...
  Scanned 660,000 files...
  Scanned 670,000 files...
  Scanned 680,000 files...
  Scanned 690,000 files...
  Scanned 700,000 files...
  Scanned 710,000 files...
  Scanned 720,000 files...
  Scanned 730,000 files...
  Scanned 740,000 files...
  Scanned 750,000 files...
  Scanned 760,000 files...
  Scanned 770,000 files...
  Scanned 780,000 files...
  Scanned 790,000 files...
  Scanned 800,000 files...
  Scanned 810,000 files...
  Scanned 820,000 files...
  Scanned 830,000 files...
  Scanned 840,000 files...
  Scanned 850,000 files...
  Scanned 860,000 files...
  Scanned 870,000 files...
  Scanned 880,000 files...
  Scanned 890,000 files...
  Scanned 900,000 files...
  Scanned 910,000 files...
  Scanned 920,000 files...
  Scanned 930,000 files...
  Scanned 940,000 files...
  Scanned 950,000 files...
  Scanned 960,000 files...
  Scanned 970,000 files...
  Scanned 980,000 files...
  Scanned 990,000 files...
  Scanned 1,000,000 files...
  Scanned 1,010,000 files...
  Scanned 1,020,000 files...
  Scanned 1,030,000 files...
  Scanned 1,040,000 files...
  Scanned 1,050,000 files...
  Scanned 1,060,000 files...
  Scanned 1,070,000 files...
  Scanned 1,080,000 files...
  Scanned 1,090,000 files...
  Scanned 1,100,000 files...
  Scanned 1,110,000 files...
  Scanned 1,120,000 files...
  Scanned 1,130,000 files...
  Scanned 1,140,000 files...
  Scanned 1,150,000 files...
  Scanned 1,160,000 files...
  Scanned 1,170,000 files...
  Scanned 1,180,000 files...
  Scanned 1,190,000 files...
  Scanned 1,200,000 files...
  Scanned 1,210,000 files...
  Scanned 1,220,000 files...
  Scanned 1,230,000 files...
  Scanned 1,240,000 files...
  Scanned 1,250,000 files...
  Scanned 1,260,000 files...
  Scanned 1,270,000 files...
  Scanned 1,280,000 files...
  Scanned 1,290,000 files...
  Scanned 1,300,000 files...
  Scanned 1,310,000 files...
  Scanned 1,320,000 files...
  Scanned 1,330,000 files...
  Scanned 1,340,000 files...
  Scanned 1,350,000 files...
  Scanned 1,360,000 files...
  Scanned 1,370,000 files...
  Scanned 1,380,000 files...
  Scanned 1,390,000 files...
  Scanned 1,400,000 files...
  Scanned 1,410,000 files...
  Scanned 1,420,000 files...
  Scanned 1,430,000 files...
  Scanned 1,440,000 files...
  Scanned 1,450,000 files...
  Scanned 1,460,000 files...
  Scanned 1,470,000 files...
  Scanned 1,480,000 files...
  Scanned 1,490,000 files...
  Scanned 1,500,000 files...
  Scanned 1,510,000 files...
  Scanned 1,520,000 files...
  Scanned 1,530,000 files...
  Scanned 1,540,000 files...
  Scanned 1,550,000 files...
  Scanned 1,560,000 files...
  Scanned 1,570,000 files...
  Scanned 1,580,000 files...
  Scanned 1,590,000 files...
  Scanned 1,600,000 files...
  Scanned 1,610,000 files...
  Scanned 1,620,000 files...
  Scanned 1,630,000 files...
✅ Scanned 1,638,692 files
ℹ️    Matched: 1,311,521
ℹ️    Orphans: 327,171
ℹ️    Protected: 0

============================================================
  Folder Summary
============================================================

  ⚠️  #recycle/
      11 files | 11 orphans | NOT in DB
  ⚠️  1and1/
      3 files | 3 orphans | NOT in DB
  ⚠️  Chapman-Christmas-2019/
      128 files | 12 orphans | FROM: MyCloud
  ⚠️  Chapman-Christmas-2020/
      126 files | 14 orphans | FROM: MyCloud
  ⚠️  Chapman-Christmas-2021/
      28 files | 4 orphans | FROM: MyCloud
  ⚠️  Chapman-Fall-2019/
      193 files | 15 orphans | FROM: MyCloud
  ⚠️  Chapman-Fall-2020/
      155 files | 17 orphans | FROM: MyCloud
  ⚠️  ChapmanFloodData/
      292 files | 64 orphans | FROM: MyCloud
  ⚠️  DSLR-Backups/
      1,369 files | 257 orphans | FROM: MyCloud
  ⚠️  DisneyCruise-2025/
      591 files | 591 orphans | NOT in DB
  ⚠️  Eric-iPhone11-ProMax Camera Roll Backup/
      42,450 files | 6,980 orphans | FROM: MyCloud
  ⚠️  Eric-iPhone7-Plus Camera Roll Backup/
      7,566 files | 1,268 orphans | FROM: MyCloud
  ⚠️  MasterServicesMaintenanceReports/
      1 files | 1 orphans | NOT in DB
  ⚠️  MediaCenter/
      4,401 files | 944 orphans | FROM: MyCloud
  ⚠️  OSxBackup/
      1,544 files | 1,540 orphans | FROM: MyCloud
  ⚠️  OSxData/
      1,139,674 files | 242,569 orphans | FROM: MyCloud
  ⚠️  Plex/
      6 files | 2 orphans | FROM: MyCloud
  ⚠️  Ry's IPhone Camera Roll Backup/
      1,013 files | 158 orphans | FROM: MyCloud
  ⚠️  TaxData/
      284 files | 284 orphans | NOT in DB
  ⚠️  iOSBackup/
      159,740 files | 26,408 orphans | FROM: MyCloud
  ⚠️  iPhone-Ash Camera Roll Backup/
      279,100 files | 46,022 orphans | FROM: MyCloud

============================================================
  Classify Folders
============================================================


📁 '#recycle/' is NOT in MyCloud database.
   This may be a folder you added manually.

  [P]rotect (never delete)  [C]leanup (delete orphans)  [S]kip for now
  Your choice: Traceback (most recent call last):
  File "/home/chapman/projects/wd-mycloud-rsync-recovery/rsync_restore.py", line 1635, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/chapman/projects/wd-mycloud-rsync-recovery/rsync_restore.py", line 1589, in main
    return run_cleanup_wizard(args.dest, args.db, args.config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chapman/projects/wd-mycloud-rsync-recovery/rsync_restore.py", line 535, in run_cleanup_wizard
    response = input(colorize("  Your choice: ", Colors.BOLD)).strip().upper()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 9] Bad file descriptor



chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ sudo $(poetry env info --executable) scripts/diagnose_paths.py --db /mnt/backupdrive/restsdk/data/db/index.db --source /mnt/backupdrive/restsdk/data/files/ --samples 200

============================================================
  Path Reconstruction Diagnosis
============================================================

Checking database...
  Files in database: 1,655,973

Sampling 200 canonical paths from database...
ℹ️  Loading canonical paths from database...
✅ Loaded 1,655,972 canonical paths
  Got 200 sample paths

Sample canonical paths from DB:
------------------------------------------------------------
  OSxData/iPhotos/iPhotoLibrary/Database/Faces/Detected/aWFnzuthR3WV43gNoTbJ+Q.apdetected
  OSxData/iPhotos/iPhotoLibrary2/Thumbnails/2014/05/22/20140522-121350/OJkGf2H%ShSU7kX81W%beA
  OSxData/iPhotos/Photos Library.photoslibrary/resources/media/face/45/00/facetile_4575.jpeg
  OSxData/iPhotos/Photos Library.photoslibrary/resources/proxies/derivatives/34/00/34b0
  OSxData/iPhotos/iPhotoLibrary/Database/Versions/2016/07/25/20160725-144452/1wpF1MzISjKAxzCa2rCBqA/Version-0.apversion
  OSxData/iPhotos/iPhotoLibrary2/Masters/2016/11/13/20161113-023725/IMG_4591.JPG
  OSxData/iPhotos/iPhotoLibrary2/Database/Versions/2014/08/04/20140804-013619
  iOSBackup/24a6451efc4c87aef1732130d05bd84086533164/8f/._8f15e0d3986b7ca7c540ac95c6e37bfab62803ce
  OSxData/iPhotos/iPhotoLibrary/Database/Faces/Detected/SdH+MUczRQiItug8ZexB8w.apdetected
  OSxData/iPhotos/iPhotoLibrary/Database/Versions/2013/06/10/20130610-075224/v%UVvBVNSZiIjqwWkdSVbw/Master.apmaster

Analyzing source directory structure...
  Top-level directories: 32
  Top-level files: 0

Top-level directories in source:
  📁 2/
  📁 3/
  📁 4/
  📁 5/
  📁 6/
  📁 7/
  📁 a/
  📁 b/
  📁 c/
  📁 d/
  📁 e/
  📁 f/
  📁 g/
  📁 h/
  📁 i/
  ... and 17 more

Sample actual file paths in source:
------------------------------------------------------------
  l/lmwgejlyxftrfahgqld4iy5z
  l/l7n3bhrnowlamtmofnjogq7q
  l/lhe7s3on3f4xcfzznhellzvh
  l/lok3wzt7udyqmlry6rpkxfnv
  l/lwpjmarf3ohd4gxrpj76oqg6
  l/lnhid3ald2c3saa7qq7agbxs
  l/l5gii5upl6rrbmhmihcgarop
  l/luepjv3dh47f7qxsb3ta74h3
  l/lmb3cr74rmsqsb4756or4hg5
  l/ll4p274j562amzaiiergt6tx

Checking if canonical paths exist in source...

  ✅ Found: 0/200 (0.0%)
  ❌ Not found: 200/200 (100.0%)
  ⚠️  Partial match: 0/200 (0.0%)

Paths not found at all:
------------------------------------------------------------
  OSxData/iPhotos/iPhotoLibrary/Database/Faces/Detected/aWFnzuthR3WV43gNoTbJ+Q.apdetected
  OSxData/iPhotos/iPhotoLibrary2/Thumbnails/2014/05/22/20140522-121350/OJkGf2H%ShSU7kX81W%beA
  OSxData/iPhotos/Photos Library.photoslibrary/resources/media/face/45/00/facetile_4575.jpeg
  OSxData/iPhotos/Photos Library.photoslibrary/resources/proxies/derivatives/34/00/34b0
  OSxData/iPhotos/iPhotoLibrary/Database/Versions/2016/07/25/20160725-144452/1wpF1MzISjKAxzCa2rCBqA/Version-0.apversion

============================================================
  Analysis
============================================================

DB path prefixes (first folder):
  OSxData: 82
  iPhone-Ash Camera Roll Backup: 9
  iOSBackup: 5
  Eric-iPhone11-ProMax Camera Roll Backup: 2
  Eric-iPhone7-Plus Camera Roll Backup: 1
  OSxBackup: 1

Source path prefixes (first folder):
  l: 20

DB path patterns:
  folder_style: 100

============================================================
  Recommendations
============================================================

⚠️  CRITICAL: No canonical paths found in source!
   This suggests a fundamental path reconstruction mismatch.
   The DB paths don't match the actual source file structure.

   Possible causes:
   - Source directory is wrong
   - Path reconstruction algorithm needs adjustment
   - Files are stored with contentID names, not folder paths

chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ 


chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ sudo $(poetry env info --executable) scripts/debug_contentid.py --db /mnt/backupdrive/restsdk/data/db/index.db --source /mnt/backupdrive/restsdk/data/files/ --limit 20
[sudo] password for chapman: 

======================================================================
  ContentID Debug - Checking 20 samples
======================================================================

File: auth0|5e62c7d40ec9700d5c82bb89
  contentID: 'zaSKH84Ojf8ijhEvPTu-D1GpAwIjYXQ6v-txLxSo'
  contentID[0]: 'z'
  Expected path: /mnt/backupdrive/restsdk/data/files/z/zaSKH84Ojf8ijhEvPTu-D1GpAwIjYXQ6v-txLxSo
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/z/zaSKH84Ojf8ijhEvPTu-D1GpAwIjYXQ6v-txLxSo

File: Eric-iPhone7-Plus Camera Roll Backup
  contentID: 'y445lvr634hfn6wtfvi5xlm3'
  contentID[0]: 'y'
  Expected path: /mnt/backupdrive/restsdk/data/files/y/y445lvr634hfn6wtfvi5xlm3
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/y/y445lvr634hfn6wtfvi5xlm3

File: IMG_7524.HEIC
  contentID: 'simpcvssayesejvqg226mzwu'
  contentID[0]: 's'
  Expected path: /mnt/backupdrive/restsdk/data/files/s/simpcvssayesejvqg226mzwu
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/s/simpcvssayesejvqg226mzwu

File: IMG_7523.PNG
  contentID: 'd3lcv7jaepvfygweagmvbgkn'
  contentID[0]: 'd'
  Expected path: /mnt/backupdrive/restsdk/data/files/d/d3lcv7jaepvfygweagmvbgkn
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/d/d3lcv7jaepvfygweagmvbgkn

File: IMG_7518.PNG
  contentID: 'w2bf74rtt7oeybpdr2676mr6'
  contentID[0]: 'w'
  Expected path: /mnt/backupdrive/restsdk/data/files/w/w2bf74rtt7oeybpdr2676mr6
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/w/w2bf74rtt7oeybpdr2676mr6

File: 60511087764__05F069DD-4373-4905-B8E4-E515D7F8EBCB.JPG
  contentID: 'u3mstuwrem4sdnflofhzrbzf'
  contentID[0]: 'u'
  Expected path: /mnt/backupdrive/restsdk/data/files/u/u3mstuwrem4sdnflofhzrbzf
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/u/u3mstuwrem4sdnflofhzrbzf

File: IMG_7522.HEIC
  contentID: 'a65nt5j7rnb3ybuzcio3ukve'
  contentID[0]: 'a'
  Expected path: /mnt/backupdrive/restsdk/data/files/a/a65nt5j7rnb3ybuzcio3ukve
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/a/a65nt5j7rnb3ybuzcio3ukve

File: 60498182809__645B5447-0872-4A4E-8123-3680F1147DA2.JPG
  contentID: '23uqt57ahk2xllymecuuwx7e'
  contentID[0]: '2'
  Expected path: /mnt/backupdrive/restsdk/data/files/2/23uqt57ahk2xllymecuuwx7e
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/2/23uqt57ahk2xllymecuuwx7e

File: IMG_7514.HEIC
  contentID: 'hhbcs7dmu3vmowly6tvpzret'
  contentID[0]: 'h'
  Expected path: /mnt/backupdrive/restsdk/data/files/h/hhbcs7dmu3vmowly6tvpzret
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/h/hhbcs7dmu3vmowly6tvpzret

File: IMG_7519.HEIC
  contentID: 'co7hwaa4wlmldd4mqvixfrqa'
  contentID[0]: 'c'
  Expected path: /mnt/backupdrive/restsdk/data/files/c/co7hwaa4wlmldd4mqvixfrqa
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/c/co7hwaa4wlmldd4mqvixfrqa

File: IMG_7520.HEIC
  contentID: 'czxwkvdedxjqpyux7xpxwnpm'
  contentID[0]: 'c'
  Expected path: /mnt/backupdrive/restsdk/data/files/c/czxwkvdedxjqpyux7xpxwnpm
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/c/czxwkvdedxjqpyux7xpxwnpm

File: IMG_7515.HEIC
  contentID: 'fq2np3fs47633d7gh7pcqjwl'
  contentID[0]: 'f'
  Expected path: /mnt/backupdrive/restsdk/data/files/f/fq2np3fs47633d7gh7pcqjwl
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/f/fq2np3fs47633d7gh7pcqjwl

File: IMG_7512.HEIC
  contentID: 'pocxyfq4rooedq552k6tl3ff'
  contentID[0]: 'p'
  Expected path: /mnt/backupdrive/restsdk/data/files/p/pocxyfq4rooedq552k6tl3ff
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/p/pocxyfq4rooedq552k6tl3ff

File: IMG_7513.HEIC
  contentID: '4h4ypxmikn6gp57r2rrtuacy'
  contentID[0]: '4'
  Expected path: /mnt/backupdrive/restsdk/data/files/4/4h4ypxmikn6gp57r2rrtuacy
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/4/4h4ypxmikn6gp57r2rrtuacy

File: IMG_7511.HEIC
  contentID: 'et3sycrbtoiryu7wqac2bnc4'
  contentID[0]: 'e'
  Expected path: /mnt/backupdrive/restsdk/data/files/e/et3sycrbtoiryu7wqac2bnc4
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/e/et3sycrbtoiryu7wqac2bnc4

File: IMG_7510.HEIC
  contentID: '4llydrbijkxxcmc63ojm4ilp'
  contentID[0]: '4'
  Expected path: /mnt/backupdrive/restsdk/data/files/4/4llydrbijkxxcmc63ojm4ilp
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/4/4llydrbijkxxcmc63ojm4ilp

File: IMG_7509.HEIC
  contentID: 'dc3iotiyce7hybhchzg4igcc'
  contentID[0]: 'd'
  Expected path: /mnt/backupdrive/restsdk/data/files/d/dc3iotiyce7hybhchzg4igcc
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/d/dc3iotiyce7hybhchzg4igcc

File: IMG_7521.HEIC
  contentID: 'ucfy76kfa5w3hlmviwwkcfyp'
  contentID[0]: 'u'
  Expected path: /mnt/backupdrive/restsdk/data/files/u/ucfy76kfa5w3hlmviwwkcfyp
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/u/ucfy76kfa5w3hlmviwwkcfyp

File: IMG_7497.MOV
  contentID: 'bf2wi7gpqr6klkqdu34zbnto'
  contentID[0]: 'b'
  Expected path: /mnt/backupdrive/restsdk/data/files/b/bf2wi7gpqr6klkqdu34zbnto
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/b/bf2wi7gpqr6klkqdu34zbnto

File: IMG_7499.MOV
  contentID: '7sdfcqaltc7tkqioaqsha3id'
  contentID[0]: '7'
  Expected path: /mnt/backupdrive/restsdk/data/files/7/7sdfcqaltc7tkqioaqsha3id
  Status: ✅ FOUND
  Found at: /mnt/backupdrive/restsdk/data/files/7/7sdfcqaltc7tkqioaqsha3id

======================================================================
  Summary: 20 found, 0 not found
======================================================================

Sample files in source directory:
----------------------------------------------------------------------
  l/lmwgejlyxftrfahgqld4iy5z
  l/l7n3bhrnowlamtmofnjogq7q
  l/lhe7s3on3f4xcfzznhellzvh
  l/lok3wzt7udyqmlry6rpkxfnv
  l/lwpjmarf3ohd4gxrpj76oqg6
  z/zueky2bwbz3nivgip4nzblji
  z/zoxbg3kttp7epyr7wpmtcyr6
  z/z3dnug67bdknnycelybrn4kc
  z/zaojbhjvpvgaqdk5dourwag5
  z/z4dz4e5kqnmtrw6saoakose2
chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ 

chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ sudo $(poetry env info --executable) scripts/debug_contentid.py --db /mnt/backupdrive/restsdk/data/db/index.db --source /mnt/backupdrive/restsdk/data/files/ --deep

======================================================================
  Deep ContentID Analysis - Checking ALL files
======================================================================

Total files with contentID: 1,655,973

Results:
  ✅ Found (files):     1,655,973 (100.0%)
  📁 Found (dirs):      0 (0.0%)
  ❌ Not found:         0 (0.0%)
  ⚠️  Empty contentID:  0
chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ 

chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ sudo $(poetry env info --executable) rsync_restore.py --db /mnt/backupdrive/restsdk/data/db/index.db --source /mnt/backupdrive/restsdk/data/files/ --dest /mnt/nfs-media/ --farm /tmp/restore-farm --force-rebuild --limit 100 --dry-run

============================================================
  Pre-flight Checks
============================================================

✅ Source directory exists: /mnt/backupdrive/restsdk/data/files/
ℹ️    Files: 3,583,981 | Size: 1.76 TB
✅ Destination directory exists: /mnt/nfs-media/
ℹ️    Files: 1,638,692 | Size: 1.92 TB
ℹ️    Free space: 2.90 TB
✅ Database found: /mnt/backupdrive/restsdk/data/db/index.db
ℹ️    Total files in DB: 1,655,973
ℹ️    Files to recover: 1,655,973
✅ Symlink farm exists: /tmp/restore-farm
ℹ️    Symlinks: 27
✅ rsync found: /usr/bin/rsync
ℹ️  Memory: 25.0% used (8.39 GB available)
ℹ️  Load average: 1.25 1.25 1.26

============================================================
  Symlink Farm
============================================================

ℹ️  Force rebuild requested - purging existing farm...
ℹ️  Creating symlink farm (streaming from database)...
⚠️  ⚠️  Testing mode: Processing only 100 files (--limit=100)
⚠️  
⚠️  Reached --limit of 100 files. Stopping...
✅ Created 100 symlinks

============================================================
  Starting rsync
============================================================

[2026-02-17 23:18:41] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 26.7% | Load: 1.21 1.24 1.26
ℹ️  Running: rsync -avL --progress --checksum --dry-run /tmp/restore-farm/ /mnt/nfs-media/


============================================================
  Summary
============================================================


  Source:        3583981 files | 1.76 TB
  Destination:   1638692 files | 1.92 TB

  Started:       2026-02-17 22:15:48
  Finished:      2026-02-18 00:00:59
  Duration:      1h 3m 7s

  Files transferred: 0
  Data transferred:  354.13 MB
  Average speed:     95.74 KB/s

✅   Errors:            0

ℹ️  Full log written to: rsync_restore.log
chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ 

chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ cat restore_full.log
nohup: ignoring input
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required
 exists: /mnt/backupdrive/restsdk/data/files/
ℹ️    Files: 3,583,981 | Size: 1.76 TB
✅ Destination directory exists: /mnt/nfs-media/
ℹ️    Files: 1,638,692 | Size: 1.92 TB
ℹ️    Free space: 2.90 TB
✅ Database found: /mnt/backupdrive/restsdk/data/db/index.db
ℹ️    Total files in DB: 1,655,973
ℹ️    Files to recover: 1,655,973
✅ Symlink farm exists: /tmp/restore-farm
ℹ️    Symlinks: 100
✅ rsync found: /usr/bin/rsync
ℹ️  Memory: 25.3% used (8.36 GB available)
ℹ️  Load average: 1.19 1.27 1.26

============================================================
  Symlink Farm
============================================================

ℹ️  Force rebuild requested - purging existing farm...
ℹ️  Creating symlink farm (streaming from database)...
  Progress: 5% (65,577/1,311,539)
  Progress: 10% (131,154/1,311,539)
  Progress: 15% (196,731/1,311,539)
  Progress: 20% (262,308/1,311,539)
  Progress: 25% (327,885/1,311,539)
  Progress: 30% (393,462/1,311,539)
  Progress: 35% (459,039/1,311,539)
  Progress: 40% (524,616/1,311,539)
  Progress: 45% (590,193/1,311,539)
  Progress: 50% (655,770/1,311,539)
  Progress: 55% (721,347/1,311,539)
  Progress: 60% (786,924/1,311,539)
  Progress: 65% (852,501/1,311,539)
  Progress: 70% (918,078/1,311,539)
  Progress: 75% (983,655/1,311,539)
  Progress: 80% (1,049,232/1,311,539)
  Progress: 85% (1,114,809/1,311,539)
  Progress: 90% (1,180,386/1,311,539)
  Progress: 95% (1,245,963/1,311,539)
  Progress: 100% (1,311,539/1,311,539)
✅ Created 1,311,537 symlinks
ℹ️  Source not found: 2 (already copied or missing - this is normal)

============================================================
  Starting rsync
============================================================

[2026-02-18 06:32:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 25.7% | Load: 1.61 1.45 1.34
ℹ️  Running: rsync -avL --progress --checksum --dry-run /tmp/restore-farm/ /mnt/nfs-media/

[2026-02-18 06:33:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.49 1.45 1.34
[2026-02-18 06:34:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.2% | Load: 1.31 1.40 1.33
[2026-02-18 06:35:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.57 1.43 1.35
[2026-02-18 06:36:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.89 1.54 1.39
[2026-02-18 06:37:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 2.41 1.76 1.48
[2026-02-18 06:38:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 2.20 1.86 1.53
[2026-02-18 06:39:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.71 1.78 1.52
[2026-02-18 06:40:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.2% | Load: 1.55 1.71 1.52
[2026-02-18 06:41:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.94 1.78 1.56
[2026-02-18 06:42:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.6% | Load: 1.52 1.69 1.54
[2026-02-18 06:43:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.2% | Load: 1.55 1.67 1.54
[2026-02-18 06:44:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.7% | Load: 1.72 1.69 1.56
[2026-02-18 06:45:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.57 1.66 1.55
[2026-02-18 06:46:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.6% | Load: 1.64 1.66 1.56
[2026-02-18 06:47:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.5% | Load: 1.84 1.68 1.57
[2026-02-18 06:48:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.44 1.60 1.55
[2026-02-18 06:49:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.47 1.57 1.54
[2026-02-18 06:50:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.56 1.57 1.54
[2026-02-18 06:51:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.25 1.48 1.51
[2026-02-18 06:52:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 26.1% | Load: 1.42 1.50 1.52
[2026-02-18 06:53:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.46 1.49 1.51
[2026-02-18 06:54:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.45 1.49 1.51
[2026-02-18 06:55:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.62 1.54 1.53
[2026-02-18 06:56:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.87 1.60 1.55
[2026-02-18 06:57:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.44 1.52 1.53
[2026-02-18 06:58:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.49 1.53 1.53
[2026-02-18 06:59:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.65 1.56 1.55
[2026-02-18 07:00:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.45 1.53 1.54
[2026-02-18 07:01:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.45 1.51 1.53
[2026-02-18 07:02:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.58 1.53 1.54
[2026-02-18 07:03:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.75 1.57 1.55
[2026-02-18 07:04:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.49 1.53 1.54
[2026-02-18 07:05:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.60 1.58 1.55
[2026-02-18 07:06:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.2% | Load: 1.53 1.55 1.54
[2026-02-18 07:07:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.45 1.51 1.53
[2026-02-18 07:08:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.43 1.49 1.53
[2026-02-18 07:09:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.49 1.49 1.53
[2026-02-18 07:10:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.47 1.48 1.52
[2026-02-18 07:11:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.44 1.48 1.52
[2026-02-18 07:12:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.26 1.42 1.50
[2026-02-18 07:13:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.8% | Load: 1.54 1.47 1.51
[2026-02-18 07:14:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.51 1.47 1.51
[2026-02-18 07:15:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.55 1.48 1.51
[2026-02-18 07:16:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.35 1.44 1.49
[2026-02-18 07:17:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.5% | Load: 1.34 1.42 1.48
[2026-02-18 07:18:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.40 1.40 1.46
[2026-02-18 07:19:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.6% | Load: 1.77 1.50 1.49
[2026-02-18 07:20:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.4% | Load: 1.65 1.50 1.49
[2026-02-18 07:21:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.6% | Load: 1.66 1.53 1.50
[2026-02-18 07:22:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.9% | Load: 1.58 1.53 1.50
[2026-02-18 07:23:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.84 1.60 1.53
[2026-02-18 07:24:15] Files: 0 | Data: 0.00 B | 0.0% | Speed: 0.00 B/s | ETA:  | Mem: 24.3% | Load: 1.96 1.68 1.56
chapman@chapman-htpc:~/projects/wd-mycloud-rsync-recovery$ jobs -l