halo/docs/icloud-upload-state.md at main · YordiLorenzo/halo · GitHub
//blob/show" data-turbo-transient="true" />
Skip to content
Type / to search
Sign in<br>Sign upAppearance settings
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
YordiLorenzo
halo
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star
FilesExpand file tree
main
/icloud-upload-state.md
Copy path
Blame<br>More file actions
Blame<br>More file actions
Latest commit
History<br>History<br>History
148 lines (106 loc) · 6.64 KB
main
/icloud-upload-state.md
Copy path
Top
File metadata and controls<br>Preview
Code
Blame
148 lines (106 loc) · 6.64 KB
Raw<br>Copy raw file<br>Download raw file
OutlineEdit and raw actions
How to tell whether iCloud Photos has actually uploaded your library
Every photo migration ends with the same decision: is it safe to erase the drive holding the<br>only other copy? Answering that turns out to be harder than it should be, because a Photos<br>library contains two records of what has been uploaded and they do not agree.
On the library measured here, 345.9 GB of original photos and videos were sitting in<br>CloudPhotos' upload queue while the library index flagged them as already available in iCloud.
Everything below is reproducible on your own library. All queries are read only.
Check your own library first
Takes about a second, writes nothing. Terminal needs Full Disk Access, under<br>System Settings, Privacy & Security.
LIB="$HOME/Pictures/Photos Library.photoslibrary"<br>sqlite3 "file:$LIB/resources/cpl/cloudsync.noindex/storage/store.cloudphotodb?mode=ro" "<br>ATTACH \"file:$LIB/database/Photos.sqlite?mode=ro\" AS p;<br>WITH uniq AS (SELECT ZFINGERPRINT f, MAX(ZREMOTEAVAILABILITY) avail FROM p.ZINTERNALRESOURCE<br>WHERE ZFINGERPRINT IS NOT NULL GROUP BY 1 HAVING count(*)=1),<br>q AS (SELECT DISTINCT fingerPrint f, fileSize FROM outgoingResources)<br>SELECT printf('%d files (%.1f GB) are queued for upload but flagged as already in iCloud',<br>count(*), COALESCE(SUM(q.fileSize),0)/1e9)<br>FROM q JOIN uniq u ON u.f=q.f WHERE u.avail=1;"
A fully synced library reports 0 files. unable to open database file means that library is<br>not the one signed into iCloud Photos; the cloudsync directory only exists on the system<br>library.
One note on the connection string. mode=ro matters, because these are live databases that<br>CloudPhotos is writing to. Do not substitute immutable=1. That tells SQLite to ignore the<br>write ahead log and will report a perfectly healthy library as malformed.
Where the two numbers come from
A .photoslibrary bundle contains, among much else:
Path<br>What it is
database/Photos.sqlite<br>The library index. Assets, albums, faces, edits: the model behind the Photos app.
resources/cpl/cloudsync.noindex/storage/store.cloudphotodb<br>CloudPhotos' own state. CPL is the sync layer, and this is its work list, kept independently of the index.
Two tables matter.
ZINTERNALRESOURCE in the index has one row per file. An asset owns several, since a RAW+JPEG<br>pair, an edited rendition and the original are separate resources. It carries a<br>ZREMOTEAVAILABILITY column.
outgoingResources in the sync store is the upload queue: one row per file CloudPhotos still has<br>to send, with fileSize, resourceType and a fingerPrint. That fingerprint also appears as<br>ZINTERNALRESOURCE.ZFINGERPRINT, which is what makes the two databases comparable.
The measurement
One 2.06 TB library, mid migration, iCloud Photos enabled and actively uploading.
What the index claims:
ZREMOTEAVAILABILITY = 1 247,964 files 1,869.8 GB<br>ZREMOTEAVAILABILITY = 0 49,902 files 306.0 GB
Read at face value, 306 GB left to go.
What the sync engine's own queue says:
32,441 files 680.9 GB (20,742 distinct assets)
The same library, at the same moment, disagreeing with itself by 375 GB about how much work<br>remains.
Joining the two on fingerprint labels each queued file with what the index claims about it:
still queued for upload, by ZREMOTEAVAILABILITY:
flagged 1, "available remotely" 22,037 files 420.6 GB 21,229 assets<br>flagged 0 12,481 files 324.4 GB
Ruling out the boring explanations
"Those are duplicates. An identical file was already uploaded under a different asset."<br>Plausible, since the join is on content fingerprint. Excluded by restricting to fingerprints that<br>appear exactly once in the entire index, so no already uploaded twin can exist:
19,576 files 345.9 GB unique fingerprint, queued for upload, flagged as in iCloud
"Those are thumbnails and derivatives, not your originals." The queue is 99.9% originals:
resourceType 17, RAW 11,706 files 299.4 GB<br>resourceType 1, photo 20,456 files 232.1 GB<br>resourceType 1 fileKind 1, video 250 files 149.2 GB<br>everything else 25 files 0.0 GB
"The flag means something other than the bytes being in iCloud."...