Download to a partial filename and rename on checksum success

[yashi]

Currently download_thread() (via get_binary()) writes the artifact directly to bundle_download_location using g_fopen(file, "wb+") or "ab+" for resume. This makes it hard to reliably detect "download completed and verified" from outside the process.

In my case, I want to trigger an action after the updater has downloaded a bundle. There is no dedicated download-complete hook, so I tried using systemd.path. But because the final filename exists while the download is still in progress (and can be resumed later), and because checksum verification may fail after the download finishes, systemd.path cannot reliably tell when the file is actually ready.

Proposal

Add support for downloading to an in-progress filename (for example ${bundle_download_location}.part) and only renaming it to
${bundle_download_location} after the checksum has been verified successfully.

Suggested behavior

1. Download and resume use the partial file path (stat and append the .part file).
2. Compute checksum from the partial file.
3. If checksum OK, rename .part to the final path (atomic rename on the same filesystem).
4. If checksum fails, either keep the .part file for debugging, or delete it, but do not rename.

This would allow external watchers to safely use systemd.path on the final filename, because the final filename would only appear once the bundle is complete and verified.

References

• systemd path units:
  https://www.freedesktop.org/software/systemd/man/systemd.path.html
• POSIX rename semantics:
  https://pubs.opengroup.org/onlinepubs/000095399/functions/rename.html
• Linux rename(2):
  https://man7.org/linux/man-pages/man2/rename.2.html
• GLib g_rename():
  https://docs.gtk.org

We are happy to implement this feature if the overall design is OK.

[Bastian-Krause]

Acting on some file appearing in the file system does not sound like a good approach to me. Whatever it is you're trying to do with it should probably happen after the download is complete and verified and before installation/removal happens.

Related: #160 (review)

If you use the downloaded RAUC bundle outside of rauc-hawkbit-updater, how can you be sure that it the download is complete and that it won't be replaced by another download while you're using it?

Have you thought about using RAUC's pre-install handler?

[yashi]

Have you thought about using RAUC's pre-install handler?

Yes. IIRC, it would not work for a download-only use case, right? Would it make sense to have a post-download hook?

[Bastian-Krause]

So RAUC is not used at all in your use case? Could you explain what you're trying to achieve with this?

[yashi]

We use RAUC and hawkBit for OTA updates. However, unlike the more typical RAUC A/B setup, we use a Golden/Main partition layout. In this scheme, the Main partition is the system that normally runs. It checks for available updates and downloads the update bundle. Because RAUC cannot update the currently active partition, we switch to the Golden partition to update Main. Once Golden is active, it installs the downloaded bundle to the Main partition and then switches back.

ref) spacecubics/meta-scobc#18