You need to start by separating the concept of updating from the concept of repairing.
The algorithm is roughly the following:
- Get the claimed version of the client.
- Ask the server for that version.
- The server returns the checksums for that version, and whatever or not there is a new version.
- Check the integrity of the files with the provided checksums.
Now, you have either a good or a corrupted copy. In addition, there may or may not be a newer version. Depending on those two variables, you pick:
- Good Copy, No new version: Nothing to do.
- Good Copy, New version: Patch update.
- Corrupted Copy, No new version: Repair.
- Corrupted Copy, New Version: Potential Full update.
Note: you can do this decision file per file.
Let us see how each one works:
Nothing to do.
You need to have precomputed, on the server, the patches from the old versions to the current one (e.g. with bsdiff, xdelta, etc...). Since this is on the server, both the last and the old version are available for comparison, so you can create the patch files. You would keep them around because a client may fall behind more than one update.
It works like this:
- Ask the server for the patch notes starting from the client version.
- The server looks for the patch files...
- If there are not patch files, return failure and fallback to Full update.
- If the patch files are available, return the list of patch files from the requested version to the current one.
- The client then downloads and applies the patches in order.
- Check version and integrity again at the end of the process.
To find the corruption, you can split the file in pieces of fixed size (e.g 4MB) and compute checksum corresponding to each piece. In the server, you would have detailed data about the files; this includes the length of the file and the checksums for the pieces.
This is the process:
- For the file that failed checksum, ask the server for detailed data.
- The server returns the size of the file and the checksums of the pieces.
- The client compares the size, if it does not match then truncate (or grow) the file to the right length.
- Compute the checksum of each piece of the file and compare it with the one you got from the server.
- If the checksum does not match, download that piece from the server and update it in the client.
Potential Full Update
You will run the full update (download the full file and replace it) for a file that is missing, or for a file for which there are no patch files.
However, if you have a corrupted file, you need to decide if you want to repair it and the patch update or you want to download the full file and replace it. So, consider computing the piece checksums as part of the process that computes the full checksum of the initial check^... this way you will have them available already for the repair process.
^: For instance, the global checksum of the file can be the hash of the checksum of the pieces. Do not use partial checksums, doing that will make the checksums not match after the first corruption is found, and we don't want that.
This optimization will allow you to see how corrupted is the file before deciding if you want to repair it.
Now your criteria need to be based on:
- How corrupted is the file
- How much have the file changed (how big are the patches)
Compute how much do you have to download to fix the damaged pieces plus the patches, it is accounts for more than the full size of the file... download the full file instead. Otherwise, do a repair and then a patch update.
You may want to add rules to avoid repairing and patching certain files. For example, do not bother repairing or patching files that are too small. In addition, you want to black list executable files, because many antivirus will not be happy with your updater editing executables (besides, executables should be small; the bulk should be in the assets).