Commit graph

5 commits

Author SHA1 Message Date
Raymond Hill
1cf634cce0
Leave out hash part when extracting patch file name
Related commit:
aeff955667
2023-11-05 08:03:43 -05:00
Raymond Hill
aeff955667
Extract list name from hash portion of ! Diff-Path: 2023-11-03 18:39:14 -04:00
Raymond Hill
abd8577096
Distinguish diff patch not-available from not-yet-available
A not-yet-available won't cause a list to become candidate for
full update.

A not-available will cause the list to be candidate for full
update.
2023-10-31 10:52:03 -04:00
Raymond Hill
2344cbdeca
Harden diff-updater against unexpected errors
The diff-updater worker will terminate upon unexpected
error, in order to avoid a stalled updater.
2023-10-30 13:47:24 -04:00
Raymond Hill
d05ff8ffeb
Add support for diff-patching filter lists
Related discussion:
https://github.com/ameshkov/diffupdates

The benefits of diff-patching filter lists is much shorter update
schedule and significantly less bandwidth consumed.

At the moment, only default filter lists are subject to be
diff-patched.

External filter lists can make their lists diff-patchable by
following the specification link above.

Only filter lists fetched by the auto-updater are candidate for
diff-patching.

Forcing a manual update of the filter lists will prevent the
diff-patcher from kicking in until one or more lists are
auto-updated.

Some back-of-the-envelop calculations regarding the load on free
CDN solutions used by uBO to distribute its own filter lists:

Currently, for each CDN (with lists updating after days):
~560 M req/month, ~78 TB/month

With diff-patching lists on a 6-hour schedule:
~390 M req/month, 1 TB/month

Those estimates were done according to statistics shown by
jsDelivr, which is one of 4 CDNs picked randomly when a list
updates:
https://www.jsdelivr.com/package/gh/uBlockOrigin/uAssetsCDN?tab=stats
2023-10-30 12:30:46 -04:00