Turns out it's currently the fastest among the three currently
implemented (Cache, browser.storage.session, indexedDB). Possibly
because indexedDB can natively persist structure-cloneable data,
something uBO can now benefit with the work on abstracting away
the limitations of various storages being limited to persist only
text or JSON data.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2969
In uBO, the "cache storage" is used to save resources which can
be safely discarded, though at the cost of having to fetch or
recompute them again.
Extension storage (browser.storage.local) is now always used as
cache storage backend. This has always been the default for
Chromium-based browsers.
For Firefox-based browsers, IndexedDB was used as backend for
cache storage, with fallback to extension storage when using
Firefox in private mode by default.
Extension storage is reliable since it works in all contexts,
though it may not be the most performant one.
To speed-up loading of resources from extension storage, uBO will
now make use of Cache API storage, which will mirror content of
key assets saved to extension storage. Typically loading resources
from Cache API is faster than loading the same resources from
the extension storage.
Only resources which must be loaded in memory as fast as possible
will make use of the Cache API storage layered on top of the
extension storage.
Compiled filter lists and memory snapshot of filtering engines
(aka "selfies") will be mirrored to the Cache API storage, since
these must be loaded into memory as fast as possible, and reloading
filter lists from their compiled counterpart is a common
operation.
This new design makes it now seamless to work in permanent private
mode for Firefox-based browsers, since extension storage now
always contains cache-related assets.
Support for IndexedDB is removed for the time being, except to
support migration of cached assets the first time uBO runs with
the new cache storage design.
In order to easily support all choices of storage, a new serializer
has been introduced, which is capable of serializing/deserializing
structure-cloneable data to/from a JS string.
Because of this new serializer, JS data structures can be stored
directly from their native representation, and deserialized
directly to their native representation from uBO's point of view,
since the serialization occurs (if needed) only at the storage
interface level.
This new serializer simplifies many code paths where data
structures such as Set, Map, TypedArray, RegExp, etc. had to be
converted in a disparate manner to be able to persist them to
extension storage.
The new serializer supports workers and LZ4 compression. These
can be configured through advanced settings.
With this new layered design, it's possible to introduce more
storage layers if measured as beneficial (i.e. maybe
browser.storage.session)
References:
- https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/local
- https://developer.mozilla.org/en-US/docs/Web/API/Cache
- https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
Purging all the lists from cache storage is detrimental to
differential update, and cause filter lists to be updated less
often and consequently to be less up to date then when letting
differential updater do its work.
Possibly related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2136
Also reported internally, steps to reproduce the issue fixed here:
- Open uBO's dashboard through 3-dot > Add-ons > uBO > Settings
- Bring forth "Filter lists" pane
We want the tab to be already opened at next launch
- Quit Firefox for Android
- Launch Firefox for Android
Result:
Very long launch time, lists marked as out of date.
Broadcast channels are more suited to uBO than DOM events to dispatch
notifications to different parts of uBO.
DOM events can only be dispatched to local context, broadcast channels
dispatch to all contexts (i.e. background process, workers, auxiliary
pages) -- this last behavior is better suited to uBO to communicate
internal changes to all potential listeners, not just those in the local
context.
Additionally, broadcasting to content scripts is now done through
tabs.sendMessage() instead of through potentially opened message
ports, this simplifies broadcasting to content scripts, and this
doesn't require to have long-lived message ports in content
scripts.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2896
TODO: Eventually, distinguish between filtering profile increasing
or decreasing so as to avoid flushing caches when increasing
filtering, which should not affect the scriptlets cache.
uBO will now verify that at least one unprocessed network requests
at launch should have been blocked in order to warn users of
unprocessed network requests through the `!` toolbar icon badge.
For example, with default filter lists, there is nothing to block
on `wikipedia.org`, and hence in this case it's not useful to
present the user with the `!` badge.
Therefore uBO will not show the badge *only* when at least one
unprocessed network requests should have been blocked had uBO been
ready when it was fired by the browser.
Related commit:
- https://github.com/gorhill/uBlock/commit/769b8da664be
Related discussion:
- https://github.com/uBlockOrigin/uAssets/discussions/16939
Until uBO's filtering engines are properly initialized, there will
be a distinct toolbar icon to help users understand that uBO may
not be fully initialized when a webpage is loaded -- often the
cause of improper filtering of trackers/ads at browser launch.
Related discussion:
- a0a9497b4a (commitcomment-62560291)
The new setting, when disabled (enabled by default), allows a user
to prevent uBO from waiting for all filter lists to be loaded
before allowing network activity at launch. The setting is enabled
by default, meaning uBO waits for all filter lists to be loaded in
memory before unsuspending network activity. Some users may find
this behavior undesirable, hence the new setting.
This gives the option to potentially speed up page load at launch,
at the cost of potentially not properly filtering network requests
as per filter lists/rules.
For platforms not supporting the suspension of network activity,
the setting will merely prevent whatever mechanism exists on the
platform to mitigate improper filtering of network requests at
launch. For example, in Chromium-based browsers, unchecking the
new setting will prevent the browser from re-loading tabs for
which there was network activity while in "suspended" state at
launch.
This commit will force-reload active tabs at launch for
environments not supporting suspend network request listeners,
or configured to not suspend network request listeners.
Name: modifyWebextFlavor
Value: A list of space-separated tokens to be added/removed from the
computed default webext flavor.
The primary purpose is to give filter list authors the ability to
test mobile flavor on desktop computers. Though mobile versions of
web pages can be emulated using browser dev tools, it's not
possible to do so for uBO itself.
By using `+mobile` as a value for this setting will force uBO
to act as if it's being executed on a mobile device.
Important: this setting is best used in a dedicated browser
profile, as this affects how filter lists are compiled. So best
to set it in a new browser profile, then force all filter lists
to be recompiled, and use the profile in the future when there
is a need to test the specific webext flavor.
The code exported to nodejs package was revised to use modern
JavaScript syntax. A few issues were fixed at the same time.
The exported classes are:
- DynamicHostRuleFiltering
- DynamicURLRuleFiltering
- DynamicSwitchRuleFiltering
These related to the content the of "My rules" pane in the
uBlock Origin extension.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
This change allows to add the redirect engine into the
nodejs package. The purpose of the redirect engine is to
resolve a redirect token into a path to a local resource,
to be used by the caller as wished.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
The changes are enough to fulfill the related issue.
A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:
./tools/make-nodejs
This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.
From the root of the package, you can try:
node test
This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.
The test.js file contains code which is typical example
of usage of the package.
Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.
This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.
This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
The title of tabs in uBO is solely to have a better
presentation in the logger -- no other purpose.
This commit simplify keeping track of the titles, from
an active approach by directly querying it from tabs
whenever a change occurs, to a passive approach by
storing it when the title string become available in
some tab event handlers.
