Default to `true`.
Set to `false` to wholly disable calls to `dns.resolve()` (Firefox-
only). Disabling calls to `dns.resolve()` will prevent cname-uncloaking
and will limit ability to enforce `ipaddress` filter option.
This commit makes the DNS resolution code better suited for both
filtering on cname and ip address. The change allows early availability
of ip address so that `ipaddress=` option can be matched at
onBeforeRequest time.
As a result, it is now possible to block root document using
`ipaddress=` option -- so long as an ip address can be extracted
before first onBeforeRequest() call.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2792
Caveat
------
the ip address used is the first one among the list of ip
addresses returned by dns.resolve() method. There is no way for uBO
to know which exact ip address will be used by the browser when
sending the request, so this is at most a best guess. The exact IP
address used by the browser is available at onHeadersReceived time,
and uBO will also filter according to this value, but by then the
network request has already been sent to the remote server.
Possibly a future improvement would make available the whole list
of ip addresses to the filtering engine, but even then it's impossible
to know with certainty which ip address will ultimately be used by the
browser -- it is entirely possible that the ip address used by the
browser might not be in the list received through dns.resolve().
If an IP address can be extracted from the hostname portion of
a URL, the IP address matching will be performed at onBeforeRequest()
time.
Regardless, IP address matching will subsequently always be performed
at onHeadersReceived() time as the request details at that point
contain a reliable IP address value on supported platforms (Firefox-
only as of now).
The `cap_ipaddress` now evaluates to `true` in Chromium-based
browsers. Even though these browsers are unable to provide reliable
IP address value at onHeadersReceived() time, they can still
perform IP address matching for IP address extracted from hostname
portion of a URL.
The purpose is to block according to the ip address of a network
request. In the current implementation, the filter option can only
be enforced at onHeadersReceived time.
The new filter option cannot be enforced in Chromium-based browsers
since the ip address of network requests is available only at
onResponseStarted time, which is not blocking.
The value assigned to `ipaddress` can either be a plain string which
must match exactly a given ip address, or a regex which will be
matched against the ip address.
The `ipaddress` option can only be enforced when the extension
framework does provide a valid ip address in a onHeadersReceived
listener. For instance, cached resources do not have a valid ip
address and thus can't be a match to `ipaddress` option.
Example:
*$script,ipaddress=93.184.215.14
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/3161
- https://github.com/uBlockOrigin/uBlock-issues/discussions/2895#discussioncomment-8504374
Two checkboxes have been added to the "My filters "pane:
1. A checkbox to wholly disable/enable "My filters". This is equivalent
to the checkbox for "My filters" in "Filter lists" pane.
2. A checkbox to enable/disable the trustworthiness of the content
of "My filters". Default to untrusted.
Since toggling these checkboxes requires reloading all filter lists,
their new state must be committed through the "Apply changes" button.
Additionally: a "book" icon has been added to the top-right of the
dashboard, which is a link to the wiki according to whichever pane is
currently active.
To disable collating global blocked/allowed counts.
Boolean, default to `false`.
Setting to `true` will prevent uBO from loading/saving global
blocked/allowed counts, and in such case the "Blocked since
install" count instead reflects the count since uBO launched.
Setting back to `false` will cause the counts to resume from
last time they were saved.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/3100
In uBO, the "cache storage" is used to save resources which can
be safely discarded, though at the cost of having to fetch or
recompute them again.
Extension storage (browser.storage.local) is now always used as
cache storage backend. This has always been the default for
Chromium-based browsers.
For Firefox-based browsers, IndexedDB was used as backend for
cache storage, with fallback to extension storage when using
Firefox in private mode by default.
Extension storage is reliable since it works in all contexts,
though it may not be the most performant one.
To speed-up loading of resources from extension storage, uBO will
now make use of Cache API storage, which will mirror content of
key assets saved to extension storage. Typically loading resources
from Cache API is faster than loading the same resources from
the extension storage.
Only resources which must be loaded in memory as fast as possible
will make use of the Cache API storage layered on top of the
extension storage.
Compiled filter lists and memory snapshot of filtering engines
(aka "selfies") will be mirrored to the Cache API storage, since
these must be loaded into memory as fast as possible, and reloading
filter lists from their compiled counterpart is a common
operation.
This new design makes it now seamless to work in permanent private
mode for Firefox-based browsers, since extension storage now
always contains cache-related assets.
Support for IndexedDB is removed for the time being, except to
support migration of cached assets the first time uBO runs with
the new cache storage design.
In order to easily support all choices of storage, a new serializer
has been introduced, which is capable of serializing/deserializing
structure-cloneable data to/from a JS string.
Because of this new serializer, JS data structures can be stored
directly from their native representation, and deserialized
directly to their native representation from uBO's point of view,
since the serialization occurs (if needed) only at the storage
interface level.
This new serializer simplifies many code paths where data
structures such as Set, Map, TypedArray, RegExp, etc. had to be
converted in a disparate manner to be able to persist them to
extension storage.
The new serializer supports workers and LZ4 compression. These
can be configured through advanced settings.
With this new layered design, it's possible to introduce more
storage layers if measured as beneficial (i.e. maybe
browser.storage.session)
References:
- https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/local
- https://developer.mozilla.org/en-US/docs/Web/API/Cache
- https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
This commit brings the following changes to the logger:
All logging output generated by injected scriptlets are now sent to
the logger, the developer console will no longer be used to log
scriptlet logging information.
When the logger is not opened, the scriplets will not output any
logging information.
The goal with this new approach is to allow filter authors to
more easily assess the working of scriptlets without having to
go through scriptlet parameters to enable logging.
Consequently all the previous ways to tell scriptlets to log
information are now obsolete: if the logger is opened, the
scriptlets will log information to the logger.
Another benefit of this approach is that the dev tools do not
need to be open to obtain scriptlets logging information.
Accordingly, new filter expressions have been added to the logger:
"info" and "error". Selecting the "scriptlet" expression will also
keep the logging information from scriptlets.
A new button has been added to the logger (not yet i18n-ed): a
"volume" icon, which allows to enable verbose mode. When verbose
mode is enabled, the scriptlets may choose to output more
information regarding their inner working.
The entries in the logger will automatically expand on mouse hover.
This allows to scroll through entries which text does not fit into
a single row.
Clicking anywhere on an entry in the logger will open the detailed
view when applicable.
Generic information/errors will now be rendered regardless of which
tab is currently selected in the logger (similar to how tabless
entries are already being rendered).
Possibly related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2136
Also reported internally, steps to reproduce the issue fixed here:
- Open uBO's dashboard through 3-dot > Add-ons > uBO > Settings
- Bring forth "Filter lists" pane
We want the tab to be already opened at next launch
- Quit Firefox for Android
- Launch Firefox for Android
Result:
Very long launch time, lists marked as out of date.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2969
Changes:
Use browser.alarms to trigger selfie creation. Presence of a selfie
improve markedly time to readiness when uBO is unsuspended.
Mirror content of storage.local to (in-memory) storage.session for
faster load to readiness when uBO is ususpended.
Reference documentation:
https://adguard.com/kb/general/ad-filtering/create-own-filters/#replace-modifier
This is a network filter option which can only be loaded from a
trusted source.
Since this filter is about modifying the response body, it currently
only works in Firefox.
As discussed with filter list maintainers.
Manual update of one or more lists will cause the most recent version
of these lists to be fetched from the "origin" server, and since the
lists from "origin" servers cannot be updated through differential
update, the lists will be subsequently updated according to their
`Expires` directive.
When the lists are auto-updated, the "CDN" servers will be used,
and as a result the lists will start to be updated trhough
differential updates every 6-hour (currently).
Thus it is recommended and optimal to let the lists auto-update,
since you will benefit from a much shorter delay to get up-to-date
lists (i.e. every 6-hour instead of every 6-day).
You can force the auto-updater to fetch all the lists by clicking
"Purge all caches", then restart uBO without clicking "Update".
This will cause uBO to perform an emergency auto-update at restart
time, after which you will have all the lists which are candidates
for differential update.
The "Update now" button in the "Support" pane will also cause lists
to be fetched from their "origin" server.
Related discussion:
https://github.com/uBlockOrigin/uBlock-issues/discussions/2895
Changes:
The _content of the My filters_ pane is now considered untrusted by
default, and only uBO's own lists are now trusted by default.
It has been observed that too many people will readily copy-paste
filters from random sources. Copy-pasting filters which require trust
represents a security risk to users with no understanding of how the
filters work and their potential abuse.
Using a filter which requires trust in a filter list from an untrusted
source will cause the filter to be invalid, i.e. shown as an error.
A new advanced setting has been added to control which lists are
considered trustworthy: `trustedListPrefixes`, which is a space-
separated list of tokens. Examples of possible values:
- `ublock-`: trust only uBO lists, exclude everything else including
content of _My filters_ (default value)
- `ublock- user-`: trust uBO lists and content of _My filters_
- `-`: trust no list, essentially disabling all filters requiring
trust (admins or people who don't trust us may want to use this)
One can also decide to trust lists maintained elsewhere. For example,
for stock AdGuard lists add ` adguard-`. To trust stock EasyList lists,
add ` easylist-`.
To trust a specific regional stock list, look-up its token in
assets.json and add to `trustedListPrefixes`.
The matching is made with String.startsWith(), hence why `ublock-`
matches all uBO's own filter lists.
This also allows to trust imported lists, for example add
` https://filters.adtidy.org/extension/ublock/filters/` to trust all
non-stock AdGuard lists.
Add the complete URL of a given imported list to trust only that one
list.
URLs not starting with `https://` or `file:///` will be rejected,
i.e. `http://example.org` will be ignored.
Invalid URLs are rejected.
Related issue:
- https://github.com/AdguardTeam/Scriptlets/issues/332
Additionally, uBO's own scriplet syntax now also accept quoting
the parameters with either `'` or `"`. This can be used to avoid
having to escape commas when they are present in a parameter.
Caused by the fact that external filter lists do not have an
`off` property even when they are not enabled.
Additionally, set the default update cycle check period to 2hr.
This commit fix properly handling toggling off the default
status of a list such that the list will be automatically
turned off when its status change from default to non-default.
Additionally, imported lists which become stock lists will
be properly migrated from imported lists section.
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/discussions/2234
Example of usage:
@@*$ghide,domain=/img[a-z]{3,5}\.buzz/
Regex-based domain values can be negated just like plain or
entity-based values:
*$domain=~/regex.../
This new syntax does not apply to static extended filters.
This commit is a rewrite of the static filtering parser into a
tree-based data structure, for easier maintenance and better
abstraction of parsed filters.
This simplifies greatly syntax coloring of filters and also
simplify extending filter syntax.
The minimum version of Chromium-based browsers has been raised
to version 73 because of usage of String.matchAll().
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/discussions/2412#discussioncomment-4421741
The new option is `to=` and the value is a list of domain list with
similar syntax as `domain=` option. Entity-based syntax is supported,
and also negated hostname.
The main motivation is to give uBO's static network filtering engine
with an equivalent of DNR's `requestDomains` and `excludedRequestDomains`.
Essentially `to=` is a superset of `denyallow=`, but for now I decided
against deprecating `denyallow=`, which still does not support entity-
based syntax and for which negated domains are not allowed.
This commit also introduces the `from=` option, which is just an alias
for the `domain=` option. The logger will render network filters using
the `from=` version.
