Related bugzilla issue:
https://bugzilla.mozilla.org/show_bug.cgi?id=1736575
The issue was fixed months ago.
The removed code block is causing uBOL to be flagged as "including
remotely hosted code".
To be clear, the removed obsolete code block was not related to
executing remote code. The referenced code was in the file itself,
not remote, and this was a workaround for when Firefox was not
supporting injecting script in the `MAIN` world.
The issue was fixed months ago in Firefox, so there is no point for
the workaround.
Related discussion:
https://github.com/uBlockOrigin/uBOL-home/discussions/35#discussioncomment-11157444
New policy setting: `rulesets`
Type: array
Type of array items: string
Each item in the list is a list id (as seen in `rulesets/ruleset-details.json`),
prefixed with either `+` to enable the ruleset, or `-` to disable the ruleset.
Users will not be able to enable or disable rulesets present in the `rulesets`
policy. Disabled rulesets will not appear in the dashboard.
Use `-*` to remove all non-default rulesets, except for those added
using `+[ruleset_id]`.
Additionally, some work has been done to properly handle policy changes in
a non-blocking and deferred manner, as I observed that it often takes long
for calls to `storage.manage.get` to resolve. This potentailly takes care
of the following issue:
https://github.com/uBlockOrigin/uBOL-home/issues/174
Ensure serialization returns copy of data rather than live
references to data. This allows to immediately deserialize() the
result of serialize().
Also, adjust code to modified behavior of filterQuery().
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/3394
When the "No large media elements" per-site switch is toggled on,
it will also act to prevent autoplay of video/audio media, regardless
of their size. This also works for xhr-based media streaming.
If blocking by size is not desirable while blocking autoplay is
desired, one can toggle on "No large media elements" switch while
setting "Block media elements larger than ..." to a very high value.
Default to `true`.
Set to `false` to wholly disable calls to `dns.resolve()` (Firefox-
only). Disabling calls to `dns.resolve()` will prevent cname-uncloaking
and will limit ability to enforce `ipaddress` filter option.
This commit makes the DNS resolution code better suited for both
filtering on cname and ip address. The change allows early availability
of ip address so that `ipaddress=` option can be matched at
onBeforeRequest time.
As a result, it is now possible to block root document using
`ipaddress=` option -- so long as an ip address can be extracted
before first onBeforeRequest() call.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2792
Caveat
------
the ip address used is the first one among the list of ip
addresses returned by dns.resolve() method. There is no way for uBO
to know which exact ip address will be used by the browser when
sending the request, so this is at most a best guess. The exact IP
address used by the browser is available at onHeadersReceived time,
and uBO will also filter according to this value, but by then the
network request has already been sent to the remote server.
Possibly a future improvement would make available the whole list
of ip addresses to the filtering engine, but even then it's impossible
to know with certainty which ip address will ultimately be used by the
browser -- it is entirely possible that the ip address used by the
browser might not be in the list received through dns.resolve().
If an IP address can be extracted from the hostname portion of
a URL, the IP address matching will be performed at onBeforeRequest()
time.
Regardless, IP address matching will subsequently always be performed
at onHeadersReceived() time as the request details at that point
contain a reliable IP address value on supported platforms (Firefox-
only as of now).
The `cap_ipaddress` now evaluates to `true` in Chromium-based
browsers. Even though these browsers are unable to provide reliable
IP address value at onHeadersReceived() time, they can still
perform IP address matching for IP address extracted from hostname
portion of a URL.
A new icon has been added to the popup panel, to open a popup
window with a terse list of DNR events for the current tab, in
reverse chronological order (most recent DNR event appears at
the top).
The new ability is available only when the extension is sideloaded,
as per `declarativeNetRequestFeedback` documentation. Ref:
https://developer.chrome.com/docs/extensions/reference/api/declarativeNetRequest#event-onRuleMatchedDebug
Purposefully minimal, so as to have something rather than nothing
when having to diagnose filtering issue with the DNR API. Example:
https://github.com/uBlockOrigin/uBOL-home/issues/156
The content of the popup window does not dynamically update, force
a refresh (F5) to get the most recent DNR events. This might be
improved in the future.
The DNR event buffer is not persisted, so the buffer is empty when
service worker is restarted. This might be improved in the future
by using session storage API.
There is no output filtering ability in this first draft. This
might be improved in the future.
DNR rules are reported. The filter from which a DNR rule
originates is not reported. Given that the rulesets are optimized
after conversion from original filter lists to reduce the DNR rule
count, this is unlikely to ever be possible.
For internal use by filter list maintainers, do not open issues
about this. Left undocumented on purpose.
This new procedural operator allows to target elements in the
shadow root of an element.
subject:shadow(arg)
- Description: Look-up matching elements inside the shadow root (if
present) of _subject_.
- Chainable: Yes
- _subject_: Can be a plain or procedural selector.
- _arg_: A plain or a procedural selector for the elements to target
inside the shadowroot.
Example:
..##body > div:not([class]):shadow(div[style]):has(:shadow([data-i18n^="#ad"]))
In uBO, the "cache storage" is used to save resources which can
be safely discarded, though at the cost of having to fetch or
recompute them again.
Extension storage (browser.storage.local) is now always used as
cache storage backend. This has always been the default for
Chromium-based browsers.
For Firefox-based browsers, IndexedDB was used as backend for
cache storage, with fallback to extension storage when using
Firefox in private mode by default.
Extension storage is reliable since it works in all contexts,
though it may not be the most performant one.
To speed-up loading of resources from extension storage, uBO will
now make use of Cache API storage, which will mirror content of
key assets saved to extension storage. Typically loading resources
from Cache API is faster than loading the same resources from
the extension storage.
Only resources which must be loaded in memory as fast as possible
will make use of the Cache API storage layered on top of the
extension storage.
Compiled filter lists and memory snapshot of filtering engines
(aka "selfies") will be mirrored to the Cache API storage, since
these must be loaded into memory as fast as possible, and reloading
filter lists from their compiled counterpart is a common
operation.
This new design makes it now seamless to work in permanent private
mode for Firefox-based browsers, since extension storage now
always contains cache-related assets.
Support for IndexedDB is removed for the time being, except to
support migration of cached assets the first time uBO runs with
the new cache storage design.
In order to easily support all choices of storage, a new serializer
has been introduced, which is capable of serializing/deserializing
structure-cloneable data to/from a JS string.
Because of this new serializer, JS data structures can be stored
directly from their native representation, and deserialized
directly to their native representation from uBO's point of view,
since the serialization occurs (if needed) only at the storage
interface level.
This new serializer simplifies many code paths where data
structures such as Set, Map, TypedArray, RegExp, etc. had to be
converted in a disparate manner to be able to persist them to
extension storage.
The new serializer supports workers and LZ4 compression. These
can be configured through advanced settings.
With this new layered design, it's possible to introduce more
storage layers if measured as beneficial (i.e. maybe
browser.storage.session)
References:
- https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/local
- https://developer.mozilla.org/en-US/docs/Web/API/Cache
- https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
This is to reduce the diff size of rulesets in new
releases. Beside smaller diff size, this also makes it
easier to investigate rule changes across releases.
This commit brings the following changes to the logger:
All logging output generated by injected scriptlets are now sent to
the logger, the developer console will no longer be used to log
scriptlet logging information.
When the logger is not opened, the scriplets will not output any
logging information.
The goal with this new approach is to allow filter authors to
more easily assess the working of scriptlets without having to
go through scriptlet parameters to enable logging.
Consequently all the previous ways to tell scriptlets to log
information are now obsolete: if the logger is opened, the
scriptlets will log information to the logger.
Another benefit of this approach is that the dev tools do not
need to be open to obtain scriptlets logging information.
Accordingly, new filter expressions have been added to the logger:
"info" and "error". Selecting the "scriptlet" expression will also
keep the logging information from scriptlets.
A new button has been added to the logger (not yet i18n-ed): a
"volume" icon, which allows to enable verbose mode. When verbose
mode is enabled, the scriptlets may choose to output more
information regarding their inner working.
The entries in the logger will automatically expand on mouse hover.
This allows to scroll through entries which text does not fit into
a single row.
Clicking anywhere on an entry in the logger will open the detailed
view when applicable.
Generic information/errors will now be rendered regardless of which
tab is currently selected in the logger (similar to how tabless
entries are already being rendered).
Procedural filters with `:xpath` operator were silently rejected
at conversion time because the parser was failing to evaluate the
xpath expression due to the absence of a `document` object in
nodejs.
If `document` object is not present, the parser will assume the
xpath expression is valid.
The idea is to remove as many dependencies as possible for
low-level ScriptletFilteringEngine in order to make it easier
to reuse the module outside uBO itself.
The high-level derived class takes care of caching and
injection of scriptlets into documents, which requires
more knowledge about the environment in which scriptlets
are to be used.
Also improve scriptlet cache usage to minimize overhead of
retrieving scriptlets.
Related issue:
https://github.com/uBlockOrigin/uBlock-issues/issues/2969
Changes:
Use browser.alarms to trigger selfie creation. Presence of a selfie
improve markedly time to readiness when uBO is unsuspended.
Mirror content of storage.local to (in-memory) storage.session for
faster load to readiness when uBO is ususpended.
Broadcast channels are more suited to uBO than DOM events to dispatch
notifications to different parts of uBO.
DOM events can only be dispatched to local context, broadcast channels
dispatch to all contexts (i.e. background process, workers, auxiliary
pages) -- this last behavior is better suited to uBO to communicate
internal changes to all potential listeners, not just those in the local
context.
Additionally, broadcasting to content scripts is now done through
tabs.sendMessage() instead of through potentially opened message
ports, this simplifies broadcasting to content scripts, and this
doesn't require to have long-lived message ports in content
scripts.
When the target world of a scriptlet is the ISOLATED one,
skip Blob-based injection in Firefox, as the current world
is always the ISOLATED one. This should make ISOLATED
world-based scriptlets more reliable (i.e. execute sooner)
in Firefox.
The `urltransform` option allows to redirect a non-blocked network
request to another URL. There are restrictions on its usage:
- require a trusted source -- thus uBO-maintained lists or user
filters
- the `urltransform` value must start with a `/`
If at least one of these conditions is not fulfilled, the filter
will be invalid and rejected.
The requirement to start with `/` is to enforce that only the path
part of a URL can be modified, thus ensuring the network request
is redirected to the same scheme and authority (as defined at
https://en.wikipedia.org/wiki/Uniform_Resource_Identifier#Syntax).
Usage example (redirect requests for CSS resources to a non-existing
resource, for demonstration purpose):
||iana.org^$css,urltransform=/notfound.css
Name of this option is inspired from DNR API:
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest/URLTransform
This commit required to bring the concept of "trusted source" to
the static network filtering engine.
As per discussion with uBO volunteers.
Volunteers offering support for uBO will be able to craft links with
specially formed URLs, which once clicked will cause uBO to automatically
force an update of specified filter lists.
The URL must be crafted as shown in the example below:
https://ublockorigin.github.io/uAssets/update-lists.html?listkeys=ublock-filters,easylist
Where the `listkeys` parameter is a comma-separated list of tokens
corresponding to filter lists. If a token does not match an enabled
filter list, it will be ignored.
The ability to update filter lists through a specially crafted link
is available only on uBO's own support sites:
- https://github.com/uBlockOrigin/
- https://reddit.com/r/uBlockOrigin/
- https://ublockorigin.github.io/
Additionally, a visual cue has been added in the "Filter lists" pane
to easily spot the filter lists which have been recently updated, where
"recently" is currently defined as less than an hour ago.
Additionally, finalize versioning scheme for uBOL. Since most updates
will be simply related to update rulesets, the version will from now
on reflects the date at which the extension package was created:
year.month.day.minutes
So for example:
2023.8.19.690
