This will make HTML filtering and `replace=` filter option less
likely to be bypassed by uBO, as the body response filterer
previously required an encoding to be expressly declared before
acting on the response body.
UTF-8 usage is currently reported as ~98.2%:
https://w3techs.com/technologies/history_overview/character_encoding
Reference documentation:
https://adguard.com/kb/general/ad-filtering/create-own-filters/#replace-modifier
This is a network filter option which can only be loaded from a
trusted source.
Since this filter is about modifying the response body, it currently
only works in Firefox.
As discussed with filter list maintainers.
Related discussion:
- a0a9497b4a (commitcomment-62560291)
The new setting, when disabled (enabled by default), allows a user
to prevent uBO from waiting for all filter lists to be loaded
before allowing network activity at launch. The setting is enabled
by default, meaning uBO waits for all filter lists to be loaded in
memory before unsuspending network activity. Some users may find
this behavior undesirable, hence the new setting.
This gives the option to potentially speed up page load at launch,
at the cost of potentially not properly filtering network requests
as per filter lists/rules.
For platforms not supporting the suspension of network activity,
the setting will merely prevent whatever mechanism exists on the
platform to mitigate improper filtering of network requests at
launch. For example, in Chromium-based browsers, unchecking the
new setting will prevent the browser from re-loading tabs for
which there was network activity while in "suspended" state at
launch.
This commit will force-reload active tabs at launch for
environments not supporting suspend network request listeners,
or configured to not suspend network request listeners.
The original motivation is to further speed up launch time
for either non-selfie-based and selfie-based initialization
of the static network filtering engine (SNFE).
As a result of the refactoring:
Filters are no longer instance-based, they are sequence-of-
integer-based. This eliminates the need to create instances
of filters at launch, and consequently eliminates all the
calls to class constructors, the resulting churning of memory,
and so forth.
All the properties defining filter instances are now as much
as possible 32-bit integer-based, and these are allocated in a
single module-scoped typed array -- this eliminates the need
to allocate memory for every filter being instantiated.
Not all filter properties can be represented as a 32-bit
integer, and in this case a filter class can allocate slots
into another module-scoped array of references.
As a result, this eliminates a lot of memory allocations when
the SNFE is populated with filters, and this makes the saving
and loading of selfie more straightforward, as the operation
is reduced to saving/loading two arrays, one of 32-bit
integers, and the other, much smaller, an array JSON-able
values.
All filter classes now only contain static methods, and all
of these methods are called with an index to the specific
filter data in the module-scoped array of 32-bit integers.
The filter sequences (used to avoid the use of JS arrays) are
also allocated in the single module-scoped array of 32-bit
integers -- they used to be stored in their own dedicated
array.
Additionally, some filters are now loaded more in a deferred
way, so as reduce uBO's time-to-readiness -- the outcome of
this still needs to be evaluated, time-to-readiness is
especially a concern in Firefox for Android or less powerful
computers.
The code exported to nodejs package was revised to use modern
JavaScript syntax. A few issues were fixed at the same time.
The exported classes are:
- DynamicHostRuleFiltering
- DynamicURLRuleFiltering
- DynamicSwitchRuleFiltering
These related to the content the of "My rules" pane in the
uBlock Origin extension.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
This change allows to add the redirect engine into the
nodejs package. The purpose of the redirect engine is to
resolve a redirect token into a path to a local resource,
to be used by the caller as wished.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
The changes are enough to fulfill the related issue.
A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:
./tools/make-nodejs
This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.
From the root of the package, you can try:
node test
This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.
The test.js file contains code which is typical example
of usage of the package.
Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.
This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.
This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1553
This commit ensures FLoC is opt-in. The generic filter
`*##+js(no-floc)` in "uBlock filters -- Privacy" ensures
the feature is disabled when using default settings/lists.
Users can opt-in to FLoC by adding a generic exception
filter to their custom filters, `#@#+js(no-floc)`; or they
can opt-in only for a specific set of websites through a
more specific exception filter:
example.com,shopping.example#@#+js(no-floc)
The syntax to remove response header is a special case
of HTML filtering, whereas the response headers are
targeted, rather than the response body:
example.com##^responseheader(header-name)
Where `header-name` is the name of the header to
remove, and must always be lowercase.
The removal of response headers can only be applied to
document resources, i.e. main- or sub-frames.
Only a limited set of headers can be targeted for
removal:
location
refresh
report-to
set-cookie
This limitation is to ensure that uBO never lowers the
security profile of web pages, i.e. we wouldn't want to
remove `content-security-policy`.
Given that the header removal occurs at onHeaderReceived
time, this new ability works for all browsers.
The motivation for this new filtering ability is instance
of website using a `refresh` header to redirect a visitor
to an undesirable destination after a few seconds.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1501
Exception filters for `document` option are complying with
uBO's own semantic for `document` option, i.e. an exception
filter for `document` option will only allow to bypass a
block filter for `document` (either explicit or implicit)
and nothing else.
Exception filters using `document` option are *not*
compatible with ABP's interpretation of these filters.
Whereas in ABP the purpose of a `document` exception filter
is to wholly disable content blocking, in uBO the same
filter will just cause strict-blocking to be disabled while
leaving content blocking intact.
Additionally, the logger was fixed to properly report pages
which are being strict-blocked.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/210
Additionally, a small (experimental) widget has been added
to emphasize/de-emphasize rows which have 3rd-party
scripts/frames, so as to more easily identify which rows
are "affected" by 3rd-party scripts and/or frames.
Tooltip localization for the new widget is not available
yet as I want wait for the feature to be fully settled.
Content scripts can't properly look up effective context
for sandboxed frames. This commit add ability to extract
effective context from already existing store of frames
used for each tab.
