- Make place for more dynamic regex-based rules when there is a
risk session regex-based rules could interfere
- Do not prune `allow` strict-block rules as they do not contribute
toward the overall regex-based rule count
Possibly related issue:
https://github.com/uBlockOrigin/uBOL-home/issues/556
Since permissions are now managed by the browsers, the browser will take
care whether to enforce those "usafe" rules according to the permissions
in effect on a given site.
As discussed internally.
The custom DNR rule count will be reported only when it's not zero,
and the count is only for effective DNR rules, i.e. it will not be
reported if "Developer mode" is not enabled.
This feature is hidden behind the "Developer mode" setting in
the dashboard. When "Developer mode" is enabled, a tab named
"Develop" will become available in the dashboard. This tab is
meant to contain tools for technical users.
At the moment, the "Develop" pane allows to create custom DNR
rules through a (CodeMirror-based) editor.
For the sake of convenience, the DNR rule must be entered in
YAML-like format. The format is not really full compliant YAML,
just YAML-like, and very strict in order to ensure the parser
stays simple enough.
Lines starting with `#` are comments and will be ignored by the
parser.
Any line which do not match the parser's expectation will be
marked as invalid, and the whole DNR rule containing such invalid
lines will be discarded.
There must not be empty lines inside a rule definition.
Each DNR rule must be separated with a `---` line, which is
known as a YAML document separator.
String values must not be quoted, otherwise the quotes will be
considered part of the value. There is one exception: `''` will
be parsed as "an empty string".
The editor will attempt to auto-complete known DNR keywords. That
feature will improve over time.
Though the parser will identify some errors, not all invalid DNR
rules are currently identified by the parser, and these will be
reported when the rules are registered through the DNR API. Better
identifying invalid DNR rules at edit time will improve over time.
The editor will report `regexFilter` values which are not
supported by the DNR engine on the current platform.
The editor reacts to instances of `regexFilter: ...` to report
whether a regex value is supported. This means you can test for
a regex value by using `# regexFilter: ...` so that you do not
have to create an actual DNR rules just for the sake of testing.
Custom DNR rules can be exported into a JSON file (a format
known by the DNR API as a "static ruleset").
JSON-based ruleset can be imported, the content will be converted
to YAML-like syntax.
The editor will attempt to convert to YAML pasted content which
can be JSON-parsed. It's possible to paste partially or wholly
JSON-based rulesets.
When disabling "Developer mode", all custom DNR rules will be
unregistered from the DNR API. The DNR rules content will be left
intact in such case. Existing DNR rules will be registered into
the DNR API when re-enabling "Developer mode".
Administrators can prevent "Developer mode" from being enabled
by adding `develop` token to `disabledFeatures` setting.
Related discussion:
https://github.com/uBlockOrigin/uBOL-home/discussions/323
The main motivation is to give list maintainers a tool to assist
with resolving filter issues. Custom DNR rules can assist in
crafting and validating filters meant to work with uBOL.
A secondary motivation is to provide technical users the ability
to further customize their content blocker.
More conveniences will be added over time, this is a first version.
Related issue:
https://github.com/uBlockOrigin/uBOL-home/issues/326
uBOL now asks broad host permissions by default. Users can still
choose narrow host permissions by using their browser's controls
for this.
For instance in Chromium, those host permissions controls are found
in the "Site access" section in the detailed view of an extension.
One can set "Site access" to "On click" to revoke broad host
permissions, and grant host permissions to only specific site.
In such mode, uBOL will still block through the DNR API, but no
cosmetic or scriptlet filtering will occurs, as these requires
permission to "read and change data" on websites for which higher
filtering mode is desired.
Some browsers do not automatically grant broad host permissions
even when an extension asks for broad permissions at install time,
and going forward all browsers will likely adopt this approach, and
thus it no longer made sense for uBOL to default to no broad hosts
permissions at install time, especially given this leads to issues
with no solution -- issues solved with the new approach (e.g. like
the ability to deploy uBOL in Optimal mode by default).
A bad test prevented the "no filtering" by default mode to not take
effect immediately when activated while no site had yet been set to
"no filtering". It would however take effect as soon as a specific
site would be excluded from "no filtering".
This extends coverage of strict-blocking to pattern-based filters
with `doc` filter option.
When proceeding with a URL-skip URL present, no temporary bypass
will be created when the "Don't warn me again about this site" is
left unchecked. The idea is to avoid the intermediate redirects
if we navigate again on the same strict-blocked site, while a
temporary bypass would prevent this.
uBO's "Badware risks" list has been spinned off as its own list.
The idea is that should a site be strict-blocked from that list,
we would want to know the strict-block is due to the
"Badware risks" list.
Add information about which ruleset caused a page to be strict-
blocked.
Whenever possible, add ability to URL-skip an incoming redirect
in a strict-blocked page.
Add new default list: "30-day Phishing Domain List"
Related issue:
https://github.com/uBlockOrigin/uBOL-home/issues/214
This implements basic functionality for strict blocking, i.e. the
ability to block navigation to undesirable websites. This is a
first implementation, which converts only filters that are plain
hostnames.
Unlike with uBO, it is not possible to know from which ruleset a
blocking rule originates. Nonetheless, users will have to make a
choice as to whether navigation should proceed or not.
A setting has been added to the dashboard to wholly enable/disable
strict blocking. It is enabled by default.
Potential future improvements, pending investigation on feasability
in an MV3 framework:
- Extend coverage to explicit `document` filters
- Leverage and use `urlskip=` filters in the blocking page in order
to proceed while bypassing unwanted redirects.
Related issue:
https://github.com/uBlockOrigin/uBOL-home/issues/229
Add "Filter lists" pane in dashboard
The DNR API now supports enabling 50 static rulesets put of a
maximum of 100 (instead of 10 out of 50 originally). Thus given
the potentially growing number of static rulesets, the available
stock rulesets has been moved to its own pane, with the following
improvements:
- Support sublists
- Support search
Aditionally, "RU AdList: Counter" has been added as a stock
ruleset.
Other changes:
- Do not re-evaluate regexes which failed validation
- Better reduce `removeparam` rules
Related discussion:
https://github.com/uBlockOrigin/uBOL-home/discussions/35#discussioncomment-11157444
New policy setting: `rulesets`
Type: array
Type of array items: string
Each item in the list is a list id (as seen in `rulesets/ruleset-details.json`),
prefixed with either `+` to enable the ruleset, or `-` to disable the ruleset.
Users will not be able to enable or disable rulesets present in the `rulesets`
policy. Disabled rulesets will not appear in the dashboard.
Use `-*` to remove all non-default rulesets, except for those added
using `+[ruleset_id]`.
Additionally, some work has been done to properly handle policy changes in
a non-blocking and deferred manner, as I observed that it often takes long
for calls to `storage.manage.get` to resolve. This potentailly takes care
of the following issue:
https://github.com/uBlockOrigin/uBOL-home/issues/174
A new icon has been added to the popup panel, to open a popup
window with a terse list of DNR events for the current tab, in
reverse chronological order (most recent DNR event appears at
the top).
The new ability is available only when the extension is sideloaded,
as per `declarativeNetRequestFeedback` documentation. Ref:
https://developer.chrome.com/docs/extensions/reference/api/declarativeNetRequest#event-onRuleMatchedDebug
Purposefully minimal, so as to have something rather than nothing
when having to diagnose filtering issue with the DNR API. Example:
https://github.com/uBlockOrigin/uBOL-home/issues/156
The content of the popup window does not dynamically update, force
a refresh (F5) to get the most recent DNR events. This might be
improved in the future.
The DNR event buffer is not persisted, so the buffer is empty when
service worker is restarted. This might be improved in the future
by using session storage API.
There is no output filtering ability in this first draft. This
might be improved in the future.
DNR rules are reported. The filter from which a DNR rule
originates is not reported. Given that the rulesets are optimized
after conversion from original filter lists to reduce the DNR rule
count, this is unlikely to ever be possible.
See `managed_storage.json` for available settings. Currently
only `noFiltering` setting is availale.
`noFiltering` is an array of strings, each being a domain for
which no filtering should occur.
Related discussion:
- https://github.com/uBlockOrigin/uBOL-issues/discussions/35
Avoid using updateContentScripts() as it suffers from an unexpected
behavior, causing injected content scripts to lose proper order
at injection time. The order in which content scripts are injected
is key for uBOL content scripts. Potential out of order injection
was causing cosmetic filtering to be broken.
Use actual storage API to persist data across service worker
wake-ups and browser launches. uBOL was trying to avoid using
storage API, at the cost of somewhat hacky code (using DNR API
to persist settings).
Make use of session storage if available, to speed up
initialization of waking up the service worker (which at this
point is necessary to properly implement cosmetic filtering).
Related issues:
- https://github.com/uBlockOrigin/uBOL-issues/issues/5#issuecomment-1575425913
- https://github.com/w3c/webextensions/issues/403
Currently, there is no other way to inject CSS user styles than to
wake up the service worker, so that it can inject the CSS styles
itself using the `scripting.insertCSS()` method.
If ever the MV3 API supports injecting CSS user styles directly
from a content script, uBOL will be back to be fully declarative.
At this point the service worker is very lightweight since the
filtering is completely declarative, so this is not too much of
an issue performance-wise except for the fact that waking up the
service worker for the sole purpose of injecting CSS user styles
and nothing else introduces a pointless overhead.
Hopefully the MV3 API will mature to address such inefficiency.
Specifically, avoid long list of hostnames for the `matches`
property[1] when registering the content scripts, as this was causing
whole browser freeze for long seconds in Chromium-based browsers
(reason unknown).
The content scripts themselves will sort out which cosmetic filters to
apply on which websites.
This change makes it now possible to support annoyances-related lists,
and thus two lists have been added:
- EasyList -- Annoyances
- EasyList -- Cookies
Related issue:
- https://github.com/uBlockOrigin/uBOL-issues/issues/5
These annoyances-related lists contains many thousands of specific
cosmetic filters and as a result, before the above change this was
causing long seconds of whole browser freeze when simply modifying
the blocking mode of a specific site via the slider in the popup
panel.
It is now virtually instantaneous, at the cost of injecting larger
cosmetic filtering-related content scripts (which typically should
be garbage-collected within single-digit milliseconds).
Also, added support for entity-based cosmetic filters. (They were
previously discarded).
---
[1] https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/scripting/RegisteredContentScript
Network filters with csp= option will now be enforced.
Caveat: DNR API does not have support for exception csp= rules,
so excepted csp= filters are currently rejected at conversion time.
This adds support for `redirect=` filters. As with `removeparam=`
filters, `redirect=` filters can only be enforced when the
default filtering mode is set to Optimal or Complete, since these
filters require broad host permissions to be enforced by the DNR
engine.
`redirect-rule=` filters are not supported since there is no
corresponding DNR syntax.
Additionally, fixed the dropping of whole network filters even though
those filters are still useful despite not being completely
enforceable -- for example a filter with a single (unsupported) domain
using entity syntax in its `domain=` option should not be wholly
dropped when there are other valid domains in the list.
