Commit graph

2480 commits

Author SHA1 Message Date
Raymond Hill
5eff4a027a
Fix https://github.com/gorhill/uBlock/issues/3541 2019-05-20 18:29:28 -04:00
Raymond Hill
cdfbc1ae02
Import translation work from https://crowdin.com/project/ublock 2019-05-20 14:10:21 -04:00
Raymond Hill
1888033070
Add support for all filter option
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bqnsoa/

The `all` option is equivalent to specifying all
network-based types + `popup`, `document`,
`inline-font`, `inline-script`.

Example from discussion:

    ||bet365.com^$all

Above will block all network requests, block all popups,
prevent inline fonts/scripts from `bet365.com`. EasyList-
compatible syntax does not allow to accomplish that
semantic when using only `||bet365.com^`.

If using specific negated type options along with `all`,
the order in which the options appear is important. In
such case `all` should always be first, followed by
the negated type option(s).
2019-05-20 13:46:36 -04:00
Raymond Hill
4b85838745
Import translation from https://crowdin.com/project/ublock 2019-05-19 18:42:32 -04:00
Raymond Hill
72d9758faa
Ensure the "Filter lists" pane is in sync with update status
Related issue:
- https://github.com/gorhill/uBlock/issues/2394

Additionally, I added a new advanced setting to control
how long after launch an auto-update session should be
started -- value is in seconds:

    autoUpdateDelayAfterLaunch 180
2019-05-19 18:31:12 -04:00
Raymond Hill
a0ac1b7ee8
Fix handling of data: for filtering purpose in logger
Related issue:
- https://github.com/gorhill/uBlock/issues/2469
2019-05-19 17:00:49 -04:00
Raymond Hill
f677443878
Warn when navigating away from pane with unsaved changes
Related issue:
- https://github.com/gorhill/uBlock/issues/3271

When navigating away by clicking another pane tab button,
there will be an embedded warning, which can be ignore
in order to proceed to the new pane, or dismissed by
either clicking on the "Stay" button or anywhere else
in the dashboard.

When navigating away by trying to close the tab, there will
be a built-in browser warning asking for confirmation.
2019-05-19 15:35:00 -04:00
Raymond Hill
1caff7429e
Add optional support for generic procedural cosmetic filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/131

The new advanced setting and its default value is:

    allowGenericProceduralFilters false

Whenever this setting is toggled, the user is responsible
of forcing a reload of all filter lists so as to allow uBO
to process differently any existing generic procedural
cosmetic filters.
2019-05-18 18:57:32 -04:00
Raymond Hill
e66e4496ed
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/360 2019-05-18 17:50:58 -04:00
Raymond Hill
ca34bc4f3e
Fix "Revert" button not resetting after saving changes
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/367
2019-05-18 17:48:19 -04:00
Raymond Hill
3cf71835c4
Set default delay for creating selfie to 3 minutes
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bq49zi/
2019-05-18 14:43:44 -04:00
Raymond Hill
f7bbc80717
Improve "Whitelist pane"; remove now useless built-in switch rule
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/214

Built-in whitelist directives are now rendered differently
than user-defined whitelist directives. Also, removing a
built-in whitelist directive will only cause that directive
to be commented out, so that users do not have to remember
built-in directives should they want to bring them back.

Related issue:
 https://github.com/uBlockOrigin/uBlock-issues/issues/494

The built-in per-site switch rule
`no-scripting: behind-the-scene false` has been removed,
it should not ever be needed since there will always be a
valid root context for main- and sub-frames.
2019-05-18 14:20:05 -04:00
Raymond Hill
de41c1bf53
Fix parsing of recursive !#if-`!#endif directives
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/270
2019-05-18 10:31:04 -04:00
Raymond Hill
62387fb87a
Prevent picker's preview mode from modifying style attribute
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/551

The issue fixes previewing the hiding/unhiding of targeted
elements in the element picker.

However it does not address the case of previewing
`:style(...)` operators -- this would require a much
more complex fix, which I am not sure is worth the
amount of work and increased code complexity.
2019-05-17 19:26:48 -04:00
Raymond Hill
9cbdff1a9d
Import translation work from https://crowdin.com/project/ublock 2019-05-17 15:02:00 -04:00
Raymond Hill
9bfbbfec84
Adjust visual of cosmetic exception filters in logger
The invariant prefixes `##` and `#@#` are now hidden,
allowing to reveal more of the filter itself when the
logger view is narrow.
2019-05-17 11:45:07 -04:00
Raymond Hill
0ca44b847c
Avoid duplicated strings in filterOrigin w/ new approach
The new approach is simpler and should benefit selfie
serialization/unserialization.

This renders stringDeduplicater obsolete -- it has been
removed.
2019-05-17 10:13:58 -04:00
Raymond Hill
1386429382
Fix regression in applying procedural cosmetic filters
Related commit:
- 3573b6b32c
2019-05-16 17:22:20 -04:00
Raymond Hill
3573b6b32c
Add ability to report exception cosmetic filters in the logger
Related issue:
- https://github.com/gorhill/uBlock/issues/127

Additionally, the extended exception filters in the
logger will be rendered with a line-through to more
easily distinguish them from non-exception ones.

Also, opportunistically converted revisited code to
ES6 syntax.
2019-05-16 13:44:49 -04:00
Raymond Hill
fc109c8b7c
Revisit code to benefit from ES6 syntax 2019-05-15 14:49:12 -04:00
Raymond Hill
1fe3b54acc
Fix cosmetic exception filters not applying
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/575

Regression from:
- 93f80eedfa

Specific cosmetic exception filters need to be returned so that
they can be applied to generic cosmetic filters.
2019-05-15 14:43:59 -04:00
Raymond Hill
39e2a03edb
Fix comment 2019-05-14 09:31:51 -04:00
Raymond Hill
a14dcecf8f
Do not assume wildcards fall on label boundaries
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492223980
2019-05-14 09:29:45 -04:00
Raymond Hill
93f80eedfa
Refactor runtime storage of specific cosmetic filters
This was a TODO item:
- 07cbae66a4/src/js/cosmetic-filtering.js (L375)

µBlock.staticExtFilteringEngine.HostnameBasedDB has been
re-factored to accomodate the storing of specific cosmetic
filters.

As a result of this refactoring:

- Memory usage has been further decreased
- Performance of selector retrieval marginally
  improved
- New internal representation opens the door
  to use a specialized version of HNTrie, which
  should further improve performance/memory
  usage
2019-05-14 08:52:34 -04:00
Raymond Hill
8a312b9bbb
Support cases with more than one wildcard
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492147440
2019-05-14 06:52:13 -04:00
Raymond Hill
fe0b7a0e0f
Relax destination hostname requirements in redirect filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572

Wildcards are now allowed in the hostname part of redirect
filters. There will be an attempt to find the longest
right-hand portion of the hostname with no wildcard. If
no non-empty hostname can be extracted, `*` will be used.
2019-05-13 20:19:10 -04:00
Raymond Hill
1e40f50eb3
Add benchmark method to cosmetic filtering engine
To measure retrieval of site-specific selectors. From
uBO's own dev console:

    µBlock.cosmeticFilteringEngine.benchmark();
2019-05-12 11:41:47 -04:00
Raymond Hill
57890d60ff
Fix incorrect use of this in static method
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/568

Regression from:
- 19ece97b0c
2019-05-11 17:40:55 -04:00
Raymond Hill
8a7e704080
Add support for nth-ancestor operator in HTML filtering
Also opportunitisically converted some code to
ES6's `class`.
2019-05-11 13:21:23 -04:00
Raymond Hill
915c1f1f3c
Report resources blocked by csp= option in logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/552
2019-05-11 10:40:34 -04:00
Raymond Hill
12bdd01595
Ensure "Ignore generic cosmetic filters" sticks on Fennec
Related issue:
- https://www.reddit.com/r/uBlockOrigin/comments/blkudl/

The setting was not sticking at first-install time.
2019-05-11 09:04:13 -04:00
Raymond Hill
ac1a323918
Import translation work from https://crowdin.com/project/ublock 2019-05-10 17:26:34 -04:00
Raymond Hill
e59bdb1485
Defuse fixed position on body element in element zapper
The `fixed` style property on the `body` element will be
defused if an overlay element is removed using the element
zapper.

Related:
- https://www.reddit.com/r/uBlockOrigin/comments/bktxtb/scrolling_doesnt_work/emlscyz
2019-05-06 13:32:55 -04:00
Raymond Hill
26237d6d40
Import translation work from https://crowdin.com/project/ublock 2019-05-06 11:39:41 -04:00
Raymond Hill
3692bb4ada
Add HNTrieRef.dump() and STrieRef.dump() as dev tool
To be used at the console, as an investigation tool for
development purpose.

Using it to verify the content of the largest
FilterHostnameDict instance, I spotted an all-uppercase
hostname in the HNTrieRef instance:

µBlock.staticNetFilteringEngine.categories.get(0).get(0x10000000).dict.dump();

Thus the changes to static-net-filtering.js are to fix
the erroneous insertion of filters with uppercase
characters. The single instance found was a hostname entry
in Malware Domain List (TRIANGLESERVICESLTD dot COM).
2019-05-06 11:12:39 -04:00
Raymond Hill
ce488d9fc8
Import translation work from https://crowdin.com/project/ublock 2019-05-01 20:05:16 -04:00
Raymond Hill
0e4fbefd07
Remove unecessary null placeholders FilterOriginHitSet et al.
The `null` placeholder are not necessary, we can just use
default arguments instead, and add the HNTrieContainer
references if and only if they are instanciated.
2019-05-01 18:54:11 -04:00
Raymond Hill
9e4385243c
Web accessible secrets can be used for at most one second
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/550

Related Chromium issue (I can't access it):
- https://bugs.chromium.org/p/chromium/issues/detail?id=957866

Findings so far: affects browsers based on Chromium 74.
I could not reproduce the issue with either Chromium 73 or
Google Chrome 75.

This commit is a mitigation: to prevent sites from using
uBO's internal WAR secret for tracking purpose. A secret
can be used for at most one second, after which a new secret
is generated.

The original issue related to the implementation of
secret-gated web accessible resources is:
- https://github.com/gorhill/uBlock/issues/2823
2019-04-30 14:36:07 -04:00
Raymond Hill
73e2f25e95
Add new cosmetic procedural operator: :nth-ancestor(n)
The purpose of this new `:nth-ancestor(n)` operator is to
lookup the nth ancestor relative to the currently selected
node.

It is essentially equivalent to `:xpath(..)`, where
ancestor distance is expressed as a number rather than a
sequence of slash-separated `..`.

The rationale to introduce this new procedural selector
is to have a low overhead way to accomplish ancestor
selection.
2019-04-30 09:02:14 -04:00
Raymond Hill
42bf659695
Revert "Order HNTrie nodes alphabetically to allow for early bailout"
This reverts commit f5f9e05071.
2019-04-30 07:00:52 -04:00
Raymond Hill
f5f9e05071
Order HNTrie nodes alphabetically to allow for early bailout
This commit implements the alphabetical ordering of HNTrie
nodes, so as to make it possible to bail out early at
HNTrie.matches() time.

Contrary to what I expected, there is no performance gain
observed to HNTrie.matches() as per benchmarks -- I find
the results perplexing.

Because of this I will revert this commit immediately.
The purpose of this commit is to record the changes so
that I can bring them back to life in the future whenever
I want to investigate further.
2019-04-30 06:47:54 -04:00
Raymond Hill
adabb56dc9
Do not store impossible to match filters in HNTrie
Consider the two following filters:

    example.com
    www.example.com

This commit make it so that if the first filter is
already present in a given HNTrie, the second filter
will not be stored, since HNTrie will _always_
return the first filter as a match whenever the
hostname to match is example.com or any subdomain
of example.com.

The detection of such pointless filters is
virtually free when adding a hostname to an HNTrie
instance (given how data is stored in the trie), so
in practice no overhead is incurred to detect such
pointless filters.

The ability to ignore impossible to match filters
in HNTrie instances will _especially_ benefit those
using large hosts files.

Examples of how this helps using real configurations:

- Default lists:
  444 filters out of 100,382 were ignored as a result
  of this commit.

- Default lists + "Energized Ultimate Protection":
  283,669 filters out of 903,235 were ignored as a
  result of this commit.

Side note: There was no measurable difference between
the two configurations above in the performance of
the matching algorithm as reported by the built-in
benchmark tool.
2019-04-29 13:15:16 -04:00
Raymond Hill
c4f9ae706a
Fix alternate code path introduced in 295f08da97 (oops) 2019-04-28 14:18:09 -04:00
Raymond Hill
295f08da97
Implement code path for when TextDecoder() is not available
The primary purpose is to unbreak
https://github.com/cliqz-oss/adblocker/tree/master/bench/comparison
2019-04-28 14:07:21 -04:00
Raymond Hill
ac58b8e688
Make token hashes fit within a 32-bit integer
The staticNetFilteringEngine uses token hashes to store/lookup
filters into Map objects.

Before this commit, the tokens were encoded into token hashes
as JS numbers (not exceeding MAX_SAFE_INTEGER) using at most
the 8 first characters of the token.

With this commit, token hashes are now restricted to fit
into 32-bit integers, and are derived from at most the 7 first
characters. This improves filter look-up performance as per
built-in benchmark().
2019-04-28 10:15:15 -04:00
Raymond Hill
510cda0bc5
Support in logger's filter expression picker 2019-04-28 07:23:14 -04:00
Raymond Hill
96dce22218
Increase resolution of known-token lookup table
Related commit:
- 69a43e07c4

Using 32 bits of token hash rather than just the 16 lower
bits does help discard more unknown tokens.

Using the default filter lists, the known-token lookup
table is populated by 12,276 entries, out of 65,536, thus
making the case that theoretically there is a lot of
possible tokens which can be discarded.

In practice, running the built-in
staticNetFilteringEngine.benchmark() with default filter
lists, I find that 1,518,929 tokens were skipped out of
4,441,891 extracted tokens, or 34%.
2019-04-27 08:18:01 -04:00
Raymond Hill
a8946c8d73
Fix list lookup of multi-hostname domain= filters in logger
Related commit:
- 3f3a1543ea

The regression was preventing uBO to find from which list a filter
originated. This affected only filters for which the `domain=`
option had multiple hostnames.
2019-04-27 07:04:43 -04:00
Raymond Hill
4c9e760a10
Import translation work from https://crowdin.com/project/ublock 2019-04-26 17:30:19 -04:00
Raymond Hill
69a43e07c4
Ignore unknown tokens in urlTokenizer.getTokens()
Given that all tokens extracted from one single URL are potentially
iterated multiple times in a single URL-matching cycle, it pays to
ignore extracted tokens which are known to not be used anywhere in
the static filtering engine.

The gain in processing a single network request in the static
filtering engine can become especially high when dealing with
long and random-looking URLs, which URLs have a high likelihood
of containing a majority of tokens which are known to not be in
use.
2019-04-26 17:14:00 -04:00