uBlock/tools/make-mv3.sh
Raymond Hill 9339a75952
[mv3] Add support for custom DNR rules
This feature is hidden behind the "Developer mode" setting in
the dashboard. When "Developer mode" is enabled, a tab named
"Develop" will become available in the dashboard. This tab is
meant to contain tools for technical users.

At the moment, the "Develop" pane allows to create custom DNR
rules through a (CodeMirror-based) editor.

For the sake of convenience, the DNR rule must be entered in
YAML-like format. The format is not really full compliant YAML,
just YAML-like, and very strict in order to ensure the parser
stays simple enough.

Lines starting with `#` are comments and will be ignored by the
parser.

Any line which do not match the parser's expectation will be
marked as invalid, and the whole DNR rule containing such invalid
lines will be discarded.

There must not be empty lines inside a rule definition.

Each DNR rule must be separated with a `---` line, which is
known as a YAML document separator.

String values must not be quoted, otherwise the quotes will be
considered part of the value. There is one exception: `''` will
be parsed as "an empty string".

The editor will attempt to auto-complete known DNR keywords. That
feature will improve over time.

Though the parser will identify some errors, not all invalid DNR
rules are currently identified by the parser, and these will be
reported when the rules are registered through the DNR API. Better
identifying invalid DNR rules at edit time will improve over time.

The editor will report `regexFilter` values which are not
supported by the DNR engine on the current platform.

The editor reacts to instances of `regexFilter: ...` to report
whether a regex value is supported. This means you can test for
a regex value by using `# regexFilter: ...` so that you do not
have to create an actual DNR rules just for the sake of testing.

Custom DNR rules can be exported into a JSON file (a format
known by the DNR API as a "static ruleset").

JSON-based ruleset can be imported, the content will be converted
to YAML-like syntax.

The editor will attempt to convert to YAML pasted content which
can be JSON-parsed. It's possible to paste partially or wholly
JSON-based rulesets.

When disabling "Developer mode", all custom DNR rules will be
unregistered from the DNR API. The DNR rules content will be left
intact in such case. Existing DNR rules will be registered into
the DNR API when re-enabling "Developer mode".

Administrators can prevent "Developer mode" from being enabled
by adding `develop` token to `disabledFeatures` setting.

Related discussion:
https://github.com/uBlockOrigin/uBOL-home/discussions/323

The main motivation is to give list maintainers a tool to assist
with resolving filter issues. Custom DNR rules can assist in
crafting and validating filters meant to work with uBOL.

A secondary motivation is to provide technical users the ability
to further customize their content blocker.

More conveniences will be added over time, this is a first version.
2025-05-29 09:06:02 -04:00

188 lines
5.7 KiB
Bash
Executable file

#!/usr/bin/env bash
#
# This script assumes a linux environment
set -e
shopt -s extglob
echo "*** uBOLite.mv3: Creating extension"
PLATFORM="chromium"
for i in "$@"; do
case $i in
full)
FULL="yes"
;;
firefox)
PLATFORM="firefox"
;;
chromium)
PLATFORM="chromium"
;;
edge)
PLATFORM="edge"
;;
safari)
PLATFORM="safari"
;;
uBOLite_+([0-9]).+([0-9]).+([0-9]))
TAGNAME="$i"
FULL="yes"
;;
before=+([[:print:]]))
BEFORE="${i:7}"
;;
esac
done
echo "PLATFORM=$PLATFORM"
echo "TAGNAME=$TAGNAME"
echo "BEFORE=$BEFORE"
UBOL_DIR="dist/build/uBOLite.$PLATFORM"
if [ "$PLATFORM" = "edge" ]; then
MANIFEST_DIR="chromium"
else
MANIFEST_DIR="$PLATFORM"
fi
rm -rf $UBOL_DIR
mkdir -p $UBOL_DIR
cd $UBOL_DIR
UBOL_DIR=$(pwd)
cd - > /dev/null
mkdir -p "$UBOL_DIR"/css/fonts
mkdir -p "$UBOL_DIR"/js
mkdir -p "$UBOL_DIR"/img
if [ -n "$UBO_VERSION" ]; then
UBO_REPO="https://github.com/gorhill/uBlock.git"
UBO_DIR=$(mktemp -d)
echo "*** uBOLite.mv3: Fetching uBO $UBO_VERSION from $UBO_REPO into $UBO_DIR"
cd "$UBO_DIR"
git init -q
git remote add origin "https://github.com/gorhill/uBlock.git"
git fetch --depth 1 origin "$UBO_VERSION"
git checkout -q FETCH_HEAD
cd - > /dev/null
else
UBO_DIR=.
fi
echo "*** uBOLite.mv3: Copying common files"
cp -R "$UBO_DIR"/src/css/fonts/* "$UBOL_DIR"/css/fonts/
cp "$UBO_DIR"/src/css/themes/default.css "$UBOL_DIR"/css/
cp "$UBO_DIR"/src/css/common.css "$UBOL_DIR"/css/
cp "$UBO_DIR"/src/css/dashboard-common.css "$UBOL_DIR"/css/
cp "$UBO_DIR"/src/css/fa-icons.css "$UBOL_DIR"/css/
cp "$UBO_DIR"/src/js/dom.js "$UBOL_DIR"/js/
cp "$UBO_DIR"/src/js/fa-icons.js "$UBOL_DIR"/js/
cp "$UBO_DIR"/src/js/i18n.js "$UBOL_DIR"/js/
cp "$UBO_DIR"/src/js/urlskip.js "$UBOL_DIR"/js/
cp "$UBO_DIR"/src/lib/punycode.js "$UBOL_DIR"/js/
cp -R "$UBO_DIR/src/img/flags-of-the-world" "$UBOL_DIR"/img
cp LICENSE.txt "$UBOL_DIR"/
echo "*** uBOLite.mv3: Copying mv3-specific files"
cp platform/mv3/"$MANIFEST_DIR"/manifest.json "$UBOL_DIR"/
cp platform/mv3/extension/*.html "$UBOL_DIR"/
cp platform/mv3/extension/*.json "$UBOL_DIR"/
cp platform/mv3/extension/css/* "$UBOL_DIR"/css/
cp -R platform/mv3/extension/js/* "$UBOL_DIR"/js/
cp platform/mv3/"$PLATFORM"/ext-compat.js "$UBOL_DIR"/js/ 2>/dev/null || :
cp platform/mv3/extension/img/* "$UBOL_DIR"/img/
cp -R platform/mv3/extension/_locales "$UBOL_DIR"/
cp platform/mv3/README.md "$UBOL_DIR/"
# Libraries
mkdir -p "$UBOL_DIR"/lib/codemirror
cp platform/mv3/extension/lib/codemirror/* \
"$UBOL_DIR"/lib/codemirror/ 2>/dev/null || :
cp platform/mv3/extension/lib/codemirror/codemirror-ubol/dist/cm6.bundle.ubol.min.js \
"$UBOL_DIR"/lib/codemirror/
cp platform/mv3/extension/lib/codemirror/codemirror-ubol/LICENSE \
"$UBOL_DIR"/lib/codemirror/
echo "*** uBOLite.mv3: Generating rulesets"
UBOL_BUILD_DIR=$(mktemp -d)
mkdir -p "$UBOL_BUILD_DIR"
./tools/make-nodejs.sh "$UBOL_BUILD_DIR"
cp platform/mv3/*.json "$UBOL_BUILD_DIR"/
cp platform/mv3/*.js "$UBOL_BUILD_DIR"/
cp platform/mv3/*.mjs "$UBOL_BUILD_DIR"/
cp platform/mv3/extension/js/utils.js "$UBOL_BUILD_DIR"/js/
cp -R "$UBO_DIR"/src/js/resources "$UBOL_BUILD_DIR"/js/
cp -R platform/mv3/scriptlets "$UBOL_BUILD_DIR"/
mkdir -p "$UBOL_BUILD_DIR"/web_accessible_resources
cp "$UBO_DIR"/src/web_accessible_resources/* "$UBOL_BUILD_DIR"/web_accessible_resources/
cd "$UBOL_BUILD_DIR"
node --no-warnings make-rulesets.js output="$UBOL_DIR" platform="$PLATFORM"
if [ -n "$BEFORE" ]; then
echo "*** uBOLite.mv3: salvaging rule ids to minimize diff size"
echo " before=$BEFORE/$PLATFORM"
echo " after=$UBOL_DIR"
node salvage-ruleids.mjs before="$BEFORE"/"$PLATFORM" after="$UBOL_DIR"
fi
cd - > /dev/null
rm -rf "$UBOL_BUILD_DIR"
echo "*** uBOLite.$PLATFORM: extension ready"
echo "Extension location: $UBOL_DIR/"
# Local build
if [ -z "$TAGNAME" ]; then
TAGNAME="uBOLite_$(jq -r .version "$UBOL_DIR"/manifest.json)"
# Enable DNR rule debugging
tmp=$(mktemp)
jq '.permissions += ["declarativeNetRequestFeedback"]' \
"$UBOL_DIR/manifest.json" > "$tmp" \
&& mv "$tmp" "$UBOL_DIR/manifest.json"
# Use a different extension id than the official one
if [ "$PLATFORM" = "firefox" ]; then
tmp=$(mktemp)
jq '.browser_specific_settings.gecko.id = "uBOLite.dev@raymondhill.net"' "$UBOL_DIR/manifest.json" > "$tmp" \
&& mv "$tmp" "$UBOL_DIR/manifest.json"
fi
else
tmp=$(mktemp)
jq --arg version "${TAGNAME:8}" '.version = $version' "$UBOL_DIR/manifest.json" > "$tmp" \
&& mv "$tmp" "$UBOL_DIR/manifest.json"
fi
# Platform-specific steps
if [ "$PLATFORM" = "edge" ]; then
# For Edge, declared rulesets must be at package root
echo "*** uBOLite.edge: Modify reference implementation for Edge compatibility"
mv "$UBOL_DIR"/rulesets/main/* "$UBOL_DIR/"
rmdir "$UBOL_DIR/rulesets/main"
node platform/mv3/edge/patch-extension.js packageDir="$UBOL_DIR"
elif [ "$PLATFORM" = "safari" ]; then
# For Safari, we must fix the package for compliance
node platform/mv3/safari/patch-extension.js packageDir="$UBOL_DIR"
fi
if [ "$FULL" = "yes" ]; then
EXTENSION="zip"
if [ "$PLATFORM" = "firefox" ]; then
EXTENSION="xpi"
fi
echo "*** uBOLite.mv3: Creating publishable package..."
UBOL_PACKAGE_NAME="$TAGNAME.$PLATFORM.mv3.$EXTENSION"
UBOL_PACKAGE_DIR=$(mktemp -d)
mkdir -p "$UBOL_PACKAGE_DIR"
cp -R "$UBOL_DIR"/* "$UBOL_PACKAGE_DIR"/
cd "$UBOL_PACKAGE_DIR" > /dev/null
rm -f ./log.txt
zip "$UBOL_PACKAGE_NAME" -qr ./*
cd - > /dev/null
cp "$UBOL_PACKAGE_DIR"/"$UBOL_PACKAGE_NAME" dist/build/
rm -rf "$UBOL_PACKAGE_DIR"
echo "Package location: $(pwd)/dist/build/$UBOL_PACKAGE_NAME"
fi