Request Control Filters #149

crssi · 2017-06-21T18:57:38Z

This is Request Control | GitHub
This is our Request Control Filters Wiki

Post your Request Control Filter-fu here

Note: Examples below are for discussion / testing - use/test at your leisure, post feedback. As really cool filters emerge, we will put them in the wiki

crssi · 2017-06-21T18:58:55Z

NOTE: ga_ stuff might need XHR type also
Pure URL extension replacement:

Pattern: Any URL
Types: Document, Embedded document (not sure for second one)
Action: Filter
Filter URL: Off
Trim URL Parameters: ref_, utm_source, utm_medium, utm_term, utm_content, utm_campaign, utm_reader, utm_place, ga_source, ga_medium, ga_term, ga_content, ga_campaign, ga_place, yclid, _openstat, fb_action_ids, fb_action_types, fb_ref, fb_source, action_object_map, action_type_map, action_ref_map, ws_ab_test, btsid, algo_expid, algo_pvid, sid, utm_name, utm_cid, utm_reader, utm_viz_id, utm_pubreferrer, utm_swu, icid, _hsenc, _hsmi, mkt_tok, sr_share, vero_conv, vero_id, nr_email_referer, ncid

Samples:
http://bigpicture.ru/?p=431513&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+bigpictures+%28%D0%9D%D0%9E%D0%92%D0%9E%D0%A1%D0%A2%D0%98+%D0%92+%D0%A4%D0%9E%D0%A2%D0%9E%D0%93%D0%A0%D0%90%D0%A4%D0%98%D0%AF%D0%A5%29

UPDATE:
Added @Atavic suggested "Trim URL Parameters"

crssi · 2017-06-21T19:05:19Z

Redirect to URL without REF tracking:

Pattern: Any URL
Types: Document, Embedded document (not sure for second one)
Action: Redirect
Redirect To: ~~{href/(.?)ref=./$1}~~ {href/(.*\/)ref=.*/$1}

Samples:
https://www.amazon.com/AmazonBasics-Type-C-USB-Male-Cable/dp/B01GGKYQ02/ref=sr_1_1?s=amazonbasics&srs=10112675011&ie=UTF8&qid=1489067885&sr=8-1&keywords=usb-c

crssi · 2017-06-21T19:10:40Z

Remove the crap and possible tracking over URL manipulation after images and CSS:

Pattern scheme: http/https
Pattern host: *
Pattern path: *.jpg?*, *.gif?*, *.png?*, *.svg?*, ~~*.css?*~~
Types: Document, Embedded document (not sure for second one), ~~Stylesheet~~, Image
Action: Filter
Filter URL: Off
Trim URL Parameters: Trim all

UPDATE:
Added Pattern path: *.css?*
Stylesheet breaks this page https://technet.microsoft.com/en-us/library/2009.07.cableguy.aspx

Added Types: Document, Embedded document

Samples:
http://www.24ur.com/

~~Sample that doesn't work, but it should... don know why, yet:~~ UPDATE: It works now.
https://thechive.files.wordpress.com/2017/06/dd0523324267789bb8beecc5d7914970.jpg?quality=85&strip=info&w=600

BRAINING: maybe there is same method to be added also for Font and Media types... will see.

crssi · 2017-06-21T19:17:42Z

UPDATE: Use "Skip Redirect" from AMO... its way better.
Facebook redirect without visiting facebook:

Pattern scheme: http/https
Pattern host: l.facebook.com
Pattern path: *u=*
Types: Document, Embedded document (not sure for second one)
Action: Filter
Filter URL: On

Sample:
https://l.facebook.com/l.php?u=https%3A%2F%2Fwww.fsf.org%2Fcampaigns%2F&h=ATP1kf98S0FxqErjoW8VmdSllIp4veuH2_m1jl69sEEeLzUXbkNXrVnzRMp65r5vf21LJGTgJwR2b66m97zYJoXx951n-pr4ruS1osMvT2c9ITsplpPU37RlSqJsSgba&s=1

crssi · 2017-06-21T20:40:37Z

Funny... I have to look into it too.
Anyway, I have updated for CSS too... see change up.
One site is particulary full of this crap to test... see updated post.

crssi · 2017-06-21T20:43:46Z

Hah... resolved, done.
Have phun

crssi · 2017-06-21T20:45:22Z

UPDATE: import/export is done.
The author is working now on import/export. ;)

@Thorin-Oakenpants - if you wish to clean the thread, you can delete my last 3 posts (including this one) and your "didn't work" too, since its resolved.

Cheers

Atavic · 2017-06-21T22:44:07Z

Here some Google's Urchin Tracking Modules that aren't listed in Trim URL Parameters list:

utm_name
utm_cid
utm_reader
utm_viz_id
utm_pubreferrer
utm_swu

Also, url-tracking-stripper has a few more for other trackers:

ICID
icid
_hsenc
_hsmi
mkt_tok
sr_share
vero_conv
vero_id
nr_email_referer
ncid

earthlng · 2017-06-22T13:13:50Z

Thanks @crssi !! 👍

a couple nits if you don't mind...

the ga_ stuff probably needs to include XHR
{href/(.?)ref=./$1} doesn't seem right. I'm pretty sure this always results in a single h (from "http/https"), right? (.?)
and wouldn't it (if fixed) break something like http://foo.bar/redirect?href=bar.foo/index.html

to give you an example, this is what I came up with for a bing-redirect:
{search/^\?.*&?q=(.+?)(&.*|#.*|$)/https://www.ixquick.eu/do/dsearch?query=$1&cat=web&pl=opensearch&language=english} - it covers everything I could think of, namely:

h ttp://www.bing.com/search ?q=crssi &go=Submit+Query&qs=ds&form=QBLH
h ttp://www.bing.com/search?go=Submit+Query &q=crssi &qs=ds&form=QBLH
h ttp://www.bing.com/search?go=Submit+Query&qs=ds&form=QBLH &q=crssi
h ttp://www.bing.com/search?q=crssi
h ttp://www.bing.com/search?go=Submit+Query&qs=ds&form=QBLH &q=crssi#anchor1

and even then, I'm sure I missed something - regexes are almost impossible to get right

another one from my small collection:

Any URL CSP report Block

crssi · 2017-06-22T14:20:33Z

You are right. Need to rethink the REF one, it was fishy to me also and I don't like it at all.... will put a "Under construction" note on that post.

The ga_ stuff, should we separate into another filter extended with XHR or just enable the XHR on the current one. I will enable here now and see if I will get any breakage... will also put a note in that post.

I have another "nasty" one, which could replace "Skip Redirect" extension, but it is also fishy... for now I am using Redirector, since I can use regex (not that I like it, but can't see better way.

I can place the nasty one in another post together with the Redirector approach for your critics?

earthlng · 2017-06-22T14:27:44Z

Have you installed and looked at Redirector yet?

nope

Does Request Control offer anything that Redirector doesn't or vice versa?

IDK, you tell me :) I'll try it someday. My way too complex bing regex is probably not necessary. It looks like the addon strips additional parameters automatically. I'll try to come up with a simpler solution

earthlng · 2017-06-22T14:44:19Z

I already edited my post dude - chillax mate :)
I'll probably end up using both

crssi · 2017-06-22T14:49:38Z

Redirector has some problems, when clicking URL from external program it doesn't always work as intended.
Another thingy is that Redirector parses input only by one filter (just my observation) and doesn't pass the result over filters again unitl nothing left to filter (it happens that you want to filter out more than only one thing).
Otherwise it has also exclusions which is great and better string manipulation over regex.
Both are very good.

crssi · 2017-06-22T14:52:37Z

@Thorin-Oakenpants do you have any sample for ga_ stuff, so I can dig in?
After tomorrow I will be offline for about 5 days.

crssi · 2017-06-22T15:57:42Z

Updated RedirectTo

does not cover @Thorin-Oakenpants sample: http://foo.bar/redirect?href=bar.foo/index.html

crssi · 2017-06-22T16:23:01Z

UPDATE: Use "Skip Redirect" from AMO... its way better.
Now a nasty one... trying to mimic part of "Skip redirect"
NOTE: this one can and will make some login breakage... for tests and brainstorming only
Does not interfere any of rules posted on this page.

It consists 2 filter rules.
First one does the skipping and the second one whitelists login pages to avoid breakage.

Pattern: Any URL
Types: Document
Action: Filter
Filter URL: On

Pattern scheme: http/https
Pattern host: *
Pattern path: *signin*, *signout*, *login*, *logout*, *logon*, *logoff*, *auth*, *account*, *eBayISAPI.dll*, *ServiceLogin*, *ServiceLogout*, *AccountChooser* (maybe you can add also *option*, but I don't like it)
Types: Document
Action: Whitelist

Samples:
Working redirect
https://outgoing.prod.mozaws.net/v1/b928e4237edbbdd2646a3971d2e6b514aee033c10f3f4c49415bf93096405f38/http%3A//www.google.com/chrome/%3Fi-would-rather-use-firefox=http%253A%252F%252Fwww.mozilla.org/

www.google.com/chrome/?i-would-rather-use-firefox=http%3A%2F%2Fwww.mozilla.org/

Working login pages
https://signin.ebay.de/ws/eBayISAPI.dll?SignIn&UsingSSL=1&pUserId=&co_partnerId=2&siteid=77&ru=http%3A%2F%2Fmy.ebay.de%2Fws%2FeBayISAPI.dll%3FMyEbayBeta%26MyEbay%3D%26gbh%3D1%26guest%3D1&pageType=3984

https://www.amazon.com/ap/signin?encoding=UTF8&openid.assoc_handle=usflex&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.mode=checkid_setup&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.ns.pape=http%3A%2F%2Fspecs.openid.net%2Fextensions%2Fpape%2F1.0&openid.pape.max_auth_age=0&openid.return_to=https%3A%2F%2Fwww.amazon.com%2Fgp%2Fyourstore%2Fcard%3Fie%3DUTF8%26ref%3Dcust_rec_intestitial_signin

https://manage.autodesk.com/service/authentication/login?returnUrl=https%3A%2F%2Fmanage.autodesk.com%2Fcep%2F

https://account.xiaomi.com/pass/serviceLogin?callback=http%3A%2F%2Fglobal.mi.com%2Fen%2Flogin%2Fcallback%3Ffollowup%3Dhttp%253A%252F%252Fwww.mi.com%252Fen%252F%26sign%3DZDk4YmY0MTRkMThmODExYTE0MDljYWRmYzczZmNjOGZjNDAzNzU0Mg%2C%2C&sid=mi_overseaen&_locale=en

https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=13&ct=1495461484&rver=6.7.6643.0&wp=MBI_SSL_SHARED&wreply=https:%2F%2Fmail.live.com%2Fdefault.aspx%26lc%3D1033%26id%3D64855%26mkt%3Den-us%26cbcxt%3Dmai%26lc%3D1033%26id%3D64855%26mkt%3Den-us%26cbcxt%3Dmai%26lc%3D1033%26id%3D64855%26mkt%3Den-us%26cbcxt%3Dmai%26lc%3D1033%26id%3D64855%26mkt%3Den-us%26cbcxt%3Dmai&lc=1033&id=64855&mkt=en-us&cbcxt=mai

Currently broken login pages
https://login.leagueoflegends.com/?region=eune&lang=en_PL&redirect_uri=http%3A%2F%2Feune.leagueoflegends.com%2F
I can't figure out how to match this one since "login" is part of hostname and not path and Request Control doesn't cover this... here Redirector is better.

crssi · 2017-06-22T18:30:34Z

Crap... at least "path" definition is case sensitive. I have placed an issue about that.

UPDATE: Case sensitive "bug" is actually "deal breaker" until corrected. Shame, started to love this one. :(

Atavic · 2017-06-22T19:34:04Z

👎 Redirector always loads the unmodified address.

Maybe we should add Request Control to Appendix B: Firefox-Add-ons?

crssi · 2017-06-22T20:55:42Z

^^ Ahh, this now perfectly explains my observation about Redirector and what I didn't like about it.
But how then uBo, uMatrix and Request Controls are dealing perfectly in those cases.

Atavic · 2017-06-22T21:19:35Z

URL path definition is case sensitive

The absolute URL path: http://example.com/data/ is not case sensitive, as per RFC 3986.

While all that comes after can be case sensitive, depending on web server settings, OS...

Everything after the RFC Standard is unclear.

earthlng · 2017-06-24T13:47:04Z

@crssi wrote:

do you have any sample for ga_ stuff

any site that uses GoogleAnalytics, fe ghacks. GA should already be blocked by uBO or uMatrix or whatnot and I'm not sure we really need a filter for that stuff.

earthlng wrote

My way too complex bing regex is probably not necessary. It looks like the addon strips additional parameters automatically

Yeah, it doesn't, at least not for "Action: Redirect". Redirector doesn't make it easier either.
But I realized I don't need to account for anchors because they are not part of the "search" parameter, which results in this slightly adjusted bing-redirect:

old:
{search/^\?.*&?q=(.+?)(&.*|#.*|$)/https://www.ixquick.com/do/dsearch?query=$1&cat=web&pl=opensearch&language=english}
new:
{search/(^\?|.*&)q=(.+?)(&.*|$)/https://www.ixquick.com/do/dsearch?query=$2&cat=web&pl=opensearch&language=english}
alternative format:
https://www.ixquick.com/do/dsearch?query={search/(^\?|.*&)q=(.+?)(&.*|$)/$2}&cat=web&pl=opensearch&language=english

... pick whichever format you prefer.

edit: fuck! the ^\?.*&?q= part is flawed, for example: ?testq=wrong without a q=right, however it works for ?testq=disney&q=porn but not for ?q=porn&testq=disney, ie if you want porn instead of disney :)
got it! (^\?|.*&)q=

edit2:
for Redirector:
https?://www.bing.com/search(\?|.*&)q=(.+?)(&.*|#.*|$) - this one "needs" the anchor part again!
Redirect to: https://www.ixquick.com/do/dsearch?query=$2&cat=web&pl=opensearch&language=english

please try to break this one y'all ;)

earthlng · 2017-06-24T14:30:51Z

We need some really generic ones, but I have no idea how bad the breakage could get.

I would stay away from generic ones if you care about efficiency and no breakage.

Does Request Control offer anything that Redirector doesn't or vice versa?

Redirector

pros
- example result
- control over processing -> "Process matches"
- very detailed console output
- simpler terms (iframe instead of embedded document)
- import/export rules
- Exclude Pattern (you can probably achieve the same in RC with a bit of regex-fu)
- custom description
cons
- very detailed console output (performance impact, log "pollution", no pref to disable it?)
- simpler terms ("other" instead of CSP report (which I guess is part of "other"), etc)
- no direct access to only parts of the url (ie RC's "parameters")
- doesn't support multiple domains/paths in one rule
- no "TLD's" support (fe. see default google rule in RC)
- no "block" request
- no indicator in the UI for redirects

Redirector's author said the addon is in "maintenance mode" and he won't add new features.
RC's author still seems to be very actively maintaining the addon and is open to add new features.

RC does some "magic" in the background which is great (and seems to work) but not very transparent.
fe. it strips additional parameters (&xyz=foo&bar=etc) with "Filter" rules and does urldecoding if necessary. I'd personally prefer a bit more control over this, especially the encoding/decoding stuff.
Maybe we can lobby him/her to add that (with a default "auto" option for the current behavior)

edit: Redirector allows/requires to order the rules but idk how useful that is. It probably makes it more complicated if you have a lot of rules

GitCurious · 2017-06-25T11:27:07Z

Request Control Filters #149 (comment)

This appears to be breaking some internal Amazon links (such as "Warehouse Deals" for example),
presumably it`s the ref= part.

Request Control Filters #149 (comment)

With regard to this, how exactly are these "2" filter rules applied ?
Separately ?, must they be in a specific order ?

This stuff gives me a headache at the best of times - intriguing stuff though !

Forsaked · 2017-06-26T05:41:12Z

OT: @Thorin-Oakenpants Good to know, looks like im not John Cena!

earthlng · 2017-06-26T15:23:27Z

FYI atm there's a bug that a redirected Document still gets added to the history. However the request doesn't touch the network. (that was fixed in 54 and also backported to ESR here)
re: https://www.reddit.com/r/firefox/comments/6j6w0v/how_to_automatically_clean_up_urls_by_removing/djc7w8v/

southwindcg · 2017-06-27T10:53:49Z

The *.jpg?*, *.png?* ... image filters break images on www.kickstarter.com.

Atavic · 2017-06-27T14:24:20Z

Try whitelisting https://ksr-ugc.imgix.net/

crssi · 2017-06-30T09:09:56Z

Will keep eye on this addon, since it has a lot of potential, but will hold until case insesitive isn't sorted out.

crssi · 2017-06-30T09:13:22Z

@GitCurious REF is tricky and WILL make breakage.
There are no orders in RC.
The first rule takes the last URL in the URL (damn if this makes sense :))
And the second skips this action when you hit some login pages.

crssi · 2017-06-30T13:18:11Z

There are also some other issues. I have placed an Issue on RC github.

crssi · 2017-07-11T04:30:07Z

Sure you can. Its just a JSON.
This is the same:
[{"pattern":{"scheme":"*","host":["www.imdb.com"],"path":["title/*","name/*","character/*"]},"types":["main_frame"],"action":"filter","active":true,"paramsFilter":{"values":["ref_"],"pattern":"ref_"},"skipRedirectionFilter":true}]

crssi · 2017-07-11T07:05:11Z

Damn cool... how do you do that?

crssi · 2017-07-11T08:39:58Z

Ah, ok, I see now... thx :)

Atavic · 2017-08-03T22:37:13Z

@crssi Google has &biw= and others, see RemoveGoogleTracking.

crssi · 2017-08-08T08:01:39Z

Thank you @Atavic. Will take a look into in the next days

crssi · 2017-08-08T08:10:12Z

For redirect skipping RC is no match to Skip Redirect (WE), especially now where author implemented same domain detection... see sblask-webextensions/webextension-skip-redirect#30

I will not try to mimic the same functionality on RC anymore.

For basic tracking removal the Link Cleaner does a nice job, but RC can be used as a supplement for more advanced rules or for additional tracking options removal that are not covered in LC.

Atavic · 2017-08-08T21:37:32Z

@ghacksuserjs from RemoveGoogleTracking:

'biw', // offsetWidth
'bih', // offsetHeight

Apparently related to screen fingerprint.

crssi · 2017-08-09T15:38:23Z

@Atavic
To to be sure... should I add to #149 (comment) the following:
biw, bih, ei, sa, ved, source, prmd, bvm, bav, psi, stick, dq, ech, gs_gbg, gs_rn, cp, scroll, vet, yv, ijn, iact, forward, ndsp, csi, tbnid, pbx, dpr, pf, gs_rn, gs_mss, pq, cp, oq, sclient, gs_l, aqs, psi

Observation: From what I see on the source code of RemoveGoogleTracking, its more complex than just to remove those parameters.

crssi · 2017-08-18T09:41:17Z

I have redesigned some rules and dropped some from the scratch and have issued few issues to @tumpio... specially this one.
When some more is known about the issue I will close this topic and open a new topic (referenced to this one) and post from scratch.

crssi · 2017-09-29T18:46:04Z

@Thorin-Oakenpants
If you agree I would like to close this topic, since @tumpio is not active almost 2 months and there is better approach to deal with skipping redirect and url cleaning.
I can open new "post" with all together solution with setup I am using for skip redirect and url cleaning in the next few days?

earthlng · 2017-10-14T12:51:01Z

example to skip youtu.be:
pattern: scheme: http/https host: youtu.be path: *
types: Document
action: Redirect
redirect to: https://www.youtube.com/watch?v={pathname:1}

example url: https://youtu.be/nqbUkThGlCo

Request Control supports direct access to named parameters
The named parameter pathname in the example link is /nqbUkThGlCo and {pathname:1} strips the slash using substring extraction

As detailed in arkenfox/user.js#149 (comment) Co-authored-by: crssi <[email protected]>

* Upgrade the default rules Add rules for GMX, DuckDuckGo, Tumblr, YouTube, Amazon and Bing, and replicate functionality of Neat URL webextension at Smile4ever/Neat-URL * Import rule from 0x01h/gif-tracking-protection Co-authored-by: Geeknik Labs <[email protected]> * Upgrade GIF filter to general image filter Now filters JPG, PNG, GIF and WEBP. It's also based in regular expressions, to filter URLs with different capitalizations (http://bad.site/TRACKER.GIF?id=666) and to register less false positives (http://good.site/giftrackingcounter?lang=en) Co-authored-by: crssi <[email protected]> * Revamp parameter trimming in images Every request for an image is filtered now, regardless of name or file format. Exemptions for common crop and size parameters, and special rules for particular sites (Facebook, Instagram, WhatsApp) have been added. Co-authored-by: crssi <[email protected]> Co-authored-by: Geeknik Labs <[email protected]> * Tweak image URL trimming and add Google Street View exception Also some optional whitelist rules to restore functionality in YouTube. * Add more terms to the blacklist of general parameter trimming As detailed in arkenfox/user.js#149 (comment) Co-authored-by: crssi <[email protected]> * Tweak URL parameter trimming in images Add exemptions for: - embedded interactive Google Maps in 3rd-party sites, - any kind of image shown in DuckDuckGo search results, - and map and aerial view tilesets following WMS and WMTS standards (like one would find while editing OpenStreetMap). * Unblock cas shown when registering a WikiMedia account by whitelisting two innocuous URL parameters: `title` and `wpCaptchaId`. * Tweak filter tracking parameters rule: exempt common parameters found in websites that use IDs for the picture, instead of static paths (as found in region government page www.xunta.gal). * Unblock Reddit pictures They can be later targeted by a different rule if necessary. * Add more image parameters to the whitelist Trimming any of these parameters blocks captchas when creating an email account at Microsoft * Whitelist name parameter Some webpages (mis)use it for size selection, and some pictures in Twitter don't load without this. * Remove two duplicate filters * Add whitelist rules for several sites - Dafont (font shopping and typesetting tests) - Fontstruct (idem) - Fontshop (idem) - SignBank (transcription of sign languages) * More exceptions for well-known sites GMX Mailbox makes extensive use of the SID parameter and will not work otherwise. It will refuse to load importart content or enter a redirection loop. The other rules are for Reddit, maps in Facebook, Google-based embedded maps, and the Ubuntu wiki. * Innocuous image parameters to whitelist Seen in https://incubator.wikimedia.org/wiki/Wp/ase/AS10002S1f548M519x514S1f548481x490S10002489x487_AS1f550S15a37S20e00S26502M531x512S15a37501x488S1f550507x495S20e00487x499S26502469x498 but might appear in typesetting pages and custom banner pages, too. * More general-purpose changes - Add more whitelisted URL parameters to the general image filter, in particular, PHP parameters used by Oracle's Site-Satellite cache. - Improve support for maps embedded in Facebook. - Trim unnecessary URL parameters in embedded Google maps. * Clearer and louder whitelist rules Most whitelist rules are now separate from the general image filter. They're also logged by default. The URL exclusion in the general image filter is best used for requests that will be processed by another filter. * IMPORTANT CHANGES - Whitelist rules completely reworked: they're no longer baked into the filter rules, and are now more specific, easily disabled, and more helpful (they are also always logged): * GMX web client whitelisted (uses `sid` parameter); * DuckDuckGo whitelisted (helpful, and built with privacy in mind); * CAPTCHAs are logged as a whitelist rule now, ensuring no parameters are removed when creating a new account; * user avatars and user karmas are a logged whitelist rule now for similar reasons; * Reddit external previews are whitelisted by necessity; * Youtube seekbar and thumbnail previews are folded into a single whitelist filter now. - New filter to anonymize Reddit's banner and community images. - New filter to avoid image downsamplers: when enabled, retrieves the original picture from the original domain. Can be disabled. - New filter for cdn.embedly.com (an unnecessary wrapper for embedding videos seen, by instance, in the site Know Your Meme) - Google Street View filter disabled by default as it breaks Street View in Firefox ESR (works correctly in the most current desktop Firefox). - Redundant filter deleted (for URL paramater `fbclid`). - General image filter tweaked: * whitelisted parameters for a couple systems of signed URLs, like Amazon's (https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-urls.html) and Facebook's (several URL parameters beginning with `_nc_`); * whitelisted innocuous parameters that don't break webpages, but result in a much longer log, like `*style`, `version`, `preview` and `i10c` (this last one is used for downsampling user avatars); * removed `url` from the whitelist (it will break some image in some random website but it is NOT worth it); * removed `wpCaptchaId` and `userid` as they are covered by the whitelist rules for user avatars and CAPTCHAs now. * Improvements for VK, LinkedIn, and Facebook and others - Some site-specific anti-redirector filters were merged into a generic one. This new filter is effective against social network VK's redirector. - Tags are now more descriptive and unique for every filter. - Facebook brands: * WhatsApp Web filters changed to fit the new avatar URLs. * Instagram's redirector is now accounted for. * More URL parameters used by Facebook blacklisted. `igshid` blacklisted globally. - Removed `wprov` parameter by Wikipedia. - Whitelisted more parameters for a different syntax for Amazon searches - The optional YouTube filter now also blocks the "watchtime" images (unblocked recently at EasyPrivacy and AdGuard because of problems with logged accounts) - General image filters: * Exceptions to this filter are clearer now. * Whitelisted URL parameters `bg` and `fg` (background and foreground colour) and `latex` (used for at least one LaTeX renderer, at WordPress). Also whitelisted `quality`, `sign`, `ssl`, `token-hash` and `token-time` (some of these are necessary at VK). * A second filter added for a gallery syntax (a PHP script to select an image based on numeric IDs). `uuid` is removed from the whitelist on the first filter. - Whitelist rules: * Added rules for YouTube icons on profile pages (low impact) and LinkedIn (very high impact). * Images from `outlook.office.com` (Outlook's web client) and `www.osapublishing.org` (a site publishing scientific papers) are now allowed in same-domain policy. * Images from GettyImages, iStockPhoto, ImageBank and AltMetric are now allowed globally. * Tweaked rule for whitelisting avatars. * Rule for MoinMoin-powered wikis is now global. * ReCAPTCHA rule merged into general CAPTCHA rules. * Whitelisted maps and street view for Google, Bing, and HERE * General image filters are now more granular and based on regexps Also, more whitelist rules, and tweaks to other rules. * Image server from LinkedIn is now whitelisted URLs are very different, often change, and never use optional query parameters. Whitelisting the whole server will not cause the browsear to leak extra data to LinkedIn. * Add filter for LinkedIn URLs This filter should cut tracking parameters from hyperlinks at LinkedIn. Exceptions for special pages like lost password retrieval have been made. Also, tweaks to other filters. * Unbreak Disqus and LinkedIn, and tweak other rules * Unbreak Amazon, Linkedin and Google -Amazon's cart -Linkedin's password management and recovery screens -Google's reCAPTCHA Co-authored-by: Geeknik Labs <[email protected]> Co-authored-by: crssi <[email protected]>

securingmom · 2021-01-05T12:03:13Z

My love to all of you

crssi mentioned this issue Jun 23, 2017

Case Sensitive problem tumpio/requestcontrol#16

Closed

Atavic mentioned this issue Jun 27, 2017

Investigate link obfuscation/redirection/unwanted parameters/refferal links cleaning pyllyukko/user.js#234

Closed

Atavic mentioned this issue Jul 21, 2017

Remove other generic tracking parameters from all urls idlewan/link_cleaner#16

Open

crssi mentioned this issue Sep 30, 2017

wiki: Skip Redirect : ELI5 #249

Closed

Thorin-Oakenpants closed this as completed Oct 6, 2017

Forsaked mentioned this issue Jan 27, 2018

sticky: extensions #294

Closed

crssi mentioned this issue Mar 31, 2018

SIMPLE QUESTIONS tumpio/requestcontrol#60

Closed

earthlng mentioned this issue May 29, 2018

EvilCorp™ AMP #424

Closed

ArenaL5 added a commit to ArenaL5/requestcontrol that referenced this issue May 17, 2020

Add more terms to the blacklist of general parameter trimming

7c79e87

As detailed in arkenfox/user.js#149 (comment) Co-authored-by: crssi <[email protected]>

ArenaL5 added a commit to ArenaL5/requestcontrol that referenced this issue Aug 4, 2020

Add more terms to the blacklist of general parameter trimming

1b7b0aa

As detailed in arkenfox/user.js#149 (comment) Co-authored-by: crssi <[email protected]>

ArenaL5 added a commit to ArenaL5/requestcontrol that referenced this issue Aug 4, 2020

Add more terms to the blacklist of general parameter trimming

ab399c6

As detailed in arkenfox/user.js#149 (comment) Co-authored-by: crssi <[email protected]>

Request Control Filters #149

Request Control Filters #149

Comments

crssi commented Jun 21, 2017 • edited by Thorin-Oakenpants Loading

crssi commented Jun 21, 2017 • edited Loading

crssi commented Jun 21, 2017 • edited Loading

crssi commented Jun 21, 2017 • edited Loading

crssi commented Jun 21, 2017 • edited Loading

crssi commented Jun 21, 2017

crssi commented Jun 21, 2017 • edited Loading

crssi commented Jun 21, 2017 • edited Loading

Atavic commented Jun 21, 2017 • edited Loading

earthlng commented Jun 22, 2017 • edited Loading

crssi commented Jun 22, 2017

earthlng commented Jun 22, 2017 • edited Loading

earthlng commented Jun 22, 2017

crssi commented Jun 22, 2017

crssi commented Jun 22, 2017

crssi commented Jun 22, 2017

crssi commented Jun 22, 2017 • edited Loading

crssi commented Jun 22, 2017 • edited Loading

Atavic commented Jun 22, 2017 • edited Loading

crssi commented Jun 22, 2017

Atavic commented Jun 22, 2017 • edited Loading

earthlng commented Jun 24, 2017 • edited Loading

earthlng commented Jun 24, 2017 • edited Loading

GitCurious commented Jun 25, 2017

Forsaked commented Jun 26, 2017

earthlng commented Jun 26, 2017

southwindcg commented Jun 27, 2017 • edited Loading

Atavic commented Jun 27, 2017

crssi commented Jun 30, 2017

crssi commented Jun 30, 2017

crssi commented Jun 30, 2017

crssi commented Jul 11, 2017

crssi commented Jul 11, 2017

crssi commented Jul 11, 2017

Atavic commented Aug 3, 2017

crssi commented Aug 8, 2017

crssi commented Aug 8, 2017 • edited Loading

Atavic commented Aug 8, 2017 • edited Loading

crssi commented Aug 9, 2017

crssi commented Aug 18, 2017

crssi commented Sep 29, 2017

earthlng commented Oct 14, 2017 • edited Loading

securingmom commented Jan 5, 2021

crssi commented Jun 21, 2017 •

edited by Thorin-Oakenpants

Loading

crssi commented Jun 21, 2017 •

edited

Loading

crssi commented Jun 21, 2017 •

edited

Loading

crssi commented Jun 21, 2017 •

edited

Loading

crssi commented Jun 21, 2017 •

edited

Loading

crssi commented Jun 21, 2017 •

edited

Loading

crssi commented Jun 21, 2017 •

edited

Loading

Atavic commented Jun 21, 2017 •

edited

Loading

earthlng commented Jun 22, 2017 •

edited

Loading

earthlng commented Jun 22, 2017 •

edited

Loading

crssi commented Jun 22, 2017 •

edited

Loading

crssi commented Jun 22, 2017 •

edited

Loading

Atavic commented Jun 22, 2017 •

edited

Loading

Atavic commented Jun 22, 2017 •

edited

Loading

earthlng commented Jun 24, 2017 •

edited

Loading

earthlng commented Jun 24, 2017 •

edited

Loading

southwindcg commented Jun 27, 2017 •

edited

Loading

crssi commented Aug 8, 2017 •

edited

Loading

Atavic commented Aug 8, 2017 •

edited

Loading

earthlng commented Oct 14, 2017 •

edited

Loading