Prepare for NetworkService migration

[iefremov]

Chromium team is in the middle of a big project, called servicification [1]. One of the most significant parts of it is transition to the Network Service [2].

One of the goals of moving to NS is to avoid storing information in ProfileIOData (i.e. cookies), so ProfileIOData will eventually become deprecated. At the moment, the core code of brave shields brave_shields_util.cc depends on ProfileIOData, so it needs to be reworked in the future. `

ChromeNetworkDelegate itself probably will also be deprecated in favor of NetworkServiceNetworkDelegate

Deadlines or milestone dates of [1] and [2] are not yet clear.

[1] https://www.chromium.org/servicification
[2] https://bugs.chromium.org/p/chromium/issues/detail?id=598073

[iefremov]

@tomlowenthal FYI

[zorae]

Can confirm that Shield becomes ineffective with chrome://flags/#network-service enabled.

[iefremov]

ChromeNetworkDelegate is not functioning with NetworkService, it seems we should start with replacing it witih NetworkServiceNetworkDelegate. ProfileIOData will stay with us for a while.

[bbondy]

Please revert this as part of this work:
brave/brave-core@9257ced

[bbondy]

upgraded to p1 priority

[iefremov]

As part of migration we have to fix Tor - it doesn't work with enabled NetworkService. It seems that the reason is TorProfileServiceImpl::SetNewTorCircuitOnIOThread: URLRequestContext is not compatible with NetworkService @darkdh

[iefremov]

Filed #4312

[simonhong]

Failed browser test with NS on MacOS.

    AdBlockServiceTest.AdBlockThirdPartyWorksForThirdPartyHost (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:649)
    AdBlockServiceTest.AdsGetBlockedAfterDataFileVersionUpgrade (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:393)
    AdBlockServiceTest.AdsGetBlockedByCustomBlocker (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:290)
    AdBlockServiceTest.AdsGetBlockedByDefaultBlocker (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:244)
    AdBlockServiceTest.AdsGetBlockedByRegionalBlocker (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:335)
    AdBlockServiceTest.BlockNYP (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:671)
    AdBlockServiceTest.CancelRequestOptionTest (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:792)
    AdBlockServiceTest.NewTabContinuesToBlock (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:474)
    AdBlockServiceTest.SocialButttonAdBLockTagTest (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:693)
    AdBlockServiceTest.SubFrame (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:509)
    AdBlockServiceTest.TagPrefsControlTags (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:744)
    AdBlockServiceTest.TrackerReferencedFromUntrustedDomain (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:573)
    AdBlockServiceTest.TwoDiffAdsGetCountedAsTwo (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:450)
    AdBlockServiceTest.TwoSameAdsGetCountedAsOne (../../brave/components/brave_shields/browser/ad_block_service_browsertest.cc:426)
    BraveNetworkDelegateBaseBrowserTest.ThirdPartySTS (../../brave/browser/net/brave_network_delegate_hsts_fingerprinting_browsertest.cc:93)
    BraveNetworkDelegateBrowserTest.Iframe3PCookieBlocked (../../brave/browser/net/brave_network_delegate_browsertest.cc:70)
    BraveNetworkDelegateBrowserTest.IframeJs3PCookieBlocked (../../brave/browser/net/brave_network_delegate_browsertest.cc:87)
    HTTPSEverywhereServiceTest.RedirectsKnownSite (../../brave/components/brave_shields/browser/https_everywhere_service_browsertest.cc:101)
    HTTPSEverywhereServiceTest.RedirectsKnownSiteInIframe (../../brave/components/brave_shields/browser/https_everywhere_service_browsertest.cc:127)
    TrackingProtectionServiceTest.TrackerReferencedFromUntrustedDomainGetsBlocked (../../brave/components/brave_shields/browser/tracking_protection_service_browsertest.cc:169)

[iefremov]

The current plan is to implement our own ProxyingURLLoaderFactory similar to WebRequestProxyingURLLoaderFactory, since this is the only logical interface for our Shields machinery after enabling NetworkService.

There is a draft PR with this factory, cargoculted/copypasted from two existing Proxying factories in Chromium (brave/brave-core#2701), not sure if it is possible to reduce the amount of boilerplate, but probably stuff connected to TrustedHeaderClient will not be needed.

Plumbing easiest individual shield callbacks seems working (not presented in the PR), though porting the whole loop of callbacks that occasionally redirect requests is somewhat bigger task, which I'm doing right now.

There is one thing that could be done in parallel with the current stuff - dealing with AdBlockInterceptor, which is a part of our ad-blocking mechanism. In the current shields architecture, we set some special header during the ad-blocking session in NetworkDelegate and later craft an empty 200 OK response in the interceptor. NetworkService doesn't work with URLRequest interceptors, instead, some custom URLLoaderFactory should be provided, which probably should be different from the Proxying one described above. it seems it should be replaced with custom URLLoaderRequestInterceptor or NavigationLoaderInterceptor paired with implementation of mojom::URLLoaderClient

[iefremov]

Fixed in
brave/brave-core#2988
brave/brave-core#3245