Background NSURLSession behaviour

[felix-schwarz]

Following reports by @hosy and @tomneedham, I've been investigating the behaviour of background NSURLSession, which all evidence so far pointed to as the cause for huge delays and a seemingly stuck Sync Engine.

Findings

Here's what I found after running into the issue during testing myself:

1. The download request for San Francisco.jpg is scheduled in the NSURLSession at 2019-04-10 13:41:40 by way of calling the NSURLSessionTasks resume method.

2. The download returns at 2019-04-10 13:49:35 - almost 8 minutes later! The look at metrics shows that the NSURLSession did in fact wait so long before actually sending the request:

Possible cause

Going through Apple's documentation on background NSURLSessions, this paragraph looks relevant:

(Source)

Except that the app did not actually fulfill the criteria and stayed in the foreground all the time - and requests also didn't start immediately when putting the app in the background and foreground again (which, according to this doc, should reset any penalty).

My best guess is that the FileProvider's use of background NSURLSessions is somehow counted as background activity towards the app itself - and leads to penalties even if the app is in the foreground.

The documentation also points out the existence of a Background Networking profile.

I installed that profile, reproduced the issue and then took the sysdiagnose recommended by the accompanying instructions. However, I could not find anything relevant in it yet.

As far as a solution is concerned, I'll try following this advice from Apple's documentation:

You don’t have to do all background network activity with background sessions as described in this article. Apps that declare appropriate background modes can use default URL sessions and data tasks, just as if they were in the foreground.

That is, except for the File Provider, which Apple explicitly tells us should use background sessions.

I'll update this issue as work on it progresses.

[felix-schwarz]

Another interesting observation: the background NSURLSession used by the File Provider does not seem to be affected in any way. Just the background NSURLSession that was used by the app.

[felix-schwarz]

Some interesting insights here: http://c-sharx.net/2015-11-25-ios-background-transfer-what-about-uploads

[felix-schwarz]

Upon further inspection, I found two places where the app might - other than thought previously - actually have fulfilled the criteria for being penalized:

• background item list updates: these were using requests scheduled with an earliestBeginDate date in the future and, when that future date arrived, scheduling new requests immediately after getting the response
• uploads: uploads were scheduling a PROPFIND on the uploaded item right after finishing the upload

Solutions I'm putting in place:

• background item list updates now use a local NSURLSession:
  • using OCBackgroundAction to keep the app awake in the background for already scheduled requests
  • using OCBackgroundManager to schedule sending these requests to the local queue only when the app is in the foreground (a new feature of OCBackgroundManager can queue blocks if the app is backgrounded - and runs them once the app is in foreground again)
• single-item PROPFINDs after a successful upload now also use a local NSURLSession, avoiding the danger of triggering penalties from iOS

[felix-schwarz]

More findings:

• Setting the NSURLSessionConfiguration.networkServiceType to NSURLNetworkServiceTypeResponsiveData seems to increase responsiveness
• NSURLSession seems to behave differently when run on the device via Xcode vs. when launched by the user on the device

[felix-schwarz]

One of the insights gained from the blog post at http://c-sharx.net/2015-11-25-ios-background-transfer-what-about-uploads is that background NSURLSessions seem to have a global transfer budget, so that it's desirable only to use the background NSURLSession if using a local NSURLSession is not an option.

As of cead9cf, OCHTTPPipeline saves metrics and can provide estimates (complete with confidence levels) based on that. That estimate is then used by OCConnection to make a choice between local and background NSURLSession: if a transfer is likely to take less than two minutes (safely within the boundaries of available background execution time) with at least 15% confidence, it'll use the local session. More details in the commit message of cead9cf.

[felix-schwarz]

Closing this issue as it has been addressed at the time of writing.