The Value of Background Fetch

David Smith wrote a blog post last week about Background Fetch, alerting developers to a potentially serious consequence of using it in a popular app. His points are backed up with statistics from his excellent Check The Weather app and from the server logs of his own podcast feed, Developing Perspective.

David warns developers that this new API can lead to the accidental creation of a “massively distributed botnet”. This is an important point to consider. For many apps, the issue can be mitigated if developers follow best practices. As David’s experience with his weather app shows, exceptional circumstances such as pay-per-request APIs require a more conservative approach.

Looking beyond this concern, the data also shows the effectiveness and the great potential offered by Background Fetch. It allows apps to provide users with up to date content before they ask for it. In some cases it can simplify an app’s infrastructure while removing a significant cost.

Servers are no longer mandatory.

Prior to iOS 7, creating a podcast app that could notify users of new episodes required the development of a back-end web app to regularly poll every podcast feed subscribed by any of the app’s users. When a new episode was found, the web app would contact Apple’s push notification service, which would, in turn, notify the relevant users. If any part of this system went down for any reason, the episode notification feature would be offline for all of the app’s users.

Building back-end infrastructure is a serious commitment in terms of development time and ongoing server hosting fees. These overheads matter. The economics of indie app development is a regularly discussed topic. Martin Hering, maker of Instacast posted the following on the Vemedio blog:

I think that 2014 might be the year that we’ll see more and more great apps to be discontinued. We are not immune to this development and we will need to come up with a new strategy in 2014 if we want to continue our line of products.

Our strategy with Castro has been to employ Background Fetch to help us avoid the ongoing cost of a server. Castro polls each subscribed feed from the app regularly and posts local notifications when new episodes are found. There are pros and cons to this approach.

Pros

Cons

Not all requests are created equal.

We know little about the inner workings of Background Fetch. Apps opt in but iOS decides when to initiate calls to the app, presumably taking into account variables like how often the app is used, current battery charge levels and network status. David’s post is the best evidence we’ve had so far of the effectiveness of the API in the real world. It works the way we’d hoped.

Some increased traffic should be expected if Background Fetch is working at all; the difficulty is in predicting the actual amount. In the case of Check The Weather, which uses the paid forecast.io API, this was a big concern. The opaque nature of Background Fetch meant that David was unable to predict the volume of API calls that would be made and he ended up with a larger than expected bill.

Outside of the paid API example however, these requests shouldn’t be a burden on either the device or the servers being called. We have taken care to ensure that Castro is a good citizen in this regard. Developers who ignore the responsibility to properly implement their network code do risk overloading the servers with which they communicate. Castro makes use of the "If-Modified-Since” header to ensure that every single request sent from the app results in “Not Modified” response unless the feed has been updated. The full RSS feed is only returned when a new episode is released. Amazon S3 charges about 42 cents for serving one million “Not Modified” responses, so the impact of properly implemented feed polling on podcast hosts is negligible.

In practice, caching doesn’t work.

We care about feed update speed, so we have investigated lots of other approaches to reducing unnecessary requests. Up until version 1.0.3, Castro used a far more aggressive caching strategy based on the Expires header. We changed the app when we realised that some podcast feeds are set up incorrectly, which was causing some episodes not to show up for users. All of the TWiT Netcasts, for example, set a 24 hour expiry. Users who checked for updates 1s before an episode was released wouldn’t get it until the next day.

We also investigated a predictive caching idea, where Castro figured out roughly what day and time episodes would appear and then checked the feed only at that time. This didn’t work out. 19 episodes out of 20 might come out on Friday, but if even one episode subsequently came out mid-week, Castro would miss it and users would be unhappy.

Being truly up-to-date is more important than avoiding additional requests. Most of these requests return “Not Modified” responses so feed updates are still snappy.

Conclusion

The background fetch API is a game-changer for iOS developers. It has the potential to free us of significant server and infrastructure overheads. This is particularly relevant at a time when many developers are wondering how to stay independent. For Castro, the decision was an easy one and we strongly advocate that other developers take full advantage of this new API as well.