The Unofficial Google App Engine Price Change FAQ
I don't work for Google, but I read the mailing lists and pay attention. Also, I show up at places where Google buys beer. Here's what I've learned:
Google is changing the way it charges for App Engine. Previously, you were charged for three things:
- Bandwidth in/out
- Data stored in the datastore
- "CPU time" of all your programs
The new billing model charges you for:
- Bandwidth in/out
- Data stored in the datastore
- Wall-clock time spent running application server instances
- Number (and type) of requests to the datastore
In addition, there is a 15-minute charge ($0.02) every time* an instance starts and $9 per application per month (as a minimum spend) if you enable billing.
* It isn't quite "every time". See this thread for more.
For most GAE applications, the biggest charge has always been "CPU time". This number was composed of two parts:
- CPU time directly consumed by your frontend web application instance
- A crude approximation of CPU time consumed by the datastore and other APIs ("api_cpu_ms")
In fact, api_cpu_ms has never been a real measure of CPU activity - the numbers are based on simple heuristics like "each index write costs 17ms". So the change from api_cpu_ms billing to per-request billing for the datastore really isn't much of a change.
The most significant change to the pricing model is that instead of billing you for CPU time consumed by your frontend web application, you will now be charged for every minute of wall-clock time that each instance runs, irrespective of how much CPU it consumes.
The old pricing model charged for the wrong thing. The overwhelming vast majority of web applications use very little CPU; processes spend most of their time blocked waiting for I/O (datastore fetches, url fetches, etc). The App Engine cluster is not limited by CPU power, it is limited by the number of application instances that can be fit into RAM. This is particularly problematic with single-threaded instances like the current Python servers and Java servers without <threadsafe>true</threadsafe>, which require multiple instances to serve concurrent requests.
Charging for CPU time created architectural distortions. Imagine your single-threaded Python application makes a URL fetch that takes 2s to complete (say, Facebook is having a bad day). You would need 200 instances (consuming large quantities of precious RAM) to serve 100 requests per second, but you would pay almost nothing because instances blocked waiting on I/O consume nearly no CPU time. Google's solution was simply to refuse to autoscale your application if average request latency was greater than one second, essentially taking down your application if third-party APIs slow down.
Because the new instance-hour pricing model more accurately reflects real costs, App Engine can now give you those 200 instances -- as long as you're willing to pay for them.
No. You're still being charged for resources you consume, but now you're being charged for the scarce resources that matter (occupied RAM) rather than the overabundant resources that are irrelevant (CPU time).
It's tempting to imagine that Google can just add instances to a machine until it runs out of RAM, then start adding instances to the next machine. In practice, you can't architect a system like this. Your frontend instance may use 20MB now but nothing stops it from growing to 100MB without warning. If a couple application instances did this suddenly, it could push the box into swap and effectively halt all instances running on it. An application instance must reserve not just the RAM it actually uses, but the RAM it *could* use. Oversubscribing creates the risk of incurring unpredictable performance problems, and at the huge scale of App Engine, even low-sigma events become inevitable. I suspect that Google is very conservative about oversubscribing RAM reservations, if they do it at all.
Almost certainly, especially if you are using single-threaded instances (Python, or Java without <threadsafe>true</threadsafe>). Google really was charging an absurdly low price for App Engine before, letting us occupy many hundreds of megabytes of RAM for pennies a day. It was nice, but it wasn't sustainable.
It depends. If you're looking at just the cost of computing, then yes GAE will be more expensive than services like AWS. On the other hand, you can run large-scale applications without the need to hire a system administrator or DBA - so that frees up a couple hundred thousand dollars per year from the budget.
It's also really hard to make an apples-to-apples comparison. With the high-replication datastore, GAE provides a redundant, multihomed, fault tolerant system that can transparently survive whole datacenter crashes. It's already happened. Setting up an equivalent system requires significant engineering effort, and you have to pay someone to wear a pager.
As App Engine has matured, it has gone from being a low-end hosting solution to a high-end hosting solution. Compared to a VPS at dreamhost, App Engine is very expensive. Compared to building your own HRD, App Engine is still comically cheap.
Google has created an article for this, but here's some blunt advice.
There are two aspects to this, driven by the two separate aspects of billing:
- Lower the number of instances your app needs
- Reduce your datastore footprint
Most developers freaking out about their bill on the appengine mailing list are Python users shocked by the number of instances running to serve their application. You may be able to optimize this somewhat by tuning the scheduler but this will at best provide a small improvement. The stark reality is that single-threaded web servers are a huge expensive waste of resources. Processes that occupy big chunks of RAM while they sit around blocked on I/O don't scale cost-effectively.
The only practical way to significantly lower your instance count is to use multithreading. This allows one instance to serve many concurrent requests, theoretically right up until you max out a CPU. Dozens of threads can block on I/O within a single process without consuming significant additional chunks of precious RAM.
- If you are using Java, put <threadsafe>true</threadsafe> in your appengine-web.xml NOW
- If you are using Python, beg/bribe/extort your way into the Python 2.7 Trusted Tester group
If you have turned on multithreading, it's unlikely that the scariest line item in your bill will be instance-hours. Instead you will be wondering why you are being charged so much for the datastore. The good news is that there's nothing new about optimizing datastore usage, this is what you should have been doing all along:
- Cache entities in memcache when you can.
- Remove unnecessary indexes.
- Denormalize where you can. It's much cheaper to load/save one fat entity than 20 small ones.
The scheduler is a red herring. It may very well have issues, but no amount of tweaking the scheduler will change the fact that in order to serve multiple concurrent requests with a single-threaded process, you need to run multiple instances. At best, the scheduler can trade off a crappy user experience for a lower bill.
Forget about the scheduler. Turn on multithreading ASAP.
Google says that higher prices will allow them to increase their commitment to App Engine and devote more resources to its development. This is probably true. If you're building a business (as opposed to a hobby), the new pricing is probably not going to make or break you; salaries and overhead are probably still your biggest concern. However, the addition of significant new features (say, cross-entity-group transactions or spatial indexing) could allow you to improve your product in ways that were too expensive or difficult before. To the extent that more money means more features sooner, paying more might be worth it. Time will tell.
