Last Friday as I was wrapping things up, my inbox beeped with an email. One of our customers had just received their Cloudyn cost-to-date alert, informing them of their total AWS spend up until that point in the month. They had then cross-checked our alert with the billing info available via their AWS console and discovered that while we reported their actual cost at $33,577, according to AWS it was much higher at $52,000. They (and I) wanted to know what was causing the discrepancy.
No, it wasn’t a bug
While the customer and I thought it was some sort of bug, a thorough investigation by our QA and R&D teams revealed a more interesting issue. When Cloudyn calculates and reports on current AWS costs for our customers, it does so based on actual usage and pricing associated with that usage. Amazon generally does the same…except for in one case where they include your entire upcoming month’s spend, from day 1. The question is how can they know in advance what your hourly usage will be?
An interesting fact about Heavy Utilization Reservation
Unlike light or medium reservations, for Heavy Utilization, you pay the one-time upfront fee plus the hourly usage fees even if you never use any of the reserved instances (this little factoid is buried way down in the pricing section of AWS’ reservation page). So when you sign-up for Heavy Utilization, Amazon already knows how much your bill will be, as they will charge the hourly rates irrespective of your actual usage.
Our customer with the discrepancy had just ordered some new Heavy Utilization reservations. So while AWS was reporting the entire month of January’s Heavy Utilization hourly usage costs, already in the beginning of the month, our alerts reflected his actual consumption and spend up until that point. This of course, explained the difference between AWS’ monthly activity report and our alerts.
The question that remains is how AWS’ reporting affects the user.
The boy who cried wolf
While Amazon’s reporting method for Heavy Utilization reservations is to some degree accurate, it skews things for trending purposes. Costs that are only fully incurred by the end of the month already appear on the first day, creating a spike that doesn’t really exist. This kind of noise makes it difficult to practice IT change management in a meaningful way. These artificial spikes can confuse users who then become jaded and ignore anomalies that really do represent a serious breach of usage or spending. (As a side point, the customer can also sell the Heavy Utilization reservation before the end of the month, making AWS’ cost estimation report completely inaccurate.)
Here at Cloudyn we think providing accurate trending not only helps you to pick up on any problems, but to also engage in accurate capacity planning. This is why our cost reports all have amortized versions where things like one-time payments (for all reservations) and hourly usage costs (for heavy reservations), are spread out evenly during the term of the reservation or the full month, respectively.
Let us know what you think in our comments section below.
Discover how Cloudyn can transform your cloud change management – Sign-Up for FREE Now!