Data is fast becoming a hot commodity—individuals, organizations and their devices require ever more computing capacity to process and store it, and much of the soaring demand is being met by cloud services.
According to Mark Whitby, Senior Vice President at Seagate, 2016 is set to mark the year when the world produces more digital information than it can easily store. And he predicts a minimum capacity gap of over six zettabytes by 2020. That’s one billion terabytes or approximately 1021 bytes.
Why Are We Running out of Capacity?
First off, companies are realizing the huge potential of big data, which can help them understand consumer behavior, better target their marketing strategies and build customer trust. Second, people increasingly want more interconnectivity and synchronization between personal computers and handheld devices. On top of that, the Internet of Things is driving closer integration between smart equipment, machinery and appliances—from traffic management systems, home automation and energy management to intelligent shopping systems.
However, the problem is that it’s far easier to generate data than manufacture the capacity to handle it. The cloud has been a major driving force in the recent digital data explosion. But is it now struggling to keep pace with the demand it’s created?
Can AWS Bridge the Gap?
According to latest Gartner estimates, AWS now has ten times the capacity than its 14 nearest cloud competitors combined. So if any cloud services provider is in a position to bridge the infrastructure capacity gap, it’s AWS. However, some users have discovered that even the market leader runs out of space from time to time. Although, as they report on the AWS forum, when confronted with an instance capacity error message they’re still able to request an instance elsewhere in the network—in either another Availability Zone or a different region.
At present, these incidences shouldn’t be cause for alarm, as they clearly represent transient capacity problems in specific areas of Amazon’s service rather than across its infrastructure entirely. This also makes good business sense for the company, as running to tight operational margins is more economical than providing capacity that customers aren’t actually using.
By contrast, a far greater clue to Amazon’s ability to meet demand is the phenomenal growth of AWS itself. Since its official launch in 2006, it has grown into a service with one million active customers. By 2014, EC2 usage was doubling every year. And S3 was growing even faster, with a 132% year-on-year increase in data transfers.
The pace of innovation is equally staggering. In 2014 alone, AWS added 449 new services and major features. Every day it adds more server capacity than its entire provision back in 2006. Through its partnership with Intel, it uses custom high-performance processors and server architectures that aren’t available anywhere else in the market. It even designs and builds its own power substations, as utility companies simply cannot keep up with Amazon’s rate of development.
Can AWS Sustain this Level of Growth?
Traditional silicon technology is now being pushed to the limit. It’s becoming ever more difficult to squeeze extra computing power into the same physical space. New alternatives to silicon, such as DNA, optical and quantum computers, may take years to make it into the commercial mainstream. So, in the meantime, it’ll be down to cloud service providers to make the most efficient use of what computing capacity is available. And, as market leader, Amazon is best placed to meet the challenge. The cloud giant certainly has scale on its side. In 2015, AWS took a whopping $7.88 billion in total revenue—an increase of 71.7% from the previous year. So, arguably, it should have the resources to keep investing in and developing innovative solutions at scale.
How Would a Capacity Shortage Play Out?
If it turns out AWS does struggle to cope with the demand, its charges could quite possibly skyrocket. At the same time, limited supply could help open the door to the competition—as customers go in search of available capacity elsewhere.
But much the same could happen to the cloud computing sector as it did in the oil industry. As prices went up, anything that relied on oil became more energy efficient. So, in the computing industry, we could see consumers making more efficient use of the cloud. This could see a return to the days, back in the 60s and 70s, when early computer users were confined by the limited processing resources available.
Finally, if prices do go up and availability goes down then we could see a significant change in the interrelationship between cloud services providers and their customers. Some consumers will demand more flexible contracts with reduced lock-in periods, and the market is already looking at multi-cloud adoption.
But, in its first 10 years, Amazon’s ability to scale up with demand has been nothing short of remarkable. So, given its track record of success, the cloud computing industry may well be set for an entirely different and more positive future.