Virtually anyone using the cloud has to consider the storage component of the process – to maintain record of information on networked storage at a data center or in virtual data centers managed for the public cloud. With network storage you can directly access where data is stored and how it is mapped. Public cloud storage has the same characteristics as cloud computing in terms of agility, scalability, elasticity and multi-tenancy. The challenge in the cloud is that the ability to access storage utilization information is not always simple in a shared environment.
For Amazon, the largest public cloud provider, the AWS Elastic Block Store (EBS) is a persistent storage offering, used in tandem with applications utilizing the AWS Elastic Compute Cloud (EC2). It is a service that offers users a place to store data that doesn’t go away when EC2 instances are dropped, and it provides the same mounted file system capability as the internal disks of a physical solution.
EBS storage instances are called Volumes, virtual disks that are created and then stored on a physical disk in a regional data center location. Volumes can be anywhere from 1 GB up to 1 TB and are typically paid for based on the amount of physical storage space at a location ($X per GB per month) and the amount of I/O requests made to the particular Volumes. Additionally, as information stored in a Volume changes, snapshot and image files are created for a particular point in time.
Why is it difficult to track storage usage in the cloud?
The problem that occurs in this environment is that as you utilize EBS storage, Volume instances, snapshots and Amazon Machine Images (AMI) related to your deployment seem to proliferate. Even a small organization can quickly get overwhelmed with tracking and correlating this array of information about cloud storage. Many organizations maintain complex Excel documents, which have to be updated daily to simply try keep on top of these various storage components.
For most businesses the primary source of information for public cloud usage is provided by Amazon CloudWatch. The CloudWatch API offers some basic information about AWS resource utilization (CPU, disk, network) but doesn’t offer any information on memory, disk space actual usage or load average metrics – a choice made by Amazon. There is a void in how to maintain control of this information.
A proposed solution
As part of Cloudyn’s analysis engine for Amazon, Cloudyn automates EBS visualization. The Cloudyn dashboard will help track the multiple AMIs, snapshots – locating forgotten data, information and automatically create a full listing of the storage usage. The collection of metadata about the entire cloud deployment can, at minimum, simplify discovery and maintenance of information in EBS. With this visibility, the dreaded spreadsheet is eliminated and enterprises can start to make some initial decisions about what stored information is necessary to be kept and what information is no longer relevant. We are also starting to help organizations gain some basic virtualization of the “relationships” between EBS objects.
Moving forward, Cloudyn is researching additional methods to gain even deeper insight into the stored data and helping create clear recommendations for optimizing costs around storage. The relational visualization will also be enhanced as we look deeper into the topology of an organizations storage assets.
Are you finding the management of cloud storage to be burdensome for your business? Has public cloud storage spiraled out of control? What do you feel would help minimize any frustrations with the management of storage via EBS?
Please let us know. Cloudyn would love your insight on the areas in which we could help to create greater control of your public cloud storage solutions.