Why cloud is not ‘basically’ a data center!
Data Center and Data Centers of Cloud Operators may look the same and even beep the same and feel the same and have the servers, storage, and networking gear from the same vendor and yet they are not the same thing.
Am I insane to suggest that the above two images don’t represent the same thing?
Let me explain — Data Centers are physical boxes that provide us the computing power, storage and networking resources that helps us run our IT systems both internal and external. The same physical boxes are in the data center of organizations and at the cloud providers. There the similarity ends. So yes if you think IT systems are about physical boxes, data centers and cloud are the same thing.
In a traditional organization, every team building their information system builds its own system from ground up, every team chooses their own compute power, operating system, storage and database and builds it from the ground up for themselves. So it is quite possible that an organization using 5 information system is using 5 different compute, OS, storage and database system. This creates multiple problems such as duplication of resources, non-standardization, non-interoperability, multiple skills requirement so on and so forth. Even where organizations agreed to standardize the duplication did not end and the results were mixed with legacy system issues and so on.
The ideal way to address this at an organization level would be to ensure that the information system builder is transparent to computing resource, storage and database. So if I want a Quad-Core 8 GB memory machine running Ubuntu OS to run my application I should be able to get it instantly without me wondering where it is going to come from physically. If I want storage space I should just be able to point my application or my compute instance to an URL and get it. Same for the database, point to a URL and get access to compute resource. These are things easier said that done. The seed of such a system was laid by Amazon in the year 2000 when it wanted to create e-commerce systems for other merchants, which later evolved to building systems that should talk to each other through APIs, which evolved to Amazon Web Services (AWS).
Jeff Bezos wrote a letter in 2002 to all his technical team at Amazon which essentially mandated
1) All teams will henceforth expose their data and functionality through service interfaces.
2) Teams must communicate with each other through these interfaces.
3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
4) It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter. Bezos doesn’t care.
5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
6) Anyone who doesn’t do this will be fired.
And that is how cloud was born. Read the genesis of AWS here.
Amazon started creating virtual machines using Xen a hypervisor (a system which lets us run multiple operation systems on a single machine, a technology created by IBM in 1967 and popularized by VMWare), and started renting out compute power to anyone who wanted it. At the same time Amazon created a object storage technology which it called S3, that allows you to store and access objects using HTTP URLs, which essentially let you store and access data as you would do in a file system. What Amazon ended up creating through AWS was a operating system for a cloud, where the user of the cloud became oblivious to where the hardware was, what the hardware was and how the hardware was being managed. The user got the computing power and storage they required as and when they required. These systems can also be setup to do automatic backups and recovery and be made to have high availability across geographical boundaries, a very very expensive proposition in the traditional setup.
As you can see the multiple servers of data center were replaced with compute instances on cloud, where you could possibly run your instances at 50% capacity utilization and replace your multiple storage paraphernalia with URL based object storage. You can increase or decrease the size of the compute instances as and when required so you don’t have to to buy the hardware keeping in view the workload 3 years from now, which may never materialize. If you are wondering why cant I do this in my data center, the answer is you can, there are copy cat cloud operating systems that are available in the market which you can use to transform your data center to a private cloud. These softwares are however not as sophisticated as the ones created and operated by Amazon AWS and Microsoft Azure and you may end up spending a lot of money and energy on a transformation project that may never yield the results you wanted. With the passage of time these softwares are likely to mature and yield the desired results.
The best thing about this system was that it could be paid for by the minute, by the hour and increased on demand, so if you were a startup looking to stream movies, or create a file storage application, you did not have to invest $10 million in creating a data center to stream the movie, you could just rent it from Amazon and be on your way. This system led to the launch of thousands of startups who were freed from the worries of buying hardware infrastructure to test their ideas.
Contrast this with countless IT projects in enterprise that were killed at budgeting stage when they put in $500,000 request for hardware and software and three full time resources to man those boxes, even though the ROI was quite high.
Almost 15 years later, cloud no longer remains about provisioning a compute server and storage on the fly, it has evolved into a much bigger beast, it has become a technology vending machine where at the insertion of a coin and press of a button you can access the best technology platform to perform any computing job on the fly without making any upfront investment, unimaginable just 5 years ago.
Next Up — How to architect for cloud and use the vending machine!
