Drivers Of Cloud Technology

Who ultimately makes the decisions on what features cloud service providers make available?  It isn't as though these folks base these decisions on what customers actually need, is it?  We'd certainly like to hope so anyway.  At the very least, service providers should build features that reflect the characteristics of our favourite applications and frameworks.  Maybe this is in the form of appliances — pre-configured components that play a small role in an application's architecture.  Software should be the driver of cloud technology.

Amazon's EC2 is the best example of a cloud technology derived from another domain.  Amazon built EC2 as a product because they were able to reflect on the challenges they themselves faced with building large and complex applications.  They saw what the future of cloud computing looked like not only for Amazon, but for almost every other technology company in the world.  Only now is Amazon associated with technological prowess — a far cry from their core business only a few years ago.  Go figure — a book store played a key role in what cloud offerings look like today.

Cloud technology derived from a domain — an online store — is one avenue to innovation.  Another is the technology that utilizes the cloud.

But is this in fact a motivator for mature cloud service providers?  Are they paying attention to the consumer technology taking advantage of cloud services?  See, a key issue with cloud stuff is over-generalization — trying to satisfy every single use case with only a handful of features.  I think part of this conundrum is with virtualization — how the providers can opperate profitably.  It doesn't cost providers much in the way of overhead to spin up new virtual machines.

These virtualized, on-demand environments are ideal if you need these resources and that the provisioning of these resources can be automated.

What I think is missing from the big picture here is the consumer application.  Actually, forget about that — what about the application framework?  Since applications typically sit on top of frameworks, shouldn't the framework have a say with regard to what the cloud proffers?  I think so.  Because ultimately, no developer with framework expertise is going to abandon the bona fide methodologies.  This of course presupposes that the cloud can offer something application frameworks want.  What?

Virtual machines are just one way to make use of on-demand resources.  The cloud is our application as this is where it runs.  However, one single virtual machine isn't going to be of great benefit.  Why not just run it on a dedicated host somehwere?  We've got to utilize the cloud resources made available to us, and this is exactly why virtual machines are handy — they consume their own resources.  An obvious example of this is running your database server in it's own virtual machine.

These specialized virtual machines imply that we've dissected our application — if not the framework too — insulating components from one another.  Of course, this isn't easy to do.  Some frameworks have separately deployed components by nature.  Take Django for instance.  A typical Django deployment might have three virtual machines — one for the Django application itself, one for the database, and one for a memcache server.  Each of these components can be expanded further — maybe the database has one or more slave virtual machines or maybe there are 3 memcached servers.  Django's architecture makes it easy to discern the separable components that can safely run in an isolated environment.  But beyond this, we've got a challenging task ahead of us.

Imagine a Django development shop that specializes in Django websites and applications for companies.  They have large clients and thus need to scale their deployments.  For this, I think this shop needs an easy way to deploy the types of configurations discussed above — a Django application, along with database and cache nodes.  Instantiating this type of deployment requires extra effort by the development shop.  They can't simply click a deploy button without a lot of ancillary work in developing a deployment tool that'll go through the process of launching these virtual machines.  This is time this shop will not be spending on creating great software for their clients.

So what does the cloud offer for third-party consumer software?  A whole lot of hardware resources.  They're made available on demand as this is the reality for any application — traffic peaks and traffic slumps.  Virtual machines are great in their flexibility — we can take any component of our application and deploy it independently.  With this flexibility comes a cost however — the cost of becoming cloud experts.  I think the best thing cloud vendors can do is listen to the technology.  What is the most efficient way to deploy a typical Drupal site?  What would make the lives of Django development shops easier?  This is what should drive the next steps in making cloud computing accessible to everyone.

Problem Design Influences

When designing software, we are trying to solve some problem at hand. That problem could be as simple as writing a command line utility that does nothing more than export a database table and send it somewhere. It could also be a very complex problem involving the simulation of another complex system that exists in the real world. The more complex the problem, the more effort that is needed for designing the solution. The question is, to what degree is the design complexity influenced by the complexity of the problem?

One great feature of object-oriented software design is the ability to look at a problem from a reductionist perspective. We are free, as developers, to create abstractions at any level of specificity. The ultimate goal in software design being to take a complex problem and make it a little more understandable and approachable by designing abstractions. This provides a view into the problem from many different perspectives.

You almost never hear about great software designs for trivial problems. This is because the implemented solution is in fact trivial and takes no time at all grasp logically. Instead, you hear about how very large, complex problems were tackled and the end result was an elegant implementation. In these implementations, you're likely to find some abstractions that you don't see every day. This is because it wasn't an every day problem; the problem had a positive influence on design. The design is interesting because it is simplistic relative to the daunting problem it solves.

So does this mean that simple problems and simple solutions are any less interesting? I tend to think not. Simple problems weren't always simple. Over time, they turned into simple problems by being removed from a larger problem. It might make sense to re-evaluate whether or not you are in fact trying to solve the right problem before it becomes a complete glutton.