Clouds and Innovation

If I were to launch a new business today, I'd be bombarded with the latest and greatest cloud technologies and how they'll make my life better. I'll save on hardware costs because I can virtualize; I'll save on operational overhead because I can operate everything I need to within a single dashboard; I can scale up when demand necessitates.

We all know that marketing doesn't reflect reality — entirely. We can do a lot of the purported cloud things that we love to talk about, but there are a lot of other complexities that pop into existence as a result of the cloud culture. You've now a deluge of product and configuration options to sort out. Of these, which assist with what you're trying to accomplish from a business perspective? Not all of it, but certainly some of the cloud product capabilities are relevant to your cause.

What I find interesting about cloud is that we're in a hurry to design and build this mega-infrastructure in a box. An impossible task by most sane measurements, but we're trying. It's not as though we've an end goal — there is finish line marking the perfect cloud product that everybody must own. But we're trying. And through all this, even the development of features you're not necessarily interested in as they don't directly impact the quality of your business today, we're witnessing some really cool innovations. What impact will this race to building the cloud have on software development projects in the future?

New Contexts
As it turns out, solutions to problems are often applicable in contexts outside that of which the problem originated. Especially in IT where software is such a fluid contrivance — easily replicated and easily changeable. One would think with all the software frameworks and libraries out there today, that we would be capable of creating something generic and useful enough to tackle any problem we encounter. Just make a few configuration changes, and, you've got the answer to your problem. That, however, could never work because problems that need solving proliferate much faster than software. There is no way that we could design something generic enough to keep pace.

We do have some really good frameworks and commercial products in the technology domain that weren't born out of the desire to tackle every single problem by means of being all-encompassing. Many successful projects that are somewhat generic today, that is, they can be applied in several contexts, were conceived to solve an smaller problem.

Django is perhaps the easiest example I can point to in the Python world. This is the de facto web framework for Python programmers, but that certainly wasn't Django's goal when it was first conceived. Django, in it's early days, had the more modest goal of providing the facilities necessary to rapidly build web applications for newspaper content. I won't drill down into all the right decisions that were made over the course of Django's history that gave it such popularity in the Python web community. I will, however, reiterate that because newspaper applications needed to be developed quickly, we now have a web framework that has been used to develop a wide array of web applications.

It's always interesting to watch solutions to problems applied in new contexts. The most successful solutions to technology problems are almost always adaptations from other contexts. The innovations that are currently under way in cloud technology most certainly have practical implications for software design and development in general.

Principles
Sometimes, a technology solutions doesn't have a one-to-one translation in a context different from the problem they were meant to solve. Sometimes, you have to derive principles from the solution, and these are valuable. Principles gained from innovating could be as simple as lessons learned. We know what not to do because such and such went wrong in environment this and that. The bi products of innovation are limitless — sometimes they're real software components that have been simplified and generalized for the masses. Sometimes its general knowledge.

In the case of cloud technology, we're trying to solve a number of problems that wouldn't otherwise prevail. Scalability, for instance, is an important aspect to cloud infrastructures. How do you accommodate such vast amounts of data, coupled with fluctuating demand, and fortify architectural quality properties? Interoperability — there is a cloud ecosystem out there, and vendors need to emulate one another's API. Migratory data. Moving large chunks of data around networks while retaining consistency isn't an easy problem to solve.

What's interesting about cloud technology is that we're seeing the inverse of the regular innovation pattern. Cloud, depending how you want to define it, solves a general problem. So the long-lasting principles that will someday be beneficial in other contexts, weren't aimed at solving a specific problem as Django did in the newspaper industry. I will say though, there are a lot of hard technical lessons to be learned during this innovation period in cloud technology. We're in the incubation period for ideas and principles that will find themselves in many future software systems.