Clouds and Innovation

If I were to launch a new business today, I'd be bombarded with the latest and greatest cloud technologies and how they'll make my life better. I'll save on hardware costs because I can virtualize; I'll save on operational overhead because I can operate everything I need to within a single dashboard; I can scale up when demand necessitates.

We all know that marketing doesn't reflect reality — entirely. We can do a lot of the purported cloud things that we love to talk about, but there are a lot of other complexities that pop into existence as a result of the cloud culture. You've now a deluge of product and configuration options to sort out. Of these, which assist with what you're trying to accomplish from a business perspective? Not all of it, but certainly some of the cloud product capabilities are relevant to your cause.

What I find interesting about cloud is that we're in a hurry to design and build this mega-infrastructure in a box. An impossible task by most sane measurements, but we're trying. It's not as though we've an end goal — there is finish line marking the perfect cloud product that everybody must own. But we're trying. And through all this, even the development of features you're not necessarily interested in as they don't directly impact the quality of your business today, we're witnessing some really cool innovations. What impact will this race to building the cloud have on software development projects in the future?

New Contexts
As it turns out, solutions to problems are often applicable in contexts outside that of which the problem originated. Especially in IT where software is such a fluid contrivance — easily replicated and easily changeable. One would think with all the software frameworks and libraries out there today, that we would be capable of creating something generic and useful enough to tackle any problem we encounter. Just make a few configuration changes, and, you've got the answer to your problem. That, however, could never work because problems that need solving proliferate much faster than software. There is no way that we could design something generic enough to keep pace.

We do have some really good frameworks and commercial products in the technology domain that weren't born out of the desire to tackle every single problem by means of being all-encompassing. Many successful projects that are somewhat generic today, that is, they can be applied in several contexts, were conceived to solve an smaller problem.

Django is perhaps the easiest example I can point to in the Python world. This is the de facto web framework for Python programmers, but that certainly wasn't Django's goal when it was first conceived. Django, in it's early days, had the more modest goal of providing the facilities necessary to rapidly build web applications for newspaper content. I won't drill down into all the right decisions that were made over the course of Django's history that gave it such popularity in the Python web community. I will, however, reiterate that because newspaper applications needed to be developed quickly, we now have a web framework that has been used to develop a wide array of web applications.

It's always interesting to watch solutions to problems applied in new contexts. The most successful solutions to technology problems are almost always adaptations from other contexts. The innovations that are currently under way in cloud technology most certainly have practical implications for software design and development in general.

Principles
Sometimes, a technology solutions doesn't have a one-to-one translation in a context different from the problem they were meant to solve. Sometimes, you have to derive principles from the solution, and these are valuable. Principles gained from innovating could be as simple as lessons learned. We know what not to do because such and such went wrong in environment this and that. The bi products of innovation are limitless — sometimes they're real software components that have been simplified and generalized for the masses. Sometimes its general knowledge.

In the case of cloud technology, we're trying to solve a number of problems that wouldn't otherwise prevail. Scalability, for instance, is an important aspect to cloud infrastructures. How do you accommodate such vast amounts of data, coupled with fluctuating demand, and fortify architectural quality properties? Interoperability — there is a cloud ecosystem out there, and vendors need to emulate one another's API. Migratory data. Moving large chunks of data around networks while retaining consistency isn't an easy problem to solve.

What's interesting about cloud technology is that we're seeing the inverse of the regular innovation pattern. Cloud, depending how you want to define it, solves a general problem. So the long-lasting principles that will someday be beneficial in other contexts, weren't aimed at solving a specific problem as Django did in the newspaper industry. I will say though, there are a lot of hard technical lessons to be learned during this innovation period in cloud technology. We're in the incubation period for ideas and principles that will find themselves in many future software systems.

Turning Off Desktop Innovation

An interesting entry brings up the always controversial discussion of innovation in the open source desktop domain. I'm not entirely convinced that this topic should be nearly as controversial as it seams to be. And who knows, maybe it isn't. Putting the desktop operating system environment aside for a moment, innovation in software as a whole is hard. It is also a requirement of doing software development. Do nothing, and nothing will happen. If there were no innovation in desktop computing environments, in open source Linux distributions specifically, the end users would be stuck in the same situation. However, as the entry asks this very question, perhaps the users are stuck where they are for a reason. Maybe they have zero need to innovation that would serve their particular purpose. They use what they are using because it helps them reach their ultimate goal. Sometimes with innovative software, users are presented with features they didn't no they needed until they became available. This, not always, but often enough, translates to they don't really need it at all. However, users aren't going to be able to use the same piece of software indefinitely in the majority of cases. So, it seems that the logical thing to strive for here is a balance between stability and new features (innovation).

When attempting to strike a balance between stability and new features, developers are faced with an additional challenge. Toward the tail end of this entry, the option of turning these new innovative features off entirely is mentioned. I think an important characteristic to think about when considering new innovative features. Think about it. You ship your existing stable features along with the brand new innovative stuff. If something blows up in the new feature set, the user simply turns it off. Simply of course not being quite accurate. This ability to turn features on and off is no easy feat. Consider the notion of extension modules. The whole idea behind them is that they extend some piece of core functionality. They can also be turned off. However, this is generally done with configuration files that a typical desktop end-user should never be expected to interface with. So, there is the the technical aspect of modularity of features.

Assuming there were a robust, modular desktop architecture that allowed developers to turn features on and off, how would the desktop compel the user to use the new "better" features? Do the new features default to "on"? There is the whole usability question in addition to a very challenging technical problem.

Software, Patents, and Innovation

An interesting entry over at the open source initiative asks if patents hinder or encourage innovation. This is an interesting question regardless of the field in question. The entry talks about the centuries-old case of the steam engine. Once a patent for an idea has been established, anyone wishing to employ this idea will be in debt to the patent holder. How likely is anyone wishing to use or further develop the idea to get involved? Also, how does the patent holder benefit from this situation? They don't. In fact, the encouragement often works in the opposite direction. In many cases, there is no alternative than to use an idea that has been patented. This restriction then grows to resentment toward the patent holder and that is not something any designer, especially in the software industry wants. The idea of openness and community collaboration not withstanding, software suffers the same patent-type problems.

Early on in the development of the steam engine, the idea was patented. This brought about an era of the steam engine where there was no spectacular innovation. Years later, the steam engine designers lifted the patent. Sure enough, this brought about an era of design innovation. Such much so that the innovation rate of the steam engine doubled over the previous patented era. Designers and engineers are not turned away by the thought of patent infringements.

It seems that patents in general do not benefit anyone. The main motivation patented ideas offer is to create the patent in the first place. This is an extremely flawed approach to building anything well. The concept of "do something you love" is tossed out the window. Who loves to patent things? Why not do something well and have the rest follow. In the end it all comes down to the designer's attitude. In the case of the steam engine, once the patent was lifted, attitudes changed. That, in turn, changed everything.

In software, patents are hard to define. There is simply no way around it. Especially trying to patent an algorithm of some sort. If I were to take an existing algorithm that has been patented, completely re-factor it, and end up with the same result, would that be considered patent infringement? If so, what is really being patented is a specific output paired with a specific input. That would be very unjust.