Delivery and Display

I sometimes find myself wondering about how web application data is delivered, and how it is displayed. Well, I actually have a good understanding of the mechanics behind it all — HTTP and HTML are the prominent technologies. What I don't understand is how the thinking behind these two technologies is changing. There was a time when HTML and HTTP were all that were needed to get the job done. And by job I of course mean a functional interface to the application. The client-server architecture that occupies the vast majority of what we interact with on the web works really well. The command response style of working with resources is simple, obvious, and well tested.

Given this sound approach to web application architecture, and it's omnipresence, why challenge it? What would cause programmers to step back and second guess the customary method of doing things? As it turns out, we should probably deliberate the importance of web browsers as the de facto presentation platform for user interfaces. That is, not all applications are inherently stateless and easily transported to a web context. They have long running processes that generate UI updates. They require richer API for displaying complex graphical arrangements than one finds available with the standard HTML and CSS combination. Running these types of applications in a browser aren't given second thought — the default answer is to build a right desktop application that may or may not have some opaque networking capabilities baked in. This school of thought seems to clash with not only what users have come to expect, but what is now possible using browser technologies alone. Anytime I find myself working on a user interface and wondering — how can I make this data display in a more lively manor — I realize more and more that this is possible using just a handful of technologies.

Perhaps the best example of technology disrupting the traditional architecture of web applications is noVNC. The premise is simple — we want access to VNC console sessions without desktop VNC clients or browser plugins. This type of application is heavily dependent on state — what is happening on the server. In your typical VNC connection, your client will receive more updates than it sends. The server component wants to transfer the screen display to the client, and so every time the state of the screen changes, this must be delivered to the client. This is the delivery concept of this new breed of web application. Put another way, we could very well develop web applications that rely more on this type of real-time delivery than the browser sending a GET request and rendering the resulting markup and style rules. When this data gets delivered to the browser, it needs to be displayed. But this data is different from what we're used to. In the case of noVNC, we receiving raw, remote frame buffer data that needs to be translated into something that makes visual sense to the user. We have the capabilities in any modern browser to now handle these new deliver and display domains.

But just because something is possible, doesn't make it a good idea. We can do these things, but do we really need to? Is there added value in redefining certain aspects of our software's architecture just to use these new browser capabilities? It depends on what your software does, and what kinds of work-arounds you have in place. Or perhaps you have a backlog of features customers actually want — things that would make their lives easier and thus make them happy — but they remain unimplemented simply due to lack of feasibility. Another risk we face as client side programmers is contaminating years worth of reliable, stable HTML, CSS and Javascript code. What we're doing by adding in these new technologies is increasing our dependency burden. And that is always a risky move. You have to be confident not only in the technology standards being consistently supported across browser vendors, but also in the availability of skills to work with this stuff. Your mature long-time staff needs to figure out an integrative approach to onboard these new ideas in a way that doesn't disrupt the regular development and release pattern. More trouble than it's worth? Not if it fixes things for you. Not if it takes away the need for unnatural command-response code that is merely emulating a stream of state coming from the server. Not if it enables something real and tangible that your customers gain.

At least from a theoretical perspective, the web socket and canvas APIs can realize these domains where we can benefit from a streamlined delivery of state, and a flexible drawing canvas. From a practical perspective, these are always challenging due to the fact that they're browser technologies. New ones at that. But the newness really isn't as big an issue as it may seem. We're still fighting with cross-browser support in HTML and CSS technologies, so this is something we would have to deal with anyway. The larger challenge would stem from having to integrate these two approaches. For example, the command response style that renders HTML would certainly remain useful for submitting data. HTML is still better suited for displaying all kinds of rich application data — grids, charts, progress bars — all of which could receive streamed changes in state. So there you have the overlap between the command response and the state delivery connection over a web socket. As an added bonus, you do not have to rely on the real-time data. If it isn't supported, or if it just fails for one reason or another, your user interface just falls back to the regular way of providing the user with updates. That could prove more challenging if we were to employ a more exotic canvas-based display medium. We use the canvas for things like noVNC — displaying an entire desktop within the browser. Entirely possible to do with a command response HTTP setup, but awkward.

This all leaves me to wonder — what types of applications justify this change in web programming methodology? There is noVNC, which is a great example of something that publishes a stream of state changes over a web socket connection and requires a sophisticated graphics API to render the data on the screen. But not only does a VNC host publish changes in display state on it's own accord — it does so frequently. Infrequent changes to processes happening on the server can certainly use an Ajax approach to detecting, pulling, and displaying these changes. What no VNC does is give us a glimpse of what is possible by exhibiting some impressive properties. We can publish a stream of frequent, real-time events over a single connection. And we have the flexibility to render graphics as depicted by the server. If we can broaden this scope to enhance some of the day-to-day applications we work on, we could build some impressive stuff. We're not limited to low level RFB-style graphic languages either. Initiatives like KineticJS take a great leap in terms of graphics APIs in conjunction with flexibility we have with delivering what to display.