Server Side Embed

In this day and age, our web experience wouldn't be nearly as interactive without the objects embedded inside most pages. What is the purpose of embedding objects inside a web page? Shouldn't HTML take care of this for us? It should, especially now that HTML5 is ever more mainstream with each passing day. Flash, on the other hand, is the standard for video, or anything fancy user interactions - transitions and such. Are embeddable objects for the web, good or bad? They're so pervasive that we can't simply ignore them, even if we don't like them. From a design perspective, embedded objects seem like a good way to reduce coupling by offloading some resources such as video. Its easy to use resources from other services, like YouTube. With social networks abound, embedding objects is an important concept for building web pages in a social context. How can our stuff on the server benefit from the embed tactic?
 

Let's take a closer look at why we embed things in our web pages – videos, menus, widgets and the like. Videos are obvious – websites want to show videos and we can't do this with HTML alone. HTML5 has the video element, but even then, I think we're still embedding conceptually because the video data isn't markup language. Likewise, an iframe might be considered an embedded object – even though its data is HTML, we still need to fetch it from an external resource, one outside of the page. Social networking widgets, too, are loaded from outside the page. I consider stuff that isn't directly contained in the page an embedded object. This works for me logically, because I can classify everything else as external, even though it may reside on my own server. Everything else, the paragraphs, headings, and so on, are a tightly-integrated unit that depends on external data.
 

What does embedding objects in HTML pages have to do with low-coupling? Well, if we can think of our web pages as a skeleton structure with slots for objects, we can reuse a lot of our HTML, consequently, CSS too. These object placeholders can put whatever we need in them – flash, HTML, our own widgets. That's really all we really need to worry about with this design, fill the hole with a web object. This is different from traditional programming languages where we're filling in the blanks using objects and polymorphism. Here, you've got to make sure that each object has the interface the slot is expecting. Not so much with HTML. You can embed whatever you want, the worst outcome being a shoddy looking site. So can we take these ideas, these embeddable objects and apply them to our web applications that live on the server?
 

As an idea, of course we can. In fact we already do this, we just use different terminology. On the server, we've got components. This is how the functional aspects of our application interchange. Replaceable authentication components, graphics libraries, you name it. There is something that can step in to do the job of a misbehaving or under-performing server object fairly trivially. I say fairly trivially because it isn't always so easy. The server-side is very different from the browser, worlds apart.
 

Lets say we're designing a simple brochure site. It doesn't rely on the server in any way other than to deliver web pages. So the big requirement is that it displays some information, maybe has a mouse-over effect here and there. A project like this only requires HTML, CSS, and Javascript. We need a web server, but don't have to do anything with it aside from starting it – we don't have to program it. Our little web site, in the browser, has three things, three languages, to worry about. This can be handled easily by one person on a small scale. Even when it comes to replacing the embedded objects, there is no learning curve because there is nothing to objects in HTML. Usually just a URL. Or maybe some cut-and-paste object code with a parameter. Easily manageable.
 

The server part of an application is a different story because there is just so much involved with serving HTTP requests. We have to make sure the user is authenticated, make find the appropriate request handler, process the request in a timely manor, and perform database queries. All of this while making sure we're fault-tolerant because unlike the browser, an HTTP request has the potential to break the entire application for every other user. This is why we have web application frameworks, to handle the common stuff with serving dynamic content. So how do we embed objects, objects that extend the capabilities of our application, into the framework?
 

Django immediately comes to mind for me, when thinking about embedding on the server. Partly because I really enjoy using it and partly because it has the best example I can think of. I'm sure there are plenty others too. In Django, we've got middleware and applications, both ready to embed in our projects. Middleware is how we inject logic into each HTTP request served. Sorry, embed logic. Middleware inspects the request before it is handled and after the response has been generated by the handler. This relates to HTML embedding in that we're able to take third-party objects and use them for our purposes, just as we would copy and paste a YouTube video.

Embedding things into larger applications, be it an HTML user interface, or something on the server, is a powerful concept. Embedded objects are simply part of the larger component architecture, where stuff can be swapped out. I think the embed concept for severs has a close parallel with the kind of embedding we do in the browser. Its a powerful idea, it helps with designing loosely-coupled systems for the web. It helps with delegating the responsibilities of a feature to the appropriate party. And that's it, really. Its just a mode of thought when putting together an application.

Simple Zope HTTP Server

I've seen mention of every conceivable Python web framework as of late. Mostly performance comparisons. One I don't see that often is Zope. It is a mature framework that does everything under the sun. I think it isn't mentioned that often by developers because of the lack of documentation.

The good news is that Zope code is easy to read and understand. I decided to experiment and create a really simple HTTP server with Zope HTTPServer. I find the task-oriented view of each HTTP request both interesting and useful. Here is what I came up with:

import asyncore
from zope.server.http.httpserver import HTTPServer
from zope.server.http.httptask import HTTPTask
from zope.server.taskthreads import ThreadedTaskDispatcher

class MyServer(HTTPServer):
  
   def executeRequest(self, task):
      
       task.start()
      
       self.log_info("Starting request task.")

       task.response_headers['Content-Type'] = 'text/plain'
       task.response_headers['Content-Length'] = 5

       task.write(task.request_data.command)
       task.write(" ")
       task.write(task.request_data.uri)   

       task.finish()

       self.log_info("Rquest task finished.")
      
if __name__ == "__main__":
  
   dispatcher = ThreadedTaskDispatcher()
   dispatcher.setThreadCount(10)
  
   MyServer('', 8084, task_dispatcher = dispatcher)

   try:       
       while True:
           asyncore.poll(5)
   except KeyboardInterrupt:
       dispatcher.shutdown()

Initializing the CherryPy server object

The CherryPy Python web application framework contains several abstractions related to an HTTP server. One of which is the Server class defined in _cpserver.py. In fact, the top-level server instance of the CherryPy package is an instance of this class. The Server class inherits from the ServerAdapter class which is defined in servers.py. Interestingly, the class serves as a wrapper, for other HTTP server classes. The constructor of ServerAdapter will accept a both a bus and a httpserver parameter. The bus parameter is a CherryPy website process bus as described here. The httpserver parameter can be any instance that implements the CherryPy server interface. Although this interface isn't formally defined, it can be easily inferred by looking at the CherryPy code.

So we have a Server class that inherits from ServerAdapter. The idea here is that many server instances may be started on a single CherryPy website process bus. One question I thought to ask was "if Server inherits from ServerAdapter and ServerAdapter is expecting a httpserver attribute, how do we specify this if the ServerAdapter constructor isn't invoked with this parameter?" In other words, the ServerAdapter isn't given a parameter that it needs by the Sever class.

It turns out that the start() method is overridden by Server. This method will then ensure that a httpserver object exists by invoking Server.httpserver_from_self(). Developers can even specify an httpserver object after the Server instance has been created by setting the Server.instance attribute with the server object we would like to use. This is the first attribute checked by the Server.httpserver_from_self() method. The Server.instance attribute defaults to None, and if this is still the case once Server.httpserver_from_self() is invoked, it will simply create and return a new CPWSGIServer instance to use.

Now we can feel safe, knowing that there will always be an object available for ServerAdapter to manipulate. We left off at the Server.start() method creating the httpserver attribute. Once this is accomplished, the ServerAdapter.start() method is invoked because it is now safe to do so. One observation about the implementation; I'm not convinced that calling the ServerAdapter.start() method with an instance of self is the best design. This is the only way that Server instances can invoke behaviour on the ServerAdapter instance, even though in theory it is the same instance by inheritance. At the same time, we wouldn't be able to override the method and then call the inherited method if we were to call ServerAdaptor.__init__() from Server.__init__(). The alternative would be to have unique method names between the two classes. Then again, this might end up taking away from the design quality of ServerAdapter. So the question is, which class is more important in terms of design. Just something to think about, not that the current implementation is defective by any means. CherryPy is probably one of the more stable Python packages in existence.

Web-based applications a good idea?

An entry on infoworld doesn't think so. Apparently the web browser sucks for applications and anyone using them are reverting back to the client server model. Well, it is a client (the browser) and it is a server (whatever). The strong case for the web application in the browser is deployment. There is no easier way to distribute a GUI than the web browser. Unless that is, everyone in the world used Windows.

Of course, the case against the browser is strong too. The same interoperability problems that operating systems experience are shared in the browser domain. In short, your HTML/CSS/AJAX GUI may not work across all platforms (platforms being web browsers in this case).

In terms of GUI interoperability and deployment, the same problems persist. There is no easy way to make everyone happy. And that will always be the case as long as people use different operating systems. There exists no simple unifying solution. Again, with web browsers, you have the interoperability problem. With interpreted languages, it is quite simple to achieve interoperability amongst various operating systems. I think the problem there is deployment. It is a lot tougher to deploy desktop applications that use any kind of data center.

The main problem I see with the suggested solution in the infoworld entry is that there is no mention of distributed data. Lets say I take the advice from the entry and all our customers now use our new Java desktop application. Where did the data go? Any modern requirements will scream distributed data. And that is another complexity there is no alternative for.

All these challenges, distributed computing, distributed deployment, and interoperability, they have been around for a while now. I agree with the author in that for the most part, web browsers suck for any kind of reliable user interactivity. However, I still don't see any solutions over the immediate horizon that will make the browser go away.