Simple Zope HTTP Server

I've seen mention of every conceivable Python web framework as of late. Mostly performance comparisons. One I don't see that often is Zope. It is a mature framework that does everything under the sun. I think it isn't mentioned that often by developers because of the lack of documentation.

The good news is that Zope code is easy to read and understand. I decided to experiment and create a really simple HTTP server with Zope HTTPServer. I find the task-oriented view of each HTTP request both interesting and useful. Here is what I came up with:

import asyncore
from zope.server.http.httpserver import HTTPServer
from zope.server.http.httptask import HTTPTask
from zope.server.taskthreads import ThreadedTaskDispatcher

class MyServer(HTTPServer):
  
   def executeRequest(self, task):
      
       task.start()
      
       self.log_info("Starting request task.")

       task.response_headers['Content-Type'] = 'text/plain'
       task.response_headers['Content-Length'] = 5

       task.write(task.request_data.command)
       task.write(" ")
       task.write(task.request_data.uri)   

       task.finish()

       self.log_info("Rquest task finished.")
      
if __name__ == "__main__":
  
   dispatcher = ThreadedTaskDispatcher()
   dispatcher.setThreadCount(10)
  
   MyServer('', 8084, task_dispatcher = dispatcher)

   try:       
       while True:
           asyncore.poll(5)
   except KeyboardInterrupt:
       dispatcher.shutdown()

Popular Python Frameworks

The Python programming language is great for building web application frameworks. The two main reasons for this are that it is a very simple language to use and understand and it has fantastic networking libraries. Given these two facts, it is no wonder that there exist dozens of relatively solid web application frameworks written in Python.

The more popular web frameworks are the stable ones that have been around for some years. These frameworks have stood the test of time and have a large feature set.

I found another interesting way to look at which frameworks are the popular frameworks by using the Python package index. I browsed the available packages by framework to see which ones have the most packages. The Zope world is still dominating the Python web application framework market. Django is slowly catching up. At the time of this writing, here are the top frameworks listed by number of available packages.

• Plone (880)
• Zope3 (872)
• Zope2 (475)
• Django (295)

This list also demonstrates which frameworks are extensible because the easier it is to extend a software package, the more developers are willing to extend it with other packages and release them. What is surprising is the small number of Twisted and Trac packages. Both frameworks are well written and easily extensible. Having said that, the number of packages listed isn't entirely accurate because not all conceivable framework package lives in the Python package index. Also, there are most likely some categorization errors to take into account.

Shrinking Python Frameworks

An older entry by Ian Bicking talks about the shrinking world of Python web application frameworks. It is still a valid statement today, I think. There is no shortage of Python web application frameworks in which to choose from. Quite the contrary, it seems that a new one springs into existence every month. This often happens because a set of developers have a very niche application to develop and the existing web application frameworks don't cut it. Either that or they are missing a feature or two, or they have too many moving parts and so they will make some modifications. Whatever the difference, some developers will release their frameworks as an open source project.

The shrinking aspect refers to the number of frameworks which are a realistic choice for implementing a production grade application. Most of the newer Python web application frameworks, still in their infancy, are simply not stable enough.

Take Pylons and TurboGears for instance. Both are still OK web frameworks, you can't have TurboGears without Pylons now. However, they are somewhat problematic to implement applications with. Even if stable enough, there are complexities that shouldn't exist in a framework. Besides, I have yet to see a stable TurboGears release.

Taking complexity to a new level is Zope. This framework has been around for a long time and is extremely stable. But unless you have been using it for several years, it isn't really worth it because of the potential for misuse is so high.

The choice of which Python web application framework to use really comes down to how much programming freedom you want. If you want everything included, Django does everything and is very stable. However, if there are still many unknowns in the application in question, there are many stable frameworks that will simply expose WSGI controllers and allow the developers to do as they will.

Gaphor editor adapters

The Gaphor UML modeling tool, which is written in Python, uses a pop-up style editor widget which allows in-line editing of certain diagram elements. The widget itself isn't overly-interesting. It makes trivial changes to modeling elements quicker which is always helpful in any software solution. What we are interested in here is the method used to display the widget based on the element type. Gaphor relies heavily on the Zope interface and component frameworks. The Zope interface framework is utilized by Gaphor to define various interfaces that are provided by classes throughout the application. The component framework is utilized for the purpose of registering components and adapters. What exactly are adapters? Adapters are a type of component, automatically created by Zope when using the interface and component framework in conjunction with one another. This doesn't happen by itself; there are some carefully placed rules involved with defining adapters. When used right, Zope adapters are a very powerful tool that provide maximum usage of an interface. Gaphor defines an extensive set of Zope adapters. Here we are interested in the editor adaptor.

There are actually several adapters created for the IEditor interface, one for each diagram element that supports the editor widget. The Gaphor adapter approach is an alternative design to providing behavior that varies by type. A more traditional approach may have been to create a class hierarchy for the editor widget. Each class in the hierarchy would correspond to a different diagram element type. The differing behavior would then be implemented in each class in the hierarchy while similar behavior remains untouched and varying behavior gets replaced in a polymorphic manor. This is similar to how the editor adapters in Gaphor are defined and registered. One key difference in the design is how each class, or adapter, is instantiated when the need arises. With the class hierarchy approach, we would need extra logic to ensure that the correct instance type is created to use in conjunction with the diagram element widget. With Zope adapters, we simply instantiate the IEditor interface providing the object we are adapting to as a parameter. In the Gaphor case, the IEditor interface is instantiated with the diagram element widget as a parameter. The correct adapter instance is then returned by the Zope component framework, complete with the alternate functionality specific to that diagram element type. A similar effect can be achieved with the class hierarchy design. The class that is instantiated would accept the widget that is being adapted.

The adapter approach is a solid one because it emphasizes the importance of the interface contract provided by classes and the modularity offered by creating components. Being able to directly instantiate an interface speaks loudly in terms of what can be used in that context.

Does Plone really suck as a CMS?

Plone is a content management system written in Python. It is built on top of Zope, a web application frame also written in Python. To identify how well Plone performs as a CMS, we must identify what a CMS is supposed to do and what Plone actually does. Rather than give a rigorous definition of what a CMS is, it is safe to assume that any given CMS is supposed to manage the publishing of content for any number of users. This involves the content work-flow which specifies the various states the content will go through in the publishing cycle. All this needs to be as transparent as possible to the end user because their ultimate goal is to publish content in an orderly manor and as painlessly as possible. These same questions of CMS quality could be aimed at any CMS in existence including TYPO3, and Drupal. Plone often gets misrepresented as a bad choice of CMS, often for the wrong reasons. Many Python developers cringe at the mere thought of using Zope as a Python web application framework. Zope is probably the most powerful Python web framework in existence and the reason for the hesitation is the time investment involved in learning how to use it. Since Plone is built on top of Zope, it relies heavily on not only Zope's functionality, but also the philosophy of how it is used. Flawed software tends to flow upward so Plone will inherit the good, the bad, and the ugly. Thus, some of the complexities found in the Plone user experience are influenced by Zope.

Ultimately, the biggest driving factor in any CMS is the user experience. Everything else is secondary, including the setup of the system. Plone is strong in some user experience areas and weak in others. One of the weak spots in the Plone user experience is the back-end interface which is essentially the Zope interface. For anyone who is using the system for the sole purpose of managing content, this can be very intimidating. In a scenario where an editor of the site is temporarily given the task of managing a few users, this trivial task suddenly becomes a time-consuming challenge. However, that same editor Plone user would excel when it comes to doing his actual job of managing content. Plone offers a great user experience when it comes to general content editing and work-flow management. In fact, the work-flow Plone component is probably superior to many other mainstream content management systems simply because it is very straightforward to use for the people who need to use it. It doesn't require major customization from developers just so that it is barely usable by content editors.

When the case for new custom functionality in the CMS arises, development in Plone is hard. That is, there is a steep learning curve when coming from another CMS or web framework. Again, this stems from the challenges in developing for Zope. The most common reason for custom development in a content management system is to create new content types with new specialized behavior. These new content type may in fact extend the work-flow functionality of the CMS if the work-flow framework will allow for it. Plone also has something call archetypes for extending the available content types in the CMS. The idea behind archetypes is that developers and transform UML modules into functional Plone extensions. This is a very cutting-edge concept that no other CMS offers.

Some of the arguments against Zope and Plone are not well grounded. For instance, a common complaint about Plone is that it is a challenge to install and configure. This may have been the case five years ago but today it is one of the easier content management systems to install. In fact, most of the configuration is taken care of by the installer asking the user simple questions. Another argument is that Zope is very resource intensive. Users must look at why these resources are used and what they get in return. An alternative CMS, offering the the same functionality as a Plone CMS installation is likely to have similar resource consumption. Additionally, a content management system isn't a flimsy desktop application. For any non-trivial CMS deployment, you are going to need modest system resources. This is just a fact of life.

Finally, this isn't a marketing scheme for Plone as a CMS. I haven't used it other than experimentally for years. I think if nothing else that serves as further proof that you need to pick the right CMS for the job. Period. Focus on user experience. Which CMS can you install, setup with the required functionality for the people who manage the content, and deploy the fastest, with minimal complaints from content editors? After asking this question, you may realize that a CMS is not what you want at all and that a plain old web application framework would do the trick after a month of custom development.

Inside zope.event

The zope.event Python package is extremely straightforward. Which is why I like it. It provides a publish-subscribe mechanism that is sufficient for most applications. The entire zope.event package consists of a list for storing subscribers and a single notify function to publish events. Here is what the module looks like.

#zope.event

subscribers = []

def notify(event):
   for subscriber in subscribers:
       subscriber(event)

Very handy functionality and very easy to understand. This could event be implemented within an application should the zope.event module not be available because of its' simplicity.

Python publish subscribe

In keeping with the theme of my previous post, I'd like to mention that the publish-subscribe event pattern can also be used in Python. The zope.event package provides this functionality. For example, If I want to publish a topic when Blog objects in my application are updated, I could do something like:

import zope.event

class Blog:
 def update(self):
     self._update()
     zope.event.notify('blogupdate')

This will send a notification to anyone in my application that has a subscription to blogupdate. To subscribe to this topic, we could do something like:

import zope.event

def sub_blogupdate():
  log('Blog updated.')
zope.event.subscribers.append(sub_blogupdate)

The subscription would obviously do something more important but this serves well to illustrate the general concept. Publish-subscribe comes in handy when there is a chain of complex events. The zope.event package, although very simplistic, allows us to produce higher quality code.

Interfaces in Python.

Defining interfaces that your classes will provide is an essential programming practice if you plan to develop extensible code. That is, when you define interfaces, you are defining your internal architecture. The interfaces you define will give you a starting point in which you can discover more meaningful abstractions in your application domain.

Interfaces are not a replacement for object-oriented analysis. You can't just put together a set of interfaces that you think might work and hope for the best. Rather, your interfaces should be derived from concepts created by doing analysis and development. That being said, you will discover how your interfaces need to be refined and expanded upon as you are implementing classes that realize your interfaces. Don't get stuck in the waterfall approach to software development.

Zope provides an interface package which allows Python developers to define interfaces and implement classes that realize them. For example, say I want to define a blog entry interface. Using the zope.interface package, I might do something similar to:

import zope.interface

class IBlogEntry(zope.interface.Interface):
  """A simple blog account interface."""
  title = zope.interface.Attribute("""The title of the blog entry.""")
  content = zope.interface.Attribute("""The main content of the blog entry.""")
  date = zope.interface.Attribute("""The publish date of the blog entry.""")

  def setTitle(self, title):
      """Set the title of the blog entry."""

  def setContent(self, content):
      """Set the content of the blog entry."""

  def setDate(self, date):
      """Set the date of the blog entry."""

  def getTitle(self):
      """Return the title of the blog entry."""

  def getContent(self):
      """Return the content of the blog entry."""

  def getDate(self):
      """Return the date of the blog entry."""

  def publish(self):
      """Publish the blog entry."""

class BlogEntry:
  zope.interface.implements(IBlogEntry)

And of course we would then have to implement all the attributes and methods defined in IBlogEntry. We can then test if IBlogEntry is implemented by any given BlogEntry instance as follows:

blog_entry_obj=BlogEntry()
IBlogEntry.providedBy(blog_entry_object)

Of course, this interface gives us a good idea of what an blog entry implementation might look like. We can easily implement a different blog entry class because the interface provides us with an interchangeable template. As long as these methods an attributes are implemented, our class may be used where the IBlogEntry interface is required.

Is there a way we could re-factor our interface to get more use out of it? I think so. In a system which incorporates blog entry abstractions, it is most probable that there will be similar abstractions. Similar in the sense that these other abstractions will have a title, content, and date. Lets take a look at an alternative interface implementation of interfaces provided by BlogEntry.

import zope.interface

class ITitle(zope.interface.Interface):
   """A simple title interface for abstractions with a title."""
   title = zope.interface.Attribute("""The title of the abstraction.""")

   def setTitle(self, title):
      """Set the title of the abstraction."""

   def getTitle(self):
      """Return the title of the abstraction."""

class IContent(zope.interface.Interface):
   """A simple content interface for abstractions with content."""
   content = zope.interface.Attribute("""The content of the abstraction.""")

   def setContent(self, content):
       """Set the content of the blog entry."""

   def getContent(self):
       """Return the content of the blog entry."""

class IDate(zope.interface.Interface):
   """A simple date interface for abstractions with dates."""
   date = zope.interface.Attribute("""The date of the abstraction.""")

   def setDate(self, date):
       """Set the date of the blog entry."""

   def getDate(self):
       """Return the date of the blog entry."""

class IPublish(zope.interface.Interface):
   """A simple publishing interface for publishing abstractions."""

   def publish(self):
       """Publish the abstractions."""

class BlogEntry:
   zope.interface.implements(ITitle)
   zope.interface.implements(IContent)
   zope.interface.implements(IDate)
   zope.interface.implements(IPublish)

In this implementation, we have removed all references to blog entry in the interfaces we have defined. This is a good practice when there are several non-trivial abstractions involved in the system being implemented. We can then reuse our interfaces for several class implementations rather than keeping them specific to blog entry.

Finally, keep your interfaces simple. The goal in using interfaces is to keep the implementation of abstractions consistent. This is not accomplished be further complicating the code by adding interfaces that are more complex than necessary.