Less Is More

When it comes to representations returned from a RESTful API, is there a preferred format? If JSON better than XML? More often than not, JSON is the preferred representation format because Ajax web applications are usually making the API requests. And when it is some other application making the request, it is usually trivial to make use of the returned data.

What about returning HTML as a RESTful representation? Isn't it as useful as JSON or XML? I would like to think so, just as long as it is consistent and simple. For instance, returning HTML lists is probably just as easy for clients to parse as retuning some other format. The added benefit is that HTML can be inserted directly into a browser. It can also be styled any any way imaginable.

Using HTML is less because it involves less work on the client end in most cases. It is more because it offers more flexibility.

File System Resources

RESTful web services often employ the concept of resources. When reading about RESTful web services, you will often here the term resource or resource-oriented. This is because a key principle of a RESTful system is that of the URI. The unique resource identifier is used to point to some resource, as the name suggests.

The concept of a unique resource identifier says nothing about the context in which it is used. That is, a URI can point to a resource on the web, or it can point to a resource locally on the file system. When using a URI on the local file system, the URIs will only be unique within the local context. For instance, the URI file:///home/ probably isn't unique within the context of the web but would most surely be unique within the local system.

There are two types of resources we are interested in when constructing RESTful applications. There are remote resource that the application might be interested in that live on the web. And, there are local resources the application might be interested in that exist locally within the file system. These two resource types really aren't all that different. The obvious difference of course being the context in which the resource is considered unique. The other difference is at a level lower than that of a RESTful design is how the actual IO functionality is implemented. For instance, you can't perform read operations on remote resource by invoking traditional file system functionality. The same is also true for performing read functionality with remote resources.

One of the similarities between remote resources and local resources is the URI. The URI differs only slightly between the remote resource, typically using HTTP as the protocol, and the local resource which uses a file IO protocol.

Illustrated below is a simple class hierarchy that models a flexible resource. It is flexible in the sense that instantiated resources can be either remote or local in the application.



Here, the base class is Protocol. Inheriting from this class is the base Resource class, with the children resources, File and HTTP. The classes are purposefully incomplete in definition because this hierarchy allows for many implementation variations. The Protocol class is high level and probably serves as an interface. The reason we want to define the Protocol class in the first place is that in this context, where resources may not be using the same protocol, resources may be considered a protocol type.

The Resource class is what should define the higher-level resource functionality. This is where the uniform methods that should be functional for any resource type should be defined. These could map closely to HTTP methods or to some other consistent interface. The File and HTTP class provide the lower level implementations that are invoked by the Resource interface. This enables an application to use resource abstractions, both local and remote, with no regard for context as the behavior can be invoked in a polymorphic way.

RESTful Python Objects.

Designing RESTful resources that behave like the web in this day and age makes for good API design practice. The resulting API has a very high level of simplicity that is sought after and valued by many developers. However, what about the client end? Can they too benefit from this elegant design? They sure can. Just like anything else in software design, APIs can be abused or they can be used as intended. So, why not make the client work similarly to how the resources themselves behave? Better yet, why not make them identical?

This can be achieved by mapping individual resources to Python instances. This makes for a good abstraction mapping. One resource, one Python instance. But this doesn't really help the Python developer if there are "special" methods they need to invoke on the instance just to interact with the actual resource. This instances acts as a proxy and so both the instance data and the instance behavior should be the same as the resource. This can be done by using the following principles:

• The constructor could issue a POST HTTP request to create a new resource using the constructor parameters.
• The attribute retrieval could be overridden to issue a HTTP GET request.
• The attribute setting could be overridden to issue a HTTP PUT request.
• The object deletion functionality could be overridden to issue a HTTP DELETE request.

That's it, the instance can then behave like a regular Python instance and be a RESTful resource at the same time. Mind you, these are just principles and not and ideal implementation, obviously. So, what is needed is an HTTP library of some kind to fully implement each of these methods. There will no doubt be variations to these methods as well. For instance, there is often the requirement of retrieving lists of resources as opposed to a single resource.

The following is a simple example illustrating these principles.

#Example; RESTful Python instances.

class RESTful(object):
   def __init__(self, uri, **kw):
       #Issue a HTTP POST request construct a new resource.
       print "POSTING..."
      
   def __getattr__(self, name):
       #Issue a HTTP GET, possibly a conditional GET to 
       #retrieve the resource attribute.
       print "GETTING..."
      
   def __setattr__(self, name, value):
       #Issue a HTTP PUT request to update the resource
       #attribute.
       print "PUTTING..."
      
   def __del__(self):
       #Issue a HTTP DELETE request to destroy the resource.
       print "DELETING..."
      
class BlogEntry(RESTful):
   def __init__(self, uri, **kw):
       RESTful.__init__(self, uri, **kw)
      
if __name__=="__main__":
   entry=BlogEntry("/blog", title="Hello", body="World")
   entry.body="There"
   body=entry.body

Addressability Lost Within The Realm Of Ajax

With RESTful APIs and applications, and with HTTP in general, addressability plays a huge role. With web applications that project an Ajax user interface, this addressability is typically gone. There is no notion of URI as far as the end user is concerned. This is because Ajax web applications often have a single URI; the application itself. From this point forward, the end user navigates through the various application states while all the addressable URIs are contacted beneath the fancy Ajax interfaces. If you are an end user using these types of applications, do you necessarily care? That would depend what you are using the application for and the type of user you are. For most end users, if the use interface is well designed, the addressability of the resources involved with the application are a non-issue. Even developers, the type one would think would be interested in the underlying application data, might be too enthralled with how useful the user interface is. If the user interface of an Ajax web application is poorly designed, things change. The data at the other end of the URI suddenly becomes much more interesting.

In more traditional web applications, the ones without the fancy asynchronous javascript toolkits, the URI of almost every resource is accessible from the address bar in the web browser. This is obviously the benefit to having addressable resources in a web application. The URIs are immediately apparent to the end user via the web browser address bar. With Ajax web applications, end users cannot point there web browser to a specific URI and expect the application to behave accordingly. This, I find, to be one of the more annoying drawbacks to using Ajax web applications. Although the user experience of Ajax web applications is improving at an ever increasing rate, the sacrifice of addressability, a powerful concept, has to be made.

But does addressability really need to be lost completely in Ajax web applications? Just because the URI isn't in its' normal location, the web browser address bar, doesn't mean the end user can't know about it. In fact, many web applications that provide an Ajax user interface will also provide a public RESTful API used by that interface, complete with documentation. However, this doesn't solve the problem of the end user that doesn't care about API documentation and just wants page X of the application to appear when they point their web browser to URI Y. I think something like this could potentially work. There would have to be two separate RESTful APIs; one for the application resource data, and one for the user interface. The user interface would interest the end users because they could use these URIs to point the web browser to a specific application state. These user interface application URIs wouldn't even need to be reflected in the web browser address bar. As long as the current user interface application state is advertised somewhere in the user interface, it could work. And it would be incredibly useful.

Restish Resources In Python.

RESTful resource design is not only a common theme in the Python web application framework world, but in countless other languages and frameworks. However, Python has the advantage of rapid development. Not just for the developers who use the frameworks in question, but also the developers creating these frameworks. The lessons learned from previous previous framework implementations have a very rapid turnaround. The restish Python web application framework is an example of just this. The package started out as the Pylons web application framework. However, the developers of restish soon realized that Pylons had shortcomings for what they were trying tho achieve. They then started to remove the problematic components from Pylons and what they eventually ended up with was a web application framework that was stripped down significantly. Just like Pylons, the restish package relies on the paster utility to create restish project and to execute restish projects. As the name of the project indicates, the framework is best suited for designing RESTful resources.

With restish, there is more than one way to define a resource, as is commonly the case with any software component. The best design is to represent a resource by building a class for that resource. Luckily, this is a no-brainer with restish. As a developer, all that needs to be built for a resource is a class that extends resource.Resource. The methods defined by this class are then responsible for returning the appropriate content. These resource methods are then decorated to indicate which HTTP method they correspond with. This is incredibly powerful and insightful; methods corresponding to methods. That is not all the decorated resource methods are capable of, however, I won't dig any further here. Resources can also be nested. The parent controller simply returns an instance of the child controller.

The abstractions provided by the restish package are very valuable to developers who want to design RESTful resources. This is a common design requirement that is elegantly implemented here. The sheer simplicity involved puts a smile on my face.

The need for a REST interface in object-oriented applications

REST is a set of design criteria used for designing web-centric architectures. Much of the HTTP protocol incorporates ideas found in REST such as being connectible, resources, and a uniform interface. This uniform interface consists of methods that can operate on resources such as GET, POST, PUT, and DELETE. These are the most common method employed by RESTful applications. The idea of resources states that each resource within a system is uniquely addressable. In fact, this is also part of the uniform interface found in RESTful designs. Many web clients, other than the web browser, use SOAP as the message transformation framework. However, SOAP is not as flexible as a RESTful design and yet there exist many clients and client libraries, in several languages for SOAP services. There are also RESTful clients and client libraries, although, no nearly as many. By the very nature of a RESTful design, objects in an object-oriented system map well to resources of a RESTful architecture. Perhaps developers should keep this in mind and have classes provide a RESTful interface.

What would a RESTful object-oriented interface look like? That is, what would the methods and attributes be? The first step to implementing a REST interface would be define methods that map to the HTTP methods. For example, consider the following example.

#Example; A RESTful Python interface.

class REST(object):
   def GET(self):
       raise NotImplementedError("GET()")
  
   def POST(self):
       raise NotImplementedError("POST()")
  
   def PUT(self):
       raise NotImplementedError("PUT()")
  
   def DELETE(self):
       raise NotImplementedError("DELETE()")

Here, we have a Python class called REST. This class defines the GET(), POST(), PUT(), and DELETE() HTTP methods. Each method, when invoked will raise a NotImplementedError exception because this class is meant to be an interface. For a class to provide this interface, it would inherit from this class and redefine all methods, providing an implementation. What about attributes? If a instance of REST were to act as a proxy to some RESTful resource, it would need to know its URI. So uri would be a good candidate for an interface attribute. There are many other meta-data attributes associated with the HTTP protocol they we aren't concerned with here. What we want to highlight is the REST interface developers could potentially use when designing objects. On the topic of attributes, another question springs to mind. What about resource attributes. If all we know about a particular resource is the methods it supports and its uri, how can we represent the resource in the context of an object-oriented system? This would most likely be another interface that we would use in conjunction with the REST interface, used to interpret the representation of the remote resource. An alternative is to use WADL to define what resources should look like. However, WADL is too much like SOAP. The rigidity involved defeats the purpose of a RESTful architecture.

The REST interface discussed so far is really only useful as a proxy to a remote resource. That is, the object we are designing that provides this interface would use this interface to make an HTTP request to the HTTP server providing the resource. An analog would be the web browser application providing the REST interface and invoking the GET() method to retrieve a web page.

The "REST" interface could also be the resource itself. If the developer is designing an object-oriented HTTP web application, they could design object within that system, exposed to the web, that provide the REST interface. The method information is always encoded in the HTTP request, otherwise it wouldn't be HTTP. If the base HTTP server forwards this request to an object that provides this interface, that object will always know what to do with the request. This same object can also act as a proxy and so on, forming a chain of RESTful resources.

However, as with all distributed computing, this chain of resources poses a design challenge. How does the system manage new resource locations? If the system is to scale at all, it will need to. However, this problem will come down the road. Right now, the problem is the RESTful implementation at the design level in object-oriented systems. With a RESTful interface, these problems would be much easier to solve.

Resource design with twisted.web

The twisted.web Python package offers several resource abstractions that make for a quality RESTful design in my opinion. Putting aside the the Python web framework requirement of "does it do everything I could possibly need ever need?" and focusing on designing an API that cohesive and easily understandable, the twisted.web package is very useful indeed.

The package offers a Resource class that serves as the base class for all resources in your application. As a developer, you will extend (and redefine if needed) this class to provide the behavior of your resources. To create a uniform API, from the RESTful perspective, we implement render_GET(), render_POST(), etc. These the value returned from these methods is the response given to the client.

Here is a trivial example of using the Resource class to implement a blogging system.

#Resource design with twisted.web

from twisted.web import server, resource, util
from twisted.internet import reactor

blog_data=[{'title':'First Entry', 'body':'First Entry Body...'},\
         {'title':'Second Entry', 'body':'Second Entry Body...'}]

class Blog(resource.Resource):
  def getChild(self, path, request):
      return util.Redirect('index')   

class BlogIndex(resource.Resource):
  isLeaf=True
  def render_GET(self, request):
      content=''
      template='<a href="/entry/%s/">%s</a><br/>'
      cnt=0
      for i in blog_data:
          content+=template%(cnt, i['title'])
          cnt+=1
      return '<html>%s</html>'%(content)

class BlogEntry(resource.Resource):
  isLeaf=True
  def render_GET(self, request):
      template='<html><h1>%s</h1><p>%s</p></html>'
      blog_id=int(request.postpath[0])
      blog_entry=blog_data[blog_id]
      return template%(blog_entry['title'], blog_entry['body'])

blog_obj=Blog()
blog_obj.putChild('index', BlogIndex())
blog_obj.putChild('entry', BlogEntry())
reactor.listenTCP(8080, server.Site(blog_obj))
reactor.run()

Some things to note in this example:

• The Blog resource is the root resource. It redefines the getChild() method in order to return the Redirect resource which will take us to the BlogIndex resource.
• The BlogIndex resource simply displays a list of sample blog entries.
• The BlogEntry resource represents a specific blog entry. This resource requires a blog id (list index). It will then display the blog entry.
• The main program first creates the Blog resource and adds the child resources. The server is then started using the Blog resource as the root resource.
  

Trac provides a good example of RESTful resources

The Trac issue tracking and wiki system provides a good example of a RESTful web resource. Good RESTful resources are connected. This means that the associations between resources are within the resource. If a given resource is associated with another resource, the second resource should be navigable from the first.

This is the basic concept behind links in hypermedia.

Trac, however takes this a step further with a trivial feature that is much more valuable than it may seem at first glance. Trac integrates very nicely with subversion. In a large percentage of cases, one ore more changesets are associated with a ticket. It is also fairly trivial to link to a changeset from within a ticket. Here is the interesting feature. The title attribute of the anchor that links to the changeset is the message associated with the changeset. Here is what I mean.



This subtle feature has saved me the time required to actually follow a link to view the changeset resource on numerous occasions. Most of the time, we're only interested in the message. Trac has realized this usability issue and addressed it.

Enomaly ECP update

Just this week, I've started writing some new unit tests for Enomaly ECP. I decided to take a different approach than the previous unit tests. These new tests are designed to test the RESTful API of the software. The new testing module, which I'm using to test version 2.2, establishes a connection with a running ECP instance and makes several HTTP requests to the API. The results for each test pass or fail based on the HTTP status, the internal error code returned, and ensuring that modifications actually took place.

Another nice little tool I've incorporated into the new unit testing is sptest. It gives us some nice colourful output.

This shift in testing focus will hopefully yield a more robust API for the Enomaly ECP platform that can be verified with test results. I hope to actually ship this new testing module with 2.2 so users can run the unit tests for themselves.

REST is better than all the REST

Representational State Transfer (REST) is by far the superior architecture when building web-based applications. What exactly is REST? Put simply, it is a means in which disparate software objects communicate state information to one another. REST can be used by practically any programming environment, and it is fast because it is lightweight.

The most powerful aspect of REST is that it is extremely easy to understand and use. By contrast, using SOAP (Simple Object Access Protocol) is anything but simple to use and understand. REST is most commonly used to access remote objects, similar to what SOAP is intended to do. Each object can be accessed using a URI for the object. An example REST URI might look something like:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0/

Here, the unique aspect of the URL is the trailing user identifier. Using REST, we can tell the URI what action we want to take place by specifying the HTTP method. The HTTP method represents the action we want to perform on the object (the URI). And we can pass parameters to these actions using the typical HTTP request approach.

Here are some examples of how you can use the HTTP method to manipulate remote objects using REST.

Using GET to retrieve an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0/

Using POST to update an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0?username=newname

Using PUT to create an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0?username=newname

Using DELETE to delete an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0/

Enomalism has a fully-featured REST API that will allow any client that has the necessary permissions defined, to access the entire Enomalism functionality.