jQuery UI dialogs aid in gathering input from the user. Dialogs jump out from page, usually in response to some user-triggered event such as a button click. When a dialog is displayed, the page layout — that is, the way the page looked before the dialog was displayed — remains unchanged. It's almost like a secondary browser window, harmonized with the rest of the user interface thanks to the theme framework.
The dialog widget isn't a perfect fit for everyone — some might prefer the approach of displaying forms in-line on the page as opposed to having the form jump out at them. But aside from individual user preferences, there are a few implementation issues with displaying data in dialog widgets. Some major API redesign is lined up for the 1.9 version of the dialog widget, aimed at solving some of these imperfections.
One issue I'm interested in seeing worked out is proper disabling of the dialog widget. Subtle cases, where not having the dialog entirely disabled, cause some weird side-effects. And then there are not so subtle breakages where we need to fetch remote data.
Forms and remote data
Forms wouldn't be vary useful in the context of web applications if they didn't somehow interact with the server. For example, submitting a forms often means dispatching a POST HTTP request from the browser to the application server, resulting in a new resource. Sending data to the server is one avenue to interacting with remote data — but the user who submitted the form will need feedback — how else will they know if the action succeeded or not?
The response generated from the form submission should be presented to the user. More often than not, the user will have made an error in one or more fields, so the user interface will need to convey this. The error message — or error messages — typically go beside the errant field.
Dialog widgets that display forms to users have another potential use for remote application data and that is the form itself. Imagine a user clicks a registration button, located on the homepage, which then displays the form in a dialog widget. Now, imagine that this website has several other public informational pages that do not require a login, but nonetheless, we still to display the registration button on these pages. The registration form is the same on all pages.
So where does the dialog widget get the actual markup for this form? The jQuery UI dialog widget uses a div element as it's template — whatever is inside the div is displayed when the dialog is opened. This means that we'll have to insert the form markup on every page where the registration button is present. A solution to avoid duplicate registration forms is to make an Ajax call when the button is clicked. Instead of just opening the dialog, we'll populate the div with remote data first — this way we're only duplicating the URI pointing to the form on each page instead of the form itself.
Three methods in which the form displayed inside a jQuery UI dialog can communicate with the application server — through submitting data, through receiving responses, and through loading the initial form. On their own, the methods aren't difficult to comprehend and implement, but latency and usability are always a concern. An Ajax-based form can take on several states when presented inside a dialog, some of which require the form be disabled.
Disabling form elements
There comes a point, during the communication from the form to the application server, where it'll need to be disabled. Disabling means giving the user a visual cue that the element is temporarily paralyzed. Also, disabled widgets shouldn't respond to user events.
For example, let's say the user has just opened up a registration form inside a dialog widget. Before it can be displayed, we need to retrieve the actual form HTML and insert it into the dialog. In the mean time, while the HTTP request is happening in the background, there isn't anything for the user to see — there is nothing to display in the dialog yet. If we defer displaying the dialog until the form is ready, it appears to the user as though nothing has happened yet — sluggish responses are discouraging.
Display the dialog in a disabled state while the form loads.
What about when the user has actually submitted form data and is awaiting the result? We could hide the dialog entirely while the Ajax response churns. But this is clumsy as there is high probability of having to display the form again for the user to correct errors. It seems the logical solution is to keep the dialog visible, but disabled, just while processing happens. This is a friendly way, a responsive way, to indicate that stuff is happening.
Simple solution to a difficult problem
There are scenarios when using jQuery UI dialog widgets where it makes sense to disable it. Especially when dealing with forms as mentioned above. Sending and receiving remote data while the dialog is open means users can interact with it — unless it's properly disabled. However, while data is being transferred — from the browser to the server — or vice versa — there is a short latency duration.
Ajax-style applications cannot avoid latency during HTTP requests that take place in the background. Or, at least they cannot work under the assumption that the network response time will be faster than user actions. Preventing the user from performing certain actions is easy — it becomes difficult when we take into account the perceived responsiveness of the user interface. There is a delicate balance between ensuring correct behavior while remaining intuitive.
The solution to this problem, as we'll see in the 1.9 release of the jQuery UI dialog widget is to place a div element over top of the dialog, preventing the user from making changes during background processing. The best part about this enhancement, however, is that there is now a simple API for the dialog widget that'll make it disabled. This greatly simplifies many usage scenarios where we're waiting on the application server — we can simply disable the dialog and re-enable it instead of stringing together a boisterous solution.
Showing posts with label api. Show all posts
Showing posts with label api. Show all posts
Thursday, September 29, 2011
Friday, September 23, 2011
Making Data Public
What sort of information should be made public by organizations? The question, phrased like this, means what type of information is displayed on the company's web site? Information for the web has been prepackaged, so to speak, for consumption by human readers. Perhaps the more interesting question is this — what I'm interested in anyway — what type of raw data should be exposed through an API? Who would use such data and for what purpose?
With pre-formatted information, it's difficult to make sense of it in large volumes. Google does this for their search index by crawling monumental assemblages of HTML pages and other web resources. But for those of us who don't have enormous computing centers and the software capable of deciphering these data sets, we need something a little more primitive. Something that'll let our software draw it's own conclusions about public data.
Who wants to expose their data?
The internet changed things for all organizations — large and small. Having a place on the web is no longer a nice-to-have — you simply cannot compete without one. For one thing, consumers have come to expect this — a place where they can gather information on products and or services. Also, the web is a social animal now. Companies need a social presence on the web if they're to engage with their customers.
This is just web technology — read-only web technology. Information on what the business does, how they're a cut above the competition, and so forth. Just the ignition for the social correspondence that follows. I won't bother going into the whole social end of it because it's fairly obvious — it's important.
What isn't so obvious, however — how the specific information organizations publish on their websites is chosen. Obviously, publishing damaging information will come back to haunt the organization commodiously. Hence the reluctancy to publish anything at all. So why take the chance in making more information available to consumers when it's much safer to make less data available?
A competitive edge
Making data public — useful data — will absolutely give your organization a competitive advantage. That of course assumes that you have both interesting data to make available and the know-how to design an API that other software makers can accommodate. If you've got those two things — you're ahead of the game because you're enabling third party software to be developed on your behalf. This software directly impacts how existing and potential customers connect with your organization.
Take Apple's App Store for instance. There is obviously no shortage of applications available for users to install and use. In fact, it is amazingly imbalanced — the number of applications available in the App Store relative to other device maker's software markets.
Developers who make these applications aren't doing it because of the interesting data Apple is exposing through an API — they're doing it because of the iPhone's popularity. There is a much larger potential user base. So how does this relate to making your data available to the general public through an API? Because this is the type of following you want from developers. How useful would the iPhone be without plethora third-party applications? It would still be great device, but it would be missing a lot of things that users want.
Another aspect to the competitive advantage of having third-party software built for you — they'll use your data in ways you haven't thought of. Could Apple really have thought of all the applications available for their devices? Probably, but not in a time frame of just a few years.
Fear of over-exposure
If our companies weren't scared of exposing data items they shouldn't, they probably would have done it years ago, right? I don't think that's necessarily true because we're only still discovering the neat things we can build with other organization's data.
How do you protect yourself from exposing too much information, perhaps leading to a competitive edge for your competition? I'll tell you what won't protect you from it — not making an API for interesting information. The reality is, it's easy to spot weaknesses — every company in the world has at least one. If the competition is smart enough to exploit these weaknesses, they won't need an API to do it.
So listen to your customers — what kind of applications would they find useful? Or, more generally, what information would they find beneficial? Expose that information through an API. Developers will build cool things if the data is valuable. Who knows what new ideas these applications will in turn bring forth.
With pre-formatted information, it's difficult to make sense of it in large volumes. Google does this for their search index by crawling monumental assemblages of HTML pages and other web resources. But for those of us who don't have enormous computing centers and the software capable of deciphering these data sets, we need something a little more primitive. Something that'll let our software draw it's own conclusions about public data.
Who wants to expose their data?
The internet changed things for all organizations — large and small. Having a place on the web is no longer a nice-to-have — you simply cannot compete without one. For one thing, consumers have come to expect this — a place where they can gather information on products and or services. Also, the web is a social animal now. Companies need a social presence on the web if they're to engage with their customers.
This is just web technology — read-only web technology. Information on what the business does, how they're a cut above the competition, and so forth. Just the ignition for the social correspondence that follows. I won't bother going into the whole social end of it because it's fairly obvious — it's important.
What isn't so obvious, however — how the specific information organizations publish on their websites is chosen. Obviously, publishing damaging information will come back to haunt the organization commodiously. Hence the reluctancy to publish anything at all. So why take the chance in making more information available to consumers when it's much safer to make less data available?
A competitive edge
Making data public — useful data — will absolutely give your organization a competitive advantage. That of course assumes that you have both interesting data to make available and the know-how to design an API that other software makers can accommodate. If you've got those two things — you're ahead of the game because you're enabling third party software to be developed on your behalf. This software directly impacts how existing and potential customers connect with your organization.
Take Apple's App Store for instance. There is obviously no shortage of applications available for users to install and use. In fact, it is amazingly imbalanced — the number of applications available in the App Store relative to other device maker's software markets.
Developers who make these applications aren't doing it because of the interesting data Apple is exposing through an API — they're doing it because of the iPhone's popularity. There is a much larger potential user base. So how does this relate to making your data available to the general public through an API? Because this is the type of following you want from developers. How useful would the iPhone be without plethora third-party applications? It would still be great device, but it would be missing a lot of things that users want.
Another aspect to the competitive advantage of having third-party software built for you — they'll use your data in ways you haven't thought of. Could Apple really have thought of all the applications available for their devices? Probably, but not in a time frame of just a few years.
Fear of over-exposure
If our companies weren't scared of exposing data items they shouldn't, they probably would have done it years ago, right? I don't think that's necessarily true because we're only still discovering the neat things we can build with other organization's data.
How do you protect yourself from exposing too much information, perhaps leading to a competitive edge for your competition? I'll tell you what won't protect you from it — not making an API for interesting information. The reality is, it's easy to spot weaknesses — every company in the world has at least one. If the competition is smart enough to exploit these weaknesses, they won't need an API to do it.
So listen to your customers — what kind of applications would they find useful? Or, more generally, what information would they find beneficial? Expose that information through an API. Developers will build cool things if the data is valuable. Who knows what new ideas these applications will in turn bring forth.
Friday, November 12, 2010
API Documentation
Creating API documentation is a tedious task at best. Automating this process removes most drudgery involved. Programmers glimpse at API documentation at several points throughout the day. We do this to confirm doubts or to make sure that the function was in fact returning an integer type. Not only does automated documentation generation safe us effort, it is also a time saver. Is there a need for human intervention, given that we have tools at our disposal to write API documentation for us? Are the tools really that good? And if not, if they're still lacking in one respect or another, will they ever replace us entirely?
What exactly do automatic code generators do? Put simply, they collect oodles of software system information and render it in a programmer-friendly format. The basic idea is to parse the source code tree of the project and write the documentation output. The format of the documentation can be anything really, but common forms are PDF and HTML. HTML is probably the best in most situations as it is easier to navigate and usually looks better.
Automatically generated API documentation depends largely on code higher-level code structures, not necessarily the low-level file operations and so-on. Modules, classes, and functions are the preeminent descriptions that make API documentation useful. Finer details about these high-level units can also be included by the documentation generator. Function parameters, return types, and class attributes - all relevant things programmers want to read about.
We annotate these these high-level language constructs using comments which are also captured by some of these documentation-generating tools. If you use Python, you have the added privilege doc-strings, which are part of the code objects themselves. In either case, you, the developer, have the ability to extend the content of the generated API documentation. You're effectively writing documentation and code at the same time. Sounds great, doesn't it? The difficulty with this approach is intent. Are code documentation and API documentation the same thing?
Documenting code is all about explaining the "nitty-gritty" to those with the misfortune of reading your code. You've agonized over strenuous details and need to walk any reading eyes through it. Probably this isn't the kind of thing we care to see in API documentation. If I need to look up something specific in the API documentation, I don't want to be bothered by big chunks of "this attribute needs to be reset after the....". Encapsulation also applies to documentation.
How do programmers format the comments lifted from the code by generators? Documentation tools do a fine job of formatting language constructs like classes, functions, variables, etc. What they can't do is format text embedded inside an explanation. In the documentation of a function, I'll sometimes want to see a brief paragraph that tells me why the state of the file object stored in an output stream queue changes. It is helpful to format text inside these explanations as well as link to other documents. A popular format for writing API documentation seems to be reStructuredText. What this amounts to is programmers using a markup language within the programming language. This adds an unnecessary burden - especially since comments should be geared more toward the low-level.
There really is no substitute for automatically generating the code structure used in API documentation. The only alternative is to do it manually, which is excessively tedious. But at the same time manually writing the API documentation gives you freedoms you don't necessarily have when generating it via code comments. For one thing, you're not legally bound for life to a single tool. You can format as you deem necessary. You're also not restricted by the configuration constraints generation tools impose - you can document as much or as little code as you want. Whichever method you choose, just be sure that good documentation is the result.
What exactly do automatic code generators do? Put simply, they collect oodles of software system information and render it in a programmer-friendly format. The basic idea is to parse the source code tree of the project and write the documentation output. The format of the documentation can be anything really, but common forms are PDF and HTML. HTML is probably the best in most situations as it is easier to navigate and usually looks better.
Automatically generated API documentation depends largely on code higher-level code structures, not necessarily the low-level file operations and so-on. Modules, classes, and functions are the preeminent descriptions that make API documentation useful. Finer details about these high-level units can also be included by the documentation generator. Function parameters, return types, and class attributes - all relevant things programmers want to read about.
We annotate these these high-level language constructs using comments which are also captured by some of these documentation-generating tools. If you use Python, you have the added privilege doc-strings, which are part of the code objects themselves. In either case, you, the developer, have the ability to extend the content of the generated API documentation. You're effectively writing documentation and code at the same time. Sounds great, doesn't it? The difficulty with this approach is intent. Are code documentation and API documentation the same thing?
Documenting code is all about explaining the "nitty-gritty" to those with the misfortune of reading your code. You've agonized over strenuous details and need to walk any reading eyes through it. Probably this isn't the kind of thing we care to see in API documentation. If I need to look up something specific in the API documentation, I don't want to be bothered by big chunks of "this attribute needs to be reset after the....". Encapsulation also applies to documentation.
How do programmers format the comments lifted from the code by generators? Documentation tools do a fine job of formatting language constructs like classes, functions, variables, etc. What they can't do is format text embedded inside an explanation. In the documentation of a function, I'll sometimes want to see a brief paragraph that tells me why the state of the file object stored in an output stream queue changes. It is helpful to format text inside these explanations as well as link to other documents. A popular format for writing API documentation seems to be reStructuredText. What this amounts to is programmers using a markup language within the programming language. This adds an unnecessary burden - especially since comments should be geared more toward the low-level.
There really is no substitute for automatically generating the code structure used in API documentation. The only alternative is to do it manually, which is excessively tedious. But at the same time manually writing the API documentation gives you freedoms you don't necessarily have when generating it via code comments. For one thing, you're not legally bound for life to a single tool. You can format as you deem necessary. You're also not restricted by the configuration constraints generation tools impose - you can document as much or as little code as you want. Whichever method you choose, just be sure that good documentation is the result.
Saturday, March 20, 2010
Reinventing The Standard
Lets face it. If you want to write code today, you are going to have to understand and interact with one or more standards. Probably more. Standards can more often than not be viewed as a necessary evil. Well, standards aren't evil, they just seem that way because they are quite good at pointing out human error.
As an example, if you are putting together even a simple networked application, it makes sense to use some kind of application-level data transfer standard such as HTTP. Even if it is a simple application, the development effort isn't likely to move any faster by using non-standardized protocols.
Sometimes it does make sense to reinvent standards. All the developers in the world are never going to agree on what the standard programming language that everyone should use. Not going to happen. There are too many cases where one is better than the other. This is fine because we don't need to standardize programming languages. The protocols and data representations are all that really matter.
This also means that any developer isn't restricted to using a popular library API. If it doesn't fit in well with what they are trying to do, it isn't worth it. The code saved from being written by using the library could cause more harm than good. This shouldn't matter as long as the publicly accessible components of the application use well understood standards.
As an example, if you are putting together even a simple networked application, it makes sense to use some kind of application-level data transfer standard such as HTTP. Even if it is a simple application, the development effort isn't likely to move any faster by using non-standardized protocols.
Sometimes it does make sense to reinvent standards. All the developers in the world are never going to agree on what the standard programming language that everyone should use. Not going to happen. There are too many cases where one is better than the other. This is fine because we don't need to standardize programming languages. The protocols and data representations are all that really matter.
This also means that any developer isn't restricted to using a popular library API. If it doesn't fit in well with what they are trying to do, it isn't worth it. The code saved from being written by using the library could cause more harm than good. This shouldn't matter as long as the publicly accessible components of the application use well understood standards.
Thursday, March 18, 2010
jQuery API Attributes
jQuery is an excellent Javascript toolkit for interacting with server APIs. Especially for RESTful, resource-oriented APIs. Each resource returned from such an API generally has a unique ID associated with it. This could be a database primary key or a UUID. Regardless, it is used to uniquely identify the resource so it may be referred to as a URI such as /resource/3495/.
jQuery web applications often build lists of user interface elements from resource lists. For example, /resource/list/ might return a list of resources in the for of (id, name). Once the jQuery callback has this list of id-name pairs, it can build an HTML list. The question is, how should the resource ID be stored in the user interface so that it can be used again as part of a resource URI if the user clicks a list element?
One solution is to store the ID directly in the DOM element when it is created. The benefit here is that the URI can be constructed from the click event. The event object itself has a currentTarget attribute which is our list element. Lets say we stored a uuid attribute as part of the list element. Inside the click event handler, we could do something like jQuery(event.currentTarget).attr("uuid"). This is all we need to build a URI for this specific resource.
jQuery web applications often build lists of user interface elements from resource lists. For example, /resource/list/ might return a list of resources in the for of (id, name). Once the jQuery callback has this list of id-name pairs, it can build an HTML list. The question is, how should the resource ID be stored in the user interface so that it can be used again as part of a resource URI if the user clicks a list element?
One solution is to store the ID directly in the DOM element when it is created. The benefit here is that the URI can be constructed from the click event. The event object itself has a currentTarget attribute which is our list element. Lets say we stored a uuid attribute as part of the list element. Inside the click event handler, we could do something like jQuery(event.currentTarget).attr("uuid"). This is all we need to build a URI for this specific resource.
Thursday, December 3, 2009
Software Maintenance
This entry talks about some of the problems faced by current software maintenance practices. It highlights some of the various maintenance methods used to maintain deployed software. Of course, not many seem to do it right.
Problems arise mainly because of incompatibility between software versions. Typically, the entire software package has a version assigned to it. This includes all the constituent parts of the application such as modules and data structures. What if this individual components were giving a version instead of the whole? Well, there are systems out there in existence that do just that according to the entry.
What about taking this atomic version schema idea to the URIs of RESTful APIs. Indeed, this idea isn't anything new as many APIs support this feature. The main problem faced by web applications when performing server-side upgrades is the cached clients. Javascript that interacts with the URIs on the client's behalf may be stuck using an old API version. This is fine if the API version number is part of the URI. But backward compatibility can only be maintained so far back. This can be dealt with much easier if the expected version is part of the URI. If an unexpected version is requested, a message can be displayed to the client telling them to download a newer client. Alternatively, the new client code could be transparently delivered to the client as a response to using an incorrect version number.
Problems arise mainly because of incompatibility between software versions. Typically, the entire software package has a version assigned to it. This includes all the constituent parts of the application such as modules and data structures. What if this individual components were giving a version instead of the whole? Well, there are systems out there in existence that do just that according to the entry.
What about taking this atomic version schema idea to the URIs of RESTful APIs. Indeed, this idea isn't anything new as many APIs support this feature. The main problem faced by web applications when performing server-side upgrades is the cached clients. Javascript that interacts with the URIs on the client's behalf may be stuck using an old API version. This is fine if the API version number is part of the URI. But backward compatibility can only be maintained so far back. This can be dealt with much easier if the expected version is part of the URI. If an unexpected version is requested, a message can be displayed to the client telling them to download a newer client. Alternatively, the new client code could be transparently delivered to the client as a response to using an incorrect version number.
Friday, October 23, 2009
Python Dictionary Generators
The dictionary type in Python is what is referred to as an associative array in other programming languages. What this means is that rather than looking up values by position, values may be retrieved by a key value. With either type of array, the values are indexed, meaning that it each value may be referenced individually within the collection of values. Otherwise, we would have nothing but a collection of values that cannot be referenced in any meaningful way.
The dictionary type in Python offers higher-level functionality than most other associative array types found in other languages. This is done by providing an API on top of the primitive operators that exist for traditional style associative arrays. For instance, developers can retrieve all the keys in a given dictionary instance by invoking the keys() method. The value returned by this method is an array, or list in Python terminology, that may be iterated. The Python dictionary API also offers other iterative functionality.
With the introduction of Python 3, the dictionary API has seen some changes. Namely, the methods that return lists in Python 2 now return generators. This is quite different from invoking these methods and expecting a list. For one thing, the return value is not directly indexable. This is because generators do not support indexing. The following example shows an example of how the dictionary API behaves in Python 3.
In this simple example, we start by creating a simple dictionary instance, my_dict. The idea is that this dictionary be a simple one as we aren't interested in the content. Next, we create three new variables, each of which, store some aspect of the my_dict dictionary. The my_keys variable stores all keys that reference the values in the dictionary. The my_items variable stores key-value pairs that make of the dictionary, each item being a tuple. The my_values variable stores the actual values stored in my_dictionary with no regard for which key references them. The important thing to keep in mind here is that these variables derived from the my_dict dictionary were created using the dictionary API.
Up to this point, we have my_dict, the main dictionary, and three variables, my_keys, my_items, and my_values, all created using the dictionary API. Next, we purposefully invoke behavior that isn't supported in Python 3. We do this by acting as if the values returned by the dictionary API are list values when they are in fact generators. This produces a TypeError each time we try to do it because the generators stored in my_keys, my_items, and my_values do not support indexing.
Finally, we simply iterate over each generator containing data derived from my_dict. This works just as expected and is in fact the main use of the data returned by the dictionary methods shown here. Sure, the direct indexing doesn't work on the returned generators, but is that really a common use of this data? I would certainly think not. The key aspect of this API change is that the API now returns a structure that is good at iterative functionality and that happens to be the intended use. And, if indexing these values that are returned from the dictionary API are absolutely necessary, these generators can easily be turned into lists. It is just an extra step involved for the rare use of the data.
The dictionary type in Python offers higher-level functionality than most other associative array types found in other languages. This is done by providing an API on top of the primitive operators that exist for traditional style associative arrays. For instance, developers can retrieve all the keys in a given dictionary instance by invoking the keys() method. The value returned by this method is an array, or list in Python terminology, that may be iterated. The Python dictionary API also offers other iterative functionality.
With the introduction of Python 3, the dictionary API has seen some changes. Namely, the methods that return lists in Python 2 now return generators. This is quite different from invoking these methods and expecting a list. For one thing, the return value is not directly indexable. This is because generators do not support indexing. The following example shows an example of how the dictionary API behaves in Python 3.
#Example; Python 3 dictionary keys.
#Initialize dictionary object.
my_dict={"one":1, "two":2, "three":3}
#Main.
if __name__=="__main__":
#Invoke the dictionary API to instantiate generators.
my_keys=my_dict.keys()
my_items=my_dict.items()
my_values=my_dict.values()
#Display the generator objects.
print("KEYS: %s"%(my_keys))
print("ITEMS: %s"%(my_items))
print("VALUES: %s"%(my_values))
#This would work in Python 2.
try:
print(my_keys[0])
except TypeError:
print("my_keys does not support indexing...")
#This would work in Python 2.
try:
print(my_items[0])
except TypeError:
print("my_items does not support indexing...")
#This would work in Python 2.
try:
print(my_values[0])
except TypeError:
print("my_values does not support indexing...")
#Display the generator output.
print("\nIterating keys...")
for i in my_keys:
print(i)
print("\nIterating items...")
for i in my_items:
print(i)
print("\nIterating values...")
for i in my_values:
print(i)
Up to this point, we have my_dict, the main dictionary, and three variables, my_keys, my_items, and my_values, all created using the dictionary API. Next, we purposefully invoke behavior that isn't supported in Python 3. We do this by acting as if the values returned by the dictionary API are list values when they are in fact generators. This produces a TypeError each time we try to do it because the generators stored in my_keys, my_items, and my_values do not support indexing.
Finally, we simply iterate over each generator containing data derived from my_dict. This works just as expected and is in fact the main use of the data returned by the dictionary methods shown here. Sure, the direct indexing doesn't work on the returned generators, but is that really a common use of this data? I would certainly think not. The key aspect of this API change is that the API now returns a structure that is good at iterative functionality and that happens to be the intended use. And, if indexing these values that are returned from the dictionary API are absolutely necessary, these generators can easily be turned into lists. It is just an extra step involved for the rare use of the data.
Thursday, October 15, 2009
Documenting With Sphinx
Whether a developer is writing an application in Python, or any other language, API documentation is an absolute must. The API documentation should go without saying these days. If a developers asks if they are a requirement, the this fact hasn't been driven home hard enough. Other developers shouldn't need to sift through a mountain of code just to find a function signature. Especially with the nice output that is available with the generation tools available.
Sphinx is just such a tool, geared toward generating API documentation output for Python applications. One of the nice things about Sphinx is its' own API documentation. There isn't much on the how-to end but every supported language construct is there and it is made clear how to use it.
Another quality of Sphinx is that it is general purpose enough to use with languages other than Python. As long as the concepts are somewhat similar to those found in Python, it can work. And this is helpful for developers if a single documentation generation tool can be used for all output.
Developers do, however, have a big decision to make when considering Sphinx as the documenter of choice. Sphinx requires that RST documents that contain the actual API documentation to be maintained. That is, Sphinx cannot generate API documentation based on the doc-strings of the source code. Many other documentation generation tools can do this, but the overall quality of the Sphinx output is far above anything else.
Sphinx is just such a tool, geared toward generating API documentation output for Python applications. One of the nice things about Sphinx is its' own API documentation. There isn't much on the how-to end but every supported language construct is there and it is made clear how to use it.
Another quality of Sphinx is that it is general purpose enough to use with languages other than Python. As long as the concepts are somewhat similar to those found in Python, it can work. And this is helpful for developers if a single documentation generation tool can be used for all output.
Developers do, however, have a big decision to make when considering Sphinx as the documenter of choice. Sphinx requires that RST documents that contain the actual API documentation to be maintained. That is, Sphinx cannot generate API documentation based on the doc-strings of the source code. Many other documentation generation tools can do this, but the overall quality of the Sphinx output is far above anything else.
Friday, October 9, 2009
Trac Web Interface
The Trac project management system provides a nice set of web interfaces. What makes these interfaces nice to work with is that they are concise and to the point. The make defining new Trac web components an intuitive process. There are two interfaces of note in the set of Trac web interfaces. These are IRequestHandler and IRequestFilter. A simple illustration of what these interfaces look like conceptually is shown below.
Both interfaces obviously need to inherit from the Interface class since they themselves are interfaces.
The IRequestHandler interface is responsible for handling HTTP requests. Developers cannot define Trac web components that do not implement this interface. The two responsibilities of components that implement this interface are two determine if the component can handle the request and to return a response. This is different from some frameworks in that the component itself carries the responsibility of matching a URL.
The IRequestFilter interface is responsible for pre and post request handler processing. The pre processing occurs once a handler has been selected but before a response is generated by the handler. The post processing occurs once a response is generated.
Web components in Trac generally implement both of these interfaces. This means that the pre and post processing functionality uses the component itself as parameters. It is a very nice API to work with and other frameworks could certainly benefit from modeling after it.
Both interfaces obviously need to inherit from the Interface class since they themselves are interfaces.
The IRequestHandler interface is responsible for handling HTTP requests. Developers cannot define Trac web components that do not implement this interface. The two responsibilities of components that implement this interface are two determine if the component can handle the request and to return a response. This is different from some frameworks in that the component itself carries the responsibility of matching a URL.
The IRequestFilter interface is responsible for pre and post request handler processing. The pre processing occurs once a handler has been selected but before a response is generated by the handler. The post processing occurs once a response is generated.
Web components in Trac generally implement both of these interfaces. This means that the pre and post processing functionality uses the component itself as parameters. It is a very nice API to work with and other frameworks could certainly benefit from modeling after it.
Wednesday, September 30, 2009
Open Source Quality
In an interesting entry over at PC world, they mention a study that shows an overall decrease in defective open source software. Over the past three years, defects in open source software are down. This is great news, even if not entirely accurate, because I doubt the study is so flawed that there are more defects in open source software today. Every day, new open source projects poof into existence. How can all the complexities of the open source ecosystem be reliably measured? The truth is that they cannot. But some larger open source projects are much more popular than others and have been deployed in production environments. These are the interesting projects to monitor for defects because chances are, when a defect is fixed in one large project, that same defect will be fixed in countless others due to the dependency.
What I find interesting is the willingness to study and publish the results of code quality. To most users, even some developers, the code quality isn't that high on the requirement list for using the software. They don't see the code. Even most developers only see the API, and, arguably, that is all they should have to see. The code quality does effect more than just maintainability.
This brings up another question. Does the improved code quality improve the perceived used experience? Not likely, in most cases. But in some, yes. Even if it isn't obvious to the developers who fix a bug that wouldn't have any apparent effect on usability. Looking at these subtle usability enhancements in the aggregate might be interesting.
What I find interesting is the willingness to study and publish the results of code quality. To most users, even some developers, the code quality isn't that high on the requirement list for using the software. They don't see the code. Even most developers only see the API, and, arguably, that is all they should have to see. The code quality does effect more than just maintainability.
This brings up another question. Does the improved code quality improve the perceived used experience? Not likely, in most cases. But in some, yes. Even if it isn't obvious to the developers who fix a bug that wouldn't have any apparent effect on usability. Looking at these subtle usability enhancements in the aggregate might be interesting.
Monday, September 28, 2009
User Authentication Design
Most systems today, in fact, any system today in which a user interacts with a system, will have some kind of user abstraction. Whether that abstraction is the incoming request for application data or a instantiated class that lives on the server, it nonetheless exists. More often than not, the application needs to know who this user is. It can then make decisions about what, if any, data this user can see or modify. This, of course, is authentication.
There are many different approaches taken to implement user authentication. Most web application frameworks have a built-in authentication system. If that authentication system is flexible enough, it will allow for an external authentication system to be used. This is often the route that is taken by any commercial application simply because systems that were designed to authenticate, often do it well. There is no need to reinvent the wheel. Another reason for doing this might be performance.
However, most simple applications, often web applications, need only simple authentication. By simple, I mean they don't need a production-ready authentication system that can simultaneously handle millions of users. This isn't necessary and would be a waste of time. In these scenarios, simple HTTP authentication will be enough to allow the application to behave as required.
Even simple authentication needs to be designed. There are many approaches that can be taken to implement the underlying authentication one of which is a self-authenticating user abstraction. The user abstraction is necessary no matter what and should always be present in any design. The self-authenticating approach means that the authentication activity is performed by the user abstraction itself, with no need to invoke a separate party. The structure of such an abstraction is illustrated below.
Once an application receives a request for authentication to happen, the user abstraction is instantiated. Once instantiated, this abstraction is then passed the necessary data in order to authenticate itself. The result of the authentication is then passed to the controller responsible for instantiating the user. This sequence is illustrated below.
There are obvious benefits and drawbacks to using such an approach. The drawback is that the user abstraction is instantiated regardless of what the authentication outcome is. This is because the authentication can't happen without a user instance. Should a user instance, even if only momentarily, exist if not authenticated? The benefit here is the self containment. There is no need for an external authentication system since the user is able to state to the system whether or not it is who it says it is. Of course, this may not even be a good thing. An authentication system may be a desired design element.
There are many different approaches taken to implement user authentication. Most web application frameworks have a built-in authentication system. If that authentication system is flexible enough, it will allow for an external authentication system to be used. This is often the route that is taken by any commercial application simply because systems that were designed to authenticate, often do it well. There is no need to reinvent the wheel. Another reason for doing this might be performance.
However, most simple applications, often web applications, need only simple authentication. By simple, I mean they don't need a production-ready authentication system that can simultaneously handle millions of users. This isn't necessary and would be a waste of time. In these scenarios, simple HTTP authentication will be enough to allow the application to behave as required.
Even simple authentication needs to be designed. There are many approaches that can be taken to implement the underlying authentication one of which is a self-authenticating user abstraction. The user abstraction is necessary no matter what and should always be present in any design. The self-authenticating approach means that the authentication activity is performed by the user abstraction itself, with no need to invoke a separate party. The structure of such an abstraction is illustrated below.
Once an application receives a request for authentication to happen, the user abstraction is instantiated. Once instantiated, this abstraction is then passed the necessary data in order to authenticate itself. The result of the authentication is then passed to the controller responsible for instantiating the user. This sequence is illustrated below.
There are obvious benefits and drawbacks to using such an approach. The drawback is that the user abstraction is instantiated regardless of what the authentication outcome is. This is because the authentication can't happen without a user instance. Should a user instance, even if only momentarily, exist if not authenticated? The benefit here is the self containment. There is no need for an external authentication system since the user is able to state to the system whether or not it is who it says it is. Of course, this may not even be a good thing. An authentication system may be a desired design element.
Wednesday, September 23, 2009
Using Python Properties
The Python programming language is a loosely-typed language. What this essentially means is that any variable defined in a program can be assigned a value of any type. This also applies to attributes of classes. There is no need to specify the allowable types that a given attribute may hold. This offers the developer much flexibility when implementing an application. Even more flexibility may be added to a given class implementation by use of properties.
Python has a built-in property type that can be used to build complex attributes. By complex, I mean that the attributes can hold both data and behavior. This functionality can be made useful by imposing constraints on the attributes of a given instance because the behavior associated with the attribute is invoked when a value is set. This doesn't necessarily mean that the behavior is checking for the type of the value. That would defeat the purpose of a dynamically-typed language. It can, however, perform more complex testing such as checking the state of the value or making sure it falls within some range. The invoked behavior can also store and retrieve values from a non-standard location such as a database.
So why go to all this trouble? Why not just implement standard attributes and methods? That depends. The only reason to implement dynamic properties for Python instances is to provide a cleaner API for the client using the instances.
There are actually two methods in which to implement dynamic Python properties, both of which are illustrated below. The first, just overloads the __getattr__() and __setattr__() methods. These methods are invoked if the attribute requested does not exist in the standard location as a regular attribute. The benefit to this method is that these are the only methods that need to be implemented for attribute management. This means that any attributes can be set or retrieved on the instances. This can however lead to more work for these two methods because they are responsible for everything that might go wrong.
The second method, the property method, provides a better distribution of responsibilities. There is more work involved with the class implementation but is cleaner work that leads to a better client API.
Python has a built-in property type that can be used to build complex attributes. By complex, I mean that the attributes can hold both data and behavior. This functionality can be made useful by imposing constraints on the attributes of a given instance because the behavior associated with the attribute is invoked when a value is set. This doesn't necessarily mean that the behavior is checking for the type of the value. That would defeat the purpose of a dynamically-typed language. It can, however, perform more complex testing such as checking the state of the value or making sure it falls within some range. The invoked behavior can also store and retrieve values from a non-standard location such as a database.
So why go to all this trouble? Why not just implement standard attributes and methods? That depends. The only reason to implement dynamic properties for Python instances is to provide a cleaner API for the client using the instances.
There are actually two methods in which to implement dynamic Python properties, both of which are illustrated below. The first, just overloads the __getattr__() and __setattr__() methods. These methods are invoked if the attribute requested does not exist in the standard location as a regular attribute. The benefit to this method is that these are the only methods that need to be implemented for attribute management. This means that any attributes can be set or retrieved on the instances. This can however lead to more work for these two methods because they are responsible for everything that might go wrong.
The second method, the property method, provides a better distribution of responsibilities. There is more work involved with the class implementation but is cleaner work that leads to a better client API.
#Example; Python properties.
#Simple person class.
class Person(object):
#The data for the instance attributes. This serves as
#an example that these attributes can be stored elsewhere.
data={"first_name":"", "last_name":""}
#Set an attribute.
def __setattr__(self, name, value):
self.data[name]=value
#Get an attribute.
def __getattr__(self, name):
return self.data[name]
#Simple person class.
class PropertyPerson(object):
#The data for the instance attributes. This serves as
#an example that these attributes can be stored elsewhere.
data={"first_name":"", "last_name":""}
#Set the first name.
def set_first_name(self, value):
self.data["first_name"]=value
#Set the last name.
def set_last_name(self, value):
self.data["last_name"]=value
#Get the first name.
def get_first_name(self):
return self.data["first_name"]
#Get the last name.
def get_last_name(self):
return self.data["last_name"]
#Create the properties using the previously
#defined instance methods.
first_name=property(fset=set_first_name, fget=get_first_name)
last_name=property(fset=set_last_name, fget=get_last_name)
#Main.
if __name__=="__main__":
#Create the test person instances.
person_obj=Person()
pperson_obj=PropertyPerson()
#Set some values without using properties.
person_obj.first_name="FirstName"
person_obj.last_name="LastName"
#Set some values using properties.
pperson_obj.first_name="FirstName"
pperson_obj.last_name="LastName"
#Display the attribute values of both instances.
print "Non-Property: %s %s"%(person_obj.first_name, person_obj.last_name)
print "\nProperty: %s %s"%(pperson_obj.first_name, pperson_obj.last_name)
Tuesday, August 11, 2009
Transactional Javascript API Interaction
Any meaningful javascript application is going to make use of some kind of application API that lives off on a server somewhere. Generally, some user interface triggers some event which, in turn, will send a request to a remote API. These requests are generally asynchronous. That is, the request happens in the background while the user interface is still responding to events. The number of API connections and requests can add up rather quickly. Even if the data sent over the network for each request and corresponding response are relatively small, there is still the overhead involved with assembling the request.
Using a transactional javascript API access pattern can help cut down on network bandwidth. This is done by minimizing the work done by the API on a given request. Additionally, we also cut down the response size for a given request. The bandwidth consumption will grow to a much larger size if each API response is sent to the javascript application in a single HTTP response. It would be nice if a single HTTP response could contain multiple API responses. One potential solution is to only return a transaction id for each transaction request. The javascript can then store this transaction id. At the core of the javascript application would be an interval that would check the API for completed transactions and their corresponding responses. This approach is illustrated below.
Using a transactional javascript API access pattern can help cut down on network bandwidth. This is done by minimizing the work done by the API on a given request. Additionally, we also cut down the response size for a given request. The bandwidth consumption will grow to a much larger size if each API response is sent to the javascript application in a single HTTP response. It would be nice if a single HTTP response could contain multiple API responses. One potential solution is to only return a transaction id for each transaction request. The javascript can then store this transaction id. At the core of the javascript application would be an interval that would check the API for completed transactions and their corresponding responses. This approach is illustrated below.
Friday, March 13, 2009
ECP and the future extension module architecture
Developers that have been using the Enomaly Elastic Computing Platform over the past year, myself included, have encountered some bottlenecks in the ECP extension module architecture. These aren't show-stoppers in every case but sometimes, they are. It is mostly an issue of architectural design such as "what is the rationale behind this API method?" and "if I build it this way, what is the impact of change in other areas going to bring?" Once we took a step back to think about such questions, we came to the conclusion that these questions should be apparent to any developer using the platform or at least easily answered by Enomaly. Right now, much of the ECP extension module framework isn't apparent how to use and we even have a hard time explaining it.
So, this has led the ECP development team to address some of the issues highlighted here.
One of the first major problems is a problem of uniformity and consistency among the core extension modules that ship with ECP. These extension modules aren't exactly consistent with one another. Some modules will use sections of the API as intended while others will use different sections and others, still, don't use the API at all. In fact, some extension module logic in ECP is coded directly in the core system. This doesn't necessarily cause any harm to anyone who wants to install the base system because these modules are "part" of the base system. They are simply constructed as extension modules. Earlier on in ECP's lifespan, we needed to construct an extension module API and these core modules were the perfect way to test out our ideas. So, either way, these core extension modules could have been built in to the core code base. But, it would be nice if they weren't so irreplaceable.
The first step is to introduce a new level of consistency among the extension module that are distributed with ECP. If nothing else, they can serve as useful examples for extension module developers.
The next major defective area within the ECP extension module API is the API itself. Or, rather, lack thereof. What I mean here is that there are plenty of smaller areas of ECP that should be extensible but aren't. For example, if there is some thing small in the ECP front-end GUI that a developer wants to extend, they must replace the entire template, duplicating many already existing elements. This means that many of the core elements, including GUI widgets, would need to become part of a bigger extension module framework. That is, we would need to move them to extension modules. And that is fine with me. A smaller core is easier to maintain and thus more stable.
There have already been some changes introduced to the ECP extension module framework in the past year. We identified the need for extension modules to store their own static data such as javascript and CSS files. To address this, we introduced methods to register these static components.
There is still a decent amount of work to do in order to realize these changes. They have been identified and that is a very good thing. Also, this is by no means a closed list of issues with the extension module framework that need fixing. This is a good starting point. I've already begun fixing the consistency problem with the core extension modules.
So, this has led the ECP development team to address some of the issues highlighted here.
One of the first major problems is a problem of uniformity and consistency among the core extension modules that ship with ECP. These extension modules aren't exactly consistent with one another. Some modules will use sections of the API as intended while others will use different sections and others, still, don't use the API at all. In fact, some extension module logic in ECP is coded directly in the core system. This doesn't necessarily cause any harm to anyone who wants to install the base system because these modules are "part" of the base system. They are simply constructed as extension modules. Earlier on in ECP's lifespan, we needed to construct an extension module API and these core modules were the perfect way to test out our ideas. So, either way, these core extension modules could have been built in to the core code base. But, it would be nice if they weren't so irreplaceable.
The first step is to introduce a new level of consistency among the extension module that are distributed with ECP. If nothing else, they can serve as useful examples for extension module developers.
The next major defective area within the ECP extension module API is the API itself. Or, rather, lack thereof. What I mean here is that there are plenty of smaller areas of ECP that should be extensible but aren't. For example, if there is some thing small in the ECP front-end GUI that a developer wants to extend, they must replace the entire template, duplicating many already existing elements. This means that many of the core elements, including GUI widgets, would need to become part of a bigger extension module framework. That is, we would need to move them to extension modules. And that is fine with me. A smaller core is easier to maintain and thus more stable.
There have already been some changes introduced to the ECP extension module framework in the past year. We identified the need for extension modules to store their own static data such as javascript and CSS files. To address this, we introduced methods to register these static components.
There is still a decent amount of work to do in order to realize these changes. They have been identified and that is a very good thing. Also, this is by no means a closed list of issues with the extension module framework that need fixing. This is a good starting point. I've already begun fixing the consistency problem with the core extension modules.
Thursday, February 12, 2009
Trouble extending Trac navigation.
I'm in the process of building a new Trac site. I wanted to add new menu items to the main navigation. Hiding or rearranging the default menu items in Trac 0.11 is quite straightforward. It can all be done in the Trac configuration. However, new menu items need to be part of a component. Hence, the need for the NavAdd plugin.
It looks like the plugin has not yet been updated to fit the 0.11 plugin architecture, although it does work with 0.11. The Trac plugin API changes weren't too drastic. I did notice some strange behaviour though. It turns out that any menu items added with the NavAdd plugin can not be considered active.
What? No active menu items? Well, it isn't really the NavAdd plugins' fault. It turns out, that in order to have an active menu item, the current request needs to be handled by the same component that produces the menu item. I discovered this by looking at the timeline Trac component. This component actually implements the IRequestHandler interface. This is necessary because the timeline component handles requests to the /timeline URI. The menu item produced by this component becomes active anytime the /timeline URI is visited.
So, this design works well for components that have URIs. But what if I want to add menu items that point to wiki pages? And what if I want to have my menu item become active when visiting those pages? There is currently no way to do this. My suggestion would be that the INavigationContributor interface adds a new uri field to be returned from get_navigation_items(). When page corresponding to this uri becomes active, so does the custom menu item.
It looks like the plugin has not yet been updated to fit the 0.11 plugin architecture, although it does work with 0.11. The Trac plugin API changes weren't too drastic. I did notice some strange behaviour though. It turns out that any menu items added with the NavAdd plugin can not be considered active.
What? No active menu items? Well, it isn't really the NavAdd plugins' fault. It turns out, that in order to have an active menu item, the current request needs to be handled by the same component that produces the menu item. I discovered this by looking at the timeline Trac component. This component actually implements the IRequestHandler interface. This is necessary because the timeline component handles requests to the /timeline URI. The menu item produced by this component becomes active anytime the /timeline URI is visited.
So, this design works well for components that have URIs. But what if I want to add menu items that point to wiki pages? And what if I want to have my menu item become active when visiting those pages? There is currently no way to do this. My suggestion would be that the INavigationContributor interface adds a new uri field to be returned from get_navigation_items(). When page corresponding to this uri becomes active, so does the custom menu item.
Wednesday, January 28, 2009
Wednesday, January 7, 2009
Python audio scrobbler
I just experimented with the audioscrobblerws Python package and as a LastFM user, I have too say, it is pretty neat.
First, we initialize the service.
Next, if you need to retrieve a LastFM user, you can do so.
You can view the available functionality available to each user.
You will most likely see something like:
As you can see, if you are a fan of LastFM and feel like experimenting with this Python package, it offers quite a bit to experiment with.
First, we initialize the service.
from audioscrobblerws import Webservice
ws=Webservice()
user=ws.GetUser('someuser')
dir(user)
['AsDict', 'FillProfileData', 'GetCommonArtists', 'GetCompatibility', 'GetCurrentEvents', 'GetFriends', 'GetFriendsEvents', 'GetNeighbours', 'GetRecentBannedTracks', 'GetRecentLovedTracks', 'GetRecentPlayedTracks', 'GetRecentTracks', 'GetRelatedUser', 'GetSystemEventRecommendations', 'GetSystemRecommendations', 'GetTasteOmeter', 'GetTopAlbums', 'GetTopArtists', 'GetTopTags', 'GetTopTracks', 'GetWeeklyAlbumChart', 'GetWeeklyAlbumChartList', 'GetWeeklyArtistChart', 'GetWeeklyArtistChartList', 'GetWeeklyChart', 'GetWeeklyChartList', 'GetWeeklyTrackChart', 'GetWeeklyTrackChartList', '__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__getattribute__', '__hash__', '__init__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'age', 'attributeMap', 'avatar', 'cluster', 'country', 'currentEvents', 'friends', 'friendsEvents', 'fullProfile', 'gender', 'icon', 'id', 'mostRecentPlayedTrack', 'neighbours', 'playcount', 'realname', 'recentBannedTracks', 'recentLovedTracks', 'recentPlayedTracks', 'recentWeeklyAlbumChart', 'recentWeeklyArtistChart', 'recentWeeklyTrackChart', 'registered', 'systemEventRecommendations', 'systemRecommendations', 'topAlbum', 'topAlbums', 'topArtist', 'topArtists', 'topTag', 'topTags', 'topTrack', 'topTracks', 'url', 'username']
Wednesday, November 26, 2008
Enomaly ECP update
Just this week, I've started writing some new unit tests for Enomaly ECP. I decided to take a different approach than the previous unit tests. These new tests are designed to test the RESTful API of the software. The new testing module, which I'm using to test version 2.2, establishes a connection with a running ECP instance and makes several HTTP requests to the API. The results for each test pass or fail based on the HTTP status, the internal error code returned, and ensuring that modifications actually took place.
Another nice little tool I've incorporated into the new unit testing is sptest. It gives us some nice colourful output.
This shift in testing focus will hopefully yield a more robust API for the Enomaly ECP platform that can be verified with test results. I hope to actually ship this new testing module with 2.2 so users can run the unit tests for themselves.
Another nice little tool I've incorporated into the new unit testing is sptest. It gives us some nice colourful output.
This shift in testing focus will hopefully yield a more robust API for the Enomaly ECP platform that can be verified with test results. I hope to actually ship this new testing module with 2.2 so users can run the unit tests for themselves.
Subscribe to:
Posts
(
Atom
)