Recursive List Building With Handlebars

I always end up implementing some convoluted JavaScript inside a Handlebars helper whenever I have to render lists with any depth. Think trees — there could be several nested lists. I don't like the JavaScript approach I've taken in the past, I'm using Handlebars to generate dynamic markup — it should be able to handle nested lists, right? My preferred approach these days is to use a partial template, and to call that recursively.

Building Python Lists

Building primitive Python lists is a common task in virtually any Python application. The Python list is not unlike a traditional array in many respects, only more powerful. A common scenario is building a Python list from an iterator that yields values. In this case, each iteration adds a new element to the list being built. One method used to build Python lists using an iteration construct is the append method. This approach invokes the list append() method to append a new list element. Another approach to building Python lists is to place the iteration itself within the new list. This approach can be considered inline list building. The benefit to using the inline approach is the performance gain. The benefit to using the append method is readability when logic is required within each iteration that appends elements to the list. Below is an example illustrating both approaches.

#Example; Building Python lists.

import timeit

#Build a list consisting of ten elements using the append method.
def build_append():
  list_obj=[]
  for i in range(0,10):
      list_obj.append(i)
    
#Build a list consisting of ten elements using the inline method.
def build_inline():
  list_obj=[i for i in range(0,10)]

if __name__=="__main__":
  #Setup timers.
  t_append=timeit.Timer("build_append()",\
                        "from __main__ import build_append")
  t_inline=timeit.Timer("build_inline()",\
                        "from __main__ import build_inline")

  #Execute timers.
  r_append=t_append.timeit()
  r_inline=t_inline.timeit()

  #Show results.
  print "APPEND:",r_append
  print "INLINE:",r_inline
  print "DIFF %:",(r_inline/r_append)*100

Stateful Python Lists

The Python programming language defines list objects as one of the primitive language types. Lists, are similar to arrays in more traditional programming languages. Of course, Python lists are mutable types, as are arrays. The main difference between Python lists and traditional arrays are similar in most other comparisons of Python to other languages; Python lists are more flexible and elegant. However, Python lists are not magic. One situation where Python lists have the potential to easily fall apart is when used in a multi-threaded environment. This fact isn't exclusive to lists, the same risk involved with passing data across threads applies to all types. The good news is that this can be easily remedied with lists. Python list instances support the notion of "pushing" and "pulling" data. This is a useful thought construct because this transfers directly to the design. Any given list instance can be "pushing" state when adding data to the list or a "pulling" state when retrieving data from the list.

The Set class in the boduch Python library is essentially a Python list that publishes events. These events that are published by Set instances have the potential to be handled in a multi-threaded environment. If Set instances are being both "pushed" to and "pulled" from in a multi-threaded environment, this could be very dangerous. Data that is expected to exist in the set instance when "pulled" may not have arrived yet. In this scenario, it would be useful to know what state the Set instance is in. By combining both the Set and StateMachine classes as demonstrated below, we end up with a "stateful" Python list. The implementation of the StatefulSet shown here isn't fully thread safe. This is because it is incomplete. However, the code used in the main program is thread safe.

#example; Creating a multi-threaded Python list with a boduch Set.

#Necessary imports.
from boduch.event import threaded, subscribe, EventSetPush
from boduch.handle import Handle, HandleSetPush
from boduch.data import Set
from boduch.state import StateMachine
from boduch.type import is_type
from boduch import PRIORITY_MINOR, PRIORITY_CRITICAL

#A handle for Set.push() method invocations.
class HandlePreStatefulSetPush(Handle):
   #This gets a higher than critical priority since it is
   #intended to run before the core behavior.
   priority=PRIORITY_CRITICAL+1
   def __init__(self, *args, **kw):
       Handle.__init__(self, *args, **kw)
      
   def run(self):
       #Check if we are handling a StatefulSet event.  If
       #so, set goes into a pushing state.
       set_obj=self.get_event_data("set")
       if is_type(set_obj, "StatefulSet"):
           set_obj.change_state("PUSHING")

#A handle for Set.push() method invocations.
class HandlePostStatefulSetPush(Handle):
   #This handle gets a minor priority since it is 
   #intended to run after all the pushing is complete.
   priority=PRIORITY_MINOR
   def __init__(self, *args, **kw):
       Handle.__init__(self, *args, **kw)
      
   def run(self):
       #Check if we are handling a StatefulSet event.  If
       #so, set goes into a ready state.
       set_obj=self.get_event_data("set")
       if is_type(set_obj, "StatefulSet"):
           set_obj.change_state("READY")

#A stateful version of the Set class.
class StatefulSet(StateMachine, Set):
   def __init__(self, *args, **kw):
       #Initialize the base classes and populate the 
       #potential set states.
       StateMachine.__init__(self, *args, **kw)
       Set.__init__(self, *args, **kw)
       self.add_state("READY")
       self.add_state("PUSHING")
      
   def _push(self, data):
       #Change to a pushing state before actually pushing data.
       self.change_state("PUSHING")
       self.push(data)
      
   def _get(self, index):
       #Don't attempt to return anything while in a pushing state.
       while self=="PUSHING":
           print "Hold on, still pushing data."
           pass
       return self[index]
  
#Subscribe the custom stateful handles.
subscribe(EventSetPush, HandlePreStatefulSetPush)
subscribe(EventSetPush, HandlePostStatefulSetPush)

#Main program.
if __name__=="__main__":
   #Enable threaded event mode.
   threaded(True)
   #Instantiate a stateful set.
   set_obj=StatefulSet()
   #Populate the set while retrieving data in a thread-safe manor.
   for i in range(0,10):
       set_obj._push("SET DATA"+str(i))
       print set_obj._get(i)
  

Python benchmarks

Just for fun, I decided to run some Python benchmarks that test the lookup time differences between list, tuple, and dictionary types. The list performance seems to come out on top every time. This is confusing to me because I hear that tuples are supposed to be faster because they are immutable. Here is the test I used:

from timeit import Timer

test_data1=[1,2,3,4,5]
test_data2=(1,2,3,4,5)
test_data3={0:1,1:2,2:3,3:4,4:5}

def test1():
  v=test_data1[2]

def test2():
  v=test_data2[2]

def test3():
  v=test_data3[2]

if __name__=='__main__':
  print 'Test 1:', Timer("test1()", "from __main__ import test1").timeit()
  print 'Test 2:', Timer("test2()", "from __main__ import test2").timeit()
  print 'Test 3:', Timer("test3()", "from __main__ import test3").timeit()

I'm running this on a Intel(R) Core(TM) 2 CPU T7200 @ 2.00GHz machine. I wonder if my test is flawed.