Python Yield Interleaving

Distributed, concurrent applications seems to be the hot topic of interest in computing these days. And, it should be. With an ever increasing amount of data to process, both locally, and on the web, the need to speed things up is crucial. The best way to achieve this speed is by means of concurrency, doing multiple things at the same time. But true concurrency is hard to achieve. Sometimes impossible to achieve as is the case with a single processor. This, however, does not mean that the responsiveness of applications cannot be improved.

Applications that are logically concurrent can support both true hardware concurrency and interleaving concurrency. Interleaving is a time sharing technique that is used when true concurrency is not possible due to a single processor. If interleaving were not used on single processor machines, trivial tasks would render the system useless due to the response time. If nothing else, the single processor architecture has shown how important interleaving is to responsiveness and the effects of that responsiveness.

Applications written in Python can also be designed to be logically concurrent. This can be achieved both by means of threads and by yielding data. Threads in Python are an interleaving construct due to the global interpreter lock, even on multiple processor systems. Yielding data from functions is also interleaving because each element that is yielded from a generator, creates a new process flow. Once this new flow has completed, the flow resumes with the next element to be yielded. An example showing how this interleaving by means of yielding is shown below.

#Example; Python yield interleaving.

#Take an iterable and turn it into a list return it.
#Even if it is already a list.
def _return(_iterable):
   result=[]
   for i in _iterable:
       print "RETURNING"
       result.append(i)
   return result
      
#Take an individual iterable and yield individual elements.
def _yield(_iterable):
   for i in _iterable:
       print "YIELDING"
       yield i
  
#Main.
if __name__=="__main__":
  
   #Create an interable.
   _string_list="A B C".split()
  
   #Display the serial results of returning.
   for i in _return(_string_list):
       print i   
  
   #Display the interleaving results of yielding.
   for i in _yield(_string_list):
       print i