Python processing

I've discovered an interesting Python package called processing. The name may be a little misleading but it is nonetheless useful. The reason it is misleading is that from the developer API perspective it is a threading API which mimics the built-in Python threading module. The main difference being, of course, it uses processes instead of threads.

This comes in handy when dealing with the controversial global interpreter lock (although I have no problem with the GIL). On multi-core and multi-processor systems, multi-threaded applications do not distribute threads amongst these processors. This is because of the Python GIL. The GIL is used to ensure safety with mutable data types.

However, new processes are distributed amongst processors. This is the issue the processing module aims to solve. I recommend trying it out. I also issue a warning. Spawning new processes more computationally expensive than spawning new threads no matter what platform is in use. There is no substitute for increasing application responsiveness with threads. However, longer-running algorithms may be better suited in a new process.