Integrating Python for High Throughput Science
Python (specifically CPython) includes important libraries for high throughput science such as numpy, matplotlib and scipy. It is still the best way scientists handle and analyze numerical data today. Data Analysis WorkbeNch (DAWN - http://www.dawnsci.org/) is an Eclipse based workbench for doing scientific data analysis used at synchrotrons and scientific facilities throughout Europe.
This talk details the approach adopted to provide tight integration of CPython within DAWN, and in particular to allow plotting operations on Eclipse views to be performed from a Python interactive console in the IDE. It looks at the XML-RPC layer that has been enhanced with the ability to deal with complex types (such as data sets or regions of interest) in a consistent way for both local and remote calls. The layer provides a generic way to call Python functions from Java (and Java functions from Python) and a robust way of handling exceptions.
This talk also examines the work done to enhance PyDev to provide interactive console debugging, co-debugging Java and Python, and the ability to examine numpy ndarrays, complete with metadata. We also look at integration of IPython.
Eclipse and CPython can be integrated to create an environment where scientists can easily/optimally develop and debug high throughput algorithms that integrate seamlessly with data visualisation.