IPython Interactive Computing and Visualization Cookbook
上QQ阅读APP看书，第一时间看更新
上一章目录下一章

Efficient interactive computing workflows with IPython

There are multiple ways of using IPython for interactive computing. Some of them are better in terms of flexibility, modularity, reusability, and reproducibility. We will review and discuss them in this recipe.

Any interactive computing workflow is based on the following cycle:

  • Write some code
  • Execute it
  • Interpret the results
  • Repeat

This fundamental loop (also known as Read-Eval-Print Loop or REPL) is particularly useful when doing exploratory research on data or model simulations, or when building a complex algorithm step by step. A more classical workflow (the edit-compile-run-debug loop) would consist of writing a full-blown program, and then performing a complete analysis. This is generally more tedious. It is more common to build an algorithmic solution iteratively, by doing small-scale experiments and tweaking the parameters, and this is precisely what interactive computing is about.

Integrated Development Environments (IDEs), providing comprehensive facilities for software development (such as a source code editor, compiler, and debugger), are widely used for classical workflows. However, when it comes to interactive computing, alternatives to IDEs exist. We will review them here.

How to do it...

Here are a few possible workflows for interactive computing, by increasing order of complexity. Of course, IPython is at the core of all of these methods.

The IPython terminal

IPython is the de facto standard for interactive computing in Python. The IPython terminal (the ipython command) offers a command-line interface specifically designed for REPLs. It is a much more powerful tool than the native Python interpreter (the python command). The IPython terminal is a convenient tool for quick experiments, simple shell interactions, and to find help. Forgot the input arguments of NumPy's savetxt function? Just type in numpy.savetxt? in IPython (you will first need to use import numpy, of course). Some people even use the IPython terminal as a (sophisticated) calculator!

Yet, the terminal quickly becomes limited when it is used alone. The main issue is that the terminal is not a code editor, and thus entering more than a few lines of code can be inconvenient. Fortunately, there are various ways of solving this problem, as detailed in the following sections.

IPython and text editor

The simplest solution to the not-a-text-editor problem is, perhaps unsurprisingly, to use IPython along with a text editor. The %run magic command then becomes the central tool in this workflow:

  • Write some code in your favorite text editor and save it in a myscript.py Python script file.
  • In IPython, assuming you are in the right directory, type in %run myscript.py.
  • The script is executed. The standard output is displayed in real time in the IPython terminal along with possible errors. Top-level variables defined in the script are accessible in the IPython terminal at the end of the script's execution.
  • If code changes are required in the script, repeat the process.

Tip

The IPython-text editor workflow can be made yet more efficient with adequate keyboard shortcuts. You can, for instance, automate your text editor such that, when pressing F8, the following command is executed in the running IPython interpreter:

%run <CURRENT_FILE_NAME>

This approach is described here (on Windows, with Notepad++ and AutoHotKey):

http://cyrille.rossant.net/python-ide-windows/

With a good text editor, this workflow can be quite efficient. As the script is reloaded when you execute %run, your changes will be taken into account automatically. Things become more complicated when your script imports other Python modules that you modify, as these won't be reloaded with %run. You can use a deep reload to solve this problem:

import myscript
from IPython.lib.deepreload import reload as dreload
dreload(myscript)

Modules imported in myscript will then be reloaded. A related IPython magic command is %autoreload (you first need to execute %load_ext autoreload). This command tries to automatically reload the modules imported in the interactive namespace, but it is not always successful. You may need to reload the changed modules explicitly with reload(module) (in Python 2) or imp.reload(module) (Python 3).

The IPython notebook

The IPython notebook plays a central role in efficient interactive workflows. It is a well-designed mix between a code editor and a terminal, bringing the best of both worlds within a unified environment.

You can start writing all your code in your notebook's cells. You write, execute, and test your code at the same place, thereby improving your productivity. You can put long comments in Markdown cells and structure your notebook with Markdown headers.

Once portions of your code become mature enough and do not require further changes, you can (and should) refactor them into reusable Python components (functions, classes, and modules). This will clean up your notebooks and facilitate future reuse of your code. Let's emphasize the fact that it is very important to refactor your code constantly into reusable components. IPython notebooks are currently not easily reusable by third-party code, and they are not designed for this. Notebooks are convenient for preliminary analyses and exploratory research, but they should not preclude you from regularly cleaning and refactoring your code into Python components.

A major advantage of notebooks is that they give you documents retracing everything you did with your code. They are extremely useful for reproducible research. Notebooks are saved in human-readable JSON documents, thus they work relatively well with version control systems such as Git.

Integrated Development Environments

IDEs are particularly well-adapted for classic software development, but they can also be used for interactive computing. A good Python IDE combines a powerful text editor (for example, one that includes features such as syntax highlighting and tab completion), an IPython terminal, and a debugger within a unified environment.

There are multiple commercial and open-source IDEs for most platforms. Eclipse/PyDev is a popular (although slightly heavy) open source cross-platform environment. Spyder is another open source IDE with good integration of IPython and matplotlib. PyCharm is one of many commercial environments that support IPython.

Microsoft's IDE for Windows, Visual Studio, has an open source plugin named Python Tools for Visual Studio (PTVS). This tool brings Python support to Visual Studio. PTVS natively supports IPython. You don't necessarily need a paid version of Visual Studio; you can download a free package bundling PTVS with Visual Studio.

There's more...

Here are a few links to various IDEs for Python:

See also

  • The Learning the basics of the distributed version control system Git recipe
  • The Debugging your code with IPython recipe
上一章目录下一章