Xmipp

Installation Internals (or, "how does the installation process work?")

Overview

See the installation process from the user's point of view: HowToInstall

Xmipp depends on lots of external packages. Some are prerrequisites (that is, they must be available before install, and are handled with OS package manager), others are included in Xmipp repository and built as part of the installation process.

Currently, xmipp is built with scons, which in turn depends on python. Python version is important for xmipp, so in the end it makes sense to have a custom python (xmipp_python)

One of the requisites of the installation is that it should work without any requisites (for the user). The least one can ask to users is a shell, hence the installation begins with install.sh. This script proceeds to the main steps of the installation process: build a suitable python, compile the external libraries and Xmipp BuildingWithSCons.

Let's see each step in more detail.

Stage 1: install.sh

The script is well documented, so it's the best place to look in order to understand its details. However, you can find here some relevant information about it.

Key steps performed by install.sh:

  • Create environment config files: .xmipp.bashrc / .xmipp.csh
  • Extract external packages files
  • Build and install external packages (included in xmipp repository): sqlite, fftw, jpeg, tiff, hdf5, tcl/tk, arpack...
  • Build custom xmipp_python and required python modules (numpy, matplotlib, mpi, PyCifRW...)
  • Download and install external packages (not available in repository): java
  • Proceed to phase 2: xmipp installation (setup.py)

In the build directory you can find the logs of the compilation of every package, so you can see more details when a package fails to compile.

The install.sh script has been greatly improved for iterative installation: shall some step fail, you can choose to retry that step, or go on with any of the others. The script help shows the steps available:

./install.sh --help
(...)
--alglib=[true|false]
    Execute or not selected operation over alglib library. When just --alglib is given, true is asumed.
--bilib=[true|false]
--condor=[true|false]
--fftw=[true|false]
--gtest=[true|false]
--hdf5=[true|false]
--imagej=[true|false]
--jpeg=[true|false]
--scons=[true|false]
--sqlite=[true|false]
--tiff=[true|false]
--nma=[true|false]
--cltomo=[true|false]

PYTHON-RELATED OPTIONS:
--python=[true|false]
--matplotlib=[true|false]
--mpi4py=[true|false]
--numpy=[true|false]
--psutil=[true|false]
--tcl-tk=[true|false]
--tcl=[true|false]
--tk=[true|false]
--pymodules=[true|false]

On a clean installation of xmipp, you would simply run:

./install.sh -j PROCS

which would build xmipp from scratch (including dependencies)

Stage 2: setup.py

Actions:

  • gui: display the compilation GUI, defined in protocols/compile_gui.py
  • update: update xmipp (git pull)
  • configure: scons configure (see BuildingWithSCons)
  • compile: scons compile (see BuildingWithSCons)
  • clean: scons compile with clean option
  • install: run xmipp_apropos --update

Typical execution of setup.py (see HowToInstall):

./setup.py -j 4 configure compile gui install

In .xmipp.bashrc there are some alias to setup.py. For instance, "xcompile" is the same as running "setup.py -j 2 compile"

Once Xmipp is working, one would use these alias when needed. For instance, when a developer modifies some source code, she would update the binaries with xcompile

-- JesusCuenca - 2013-09-06