Installation

Requirements

The only requirements are

  • Python (3.6 or later)

  • numpy (1.6 or later)

  • Cython (0.2 or later)

Because pyqg is a pseudo-spectral code, it realies heavily on fast-Fourier transforms (FFTs), which are the main performance bottlneck. For this reason, we try to use fftw (a fast, multithreaded, open source C library) and pyfftw (a python wrapper around fftw). These packages are optional, but they are strongly recommended for anyone doing high-resolution, numerically demanding simulations.

If pyqg can’t import pyfftw at compile time, it can fall back on numpy’s fft routines. Note that the numpy_ fallback requires a local install (see [below](#installing-pyqg)).

PyQG can also conveniently store model output data as an xarray dataset. The feature (which is used in some of the examples in this documentation) requires xarray.

Instructions

The easiest and quickest way: installing pyqg with conda

We suggest that you install pyqg using conda. This will automatically install pyfftw as well, so then you will be done and can ignore the remaining instructions on this page. To install pyqg with conda,

$ conda install -c conda-forge pyqg

Alternatives

In our opinion, the best way to get python and numpy is to use a distribution such as Anaconda (recommended) or Canopy. These provide robust package management and come with many other useful packages for scientific computing. The pyqg developers are mostly using anaconda.

Note

If you don’t want to use pyfftw and are content with numpy’s slower performance, you can skip ahead to Installing pyqg.

Installing fftw and pyfftw can be slightly painful. Hopefully the instructions below are sufficient. If not, please send feedback.

Installing fftw and pyfftw

Once you have installed pyfftw via one of these paths, you can proceed to Installing pyqg.

The easy way: installing with conda

If you are using Anaconda, we have discovered that you can easily install pyffw using the conda command. Although pyfftw is not part of the main Anaconda distribution, it is distributed as a conda pacakge through several user channels.

There is a useful blog post describing how the pyfftw conda package was created. There are currently 13 pyfftw user packages hosted on anaconda.org. Each has different dependencies and platform support (e.g. linux, windows, mac.) The conda-forge version is the most popular and appears to have the broadest cross-platform support. To install it, open a terminal and run the command

$ conda install -c conda-forge pyfftw

The hard way: installing from source

This is the most difficult step for new users. You will probably have to build FFTW3 from source. However, if you are using Ubuntu linux, you can save yourself some trouble by installing fftw using the apt package manager

$ sudo apt-get install libfftw3-dev libfftw3-doc

Otherwise you have to build FFTW3 from source. Your main resource for the FFTW homepage. Below we summarize the steps

First download the source code.

$ wget http://www.fftw.org/fftw-3.3.4.tar.gz
$ tar -xvzf fftw-3.3.4.tar.gz
$ cd fftw-3.3.4

Then run the configure command

$ ./configure --enable-threads --enable-shared

Note

If you don’t have root privileges on your computer (e.g. on a shared cluster) the best approach is to ask your system administrator to install FFTW3 for you. If that doesn’t work, you will have to install the FFTW3 libraries into a location in your home directory (e.g. $HOME/fftw) and add the flag --prefix=$HOME/fftw to the configure command above.

Then build the software

$ make

Then install the software

$ sudo make install

This will install the FFTW3 libraries into you system’s library directory. If you don’t have root privileges (see note above), remove the sudo. This will install the libraries into the prefix location you specified.

You are not done installing FFTW yet. pyfftw requires special versions of the FFTW library specialized to different data types (32-bit floats and double-long floars). You need to-configure and re-build FFTW two more times with extra flags.

$ ./configure --enable-threads --enable-shared --enable-float
$ make
$ sudo make install
$ ./configure --enable-threads --enable-shared --enable-long-double
$ make
$ sudo make install

At this point, you FFTW installation is complete. We now move on to pyfftw. pyfftw is a python wrapper around the FFTW libraries. The easiest way to install it is using pip:

$ pip install pyfftw

or if you don’t have root privileges

$ pip install pyfftw --user

If this fails for some reason, you can manually download and install it according to the instructions on github. First clone the repository:

$ git clone https://github.com/hgomersall/pyFFTW.git

Then install it

$ cd pyFFTW
$ python setup.py install

or

$ python setup.py install --user

if you don’t have root privileges. If you installed FFTW in a non-standard location (e.g. $HOME/fftw), you might have to do something tricky at this point to make sure pyfftw can find FFTW. (I figured this out once, but I can’t remember how.)

Installing pyqg

Note

The pyqg kernel is written in Cython and uses OpenMP to parallelise some operations for a performance boost. If you are using Mac OSX Yosemite or later OpenMP support is not available out of the box. While pyqg will still run without OpenMP, it will not be as fast as it can be. See Installing with OpenMP support on OSX below for more information on installing on OSX with OpenMP support.

With pyfftw installed, you can now install pyqg. The easiest way is with pip:

$ pip install pyqg

You can also clone the pyqg git repository to use the latest development version.

$ git clone https://github.com/pyqg/pyqg.git

Then install pyqg locally on your system:

$ cd pyqg && pip install --editable .

This will also allow you to make and test changes to the library. pyqg is a work in progress, and we really encourage users to contribute to its Development

Note that due to Cython build considerations, this local install method is required if you do not wish to use pyfftw.

Installing with OpenMP support on OSX

There are two options for installing on OSX with OpenMP support. Both methods require using the Anaconda distribution of Python.

  1. Using Homebrew

Install the GCC-5 compiler in /usr/local using Homebrew:

$ brew install gcc --without-multilib --with-fortran

Install Cython from the conda repository

$ conda install cython

Install pyqg using the homebrew gcc compiler

$ CC=/usr/local/bin/gcc-5 pip install pyqg

  1. Using the HPC precompiled gcc binaries.

The HPC for Mac OSX sourceforge project has copies of the latest gcc precompiled for Mac OSX. Download the latest version of gcc from the HPC site and follow the installation instructions.

Install Cython from the conda repository

$ conda install cython

Install pyqg using the HPC gcc compiler

$ CC=/usr/local/bin/gcc pip install pyqg