The only requirements are
Python (3.6 or later)
numpy (1.6 or later)
Cython (0.2 or later)
Because pyqg is a pseudo-spectral code, it realies heavily on fast-Fourier transforms (FFTs), which are the main performance bottlneck. For this reason, we try to use fftw (a fast, multithreaded, open source C library) and pyfftw (a python wrapper around fftw). These packages are optional, but they are strongly recommended for anyone doing high-resolution, numerically demanding simulations.
If pyqg can’t import pyfftw at compile time, it can fall back on numpy’s fft routines. Note that the numpy_ fallback requires a local install (see [below](#installing-pyqg)).
PyQG can also conveniently store model output data as an xarray dataset. The feature (which is used in some of the examples in this documentation) requires xarray.
The easiest and quickest way: installing pyqg with conda¶
We suggest that you install pyqg using conda. This will automatically install pyfftw as well, so then you will be done and can ignore the remaining instructions on this page. To install pyqg with conda,
$ conda install -c conda-forge pyqg
In our opinion, the best way to get python and numpy is to use a distribution such as Anaconda (recommended) or Canopy. These provide robust package management and come with many other useful packages for scientific computing. The pyqg developers are mostly using anaconda.
If you don’t want to use pyfftw and are content with numpy’s slower performance, you can skip ahead to Installing pyqg.
Installing fftw and pyfftw can be slightly painful. Hopefully the instructions below are sufficient. If not, please send feedback.
Installing fftw and pyfftw¶
Once you have installed pyfftw via one of these paths, you can proceed to Installing pyqg.
The easy way: installing with conda¶
If you are using Anaconda, we have discovered that you can easily install
pyffw using the
conda command. Although pyfftw is not part of the main
Anaconda distribution, it is
distributed as a conda pacakge through several user channels.
There is a useful blog post describing how the pyfftw conda package was created. There are currently 13 pyfftw user packages hosted on anaconda.org. Each has different dependencies and platform support (e.g. linux, windows, mac.) The conda-forge version is the most popular and appears to have the broadest cross-platform support. To install it, open a terminal and run the command
$ conda install -c conda-forge pyfftw
The hard way: installing from source¶
This is the most difficult step for new users. You will probably have to build FFTW3 from source. However, if you are using Ubuntu linux, you can save yourself some trouble by installing fftw using the apt package manager
$ sudo apt-get install libfftw3-dev libfftw3-doc
Otherwise you have to build FFTW3 from source. Your main resource for the FFTW homepage. Below we summarize the steps
First download the source code.
$ wget http://www.fftw.org/fftw-3.3.4.tar.gz $ tar -xvzf fftw-3.3.4.tar.gz $ cd fftw-3.3.4
Then run the configure command
$ ./configure --enable-threads --enable-shared
If you don’t have root privileges on your computer (e.g. on a shared
cluster) the best approach is to ask your system administrator to install
FFTW3 for you. If that doesn’t work, you will have to install the FFTW3
libraries into a location in your home directory (e.g.
add the flag
--prefix=$HOME/fftw to the configure command above.
Then build the software
Then install the software
$ sudo make install
This will install the FFTW3 libraries into you system’s library directory.
If you don’t have root privileges (see note above), remove the
will install the libraries into the
prefix location you specified.
You are not done installing FFTW yet. pyfftw requires special versions of the FFTW library specialized to different data types (32-bit floats and double-long floars). You need to-configure and re-build FFTW two more times with extra flags.
$ ./configure --enable-threads --enable-shared --enable-float $ make $ sudo make install $ ./configure --enable-threads --enable-shared --enable-long-double $ make $ sudo make install
At this point, you FFTW installation is complete. We now move on to pyfftw.
pyfftw is a python wrapper around the FFTW libraries. The easiest way to
install it is using
$ pip install pyfftw
or if you don’t have root privileges
$ pip install pyfftw --user
If this fails for some reason, you can manually download and install it according to the instructions on github. First clone the repository:
$ git clone https://github.com/hgomersall/pyFFTW.git
Then install it
$ cd pyFFTW $ python setup.py install
$ python setup.py install --user
if you don’t have root privileges. If you installed FFTW in a non-standard location (e.g. $HOME/fftw), you might have to do something tricky at this point to make sure pyfftw can find FFTW. (I figured this out once, but I can’t remember how.)
The pyqg kernel is written in Cython and uses OpenMP to parallelise some operations for a performance boost. If you are using Mac OSX Yosemite or later OpenMP support is not available out of the box. While pyqg will still run without OpenMP, it will not be as fast as it can be. See Installing with OpenMP support on OSX below for more information on installing on OSX with OpenMP support.
With pyfftw installed, you can now install pyqg. The easiest way is with pip:
$ pip install pyqg
You can also clone the pyqg git repository to use the latest development version.
$ git clone https://github.com/pyqg/pyqg.git
Then install pyqg locally on your system:
$ cd pyqg && pip install --editable .
This will also allow you to make and test changes to the library. pyqg is a work in progress, and we really encourage users to contribute to its Development
Note that due to Cython build considerations, this local install method is required if you do not wish to use pyfftw.
Installing with OpenMP support on OSX¶
There are two options for installing on OSX with OpenMP support. Both methods require using the Anaconda distribution of Python.
Install the GCC-5 compiler in
/usr/local using Homebrew:
$ brew install gcc --without-multilib --with-fortran
Install Cython from the conda repository
$ conda install cython
Install pyqg using the homebrew
$ CC=/usr/local/bin/gcc-5 pip install pyqg
Using the HPC precompiled gcc binaries.
The HPC for Mac OSX sourceforge project has copies of the latest
gcc precompiled for Mac OSX. Download the latest version of gcc from the HPC site and follow the installation instructions.
Install Cython from the conda repository
$ conda install cython
Install pyqg using the HPC
$ CC=/usr/local/bin/gcc pip install pyqg