PnetCDF-Python is a Python interface to PnetCDF, a high-performance I/O library for accessing NetCDF files in parallel. It can provide MPI-based parallel python programs to achieve a scalable I/O performance.

• Python 3.9 or later.
• numpy Python package.
• MPI C library and Python package, mpi4py.
  • Note when using mpi4py 4.0 and MPICH, MPICH version 4.2.2 and later is required.
• PnetCDF C library, built with shared libraries.

• Make sure you have a working MPI and PnetCDF-C software installed.
• Run pip command below to install PnetCDF-Python library from PyPI:

  CC=/path/to/mpicc PNETCDF_DIR=/path/to/pnetcdf/dir/ pip install pnetcdf
  

• Clone this GitHub repository
• Required software for developer installation:
  • The above mentioned dependent software are installed and additionally,
  • Cython, packaging, setuptools>=65 and wheel.
• Commands to install.

  export CC=/path/to/mpicc
  export PNETCDF_DIR=/path/to/pnetcdf/dir
  pip install --no-build-isolation -e .
  

• Testing
  • Additional packages are required: torch and torchvision
  • Add the PnetCDF and MPI installation paths to the environment variable LD_LIBRARY_PATH, for example by running command:

    export LD_LIBRARY_PATH="${PNETCDF_DIR}/lib:${MPI_DIR}/lib:${LD_LIBRARY_PATH}"
    

  • Command "make check" tests all the programs available in folders "test/" and "examples/" by running one MPI process.
  • Command "make ptests" tests all the programs by running more than one MPI process.
  • Note when using OpenMPI, use command below.

    make check  TESTMPIRUN="/path/to/OpenMPI/bin/mpirun --oversubscribe"
    make ptests TESTMPIRUN="/path/to/OpenMPI/bin/mpirun --oversubscribe"
    

• Example python programs available in folder ./examples.
• PnetCDF-python User Guide
• Data objects in PnetCDF python programming
• Comparison of PnetCDF-python to NetCDF4-python
• PnetCDF project home page
• PnetCDF of C/Fortran library repository

• Youjia Li <[email protected]>
• Wei-keng Liao <[email protected]>

Ongoing development and maintenance of PnetCDF-python is supported by the U.S. Department of Energy's Office of Science, Scientific Discovery through Advanced Computing (SciDAC) program, RAPIDS/OASIS Institute.