The EDSS/Models-3 I/O API



NOTES

The Models-3 I/O API is A programming interface, not a data format !!

I/O API files are not synonymous with "netCDF files" !!
Instead, netCDF is one of four (and a half) distinct lower layers on which the data and metadata structures for I/O API files are currently available; additional lower layers have been incorporated at times in the past, and may very well be incorporated at various times in the future. Attempts to treat the I/O API as "just a data format" have consistently failed in all cases because the persons attempting to do so have not fully understood the I/O API data structures embedded in these files. (And generally they haven't attempted to contact the I/OL API author about these data structures, either.
(Does anyone want to fund development of an MPI 2 lower layer? -- Contact the author!)

The I/O API follows a "punctuated rolling release" model: great effort has been expended to provide and maintain upwards compatibility, so that any program developed to use the I/O API will work correctly with any subsequent release of the I/O API. Newer releases will contain (a) bug-fixes, and (b) new feature and capabilities. There are only the following upward-compatibility issues:

I/O API-2.1 to I/O API-3.0: 2002
Requires re-linking due to netCDF API changes—fully source- and object-file compatible: uses netCDF-3.x which was not fully compatible with netCDF-2.x.

Targets Fortran-90 compilers only, and starts to use Fortran-90 features such as MODULEs

New features for 3.0 include MODULE M3UTILIO, containing INTERFACEs, INCLUDE-files and other declarations.

Deprecate direct use of INCLUDE-files.

Deprecate use of M3ERR() routine in favor of the more general M3EXIT()

Deprecate m3tools/utmtool, in favor of its replacement m3tools/projtool.

I/O API-3.0 to I/O API-3.1: 2013
Requires re-compile: Fully source compatible but not object compatible—fundamental PARAMETER MXVARS3 changed from 120 to 2048.

More INTERFACEs in MODULE M3UTILIO

New experimental MODULE MATXATTS supports full input-grid and output-grid descriptions for matrix files (which may be used, for example, for grid-to-grid transforms by such programs as m3tools/mtxcple).

I/O API-3.1 to I/O API-3.2:
Minor source code changes required:

More INTERFACEs in MODULE M3UTILIO

Respond to failure in netCDF-4 API backwards compatibility:
Replace netCDF-2 CALL NC*() statements throughout with the corresponding netCDF-3 IERR = NF_*(), to accommodate the fact that recent versions of netCDF-Fortran silently did away with the former.
I/O API-3.2 works with both netCDF-3 and netCDF-4 (but netCDF-4 breaks I/O API-3.1)

More m3tools programs:
findwndw, gridprobe, insertgrid, m3probe, mpasdiff, mpasstat, mpastom3, datshift, greg2jul, , jul2greg, juldiff, julshift, timeshift, wrfgriddesc, wrftom3

Experimental MODULE MATXATTS becomes MODULE MODATTS3, with additional support for various extra (CF, CMAQ, SMOKE(place-holder for future specification)) metadata.

New MODULE MODGCTP for geospatial transform and interpolation operations; now contains some INTERFACEs formerly in MODULE M3UTILIO.

New MODULE MODNCFIO safely encapsulates netCDF and PnetCDF declarations, replacing include-file NETCDF.EXT (and pnetcdf.inc)

PnetCDF/MPI distributed I/O support for CMAQ. New MODULE MODPDATA encapsulates data structures and routines for distributed CMAQ I/O.

New MODULE MODWRFIO for WRF-netCDF-format I/O, and related m3tools programs wrfgriddesc and wrftom3.

New MODULE MODMPASFIO for MPAS-format-netCDF I/O and MPAS (unstructured Voronoi-complex) grid-geometry utilities. and related m3tools programs mpasdiff, mpasstat, mpastom3

Remove previously-deprecated m3tools program utmtool (superseded by program projtool).


Abstract

The Models-3/EDSS Input/Output Applications Programming Interface (I/O API) provides the environmental model developer with an easy-to-learn, easy-to-use programming library for data storage and access, available from both Fortran and C. The same routines can be used for both file storage (using netCDF files) and model coupling (using PVM mailboxes). It is the standard data access library for both the NCSC/CMAS's EDSS project and EPA's Models-3, CMAQ, and SMOKE, as well as various other atmospheric and hydrological modeling systems. There was a external-package wrapper for the I/O API in the Weather Research and Forecasting Model [WRF], which provided the option of direct I/O API output, enabling (among other things) the use of I/O API coupling mode to couple WRF-Chem with SMOKE—an effort unfortunately dropped by NCAR.

Since it is built around an object based conception of environmental modeling, the I/O API provides a variety of data structure types for organizing the data, and a set of access routines which offer selective direct access to the data in terms meaningful to the modeler. For example,

Read layer 1 of variable 'OZONE' from 'CONCFILE' for 5:00 PM GMT on July 19, 1988 and put the result into array A.
is a direct English translation of a typical I/O API READ3() call. "Selective direct access" means that this READ3 call retrieves exactly this ozone data immediately. It does not have to read through previous hours of data, nor whatever other variables (such as NOX or PAN) are in what order in the file. Data can be read or written in any order (or not at all). This characteristic provides the following advantages: I/O API files also have the following characteristics

The I/O API also contains an extensive set of utility routines for manipulating dates and times, performing coordinate conversions, storing and recalling grid definitions, sparse matrix arithmetic, etc., as well as a set of geospatial-transform, data-manipulation and statistical analysis programs, and programs for interfacing with a number of external modeling systems, such as CAMX/UAM, WRF, and MPAS. It has an extensive documentation set, including

Various extensions and research efforts to the I/O API have been developed or are under development. Developments include the use of the I/O API interacting with PVM for model coupling, and adding operations to read or write entire time series (with multiple time steps) as single operations, and research projects include data-parallel I/O and a very powerful "geospatial cell complex" data type with polygonal-cell decompositions that may be both time independent (as for finite elememt modeling) and time dependent (as for moving-mesh plume-in-grid modeling).

Copyright
These web pages are copyright © 1992-2002 MCNC and Carlie J. Coats, Jr., © 2003-2005 by Baron Advanced Meteorological Systems (hereinafter, BAMS), © 2005-2013,2017- Carlie J. Coats, Jr., and © 2014- UNC Institute for the Environment. The I/O API source code and documentation are copyright © 1992-2002 MCNC and Carlie J. Coats, Jr., © 2003-2013 BAMS, © 2005- Carlie J. Coats, Jr., and © 2014- UNC Institute for the Environment. The library source code is distributed under the GNU Lesser (library) Public License (LGPL) Version 2.1, and the tools-program source code is distrubuted under the GNU General Public License, Version 2, subject to copyright-statement retention constraints. See the Notices: Copyright, and Acknowledgements page.

    $Id: index.html 165 2020-05-27 16:13:18Z coats $