What's New with the I/O API

New generic in-place-permutation routine PERMUTI for use with SORTI

Version 3.2.73

New generic routine PERMUTI uses indexes produced by SORTI to do easy in-place sorting of key and data tables, as in the following example:

        ...
        DO  I = 1, N	!  initialize INDX array
            INDX( I ) = I
        END DO
        CALL SORTI(   N, INDX, LIST1, LIST2, LIST3 )
        CALL PERMUTI( N, INDX, LIST1, LIST2, LIST3 )
        CALL PERMUTI( N, INDX, DATA1, DATA2 )
        ...

Support for "vertical grid" types GISLAY3 and TBLLAY3 for I/O API internal routines CKFIL3 and CKDESC3

Version 3.2.72

Subject says it.

Bug-fix for m3tools program wrfwndw, I/O API routine INIT3

Version 3.2.71

Need to set *_GRID_DIMENSION attributs for output window.
Don't you love having to deal with software systems that lack referential integrity ?!

PSCHIEN points out editor-caused line-continuation and undeclared new-tag variable bugs in I/O API routine INIT3.

Bug-fix for m3tools program wndwpoints

Version 3.2.70

Subject says it.

New m3tools programs wndwpoints, wndwptdata, and wrfwndw
MODULE MODWRFIO update to match.

Version 3.2.69

wndwpoints and wndwptdata

Subset the point-sources in SMOKE-I/O API POINTS and point-data files to a specified a grid-window and write the result to SMOKE-I/O API files.

wrfwndw

Window/extract variables from WRF-format netCDF files to a new WRF-format netCDF file.

MODULE MODWRFIO

Make new PUBLIC MODULE-variables CDFIDW and FMODEW for use in program wrfwndw.
Bug-fix for windowed-case XOFFS1, YOFFS1

MODULE MODWRFIO

Make new PUBLIC MODULE-variables has new variants of CREATENC(), DESCNCVAR(), READNCVAR(), and WRITENCVAR() with additional netCDF file-ID argument. These presume an already-open file, and do not close that file upon completion.

Bug-fixes for interpolation-mode operation of m3tools programs m3cple and mtxcalc
I/O API-4.0 bug-fixes.

Version 3.2.68

Subject says it.

I/O API-4.0: Conversion to Standard conforming "free" (.f90) source code format.
SCALEFAC for M3Tools program m3cple

Version 3.2.67

The remaining fixed-format (f77-style) Fortran codes were converted to the newer Standard conforming(.f90-style) free source code format. This was aided by Willem Vermin's findent utility: see https://gitlab.com/wvermin/findent.

Except for the source-code format, this version is completely compatible with I/O API-3.2. Minor bug-fixes were made to various prompts and error-messages but there are no usage nor interface changes.

Note that this version should not be installed "on top of" previous versions, because of name-collisions between older .f source files and their newer .f90 counterparts, and the resulting make problems that causes.

M3Tools program m3cple has a new environmewnt-variable option SCALEFAC for emissions interpolation (necessary because SMOKE gridded emissions data uses non-interpolatable, non-MKS units of the form mass per unit time per grid-cell, and one must re-scale the data by the ratio of the output cell-area to the input cell-area).

Bug-fix in gridded-output versions of BILIN, BMATVEC, m3tools program latlon
Makefiles now create ${OBJDIR} if necessary.

Version 3.2.66

There was an off-by-one single-indexing error in the gridded-output versions of generic bilinear-interpolation routines BILIN and BMATVEC, now fixed.

There was a bug in the boundary-LATLON-file creation functionality for m3tools program latlon, now fixed.

Rules in the Makefiles now check to see if the build directory ${OBJDIR} already exists; and if not, they create it (simplifying the build-process for the I/O API.

Improved error reporting in GETEFILE(), GETDFILE(), GETFFILE(), INITLOG3()
Improved debug-flags in Makeinclude.Linux2_x86_64dbg, Makeinclude.Linux2_x86_64gfortdbg; new Makeinclude.Linux2_x86_64pgdbg.
New vertical coordinate type TBLLAY3 (=-2) in PARMS3.EXT and PARMS3-LARGE.EXT for interpolation-table-valued I/O API files (as in MOVESTABLE emissions).

Version 3.2.65

Subject says it.

New pointwise-product operation for use in, e.g., m3tools program M3DIFF
Size-traps in GRIDOPS to avoid denormals in denominators.

Version 3.2.64

Calvin Arter (UNC Institute for the Environment) introduced a new pointwise-product grid-algebra operation A * B into SUBROUTINE GRIDOPS. This operation is then used by (among others) m3tools programs m3diff and mpasdiff.

In particular, operation A * B may be used in conjunction with a REAL mask-variable (=1 on the requested region, =0 elsewhere) "mask" file to "mask out" regions for selected variables.

Size traps (against EPS = 1.1755E-38 were put into SUBROUTINE GRIDOPS, to avoid denormals in denominators of operations having a divide (e.g., normalized differences) in them.

Typo-fixes in PARMS3.EXT, crtfil3.F90

Version 3.2.63

Subject says it.

Fix ADJUSTL bug reported by ZHUANSHI HE in GETNUM, GETINT8

Version 3.2.62

Subject says it.

New routines ENVINT8, BENVINT8, GETINT8, GETINT81 for (64-bit) INTEGER*8; bug-fix in MODMPASFIO

Version 3.2.61

See INTERFACE ENVGETS and INTERFACE GETVAL.

There was a bug in MPAS-to-M3 time-format conversion in MODULE MODMPASFIO for the "no seconds field" MPAS-date-format case.

New utility routine FIXNULLS to fix trailing ASCII-nulls in strings; use FIXNULLS for MPAS netCDF.

Generic GCD(), LCM()

Version 3.2.60

Kemal Gurer, California Air Resources Board, pointed out this problem with MPAS-format netCDF, which was padding character-strings with trailing nulls. (The netCDF Fortran bindings do not have a sophisticated understanding of Fortran character-strings and do not always "do the right thing.")

New SUBROUTINE FIXNULLS() in MODULE M3UTILIO replaces trailing ASCII-nulls with blanks (to deal with problems caused by the lack of sophistication of netCDF's Fortran bindings), especially for MPAS-format netCDF.

MODULE MPASFIO now uses FIXNULLS() to deal with strings coming from this defect in MPAS-format netCDF.

Generic GCD(), LCM() for INTEGER and INTEGER(8) arguments.

New Makeinclude.*gfort10* files for gfortran version 10; new 20200828 tagged version.

Version 3.2.59

Version 10 of gfortran takes a particularly idiosyncratic interpretation of the (latest) Fortran-2018 Standard. Starting with netCDF and going through the I/O API, M3Tools, CMAQ. SMOKE, and all other programs, program-builds that use this version of gfortran need to include the compile-flag

    -fallow-argument-mismatch
    

In order to get rid of the enormous number of spurious warning messages with most versions of gfortran, there are now new Makeinclude files specific to gfortran-10; the other gfortran Makeinclude files have reverted to the pre-July 2020 behavior.

This version of the I/O API has been tagged 20200828 n GitHub.

Additional compile-flag for gfortran version 10;
Bug-fixes in pnetcdf create-file routine.

Version 3.2.58

Version 10 of gfortran takes a particularly idiosyncratic interpretation of the (latest) Fortran-2018 Standard.

Thanks to Mrs. Indumathi S Iyer, (SO/D), BARC, for pointing out this compiler-problem and help with testing the fix, indicated below.—CJC

AS of July 12, 2020, the relevant ioapi/Makeinclude.${BIN} files have been modified to add Fortran compile-flag

-std=legacy

so that this interpretation does not cause a compile-error.

However, using this compiler version will cause the generation of a huge number of spurious warning-messages, as the compiler is still trying to enforce its version of the Fortran-2018 (not Fortran-90, not Fortran-95, not Fortran-2008) Standard.

Additionally, there was an argument-list error in PnetCDF/MPI enabled routine pn_crtfil3.f, now fixed.

Bug-fix in m3tools/gridprobe

Version 3.2.57

Subject says it.

Increased message-buffer length in STR2DBLE, STR2INT, STR2REAL routines

Version 3.2.56

Subject says it. This came from a situation where badly-formatted input was so bad that it caused the error-message buffer to overflow, preventing the calling program from diagnosing just what input was so badly-formatted...

Buffer-length is now 256. This should be adequate for all but the most egregious badly-formed inputs.

Bug-fix in M3Tools program m3interp

Version 3.255

skunwar on the CMAS Forum has found a bug in M3Tools program m3interp: it was not handling boundary-condition files correctly. Now fixed.

I/O API-3.2 Release 2020104

Version 3.2.54

This version established for CMAQ (etc.) tracking purposes, with enhancements and bug-fixes for WRF-CMAQ. New changes include the following:

New test suite

Subsequent to make all, the command make test runs a series of tests that ensures the I/O API and m3tools programs are correctly installed, and demonstrate various I/O API capabilities by running various m3tools programs quasi-interactively. This will display all the logs from the relevant program-runs; look for "successful completion" messages at the end of each of these logs.

Makeinclude.Linux*ifort* changes

to support Intel compilers for both pre-version-16 and post-version-16 (changes to OpenMP compile-flags: both versions of the flags are now present. This will result in successful builds, but at the cost of a warning for the compiler not recognizing the other version of this flag.

M3Tools program m3tproc

changes for better handling of default responses.

Bug-fixes for BUFFERED3 virtual file type

from Fahim Sidi, US EPA

gfortran ddebug Makeinclude

enhancements and clean-up

Documentation enhancements

Enhancement to M3Tools program GRIDPROBE

Version 3.2.53

M3Tools program GRIDPROBE now supports either M3IO file output, ASCII output, or both (instead of only ASCII output).

To turn on M3IO output

    setenv  OUTFILE    <path name for output ASCII   file> or "NONE"
    

or to turn it off

    setenv  OUTFILE    NONE
    

Likewise, to turn on ASCII output,

    setenv  REPORT    <path name for output ASCII   file> or "NONE"
    

or to turn it off

    setenv  REPORT    NONE
    

Mods to MODULE MODMPASFIO for I/O API-3.2-large compatibility

Version 3.2.52

As originally dimensioned, the set of MODULE variables for MODULE MODMPASFIO exceeded 8 GB in the I/O API-3.2-large case, which causes link errors with Linux x86_64 small-memory-module compilation.

New dimensioning PARAMETERS are defined as follows, reducing the size of the state variables to about 300 MB:

INTEGER, PUBLIC, PARAMETER :: MXMPASVARS = 2048

Maximum number of MPAS-file variables for this module.

INTEGER, PUBLIC, PARAMETER :: MXMPASFILE = 64

Maximum number of MPAS-files for this module.

Increase MXFILE3 to 256;
New PARMS3-LARGE.EXT for CMAQ-DDM

Version 3.2.51

Subject says it.

PARMS3-LARGE.EXT has

MXFILE3 = 512
MXVARS3 = 16384

for use in CMAQ-DDM and CMAQ-ISAM. It will have somewhat lower performance than the "normal" I/O API and will require substantially more memory and disk space.
Note that the I/O API built with PARMS3-LARGE.EXT must be kept carefully segregated from the normal I/O API.

At the same time, the "normal"-I/O API parameter MXFILE3 was increased to 256. This should not affect any application that does not access the internal I/O API STATE3 data structures (as applications are forbidden to do). Note that CMAQ's pario routines violate this restriction and thus are subject to hard-to-diagnose problems; therefore, any CMAQ versions that use this library should be completely re-compiled from scratch.

Subversion-ID in INIT3() is updated for this release...

Bug-fixes for m3tools programs findwndw, gridprobe, insertgrid, latlon, m3cple, m3interp, projtool

Version 3.2.50

James Beidler, US EPA, reported a problem with the above: since they did not CALL INITSPHERES() before using geospatial transforms from MODULE MODGCTP, they always did their calculations assuming the GRS80 geodetic spheroid.

Fixed.

Bugfix for LIST: file-lists

Version 3.2.49

Winston Hao (NY DEC) reported a bug which caused errors with file-set file lists (a bad recursion in internal worker-routines OPNLIST3() and OPNLOG3()). This bug is now fixed.

Bugfix for M3TOOLS program mtxcple;
ioapi-3.2-large.tar.gz

Version 3.2.48

Subject says it for mtxcple.

For CMAQ-DDM and CMAQ-ISAM, ioapi-3.2-large has expanded the maximum-number-of-files and maximum-variables-per-file parameter-values to

    MXFILE3 =  512
    MXVARS3 =16384
    

This version should be kept carefully isolated from the regular version of the I/O API, and should be used only for CMAQ-DDM and CMAQ-ISAM. It can be downloaded from the CMAS web-site:
ioapi-3.2-large.tar.gz

Bugfix for M3TOOLS programs m3diff, m3stat, m3tshift, n3wndw, vertot
New M3TOOLS program wndwdesc

Version 3.2.47

Winston Hao (NY DEC) reported an off-by-one bug in processing the default value for RUNLEN (which comes in from a user-prompt) for M3TOOLS programs m3diff, m3stat, m3tshift, n3wndw, and vertot for I/O API 3.2. This bug is now fixed.

New M3TOOLS program wndwdesc takes a grid name and the Lat-Lon coordinates for points in opposite corners of a windowed sub-grid of that grid, computes the col-row boundaries for that windowed sub-grid, and generates a single-grid GRIDDESC file for that sub-grid.

There were problems with LIST: file lists. These problems are now fixed (June 15. 2019. This extension had not been tested, and needed a bit of work). Now fixed.

RDTFLAG(*) now puts both its input argument date&time and the on-file date&time into normal form...

Version 3.2.46

...for compatibility with "unusual" date&time usage in MEGAN. (RDTFLAG(*) is an internal routine called by READ3(*) internals.)

Bug-fix in m3tools/mtxcalc;
Clean-ups in MODULE MODGCTP.

Version 3.2.45

There was a typo in the refined-grid definition for m3tools/mtxcalc that had the effect of "squashing" the output-range for the matrix that was calculated. Fixed.

Various clean-ups in I/O API MODULE MODGCTP, enhancing OpenMP parallelism, while these should have no impact upon the results.

Enhanced documentation for m3tools/m3combo—thanks, Uma Shankar! —and for available systems and libraries.

Bug-fix to Makeinclude.*pg*

Version 3.2.44

Jim Dew points out that in the Makeinclude.*pg* (the Makeinclude-files for PGI-compiler binary types), the use of double-quotes in the definition of make-variable MODI is not supported by GNU make, and creates compile-time errors.

It turns out that GNU make doesn't support backslash-escapes of a trailing blank either; however, use of single-quotes to force a trailing blank within that definition

MODI = '-module '

does work.

Note that the distinction of whether there is a trailing blank is necessary: the usual UNIX industry standard directive for module-search directives is

-Idirectory

and the Sun/Oracle variation is

-Mdirectory

with no space (a space would be a directive "delete all current module-search directives", which would subsequently be followed by new module-search directives). PGI wants

-module directory

with a space. And I need to define MODI so that sometimes has a trailing space, and sometimes not. The usual usages of either double-quotes to force the inclusion of a trailing blank or of escaping the trailing blank with a backslash (\) don't work for GNU make ;-(

New routine INDEXL1() and new generic INDEXKEY()

Version 3.2.43

New INTEGER*8-key lookup routine INDEXL1() and new generic interface INDEXKEY() in MODULE M3UTILIO:

    INTERFACE INDEXKEY

        INTEGER FUNCTION INDEX1( NAME, N, NLIST )
        CHARACTER*(*), INTENT(IN   ) :: NAME        !  Character string being searched for
        INTEGER      , INTENT(IN   ) :: N           !  Length of array to be searched
        CHARACTER*(*), INTENT(IN   ) :: NLIST(*)    !  array to be searched
        END FUNCTION INDEX1

        INTEGER FUNCTION INDEXINT1( IKEY, N, NLIST )
        INTEGER, INTENT(IN   ) :: IKEY        !  integer being searched for
        INTEGER, INTENT(IN   ) :: N           !  Length of array to be searched
        INTEGER, INTENT(IN   ) :: NLIST(*)    !  array to be searched
        END FUNCTION INDEXINT1

        INTEGER FUNCTION INDEXL1( LKEY, N, NLIST )
        INTEGER*8, INTENT(IN   ) :: LKEY        !  integer being searched for
        INTEGER,   INTENT(IN   ) :: N           !  Length of array to be searched
        INTEGER*8, INTENT(IN   ) :: NLIST(*)    !  array to be searched
        END FUNCTION INDEXL1

    END INTERFACE       !!  indexkey
    

Add more coordinate systems to m3tools/latlon

Fixes to ioapi/rdatt3,wratt3 and m3tools/mtxcalc,mtxcple; web updates

Version 3.2.42

Bug-fixes in ioapi/rdatt3 and ioapi/wratt3: fixed bug introduced by I/O API 3.2 PnetCDF changes.

Bug-fixes in m3tools/mtxcalc

Enhanced m3tools/mtxcple does not attempt to read/transform nonREAL variables.

Flag-fix in ioapi/Makeinclude.Linux2_x86_64g95dbg

Updated web-copies of I/O API INCLUDE-files.

Bug-fix to m3tools/mtxcalc; better flags for gfortran Makeincludes

Version 3.2.41

Bug-fix in m3tools/mtxcalc: fixed uninitialized NUMER, DENOM variables.

Modified debug gfortran Makeincludes to use the latest (?) version of compile-flag for "trap uninitialized"

Modified XY2XY() to pass-through "missing" location-arguments, for SMOKE.

Version 3.2.40

Modified generic routine XY2XY() from MODULE MODGCTP to &pass-through location arguments: if X_IN<AMISS3 or X_IN<AMISS3 then after the call to XY2XY( ..., X_IN,Y_IN,X_OUT,Y_OUT), X_OUT=Y_OUT=BADVAL3<AMISS3.

Enhanced (cleaned-up) documentation.

Enhanced m3tools program latlon
Changed default for environment variable IOAPI_OFFSET_64

Version 3.2.39

For I/O API-3.2 post Dec. 25, 2017, m3tools program latlon now also writes M3DBLE (double-precision) Latitude-Longitude variables LATD and LOND, in addition to the original M3REAL (single-precision) variables LAT and LON. And since the I/O API features direct access by variable (date&time, and layer), this should not break any existing application.

Additionally, it is now possible to do interactive definition of grids with map projections of types

• TRMGRD3: Transverse Mercator projection
• EQMGRD3: Equatorial Mercator projection
• POLGRD3: Polar Stereographic projection
• LEQGRD3: Lambert Azimuthal Equal Area projection
• MERGRD3: General Mercator projection
• STEGRD3: General Stereographic projection

(as well as the original Lat-Lon (LATGRD3), Lambert Conformal Conic (LAMGRD3), and and UTM (UTMGRD3; grids may of course also be specified by a GRIDDESC file).

The default for environment variable IOAPI_OFFSET_64 is now TRUE. For netCDF 3.3 or later, this allows the size of file-timesteps to exceed 2 GB. (Assuming only (4-byte) REAL and INTEGER variables, a timestep in a GRIDDED file has size 4*NVARS*NCOLS*NROWS*NLAYS + 8*NVARS bytes.)

Fixed bug reported by Ehsan Mosadegh in m3tools programs m3tshift and m3wndw.

Version 3.2.38

Ehsan Mosadegh (dri.edu) reports a problem related to a missing EFLAG initialization in m3tools program m3tshift. This bug was also present in program m3wndw.

Now fixed.

Bug-fix in UNGRIDB().

Version 3.2.37

Jeff Vukovich (US EPA) reports a missing line

CU( 4,S ) = X * Y

in the I/O API-3.2 version of UNGRIDB().

Now fixed.

MPAS INTEGER-variable support

Version 3.2.36

INTEGER-variable support for m3tools program mpastom3: uses MPAS-cell values (as for MPCELLMATX(), below) instead of barycentric-linear interpolation.

INTEGER-variable support for MODMPASFIO generic routine MPINTERP(): uses MPAS-cell values (as for MPCELLMATX(), below) instead of barycentric-linear interpolation.

New "incidence-matrix" generic routine MPCELLMATX() returns the MPAS-cell subscripts for a set of points or 2-D grid of points.

m3tools programs mpasdiff, mpasstat, mpastom3
MODMPASFIO generic routine MPINTERP(), MPBARYMATX(), MPBARYMAULT()

Version 3.2.35

New m3tools programs:

mpasdiff

Compare two variables or layer-ranges of variables from one or two MPAS-format-netCDF files, using the standard list of differencing operations

mpasstat

Compute statistics (MEAN, SIGMA, MAX, MIN, and their locations), for a variable or layer-ranges of a variable from an MPAS-format-netCDF file.

mpastom3

Interpolate a set of variables from an MPAS-format-netCDF file for a specified time step sequence to a grid specified by an I/O API gridded Lat-Lon file, and write the result to a gridded I/O API file.

New MODMPASFIO generic routines

MPINTERP()

Interpolate MPAS-grid data to a Lat-Lon specified point, set of points, or 2-D grid of points, using barycentric-linear interpolation.

MPBARYMATX()

Return a 3-band matrix for doing barycentric-linear interpolation from MPAS-grid data to a set of points or 2-D grid of points

MPBARYMULT()

Multiply a 3-band matrix from MPBARYMATX() by a (layered or non-layered) MPAS-grid variable.

Bug-fixes in programs VERTOT, VERTINTEGRAL

Version 3.2.34

Bug reported by Kathy Brehme, Edward Anderson: incorrect default RUNLEN in m3tools program vertot. Same bug existed in vertintegral; bug fixed in both.

Enhanced default for RUNLEN in m3tools programs m3stat, m3tshift, m3wndw, m3xtract.

Enhanced Makeinclude.*: make-variables MODI, E132 for module-include path, 132-column-fixed source format.

Add extra attributes to new MPAS-files, for VERDI

Version 3.2.33

VERDI uses NCAR's Java MPASConvention.java for MPAS-file recognition. This code requires global file-attributes model_name and Conventions for file recognition. These attributes are nowhere mentioned in the MPAS file specification document.

MODULE MODMPASFIO routine CREATEMPAS() now puts these attributes into the file headers.

Bug-fixes for MODULE MODMPASFIO

Version 3.2.32

Bug-fixes for incorrect read of variables lonCell and write of kiteAreasOnVertex to/from file headers, in MODULE MODMPASFIO.

Bug-fixes in EQMERC, SETEQM, SETTRM

Version 3.2.31

EQMER() had an incorrect test for P_ALP

SETEQM, SETTRM had incorrectly-worded error messages.

Bug-fix in MODMPASFIO/ARC2MPAS().

Bug-fixes for time stepped versions of read and write operations in MODULE MODMPASFIO and MODULE MODNCFIO

Version 3.2.30

Subject says it: in both of these, there was a consistent error in values for the DELS() argument for the time step dimension.

Bug-fixes in MODULE MODMPASFIO and log-message reductions for MODULE MODNCFIO

Version 3.2.29

Bug in MODMPASFIO/MPSTR2DT() which then causes bad results from MODMPASFIO/READMPSTEPS().

Eliminate (annoyingly-numerous) "Processing coordinate transform" messages in MODULE MODNCFIO.

Bug-fixes in MODULE MODNCFIO, MODULE MODMPASFIO , and m3tools program m3interp

Version 3.2.28

Subject says it: bugs in time step versions of read/write routines; with time interpolation of BNDARY3 boundary condition data structures.

Thanks to Joey Huang for reporting the m3tools bug.

MODULE MODNCFIO: add CREATENC(), WRITENCVAR(); bug-fixes

Version 3.2.27

• See CREATENC() documentation: create a new "raw" netCDF output file.
• See WRITENCVAR() documentation: generic routine for writing whole-variable or timestep-of-variable to "raw" netCDF output file.
• READNCVAR() is now generic-only (with no public type-specific routines).

Bug-fixes for argument-mismatch in error-condition calls to NF_STRERROR().

MODULE MODMPASFIO

Version 3.2.26

MPAS is a (potentially-)global unstructured-grid weather model that can be adapted for atmospheric chemistry, land-surface modeling, and other tasks. It uses its own format built on top of netCDF to store and retrieve its model-data; the file structure is quite complicated.

New MODULE MODMPASFIO addresses some of the difficulties in dealing with this complexity. It provides parameters and state-variables for MPAS grid description, routines for creating, opening, describing, reading, and writing MPAS-netCDF files, as well as a number of grid related utility routines.

Since this netCDF-format is quite complex, this can save you from having to write several thousands of lines of code on your own. Additionally, it provides the data integrity facilities and error checking that are necessary to construct models of regulatory quality, for which the "chain of custody" for the model data must be kept in a form adequate for use in court.

Timestep-read operations and bug-fixes in MODULE MODNCFIO

Version 3.2.25

Added routines READNVSTEP*(FNAME,TSTEP,VNAME,...) to MODULE MODNCFIO for reading one time step (TSTEP=1,2,3,...) of the indicated variable from time-stepped "raw netCDF" file FNAME.

Bug-fixes in some of the READNCVAR4D*() routines (REAL case was OK; others had a typo).

Bug fix in MODPDATA/SETUP_DECOMP() top-level Makefile.template

Version 3.2.24

Transcription error in MODPDATA/SETUP_DECOMP() reported by David Wong, US EPA, now fixed.

Master Makefile.template had an inadvertent and backwards netCDF libraries definition; now corrected to

-lnetcdff -lnetcdf

reported by Xiangshang Li, Unkiversity of Houston.

I/O API Snoop Mode

Version 3.2.23

For I/O API 3.2 or later: If snoop mode is configured in the Makefile when you build libioapi.a, then environment variables SNOOPTRY3, SNOOPSECS3 control the number of re-tries and the delay between re-tries (secs) for reading data under end-of-file conditions.

This is useful, for example, when trying to "pipeline" processing for the models in a forecasting system.

Support for INTEGER*1 and INTEGER*2 for READNCVAR()

Version 3.2.22

MODULE MODNCFIO had generic INTERFACE READNCVAR() augmented to support NF_BYTE==INTEGER*1 and NF_SHORT==INTEGER*2 variables.

Bug in library-order for m3tools/Makefiles, reported by Luís Carvalheiro, CESAM, Universidade de Aveiro. Fixed.

New m3tools program M3MASK

Version 3.2.21

m3tools program M3MASK reads ASCII mask-data, specified by a file having one line per column and row of the masked grid cells, and creates a GRIDDED output file with one INTEGER variable MASK for which

• MASK(C,R) = 1 for cells listed in input file; and
• MASK(C,R) = 0 otherwise

4-D READNCVAR() routines in MODULE MODNCFIO

Version 3.2.20

New routines READNCVAR4DD(), READNCVAR4DI(), READNCVAR4DR(), READNCVEC4DD(), READNCVEC4DI(), READNCVEC4DR() in the generic routine READNCVAR()in MODULE MODNCFIO

Bug-fixes (fixed bad dimension-check) for READNCVAR3DD(), READNCVAR3DI(), READNCVAR3DR(), READNCVEC3DD(), READNCVEC3DI(), READNCVEC3DR()

Bug-fixes in LASTTIME(), RUNSPEC(), CKFILE3() routines

Version 3.2.19

Missing IMPLICIT NONE and EXTERNAL declarations in LASTTIME().

Error in calculation of NRECS argumnent for RUNSPEC()

Bug-fixes by Edward Anderson, Lockheed Martin, for internal routine CKFILE3().

Bug-fixes in DBLLIST(), INTLIST(), REALIST(), STRLIST() routines

Version 3.2.18

Corrected mis-handling of delimiters in COMMA-or-BLANK delimited lists.

Copy CMAQ metadata from input to output, for relevant m3tools programs.

Version 3.2.17

Add new routines ISCMAQ() and ISSMOKE() to MODULE MODATTS3.

Copy CMAQ metadata from input to output, for m3tools programs, under control of environment variable COPY_META (default ISCMAQ(<input file>))

m3combo

m3cple

m3interp

m3tproc

m3tshift

m3wndw

m3xtract

Don't compile single_thread.F if coupling mode not enabled

Version 3.2.6

Edward Anderson (Lockheed Martin, supporting the U.S. EPA) points out that single_thread.o has unsatisfied references that are troublesome for the creation of shared libraries.

single_thread.F now compiles conditionally, with nonempty body only if IOAPICPL is defined. This source file is also commented out of the Makefile.nocpl*.

Bug-fix for I/O API 3.2 m3tools program m3tproc

Version 3.2.5

Chris Nolte, US EPA ORD, points out a bug in the I/O API 3.2 version of m3tools program m3tproc, together with a documentation problem and a m3tools/Makefile problem.
FIXED.

More CMAQ metadata in MODULE MODATTS3

Version 3.2.14

Metadata now contains values for all environment based control parameters, and path-names for input files.

Bug-fixes for MPI/PnetCDF mode in MODULE MODATTS3

Version 3.2.13

Subject says it.

Fixes for Climatology-Year IO_365 Version

Version 3.2.12

Bug-fix from Chris Nolte, US EPA, in SECSDIFF() for the Climatology-Year IO_365 Version. He also noted that the following C routines needed IO_365 support:

daymonc()
julianc()
mmddyyc()
secsdiffc
wkdayc()

Now complete.

The following Makeinclude-files were added, for building 360-day and 365-day Climatology Year versions of the I/O API, so that it will be easier to keep the Climatology-Year builds distinct from the (not-compatible) "normal" builds:

Makeinclude.Linux2_x86_64_360
Makeinclude.Linux2_x86_64_365
Makeinclude.Linux2_x86_64gfort_360
Makeinclude.Linux2_x86_64gfort_365
Makeinclude.Linux2_x86_64ifort_360
Makeinclude.Linux2_x86_64ifort_365
Makeinclude.Linux2_x86_64pg_360
Makeinclude.Linux2_x86_64pg_365

"unstructured text-file" metadata in MODULE MODATTS3

Version 3.2.11

MODULE MODATTS3 now has support for automated inclusion of "unstructured text-file" metadata into [P]netCDF file headers of all files newly-created by programs. Such files should be structured with 80-column lines; to have their contents copied into file-headers, you need to do the following command in the program-script

    setenv  IOAPI_TEXTMETA   <path-nanme>
    

If active, the text metadata can be found in the following MODULE-variables in MODULE MODATTS3

    LOGICAL,                   PUBLIC, PROTECTED, SAVE :: TEXTMETA          !!  text-file metadata active?
    INTEGER,                   PUBLIC, PROTECTED, SAVE :: TEXT_MLINES       !!  number of lines in TEXT_MDATA( :,: )
    CHARACTER*80, ALLOCATABLE, PUBLIC, PROTECTED, SAVE :: TEXT_MDATA( : )   !!  (TEXT_MLINES)
    

Makeinclude changes for gfortran

Version 3.2.10

gfortran documentation is less than clear; it turns out that -I should be used for F90 module search-path, whereas -J is used for F90 modulefile placement directory.

Note that PGI Fortran has strange and difficult rules for module search and module placement: -module works for both, but only the first such directive governs module placement.

New m3tools program findwndw

Version 3.2.9

New m3tools program findwndw to read headers from two GRIDDED files FILE1 and FILE2, and compute the smallest window into the FILE2 grid that completely encloses the grid for FILE1, if any. Program reports failure if the FILE2-grid does not completely cover the FILE1-grid.

Useful for tasks like finding the smallest window into a global Lat-Lon file (e.g. EDGAR emissions) that covers a Lambert based modeling grid, for subsequent use by m3tools program m3wndw.

Added Apr. 6, 2016: Mac OSX/gfortran/gcc support using ${BIN}=OSX_x86_gfortran; see file ioapi/Makeinclude.OSX_x86_gfortran.

Compatibility fixes for I/O API 3.2; DOWNCASE()

Version 3.2.8

B. H. Baek (UNC IE) discovered bacvkwards-compatibility problems in BILIN(), BMATVEC(), UNGRIDB(), and UNGRIDI() . for codes that do not USE M3UTILIO. Now fixed.

Added routine DOWNCASE( ASTRING ) to convert argument ASTRING to lower-case

.

Bug fixes for I/O API 3.2 from Edward Anderson, Lockheed Martin.

Version 3.2.7

Top-level Makefile.template

Fixes to treatment of make-variable BIN

ioapi/Makefile.*

Revised treatment of make fixed_src to fix initial creation of fixed-132 INCLUDE-files for CMAQ.

Multiple coupling-mode fixes.

in ddtvar3v.F, interp3v.F, intpqv.F

Multiple-definition issues

Stand-alone versions of LL2UTM() and UTM2LL()

Added March 9, 2016:
Need to add utm2ll.f to the fSRC list in the Makefiles; need to remove USE M3UTILIO from it to keep gfortran from claiming (falsely) a circular dependency.

CMAQ metadata routines now implemented in MODULE MODATTS3

Version 3.2.6

The following routines are now implemented and "active" in MODATTS3 MODATTS3:

INITCMAQ()

Initializes internal state variables holding CMAQ-standard metadata from either the standard-format text file with logical name IOAPI_CMAQMETA, from the program-environment if IOAPI_CMAQMETA=="ENV" or from the argument MDATA.

GETCMAQ()

Reads CMAQ-standard metadata from the header of file FNAME and puts it either into CMAQ-metadata object MDATA (if present) or into CMAQ-metadata module variable CMAQ_MDATA.

LOGCMAQ()

Writes CMAQ-standard metadata either from metadata-object MDATA (if present) or CMAQ-metadata module variable CMAQ_MDATA (otherwise) either to a structured-ASCII output file FNAME (if present) or to the program log

SETCMAQ()

Writes CMAQ-standard metadata either from metadata-object MDATA (if present) or CMAQ-metadata module variable CMAQ_MDATA (otherwise) to the header of the specified I/O API file

ENDCMAQ()

Turns off CMAQ-standard metadata

New "raw netCDF" routines in MODULE MODNCFIO

Version 3.2.5

Add new routines DESCNCVAR() and READNCVAR() to MODULE MODNCFIO:

DESCNCVAR( FNAME, MXVAR, NVARS, VNAMES, VTYPES, VNDIMS, VDIMS )

Returns lists giving the names, data-types, numbers of dimensions, and dimensions for the variables in the netCDF file with logical name FNAME

READNCVAR( FNAME, VNAME, NCOLS, [NROWS, [NLAYS,]] GRID )

Reads the specified 1-D, 2-D, or 3-D netCDF variable named VNAMEfrom the netCDF file with logical name FNAME (with dimension-checking).
Generic, for 1-D, 2-D, and 3-D variables of type {M3INT,M3REAL,M3DBLE}, and both fully-dimensioned and single-indexed output array GRID.

New DBLLIST(); bug-fixes and support for blank-delimited lists in *LIST() routines

Version 3.2.4

Add new routine DBLLIST() for REAL*8 valued lists, with support as part of the ENVLIST() generic.

Support for lists requiring more than 512 columns had a bug. Now fixed. These routines also now support blank-delimited lists.

New m3tools program gridprobe interpolates time series for a specified layer of a REAL variable to a specified set of Lat-Lon points from a GRIDDED input file, and write the resulting time series as columns to a human-readable ASCII REPORT-file.

I/O API v3.2 m3tools programs wrfgriddesc and wrftom3

Version 3.2.3

m3tools program wrfgriddesc creates a GRIDDESC file from the header-data in a WRF-format netCDF file.

m3tools program wrftom3 converts/extracts/windows variables from WRF-format netCDF files to GRIDDED I/O API files.

I/O API v3.2 support for 360/365-day (no leap-year) climate modeling years

Version 3.2.2

I/O API 3.2 has optional support for climate modeling with a 365-day/no-leap-year calendar with "normal" months, and weeks aligned so that Jan. 1, 1900 is a Monday. To build this version of the library, customize your ioapi/Makefile by adding -DIO_365=1 to the DEFINEFLAGS make-variable in it.

Note that since I/O API 3.0 in 2002, the I/O API has had optional support for climate modeling with a 360-day/30-day-month calendar, with weeks also aligned so that Jan. 1, 1900 is a Monday. To build this version of the library, customize your ioapi/Makefile by adding -DIO_360=1 to its DEFINEFLAGS.

I/O API v3.2 Makefiles

Version 3.2.1

Note that Makefiles always need to be customized for your system (installation-directory BASEDIR, etc.).

Starting with this release, these Makefiles will not be included in the release ioapi-3.2.tar.gz; for both of the directories ioapi and m3tools you will need to copy one of the Makefile.* templates to Makefile and customize it. This prevents the situation where an I/O API upgrade clobbers your existing customization.

I/O API v3.2-beta: Major Update.

There are major changes for this release, some of which are documented elsewhere below. There are a number of additional changes, especially for CMAQ-5.1, netCDF-4, and for MS-Windows:

PnetCDF for CMAQ-5.1

CMAQ-5.1 can use PnetCDF for distributed-parallel I/O. Note that this is for gridded files only, with the fixed data-distribution specified for CMAQ. To activate pnetCDF distributed I/O for a particular file, you need the following kind of setenv statement in your run-script.

setenv <logical name> MPI:<path (physical) name>

The ioapi/Makefile.pncf and m3tools/Makefile.pncf may be used to create I/O API libraries with PnetCDF activated, or this feature may be turned on by un-commenting the compile-flag -DIOAPI_PNCF=1 in the Makefiles.

Related data- and processor-distribution state and routines are found in new MODULE MODPDATA. See this section on environment variables needed for using distributed-parallel I/O.

The original concept and the prototype code are due to Dr. David Wong, US EPA. This code has been extensively extended and revised, to meet the needs of proper I/O API integration and software engineering standards.

Changes for netCDF-4

The I/O API was originally implemented in the early 1990's, using the netCDF-2 NC*() Fortran interfaces. As noted elsewhere, netCDF-4 split apart the C interfaces, the Fortran interfaces, and the C++ interfaces into separate libraries (which each have their own versioning). Recent (post-4.4) versions of netCDF-Fortran have dropped the NC*() interfaces used; a major and tedious re-write was necessary to re-cast all 800+ NC*() calls in terms of netCDF-3-style NF_*() interfaces.

At the same time, support was added for 64-bit-integer variables (type M3INT8 = 10, provided that you are using netCDF-4 libraries. This feature may be turned on by un-commenting -DIOAPI_NCF4=1 in the Makefiles.

Cygwin under MS-Windows

The following Makeinclude files are now available, to build the I/O API using gcc and gfortran for MS-Windows under Cygwin:

Makeinclude.WIN_x86gfort

32-bit library/executables using Cygwin gcc and gfortran

Makeinclude.WIN_x86_64mingw64

64-bit library/executables using MingW-64 gcc and gfortran

I/O API v3.2-beta: dependency-fixes; new m3tools program insertgrid

For various C-language call reasons, there need to be standalone versions of LAMBERT() and LASTTIME()

New m3tools program insertgrid replaces the REAL variable-values from GRIDDED file FILE1 by values from GRIDDED file FILE2 for those FILE1-cells completely covered by FILE2, with re-scaling option. Default scaling assumes (non-MKS SMOKE: e.g., G/S/grid-cell) emissions units rather than (properly-mathematical MKS) flux units (e.g., W/m^2).

For I/O API version3.2-beta.

Back to Contents

Bug fix in ckfile3()

Mogesh Naidoo reports failure of ckfile3() to support Equatorial Mercator projection (GDTYP3=EQMGRD3). This bug is also present for GDTYP3=TRMGRD3, ALBGRD3, LEQGRD3, SINUGRD3

Now corrected in all I/O API versions (3.0, 3.1, 3.2-beta).

Back to Contents

Fix in GCTP()for Intel ifort v-11 and later.

Introduce new scratch variables in gctp.f so that INTENT works correctly for Intel ifort version 11 and later.

Back to Contents

More gfortran hacks; SINUGRD3 map projection

Recent gfortran wasn't allowing certain USE M3UTILIO statements internally. Hacked the source to get it to where it will still work.

Added new parameter SINUGRD3 for Sinusoidal Equal Area map projection to PARMS3.EXT for I/O API versions 3.1 and 3.2., together with support for it in I/O API-3.2 MODULE MODGCTP

Back to Contents

New routine RUNSPEC()

SUBROUTINE RUNSPEC( FNAME, USEENV, SDATE, STIME, TSTEP, NRECS ) returns a time step sequence compatible with use of FNAME as an input file:

If FNAME is blank, default SDATE:STIME are from the wall-clock, and default TSTEP is 1 hour.

If FNAME is time independent, return SDATE=STIME=TSTEP=0, NRECS=1.

Else if USEENV, get SDATE, STIME, TSTEP, EDATE, ETIME from the environment and use them to calculate NRECS.

Else prompt the user for SDATE, STIME, TSTEP, EDATE, ETIME and use them to calculate NRECS.

Added March 4, 2015: Abhishek Bhat, PhD, EPI, of Trinity Consultants, points out a Makefile-bug: there needs to be a dependency line runspec.o: m3utilio.mod in the Makefile, similar to the already-present matxatts.o: m3utilio.mod dependency line.

Back to Contents

Standards-violation fixes in getyn(), getstr(), getreal(), getnum(), getmenu(), getdble(), promptmfile(), promptffile(), year4(), skipl()

The Models-3 coding standards state:

1. You will check status of systems-calls and other calls that may fail (e.g., netCDF and MPI calls) and return success/failure status; and
2. You will write meaningful log messages adequate to locate and diagnose the error for the above, in cases of failures.

Now corrected.

Back to Contents

I/O API Version 3.2-beta

Version 3.2 of the I/O API library code is available here for download in gzipped-tar source code form from the CMAS Center web-site, or is available on GitHub from <https://github.com/cjcoats/ioapi-3.2>. To install a distribution-copy from GitHub, go to the directory under which do the installation, and then do the command

    git clone https://github.com/cjcoats/ioapi-3.2
    

• New "bounded" environment-value routines BENVINT(), BENVDBLE(), BENVREAL(), with three MINVAL, MAXVAL, DEFAULT arguments corresponding to the simple DEFAULT argument of ENVINT(), ENVDBLE(), ENVREAL()
• New generic INTERFACEs in MODULE M3UTILIO:
  • BILIN(): bilinear interpolation using matrix from UNGRIDB(), forms for 1-D/2D/3D input and output data arrays.
  • BMATVEC(): vertical-transpose and bilinear interpolation using matrix from UNGRIDB(), forms for 1-D/2D-horizontal input and output data arrays.
  • ENVGET() generic get-environment-variable routine encapsulating BENVINT(), BENVDBLE(), BENVREAL(),ENVINT(), ENVDBLE(), ENVREAL(), ENVYN()
  • UNGRIDB()
  • PMATVEC() Apply incidence matrices from UNGRIDB(), forms for 1-D and 2-D input and output data arrays.
  • UNGRIDB() Compute "ungridding" bilinear-interpolation matrices, for 1-D and 2-D arrays of interpolation-points. matrices for BILIN() and BMATVEC()
  • UNGRIDI() Compute "ungridding" incidence matrices, for 1-D and 2-D arrays of interpolation-points. matrices for PMATVEC()

• Many ioapi routines and m3tools programs are now OpenMP parallel.

• CF-compliant geosapatial metadata ioapi file-creation option, turned on by environment variable IOAPI_CFMETA:
  

  setenv IOAPI_CFMETA Y

• New MODULE MODGCTP with all the coordinate-transform related routines and INTERFACEs:
  Old but modified: (moved from MODULE M3UTILIO)
  • INTERFACE for GTPZ0()
  • INTERFACEs and new implementations for GTPZ0()-spheroid initialization procedures SETSPHERE(), INITSPHERES(), SPHEREDAT()
  • INTERFACEs and new implementations for single-precision/single-point/specific-projection coordinate-transform functions INITPROJ(), ..., LAMBERT(), ..., ALB2EQM()

  New:
  • PARAMETERs STDSPHERES(0:21) and SPHERENAMES(0:21) for GCTP spheroid names and indices;
  • GTPZ0()-argument initialization procedure M3TOGTPZ()
  • Scattered-point-array generic coordinate-transform procedure XY2XY()
  • Grid-node-array generic coordinate-transform procedure GRID2XY()
  • High-performance OpenMP-Parallel generic bilinear interpolation package GRID2INDX(), PNTS2INDX(), INDXMULT()

• New MODULE MODATTS3 replaces MODULE MATXATTS (so that all USE MATXATTS statements need to become USE MODATTS3 in your codes):
  • Matrix-attribute routines (add extra netCDF file attributes to describe input and output grids for matrix-files):
    • INITMTXATT()
    • GETMTXATT()
    • SETMTXATT()
    • CHKMTXATT()
    • ENDMTXATT()
  • CF-convention geospatial-metadata routines make newly-created files CF compliant (hopefully, subject to further testing...):
    • INITCF()
    • SETCF()
    • ENDCF()
  • Place-holders for "standard" CMAQ and SMOKE metadata routines (what this metadata contains needs to be determined...), turned on by environment variables IOAPI_CMAQMETA, IOAPI_SMOKEMETA:
    

    setenv IOAPI_CMAQMETA <path> or NONE
    setenv IOAPI_SMOKEMETA <path> or NONE

    • INITCMAQ()
    • GETCMAQ()
    • LOGCMAQ()
    • SETCMAQ()
    • ENDCMAQ()
    • INITSMOKE()
    • GETSMOKE()
    • LOGSMOKE()
    • SETSMOKE()
    • ENDSMOKE()

• Updates to m3tools programs:
  • New programs:
    • dayagg,
    • m3probe,
    • m3totxt, and
    • vertintegral.
  • These programs now USE MODATTS3 for matrix-file attributes:
    • mtxbuild
    • mtxcalc
    • mtxcple
  • These programs USE MODGCTP for geospatial transforms and interpolation:
    • latlon
    • m3cple
    • m3interp
    • mtxcalc
    • projtool
  • These programs are now OpenMP parallel:
    • bcwndw
    • dayagg
    • m3agmask
    • m3agmax
    • m3combo
    • m3cple
    • m3interp
    • m3tproc
    • mtxcalc
    • mtxcple
    • presz
    • vertimeproc
    • vertintegral
    • vertot
  • Many source files have been converted to Fortran-90 "free" source format.
  • Rather than using a fail-immediately error management strategy, make a best-effort attempt at fulfilling the run-specifications, while logging errors as they occur and keeping track of final error-status.
  • Program mtxcalc adds option for output-scaling/units-conversion (e.g., Kg-to-Ton) via new environment variable SCALEFAC.
  • Program presz adds environment-variable configuration option for its MM5-reference atmosphere, and now also outputs reference temperature TA (K)

• Updated and error-checked documentation (including docs for all the "new stuff" above).

• All codes and HTML pages have Subversion IDs in their header comments.
  Use ident to see these IDs in object (.o), library (.a) and executable files.

• Default Makefile now builds for nocpl.

To-Do List:

• Define standard CMAQ and SMOKE metadata and finish the relevant routines in MODULE MODATTS3 accordingly. CMAQ metadata now in place.

• Do overloaded LL2LAM() routines, to support both single-precision and double-precision operation?

• Python and Java APIs?

• Parallel I/O for CMAQ-5.1 (now in place)

• Tension-spline interpolation module and tools-programs?

• m3tools development—standardized, overflow-proof interfaces, free-format F90, OpenMP-parallel where reasonable?

• Tutorial development? —new stuff, best practices...

Back to Contents

Bug-Fixes in m3tools/projtool

Bug-fixes in m3tools program projtool:

• error in spheroid-management found by John N. McHenry, BAMS.
• Errors found by Bhupesh Adhikary, International Centre for Integrated Mountain Development, Kathmandu, Khumaltar, Lalitpur, Nepal

Now fixed.

Back to Contents

Bug-Fix in m3tools/projtool

Bug-fix in m3tools program projtool: for grid-corner calculations, the NW and SE corners were labelled backwards. Now fixed.

Back to Contents

Enhanced m3tools/m3diff

New version of m3tools program m3diff now supports dimensions having up to 9 digits, with variable-format spacing as needed. This will be useful for comparing SMOKE vector-files (which use degenerate grids with NCOLS=NLAYS=1 and NROWS=<number of sources>) that have large (>99,999) numbers of sources.

Back to Contents

Bug-fix in m3tools/projtool: defining projections by angles.

John McHenry, BAMS, found a bug in the algorithm that m3tools program projtool uses to define input and output map projections by angles. Now fixed.

Back to Contents

INTEGER(8)-key search and sort routines.

New INTEGER*8-keytuple search and sort routines FINDL1(), FINDL2(), FINDL3(), FINDL4(), LOCATL1(), LOCATL2(), LOCATL3(), LOCATL4(), and SORTL1(), SORTL2(), SORTL3(), SORTL4() and their C counterparts qsortl1(), qsortl2(), qsortl3(), and qsortl4() were added, together with the inclusion of these routines into generic INTERFACEs FINDKEY(), LOCATE(), and SORTI() in MODULE M3UTILIO.

Back to Contents

Bugfix in m3tools program m3diff

Mohammad Omary, UNC IE, turned up a bug in using m3tools program m3diff on files with distinct time step sequences (e.g. comparing file A for July 2009 with File B for August 2014. This is fixed in the new version.

Back to Contents

Additional SORTI*C* routines

There are now the following additional sort-by-string-table routines, covered by the MODULE M3UTILIO SORTI() generic interface:

SORTINC4( N, M, INDX(N), CTABLE(N) )

arguments INTEGER N, M, INDX(N); CHARACTER(*) CTABLE(N)
Indirect-sort by CTABLE, assuming that only the first M characters in each table-entry are significant.

SORTINC8( N8, M8, INDX8(N8), CTABLE(N8) )

arguments INTEGER(8) N8, M8, INDX8(N8); CHARACTER(*) CTABLE(N8)
Indirect-sort by CTABLE, assuming that only the first M8 characters in each table-entry are significant.

Back to Contents

Generic INTERFACEs in MODULE M3UTILIO

Environment-buffer maximum size expanded to 64K-1 (=65535) for environment-variable routines ENVLIST,ENVDBLE, ENVINT, ENVREAL, ENVSTR, ENVYN, INTLIST, REALIST, STRLIST

MODULE M3UTILIO now has generic interfaces for the following:

ENVLIST

INTLIST, REALIST, STRLIST

FINDKEY

FINDC, FIND1, FIND2, FIND3, FIND4, FINDR1, FINDR2, FINDR3, FINDR4

LOCATE

LOCAT1, LOCAT2, LOCAT3, LOCAT4, LOCATC, LOCATR1, LOCATR2, LOCATR3, LOCATR4

SORTI

SORTIC4, SORTIC8, SORTI1, SORTI2, SORTI3, SORTI4, SORTR1,SORTR2, SORTR3, SORTR4

New m3tools programs designed to aid scripting, driven by command-line date and time arguments:

• greg2jul: convert Gregorian dates to Julian YYYYDDD form
• jul2greg: convert Julian dates to Gregorian YYYYMMDD form
• juldiff: Number of days between two Julian dates
• julshift: Add/subtract number of days from a Julian date
• timeshift: Add/subtract time HHMMSS to a Julian date-and-time YYYYDDD.HHMMSS.

Back to Contents

Bugfix in m3tools program m3merge

John N. McHenry, BAMS, turned up a bug in usingm3tools program m3merge on time independent files. This is fixed in the new version.

Back to Contents

Changes to deal with gfortran standards-incompatibilities.

Recent versions of gfortran insist on breaking compatibility with all other compilers we use, not accepting command-line-argument routines IARGC() and GETARG() (which are an industry standard Fortran extension), and accepting only Fortran-2003 routines GET_COMMAND_ARGUMENT() and GETARG().

This version conditionally implements IARGC() and GETARG() in init3.F, depending upon preprocessor-definition -DNEED_ARGS=1 which is now set in Makeinclude.Linux2_x86, Makeinclude.Linux2_x86gfort, Makeinclude.Linux2_x86_64, and Makeinclude.Linux2_x86_64gfort

Back to Contents

OpenMP parallel for programs M3AGMASK, M3AGMAX, M3COMBO, M3TPROC

In addition to the previously OpenMP parallel program VERINTEGRAL, programs M3AGMASK, M3AGMAX, M3COMBO, M3TPROC are now OpenMP parallel.

Environment variable OMP_NUM_THREADS controls how many parallel threads/processors are used for program execution. Note that for the most part, these programs do a lot of I/O, so that running with two threads may help performance, especially for large grids, but OMP_NUM_THREADS > 2 has severely diminishing returns for these particular programs.

Back to Contents

New SUBROUTINE LASTTIME() and its use for applications in m3tools; larger environment-buffer size

New routine LASTTIME(SDATE,STIME,TSTEP,NSTEPS,EDATE,ETIME) returns the ending date&time for time step sequence SDATE:STIME:TSTEP:NSTEPS. Safe for very long time step sequences (even multi-century!). Now used in new versions of m3tools programs m3combo, m3cple, m3interp, m3tproc, mtxblend, and mtxcple.

Internal environment buffer in subroutines SETENVVAR(), ENVYN(), ENVINT(), ENVREAL(), ENVDBLE(), INTLIST(), STRLIST(), and REALIST(), expanded from 512 bytes to 64K bytes, to support larger environments as necessary.

Back to Contents

Bug-fix for m3tools/m3combo from Bonyoung Koo, Environ.

Bonyoung Koo, Environ, found a set of bugs in m3tools/m3combo associated with the use of this tool for CUSTOM and BOUNDARY files. This bug is now fixed in both I/O API versions 3.0 and 3.1.

Back to Contents

Bug-fix for m3tools/m3xtract and m3tools/vertintegral.

Sarika Kulkarni, California ARB, found bugs in programs m3tools/m3xtract and m3tools/vertintegral and provided fixes for them. The m3tools/vertintegral bug is present only in Version 3.1 of the I/O API. These fixes are now in the distribution.

Back to Contents

Enhancement and bug-fix for m3tools/projtool.

For operation-cases 10, 11 (display input-grid corners or centers relative to output coordinate system, there was a bug: the NW corner was calculated but not reported to the user.

Additional operation 12: report output-grid column and row for input-projection location <X,y>.

Back to Contents

Bug-fix for INQATT3().

Title says it... This was a long standing bug, which (I guess) shows how no one is using extra M3IO attributes ;-(

Back to Contents

Bug-fix from Sarika Kulkarni, CA ARB, in m3xtract; new address info for m3tools programs.

Sarika Kulkarni, CA ARB, provided a bug-fix to the I/O API Version 3.1 m3xtract; this is now incorporated into the release.

The primary I/O API author, Carlie Coats, has performed "musical offices" again, and is now at

Carlie J. Coats, Jr. , Ph.D. carlie@jyarborough.com
UNC Institute for the Environment
137 E. Franklin St. Suite 602 Room 613-C / Campus Box 1105
Chapel Hill, NC 27599-1105

Back to Contents

relocation error link issues for x86_64, and new BIN=Linux2_x86_64ifort_medium

There are issues with huge arrays and COMMON blocks on x86_64 Linux (and other) platforms; these show up at link time with error messages talking about relocation errors. New BIN=Linux2_x86_64ifort_medium is designed to deal with these situations; see this note for details.

Back to Contents

Version 3.1 m3tools/Makefiles now assume netCDF Version>4.1 Library Changes

Starting with netCDF Version 4.1, linking with netCDF now requires additional libraries. As of today's downloads, I/O API Version 3.1 m3tools/Makefiles now assume at least the minimal set. See this note about netCDF.

Note that this incompatibility from netCDF will also affect all other model-builds using netCDF Version 4.1 or later.

Bug-fix in m3tools program VERTINTEGRAL

Rajesh J points out a problem with OpenMP directives in m3tools program VERTINTEGRAL. Now fixed in both the Version 3.0 and 3.1 tarballs.

Bug-fixes in SUBROUTINE CURREC()

Rajesh J points out a problem with the Version 3.0 currec.F (extraneous lines beginning with #, possibly inserted by the version control system). On comparison with the Version 3.1 code, it turns out that a bug-fix suggested by Christian Hogrefe was dropped from this version. Both are now fixed.

Fixes from Christian Hogrefe in programs JULDATE and M3XTRACT

Christian Hogrefe points out that gfortran regards IARGC() as a Fortran INTRINSIC rather than as an EXTERNAL, and PROGRAM JULDATE needs to be modified accordingly

He also found a bug in PROGRAM M3XTRACT and provided a fix.

Bug-fix for file type CUSTOM3 in PROGRAM M3STAT

There was a problem for single-variable analysis of multi-layer files of type CUSTOM3, now fixed.

Bug-fixes and format improvements in m3tools program PROJTOOL

There were menu-problems and format-readability problems, now fixed.

Add interface-block for CRLF() to MODULE M3UTILIO

At the request of US EPA, an interface block for routine CRLF() was added to MODULE M3UTILIO for use in the upcoming version of CMAQ. This may necessitate code changes for codes that already USE M3UTILIO and that use that routine (just delete the declaration and the EXTERNAL statement that accompanies it).

Code re-organization to support full strength DESC3() for BUFFERED files

At the request of US EPA, full strength DESC3() for BUFFERED files was added, for use in the upcoming version of CMAQ. To see whether the library is built on this code-version, IF(.NOT.CHKBUF3()) STOP : you get a link-failure with previous editions of the library.

Bug-fix in error-formats from Matthew Turner, University of Colorado at Boulder.

Matthew Turner of UC-Boulder points out that error formats in functions CKDESC3() and CKFILE3() have too narrow a field for the variable-count which may occur. This is now fixed in both Versions 3.0 and 3.1.

I/O API Version 3.1: 64-bit record support; gfortran support

Support is now optionally available for "large" time-step records (total timestep extent, including all variables, exceeds 2GB). This I/O API requires netCDF version at least 3.6. The option is controlled by a new environment variable:

setenv IOAPI_OFFSET_64 < YES | NO > [NO]

"YES" enables netCDF NF_64BIT_OFFSET mode for newly-created files. The default for this environment variable is "NO" since few applications require more than 2GB of array space for a single timestep. Note that applications compiled with pre-3.6 versions of netCDF (e.g., downloaded PAVE) will not be able to open and access these large-record files.

For compile-types that use gfortran (BIN=Linux2_x86_64 and BIN=Linux2_x86_64), there is a change to the Makeinclude files: make-variable OMPLIBS is now -lgomp. Notes:

• If you're running pre-4.3 gfortran, the compiler's OpenMP support is flaky, and you should disable it in the I/O API by setting OMPFLAGS and OMPLIBS to be empty.
• You may well need to install extra packages in order for this library to be available. I consider this fact to be defective package management, but on my desktop I had to install extra packages for libgomp1 and libgomp-devel; just what you'll have to do is distribution/installation specific.

I/O API Version 3.1 Beta 1

The first beta-release version of I/O API Version 3.1 is now available.

This is a "Jubilee-Day" release:

This version is source-code compatible with previous versions; however, you should not mix INCLUDE nor object files between the two versions.

Because of data-structure-dimension changes, new and old versions are not link compatible. You will need to re-build programs that use the I/O API.

Files written with previous versions should still be readable by programs built with the new version.

Files written by the new version may not be readable by programs compiled/linked with the old version, if either they take advantage of the increased MXVARS3 or large timestep-record features.

Changes for Version 3.1 are:

• Library and m3tools codes are now strictly Fortran-90 or later.

• Hacks and Fortran-extensions used to manage dynamic arrays with Cray pointers have been removed, in favor of language standard ALLOCATABLE and dynamic arrays.

• Dimension MXVARS3 for maximum number of variables in an I/O API file is now 2048. Consequently, new files with more than the old limit of 120 variables may not be readable by pre-3.1 I/O API programs.

• Binary compile types (corresponding to environment variable BIN and compiler-option files Makeinclude.${BIN} have been changed:
  • For Sun and SGI, compiler is f90 instead of f77 (and the Makeinclude.*f90 have been re-named/eliminated
  • For Linux, gfortran similarly replaces g77
  • Support for Cray vector machines has been dropped.
  Note that in both cases, the old Fortran-77 compilers never have been link-compatible with the newer Fortran-9x compilers.

• Use of new MODULE M3UTILIO is encouraged, and use of INCLUDE-files PARMS3.EXT, FDESC3.EXT, and IODECL3.EXT is deprecated. This MODULE gives access to all the I/O API PARAMETERs and declarations, including Fortran-90-style explicit interfaces for all the relevant routines.

• For programs dealing with spatial transforms (e.g. by sparse matrices), new MODULE MATXATTS provides facilities for specifying the grid parameters for both the input and output grids for the matrix.

• INTENT is specified for all subroutine and function arguments.

• Function TRIMLEN() is deprecated in favor of new Fortran-90 intrinsic routines LEN_TRIM(), which has exactly the same meaning as TRIMLEN()), and TRIM(), which allows simplification of common character-string expressions from FOO(1:TRIMLEN(FOO)) to TRIM(FOO).

Bug fix: Transverse and Equatorial Mercator output projections in m3toolsprogram MTXCALC

Dr. Steve Yim, Department of Engineering, University of Cambridge, UK, discovered a bug in m3tools program mtxcalc treatment of Transverse Mercator outpout projections. Same bug was present for Equatorial Mercator output.

Now fixed.

Bug fix: declaration of IOPARMS3() in iodecl3.h

David Borlund, UNC, discovered a C-language link problem which traced down to a bad declaration of IOPARMS3() in C include-file iodecl3.h.

Now fixed.

NetCDF flag-changes from Martin Otte, US EPA, for netCDF4 compatibility; bug fix in m3tools program PROJTOOL.

Martin Otte of US EPA reports that netCDF Version 4 expects somewhat different flags for file-opening and file-creation than previous versions: in particular, for creating files the "open for write" flag which was required for netCDF v2 and optional for v3 must not be present for v4. Similarly for the "no-clobber" flag for opening pre-existing files.

m3tools program PROJTOOL had bad argument types to GETDBLE() in the UTM-coordinate case. Now fixed.

New program VERTINTEGRAL for computing CONC-file species vertical column totals; various fixes.

Frank Binkowski and Zac Adelman of the UNC Institute for the Environment, with help from Carlie Coats, BAMS, developed program VERTINTEGRAL for computing vertical column totals of gas-phase chemistry and mass-mode aerosol species from CMAQ conc-files, writing the result to a one-layer gridded M3IO output file. This program is OpenMP parallel, and is structured for efficiency on microprocessor based (shared memory parallel) systems.

Also there are a variety of minor enhancements to I/O API routines STR2DBLE(), CHECK3(), INQATT3(), and READ3(), a bug-fix to CURREC() affecting time periods longer than five years, and minor enhancements to various m3tools programs.

Bug-fixes in MATXATTS() and M3CPLE

Angel Rodrigues Lopezof rai.usc.es found a bug in program M3CPLE's false-Easting/Northing treatment for UTM coordinates.

Also there was an attribute-type bug in routine MATXATTS()

Both are now fixed.

Bug-fix in function STR2DBLE()

B.H. Baek, Ph.D., of the Institute for the Environment at UNC-CH found a bug in the internals of function STR2DBLE(). Now fixed.

Bug-fix in subroutine INQATT3()

Fixed an argument-list mismatch in a netCDF call inside subroutine INQATT3()

Bug-fix in M3COMBO

Dr. M. Talat Odman, of the School of Civil & Environmental Engineering at Georgia Institute of Technology points out a bug in the consistency-checking code for m3tools program M3COMBO.

pgf90/gcc issues

Current versions of the gcc compiler system have changed the Fortran name-mangling style, and I've changed the Makeinclude.Linux*pg_gcc* files to match. The old-version flags are present but commented out in these files if you need the older behavior.

Makefile issues

Neil Davis (UNC) reported some further problems with the Makefiles, and then helped with diagnosing, fixing, and validation.

New MODULE MTXATTS; Makefiles now default to compiling with a f90 compiler.

As pointed out by David Wong, the Nov. 28 fix to m3tools program MTXCALC brought in a new version from the experimental branch that uses new MODULE MTXATTS. This caused Makefile problems.

MODULE MTXATTS is intended to allow much more complete integrity checking for programs that use grid/map transformation-matrix files. It provides new routines for storing and retrieving the grid description parameters for both the input and the output grid for the matrix stored in the file:

SETMTXATT

set or check input or output grid description attributes from arguments for matrix-files: if file is NEW, set the attributes; else check them.

GETMTXATT

get input or output grid description attributes for existing matrix-files and return as arguments

CHKMTXATT

check input or output grid description attributes for existing matrix-files against attributes from the argument list

and new PARAMETERs INGRD3 and OUTGRD3 to discriminate between a matrix's input-grid and its output-grid in the above routines.

Correspondingly, the default Makefiles have been modified so that they always build the MODULEs MTXATTS and M3UTILIO and include them in the library. That means the default builds require a Fortran-90 compliant compiler (which is needed for the most common I/O API clients, CMAQ, SMOKE, and MCIP, anyway). For those requiring a FORTRAN-77-only build, the old Makefiles have been renamed with a .f77only suffix; just copy the required variant to Makefile and proceed as before.

Makeinclude.${BIN} files have not been changed for reasons of maintaining compatibility with existing directory-setups, etc. This means, for example, you can expect problems with the default Makefiles and such machine/compiler types that assume a FORTRAN-77-only such as the g77 assumed by ${BIN}=Linux2_x86.

Hack/Fix for WMO-screwup Longitude conventions in programs M3CPLE and MTXCALC

It turns out that the WMO has invented a "standard" that conflicts with ISO Standard 6709 (as well as treaty obligations and centuries of common usage), by which—according to WMO—longitudes should be always positive, and North America has longitude range something like +200 to +300. We have encountered (obscure, hard-to-expect, hard-to-diagnose) problems dealing with data following these conventions, and have therefore implemented heuristics to deal with this in m3tools programs M3CPLE MTXCALC.

Be aware that you will have to deal with this WMOism in GRIB2 data, as well as a lot of other data coming out of national meteorology organizations.

Add support for new map projections in ckdesc3()

Neil Davis (UNC) reports lack of support for map projection type EQMGRD3 in ckdesc3(). Iurns out that TRMGRD3, ALBGRD3, LEQGRD3 weren't supported either, by oversight (many codes don't use this file-header-checker when creating new files...).

Now implemented, together with comments in FDESC3.EXT indicating the conventions for defining map projections of types ALBGRD3, LEQGRD3.

Bug-Fix in m3tools/Makefile.nocpl

Neil Davis (UNC) reports a bug in m3tools/Makefile.nocpl, together with the required fix. Now implemented.

Bug-Fixes for Albers

In ALBERS() and LL2ALB() in the experimental Albers Equal-Area Conic map projection support: internal map-projection parameters were not being maintained correctly.

Experimental Albers Equal-Area Conic map projection support
MM5 and WRF spheres added to GCTP() and SETSPHERE() by Steve Howard, NOAA/ASMD
Bug-fix to dayagg, for a problem reported by Alexandros Syrakos.

INCLUDE files PARMS3.EXT and parms3.h have new token-values

ALBGRD3

Albers Equal Area Conic map projection

LEQGRD3

Lambert Azimuthal Equal Areq map projection

and beta-test partial Albers-projection support has been added to I/O API subroutine LAMBERT, in new entries ALBERS(), SETALB(), ALB2LL() and LL2ALB(), and to m3tools programs LATLON, PROJTOOL, M3CPLE, MTXCALC, and M3FAKE.

Steve Howard, NOAA Atmospheric Sciences Modeling Division, added two additional sphere-types to I/O API subroutines GCTP() and SETSPHERE():

Type 20

normal sphere with R_Earth=6370000 meters, matching default MM5 and WRF-ARW.

Type 21

normal sphere with R_Earth=6371200 meters, matching default WRF-NMM.

There are consequent changes to m3tools program M3CPLE.

Alexandros Syrakos reported a bug in m3tools program DAYAGG: output starting date should be 1000*(YEAR==0)+(DAY==1) = 1 (instead of 0), in order for repeated applications to work.

Bug-fix to m3combo, from Bonyoung Koo, ENVIRON International Corporation.

Bonyoung Koo reported a further bug in m3combo, that happens with relatively long variable-names (not found in my test case), and suggested exactly what the fix needed to be. Thanks!

Also, minor fixes in m3cple: default INSPHERE value, eliminate (invalid) resolution warning for Cartesian-to-Lat/lon coordinate transformations.

Multi-decade support; some Mac OS10 support; netCDF 3.6.2 warning.

Debra Baker of the University of Maryland has put in some extensive work figuring out how Mac OS-X (and generally Carnegie-Mellon Mach-descended) linker symbols work, resulting in a gfortran build for Macs; the new Makeinclude file is Makeinclude.OSX_x86. Scott N. Spak of the Center for Sustainability and the Global Environment at the University of Wisconsin-Madison provides an Intel/Gnu ifort/gcc configuration which may now be found in Makeinclude.OSX_x86_64ifort.

Evidently, all of the compilers on Mach-based operating systems generate linker-symbols with prefixed underscores, unlike UNIX systems which take the C-language names verbatim (so that C "void foo" becomes linker symbol "_foo" instead of "foo", for example.) This is highly confusing to traditional Unix users of the nm linker-symbol utility;-( Other Mac compilers should behave similarly to these two. [Send me more working variations, and I will include them in the distribution and credit you for them...]

Debra also reports that when you build netCDF version 3.6.2 and higher, you should be careful not to use the --enable-shared flag, because it excludes Fortran symbols from the resulting libnetcdf.a.

Christian Hogrefe of the New York DEQ pointed out (and diagnosed!) problems with integer-overflow in the internal record-calculation routines which he found for multi-decade runs, and then participated in fixing and testing the improved versions, which should be OK (but maybe slow) for even multi-millenium runs.

Bug-fix in ISDSTIME()

Prakash Doraiswamy reports a bug in 2007-and-later start of daylight savings time. Now fixed.

Also: format-bug fix in SYNC3(); Makeinclude for binary type Linux2_x86path (32-bit PathScale Fortran).

Bug-fix in worker routine chkfil3()

A bug was introduced into CHKFIL() to support same-set/different-order variables-lists (at someone's request; I don't recall who). This is bogus. Revert to reqiring variable lists have the same order.

Makeinclude improvements

Improvements to Makeinclude files for Linux2_x86ifcdbg, Linux2_x86_64ifortdbg, Linux2_x86_64ifort, Linux2_ia64ifortdbg, Linux2_x86_64sun, Linux2_x86_64path, to better support current compilers -- particularly Intel version 10.x.

Improved error-checking in WRITE3(), CHKFIL().

Linux x86_64 Makeinclude support for PGI compilers

Prakash Doraiswamy, Atmospheric Sciences Research Center, SUNY Albany and New York State Department of Environmental Conservation, and Winston Hao, New York State Department of Environmental Conservation, provided flag-sets that they use with their 64-bit Opteron cluster. These may be found in the new release's Makeinclude.Linux2_x86_64pg_pgcc_nomp and imply that the user has compiled libnetcdf.a with PGI pgcc; for GNU gcc-compiled libnetcdf.a, a companion but untested flag-set is in Makeinclude.Linux2_x86_64pg_gcc_nomp.

They report that they successfully built libnetcdf.a with the following flags:

setenv CC pgcc setenv CXX pgcCC setenv FC pgf90 setenv F90 pgf90 setenv CFLAGS "-O2 -Msignextend -V" setenv CPPFLAGS "-DNDEBUG -DpgiFortran" setenv FFLAGS "-O2 -w -V"

following this advice from the PGI website.

Bug-fix/enhancement from Bonyoung Koo, Environ, to m3tools program M3COMBO; fix to CKDESC3() for boundary files

Bonyoung Koo, ENVIRON International Corporation, found a problem situation in M3COMBO, where one wanted to use variables from distinct files but having the same names to build the combinations, and provided a new version of the code with an ingenious way of handling this problem, as well as with bug-fixes dealing with defaults for time step sequence issues.

Michael Bane, U Manchester, UK), found a problem with internal routine CKDESC3() for "box-model" test cases (BugZilla Bug 2355), where boundary files could not exist for 1x1 box-models. This is now fixed (although the fix now permits very thick boundaries in some cases).

Fixes to m3tools programs M3COMBO and M3TPROC

George Pouliot, US EPA, found a bug in the "all variables" initialization for program MTXCALC and provided a fix.

Jerry Condrey, Baron Advanced Met Systems, found an inconsistent initialization in program M3COMBO and provided a fix.

#include <stdlib.h> in getdttime()

As pointed out by Dr. Frank Binkowski, gcc versions 4.x generated a warning about the call to exit() in getdttime(). This is alleviated by adding a reference to stdlib.h.

Sun compilers:s Bug-fix in program M3COMBO

Sun Studio Express compilers for Solaris and Linux are now available at no charge for both commercial and noncommercial use from http://developers.sun.com/sunstudio/index.jsp. I/O API support has been added for using these compilers under Linux in both 32-bit and 64-bit modes, with corresponding binary types Linux2_x86_64sun and Linux2_x86sun.
This is new stuff, and should be considered "beta" at this time.

Fix in program M3COMBO for a bug that was evidently introduced in the April 26, 2006 release.

bug-fix in JSTEP3()

Fix in JSTEP3() for a bug that was introduced in the "standard-year" set of mods.

Daylight savings time; program M3STAT; negative JDATEs

Subroutine ISDSTIME has been enhanced so that it supports daylight savings time determination for years both 2006 and earlier, and 2007 and later (where note that the US Congress changed the starting and ending times effective year 2007).

M3STAT had a bug in processing files of type BNDARY3. This has now been corrected.

There were bugs in time-handling for dates with negative year component. Such dates are required for INTERP3() usage with Standard-Year and Standard-Week files. Affected routines are

• CURRSTEP() and CURREC()
• DAYMON()
• JSTEP3()
• MMDDYY()
• NEXTIME()
• SECSDIFF()
• Private routine RDTFLAG()

Note that PAVE has a bug in constructing legends for files with SDATE3D < 0

Corrected bug bug in program MTXCALC

MTXCALC had a map projection-types array that incorrectly swapped Equatorial Mercator and Transverse Mercator, as reported by A. Fraser of ic.ac.uk. This has now been corrected.

Corrected Splash-Screen bug in INIT3

INIT3 incorrectly generated double-copies of the opening splash screen under some circumstances. Thanks to John McHenry for noting this issue.

Corrected Makeinclude.Linux2_x86pg_gcc_mp and Makeinclude.Linux2_x86pg_gcc_nomp

gcc and its flags were not being set correctly. Thanks to NOAA/ASMD for noting this change.

m3tools program M3CPLE Enhancement

m3tools program M3CPLE now accepts environment variable IOAPI_ISPH to control what geodetic spheroid is used for grid-to-grid interpolation operations. The default is still the GRS-80 spheroid that previously was hard-coded in the program.

C/Fortran interface utilities; MMDDYY() bug-fix

Fortran::C string conversion routines fstr2cstr(), name2cstr(), cstr2fstr() for C programmers were promoted from private (internal-only) to public (callable as part of the I/O API library.

Subroutine MMDDYY() was modified so that it handles year-argument zero correctly (needed for SMOKE-RT for WRF).

Warning-message format fix in CKDESC3()

Tanya Otte, US EPA, notes that internal I/O API worker routine CKDESC3() had a bad warning-message format. This bug is now corrected.

Added WRF vertical coordinate support to worker routine CKFILE3()

Tanya Otte, US EPA, notes that internal I/O API worker routine CKFILE3() did not support the WRF vertical coordinate types. Support was added for both WRF vertical coordinats types. Upon further examination, CKDESC3() did not support WRF-NMM; this support was also added, and the routine was restructured for greater simplicity.

Added INTERFACE block for GTPZ0() in m3utilio.f; corrected legend in utmtool

MDOULE M3UTILIO now supports interface checking for USGS-derived routine GTPZ0().

"Lower-Right" and "Upper-Left" legends were swapped for the grid-corner modes (40,41,42,43) of utmtool (although you could have figured out what you wanted from the accompanying <X,Y>::<LAT,LON> values).

Also comment-correction related to vertical grid types by Jerry Condrey in routine GRDCHK3() (no code-changes).

Support for WRF EM vertical coordinate type VGWRFEM in CKDESC3()

Tanya Otte, US EPA, notes that internal I/O API worker routine CKDESC3() did not support the WRF EM vertical coordinate type VGWRFEM defined in post-May-2005 versions of INCLUDE-file PARMS3.EXT. This bug is now corrected.

New multiple-input file version of program M3COMBO; bug-fix in M3COMBO, M3DIFF

There is a new version of "m3tools" program M3COMBO that can use variables from multiple input files (rather than from a single input file) in constructing its linear-combination output variables. The program even supports mixing time independent and time stepped input files.

Additionally, a minor bug in the specification of starting time for runs of programs M3COMBO and M3DIFF has been fixed (previously, if the input file started at date&time 2006116:120000, for example, the program would not allow you to start a run at 2006117:000000, since it thought 000000 was "before" 120000 in all cases.

Because of the complexities in specifying linear combinations of variables from multiple input files, the new multi-file-input version of M3COMBO is not script compatible with the preceeding version.

state3.h storage-order error for IOAPI_CSTATE3

David Wong points out that versn had been inserted into the wrong storage location in the IOAPI_CSTATE3 typedef in C include-file state3.h. This is now fixed.

PGI "save" flag fix

The FSFLAG in the Makeinclude.*pg* has been fixed. This is used to enforce default SAVE for variables in USGS routine GCTP. If not present, this can potentially cause geospatial transform results to be flaky.

Another M3DIFF bug-fix

As pointed out by Matthew Johnson of the Iowa DNR - Air Quality Bureau, in the January 27 bug-fix, an additional typo crept into program M3DIFF at line 560 -- an extra inadvertent

IF ( VARMODE ) THEN

crept into the code. This has now been fixed.

M3DIFF bug-fix

Program M3DIFF had an output time step sequence at about lines 653-735, as pointed out by Dr. Christian Hogrefe of the State of New York, who also suggested the correct fix.

MTXCALC bug-fix

Program MTXCALC had an off-by-one-half error at about line 732, as pointed out by Dr. Michael Bane of the University of Manchester (UK), that is now fixed. Also one interface-block fix in MODULE M3UTILIO.

LAMBERT() fixes; M3UTILIO additions

Subroutine LAMBERT() and its entries did not initialize ostensibly unused arguments to USGS routine GCTP(). Consequently, for compiler environments that initialize variables to NaN, internal GCTP() arithmetic (that is in principle unnecessary) on these array elements could lead to floating point exceptions. LAMBERT() now initializes the affected aray arguments with zeros.

MODULE M3UTILIO now has interface blocks for all 36 of the entries to LAMBERT().

Support for Absoft Pro Fortran

Support has been added for Absoft Pro Fortran for x86 and x86_64 Linux. Binary types are:

Linux2_x86af95

32-bit Linux with Absoft native linker-sumbol name-mangling (subroutine and function names are upcased; COMMON block names are prefixed with _C and then upcased)

Linux2_x86af95_

32-bit Linux with Intel/PGI/Lahey compatible linker symbols (subroutine, function, and COMMON block names are downcased and suffixed with a single underscore)

Linux2_x86af95dbg

32-bit Linux with Absoft native linker sumbols compiled for debug

Linux2_x86af95_dbg

32-bit Linux with Intel/PGI/Lahey compatible linker symbols, compiled for debug

Linux2_x86_64af95

64-bit (Opteron) Linux with Intel/PGI/Lahey compatible linker symbols

Linux2_x86_64af95_

64-bit Linux with Intel/PGI/Lahey compatible linker symbols

Linux2_x86_64af95dbg

64-bit Linux with Absoft native linker sumbols, compiled for debug

Linux2_x86_64af95_dbg

64-bit Linux with Intel/PGI/Lahey compatible linker symbols, compiled for debug

Mac ioapi package from Alex Zubrow at the University of Chicago; assorted unused variables removed

Alex writes:

Hi all.

If you want to use libioapi on a Mac, I've added the package to fink. For those who don't know fink, it is a package management system that automatically downloads and builds packages for you to work on a Mac. Through fink, you can easily install a whole series of useful scientific and graphics packages.

To get fink, go to:

http://fink.sourceforge.net/

You'll need to set it up to use the unstable branches of the distribution, see: http://fink.sourceforge.net/faq/usage-fink.php?phpLang=en#unstable Once you have fink working, you can get the ioapi package from the command line:

$ fink install ioapi

Or you can use the GUI FinkCommander.

I hope this is useful to folks out there.

Best,
Alexis

Makeinclude files are now available for the Absoft Fortran 90/95 compiler under Linux for x86 and x86_64 (64-bit mode on Opteron etc.):

Makeinclude.Linux2_x86_64af95 Makeinclude.Linux2_x86_64af95dbg Makeinclude.Linux2_x86af95 Makeinclude.Linux2_x86af95dbg

Additionally, in a number of routines (READ4D, NAME2FID, LBLANK, GETNUM, OPEN3, MMDDYY, INITLOG3, INIT3, UNGRIDI, SETSPHERE, CHECK3, CBARNESN, CBARNES1, OPNLIST3, M3UTILIO, GETMENU, DDTVAR3 ) and m3tools programs (DIFFSTEP, VERTOT, BCSTEP, STATSPARS, LATLON, FAKESTEP, DAYAGG, AIRS2M3, AGMAX, M3CPLE, AGMASK, M3INTERP, M3COMBO, MTXCALC, MTXBUILD, MTXBLEND, M3TPROC, M3MERGE, AGGVARS, SELMRG2D, PRESTERP, MTXCPLE ), unused variables (as diagnosed by the Absoft f95) were eliminated.

Back to Contents

Dr. Michael Bane of the University of Manchester (UK)

Python ioapiTools package from Alex Zubrow at the University of Chicago; UNGRIDI;

Alex Zubrow of the University of Chicago just announced release of a beta version of his ioapiTools. For those of you who didn't get a chance to hear his talk at the last CMAS conference, ioapiTools is a python interface to I/O API data. It allows you to easily import the data into python, have full access to the I/O API metadata, use the full breadth of python's scientific libraries, create plots, and write out your data to new I/O API or netCDF files. See:

http://www-pcmdi.llnl.gov/software-portal/Members/azubrow/ioapiTools/index_html

New subroutine UNGRIDI computes "incidence matrix ungridding" indices for grid-to-grid transformation from a grid to the set of target (e.g., point source) locations as specified above, i.e., so that for a variable V(NCOLS*NROWS) single-indexed on the source grid,

VNEW(S) = V(NX(S)) is the value at { <XLOC(S),YLOC(S)>

There was a trivial inconsistency in the trailing entry VERSN3 of /CSTATE3/ between ioapi/STATE3.EXT and ioapi/fixed_src/STATE3.EXT (which could easily be fixed by running make fixed_src. Thise COMMON-block entry is never referenced by any CMAQ code, and so this discrepancy should never have caused any difficulty. However, these are now fixed up to be consistent. That such a problem arises shows what a serious mistake it was for CMAQ to insist upon a nowhere-standard Fortran source format. Such failure to follow standards almost guarantees referential integrity problems such as this one.

Back to Contents

Bad M3TOOL starting time constraints

The following programs had incorrect starting time constraints (in all cases, the starting time for program-run was incorrectly constrained to be at least as late as the starting time for the input file(s)).

M3COMBO

M3CPLE

M3INTERP

MTXBLEND

MTXCPLE

These programs are now fixed.

Back to Contents

XTRACT3(); web-pages

Dr. Michael Bane of the University of Manchester (UK) pointed out an extraneous VNAME = VNAME in the code for XTRACT3(). This was left over from code unifications for I/O API Version 3, and would have appeared to be harmless, but seems to cause seg-faults on some architectures.

Dr. Joseph Vaughn of the Washington State University (Pullman, WA) pointed out errors in the documentation for NEXTIME(). This has now been fixed.

Back to Contents

F77 issue in GRDCHK3

Alex Zubrow of the University of Chicago points out an issue in grdchk3.f: the string concatenation at line 162 is not permitted in strict Fortran-77, although it is legal in Fortran-90 (and is supported as an extension by most f77 compilers). This is now replaced by a modified format and arg-list that does pass strict Fortran-77 checking.

Back to Contents

Mods to PARMS3.EXT for WRF-Chem

Two additional parameter tokens are added to INCLUDE-file PARMS3.EXT, for WRF vertical coordinate types, to support the on-going work to put the Models-3I/O API into WRF, and to create a WRF-Chem/SMOKE cooperating-model coupled system:

VGWRFEM=7

WRF mass-core sigma vertical coordinate type

VGWRFNM=8

WRF NMM vertical coordinate type

Back to Contents

m3tools program enhancements

Many thanks to Dr. Michael Bane of the University of Manchester (UK) who has pointed out a number of problems with various cases in the the large-scale make system.

• There was a bug in the interaction between the top-level make configure and make [no]cpl commands... the latter would not use the Makefiles created by the former! This bug should now be corrected... make configure now copies the Makefiles it produces to Makefile.${CPLMODE}.
• There was also a problem with make configure for the m3toolsprograms.
• There were some compile-flag problems causing warnings, with respect to the Intel compilers for Linux (both x86and ia64).
• There were problems with INSTALL vs INSTDIR for make install

Back to Contents

m3tools program enhancements

Smarter default values for user-entered NRECS are provided in the following programs. Primarily affected is the case when the input and output time steps are unequal.

• M3COMBO
• M3CPLE
• M3TPROC
• M3INTERP
• MTXBLEND
• MTXCPLE

Back to Contents

AIX Build Changes

As pointed out by David Wong, US EPA, there were3 problems for AIX with preprocessor flags for the new build system and with program M3EDHDR. The flags are now special cased for AIX, and M3EDHDR now uses subroutine M3PROMPT to get around the flush() problems for AIX.

Back to Contents

Build Enhancements

There are problems with the Fortran pre-processing for some versions of the Intel Fortran compilers; additionally, many Microsoft Windows build environments do not provide properly for Fortran pre-processing. These problems can to some extent alleviated by separating the preprocessing from the compile step for the *.F files. Makefile.cpphack are sample Makefiles in directories ioapiand m3tools for this purpose (as provided, they assume OpenMP and coupling mode).

Back to Contents

Bugfix in programs M3CPLE, M3INTERP, MTXBLEND, and MTXCPLE

These programs had a bug in the prompt for starting time: they did not permit an STIME less than the starting time for the input file(s) even if the starting date was later than the input file starting date. This bug manifested itself, for example, if one had inputs starting at 2005153:120000 but wanted a program run starting at 2005154:000000. [One could have used the workaround of requesting a run starting at 2005153:360000... the I/O API time manipulation routines are quite robust in that regard.]

Back to Contents

Bugfix in NEXTIME() and CURREC()

NEXTIME() had a bug with respect to normalizing JDATE = 0 arguments in all cases, not just the 0:0:0 case; moreover there were similar problems with CURREC().

Back to Contents

Bugfix in NEXTIME()

NEXTIME() had a bug with respect to normalizing 0:0 = JDATE:JTIME arguments (recall that NEXTIME(JDATE,JTIME,0) may be used to normalize JDATE:JTIME so that JTIME is between 0 and 235959); this is now handled correctly.

This manifested itself in the creation of time-independent files (with SDATE3D = STIME3D = TSTEP3D = 0; the file header would record SDATE3D = -635. The I/O API would handle such files correctly, but it certainly "looks" bad...

Also minor enhancements to experimental MODULE M3UTILIO.

Back to Contents

Bugfix in ENVYN()

DAVID Wong (US EPA) points out a bug in ENVYN() that occurs when the value is set to .TRUE. or .FALSE.. Now fixed.

Back to Contents

Makefile.nocpl issue: ddtvar3v.F and interp3v.F

Files ddtvar3v.F and interp3v.F are active only when coupling mode is enabled (otherwise, the preprocessor strips their contents out, leaving empty results (which some compilers, e.g., Intel ifort Version 8.1, don't like...). These routines should not be in the FSRC list in Makefile.nocpl; that file has been revised accordingly.

Back to Contents

Bugfix for file header checking routine

Dr. Michael Bane of the University of Manchester (UK) pointed out a bug in duplicate variable-name checking which had been put into file header checking in November (active when IOAPI_CHECK_HEADERS is turned on). This bug is now corrected.

Weakened architecture/optimization flags for binary type Linux2_x86_64 to get around a compiler bug, i.e., internal compiler errors in the gcc/g77 shipped with Fedora Core 3 for x86_64: the compiler has an internal "unable to find a register to spill..." error when attempting to compile certain routines. Note that this same compiler bug will bite you when trying to build lots of stuff on FC3/gcc/g77 systems, and the same fix seems to work universally.

Minor changes to Makefiles for better handling of the new experimental Fortran-90 MODULE M3UTILIO.

Following another suggestion from Dr. Bane, the command make gtar (used to create the distribution download tar-files) now creates a file VERSION.txt in the distribution root directory prior to creating the tar.gz file. VERSION.txt contains the result of running the system date command at the time make runs the gtar command.

Back to Contents

Build System Enhancements, Bugfix for CBARNES gridding routines

There were troubles with floating point exceptions due to the interaction between round-off error and the exact-distance formula in the CBARNES Barnes analysis routines. The previous approximate-distance formula (which did not have this problem) has been put back in.

PathScake compiler support for AMD Opteron processors under Linux has been added.

Coupling mode support for GNU compiled libraries has been added (with a new treatment of GNU-style Fortran name mangling...)

Back to Contents

READ4D() change; new MODULE M3UTILIO

Alex Zubrow of the University of Chicago points out that READ4D did not return a value for success.He also pointed out the duplicate potential definition of PERROR() in GETEFILE() and GETFFILE(). These are now corrected.

New file m3utilio.fcontains an experimental Fortran-90 alternative, MODULE M3UTILIO, to the INCLUDE files 'PARMS3.EXT, FDESC3.EXT, and IODECL3.EXT, as well as providing explicit INTERFACE blocks for most of the public I/O API routines (those for which argument-signature is not subject to variation).

Back to Contents

New "Edit FDESC3D" capability for program M3EDHDR

The capability of editing the file description field FDESC3D was added to m3tools program M3EDHDR in order to compensate for systematic SMOKE Version 1.x and 2.x mis-design.

Back to Contents

Bug fix for worker routine m3abort()

There was an error in worker routine m3abort() (called from OPEN3()) that would cause garbled error messages under some circumstances, when files could not be opened. Fixed.

Back to Contents

Bug fix for GCTP;
Format fix for envrealc() and envdblec()

Catherine Seppannen of UNC-Carolina Environmental Program diagnosed an obscure bug that occurs in the GCTP map-projection package from USGS:

I've traced the problem to the subroutine SPHDZ0 in gctp.f. When GTPZ0 is first called, it calls SPHDZ0 with dummy values that cause the spheroid to be set to Clarke 1866. A later call to SPHDZ0 is made that looks like it should reset the spheroid. However, at line 5475, the subroutine checks if the spheroid needs to be set and then returns. This means that the spheriod is never changed from Clarke 1866. When processing UTM coordinates, the spheroid is changed outside of SPHDZ0 (at lines 1273 and 1274) so the correct spheroid is used in later calculations. For now, simply removing line 5475 and moving the label to the next line seems to solve the problem.

Dr. Michael Bane of the University of Manchester (UK) notes that there are problemswith underflow with the existing %f format used in envrealc() and envdblec(). This has now been changed to a more-robust %G,that should not be subject to underflow.

Back to Contents

Range checks for NVARS in OPEN3

To facilitate user changes to parameter MXVARS3 in INCLUDE-file PARMS3,EXT, additional checks were inserted into OPEN3 and its worker routines to ensure that one may not inadvertently open files with NVARS too large for the current libioapi.a build.

Note that environments in which nonstandard values of parameters from PARMS3,EXT are used must be managed with great care: inconsistent values of these parameters for different objects or files within a build- or usage-scenario can lead to very hard to diagnose, obscure failures, and requesting help for such difficulties will still invoke the triple-time support penalties in the I/O API license.

Back to Contents

Fixes to M3TPROC

Dr. Michael Bane of the University of Manchester, Manchester, UK, submits changes to program M3TPROC to allow clean compilation on the new Intel Version 8.1 compilers: removal of a duplicate declaration for ASTEPS, and reversion to F77 style declarations (instead of F90 style) for MENUITMS and OPNAMES.

Back to Contents

Duplicate declaration removed from STATE3.EXT; new experimental Fortran-90 MODULE M3UTILIO

Dr. Michael Bane of the University of Manchester, Manchester, UK, noticed a duplicate declaration of variable VERSN3 in INCLUDE-file STATE3.EXT. This was clearly an error, and may have been responsible for cryptic error messages on various platforms. He also points out that Intel "ifort" Version 8 generates a "Command line warning: overriding '-auto' with '-save'" message during the compilation of GCTP; this latter message is not a symptom of an underlying problem (the build should still be correct.

MODULE M3UTILIO: Fortran-90 INTERFACE Blocks

USE M3UTILIO

INCLUDE 'PARMS3.EXT'
INCLUDE 'FDESC3.EXT'
INCLUDE 'IODECL3.EXT'

I am thinking of splitting this module into two pieces, however: an INTERFACE block/declaration module (M3IO?), and a module for the SMOKE-related utility routines (MUTIL?); I welcome feedback on that matter (and in fact on the whole idea). -- CJC

SunOS SPARC 64-Bit? Opteron 32- and 64-bit? Others?

Back to Contents

Enhancement for CHECK3()

During the transition to I/O API Version 3, the responsibility for "timestep not available" status and for messages about that status was transferred from routines CHECK3 DDTVAR3 INTERP3 READ3 READ4D XTRACT3 to worker routine RDTFLAG. However, such messages should be generated for all of these routines except CHECK3, as was discovered during the implementation of the new WRF-enabled / sub-gridscale terrain version of the SMOKE TMPMOBIL program. Therefore, a new "verbose-output" flag was added to the argument list for RDTFLAG, and the other routines were modified accordingly.

Back to Contents

Bug fix, version in sample program VERTOT

Bill Hutzell of EOA's Atmospheric Modeling Division, National Exposure Research Laboratory, pointed out a bug in sample program VERTOT, and provided the fix: when used to process all variables, the previous version of the program did not handle the list of output variable names (etc.) correctly.

Back to Contents

Bug fix, GCD() and RDTFLAG(); enhanced program M3TPROC

There was a coding bug in routine GCD() and bugs in the error messages for worker-routine RDTFLAG(). These are now corrected.

Program M3TPROC was enhanced so that it will now do time aggregation of minima, in addition to the previously supported arithmetic means and maxima (so that one can now compute daily-minimum layer 1 temperature, for example). The time-stepping logic was enhanced to support partial-input aggregation, and the program control logic was completely overhauled to get rid of the absurdly complex hierarchy of environment variables and command-line prompts used in the prior version. Consequently, scripts used with the previous version will no longer work: the new version requires re-scripting.

Back to Contents

Bug fix, version in CRTFIL3()

As pointed out by Dr. Michael Bane of the University of Manchester, Manchester, UK, worker routines CRTFIL3() and CRTKF3() were not correctly picking up the I/O API version to insert into file headers. This is now corrected.

Back to Contents

Bug fix, virtual mode INTEPR3()

The virtual-mode part of the interp3() system still had a bug that occurred for PVM-based "virtual" files (netCDF file behavior is unaffected). This has now (finally!) been fixed and the fix validated.

Back to Contents

Bug fixes; program m3interp

ungridb() had a fencepost error in dealing with the last row of the input grid, leading to errors when the output grid extended beyond that edge. Bug was reported by Catharine Sepannen, UNC CEP.

open3() now checks for repeated occurrences of variable names when opening files as "new" or "unknown", leading to more meaningful error messages than those which would otherwise come from netCDF.

The virtual-mode part of the interp3() system has been re-written to eliminate a bug which occurs forPVM-based "virtual" files (netCDF file behavior is unaffected).

New program m3interp is similar in capabilities to m3cple but in addition does time-interpolation to a potentially higher-resolution time step sequence. (This program has been moved from iotests directory to m3tools.)

Back to Contents

Bug fix in currstepc()

currstepc() was wrong -- I found the bug during attempt to rationalize/re-engineer the structure of PAVE. The previous version would almost certainly have caused a seg-fault.

Back to Contents

Bug fix in M3COMBO

M. Talat Odman of Georgia Tech points out a bug in M3COMBO: the first term in the linear combinations was wrong for multi-variable M3COMBO runs.

Back to Contents

Bug fixes in M3MSG2 and UNGRIDB

Catherine Seppanen of UNC-CEP points out a bug in UNGRIDB: several clauses of the form

    ELSE IF ( C .GE. NCOLS - 1 ) THEN !  c right of grid
    

    ELSE IF ( C .GT. NCOLS - 1 ) THEN !  c right of grid
    

M3MESG and M3MSG2 had an argument-mismatch problem if they had to call worker routine INITLOG3 (which would occur if they were called prior to callingINIT3; this has been fixed.

Back to Contents

Message fix in MTXCALC

Luca Delle Monache of the Atmospheric Science Programme at the University of British Columbia points out that the warning message for unsupported coordinate system types was incorrect in program MTXCALC. This is now fixed in the I/O API Version 3 download.

Back to Contents

I/O API Version 3.0 Pre-Release 2

David Wong of SAIC characterized a bug in the BINFIL3 version of XTRACT3(), which is now fixed. There was also an over-dimensioning of the malloc()ed input buffer in that routine, now corrected.

Back to Contents

I/O API Version 3.0 Pre-Release 1

I/O API Version 3.0 is now available for final user testing. Among its features are the following:

• New/improved build-system with a new ew top-level Makefile which allows one to build various configurations of the entire (I/O API + m3tools) system with one make command.

• More complete WRF support
  • File and variable name argumentss are now declared CHARACTER*(*) with a maximum length of 16, instead of being fixed-length CHARACTER*16). (This is still not fully compatible with WRF, which uses a maximum length of 32...)
  • Fortran INCLUDE-files are now in the ISO-suggested "hybrid format" (see below).

• INCLUDE files are now safe for both fixed-form (F77-style) and free-form (F90-style) source code format: line continuations have two ampersands, one in column 73 (where it is not visible to F77-style compilation, and one in column 6, where it is the first non-blank character in the line.

• New script fixed_src.csh and build target make fixed_src to deal with INCLUDE files for nonstandard CMAQ fixed-132 source code form.

• A new "native-binary I/O" (BINFIL3) lower layer, in addition to the current netCDF, BUFFERED, VIRTUAL, and FILE-LIST lower layers. This is the least-tested part of the code, and is primarily intended for use with ETA/CMAQ at NOAA/NCEP, where the departmental politics prevents the existence of libnetcdf.a on any NCEP computer system.

• A more thorough-going treatment of Fortran/C interface type-matching, to deal with portability to platforms where one or more of the following do not occur:
  • Fortran REAL is not C float
  • Fortran INTEGER is not C int
  • Fortran CHARACTER string length type is not C int

• Support for additional compiler/OS combinations, including Intel Fortran and C for both Linux2_x86 and Linux2_ia64, as well as the new g95 GNU Fortran-95 compiler for Linux2_x86 (see URL http://www.g95.org/)

• Various new routines, including M3PARAG, M3FLUSH, IOPARMS3 GET_ENDIAN with corresponding parameters LITTLE_ENDIAN, BIG_ENDIAN, and PDP_ENDIAN in PARMS3.EXT and parms3.h

• Messaging routines M3MESG and M3MSG2, M3WARN, and M3EXIT, have been modified to get around problems with SGI compilers that were causing duplicate copies of log output on SGIs.

• Improved OpenMP thread-safety.

• The tar-file naming scheme now matches GNU standards.

Documentation for I/O API Version 3 continues to be in preparation.

Back to Contents

New Makefiles for I/O API version 2.2; beta availability for I/O API version 3.

In the retrofitting of M3PARAG, M3FLUSH, etc., into I/O API Version 2.2, I failed to update the Makefiles to match. This error has now been corrected for the new download tar-file.

The beta release of I/O API Version 3, as described below is now available for download.

Further documentation for I/O API Version 3 is in preparation.

Back to Contents

New subroutines M3PARAG, M3FLUSH

New subroutine M3PARAG ("M3-paragraph") allows the caller to write multi-line messages (in a thread-safe manner) to the log.

New subroutine M3FLUSH allows the caller to flush a specified Fortran file to disk. This routine is useful for dealing with portability problems due to the fact that various operating system/Fortran compiler combinations define FLUSH inconsistently.

Subroutines M3MESG, M3WARN, M3EXIT, and M3ERR have been re-worked to enhance OpenMP thread-safety. (Actually, all of these except M3WARN, M3EXIT,and M3ERR are ENTRYs to the nified message-treatment now implemented in m3msg2.F.)

Back to Contents

New subroutine IOPARMS3

New subroutine IOPARMS3 allows for run-time checks in model-level code, to check the I/O API dimensioning parameters from INCLUDE-file PARMS3.EXT being used to build the model match those used to build the libioapi.a being linked against. This is made the more necessary by the tendency of some among the SMOKE emissions modeling community to change dimensioning parameters in PARMS3.EXT and not simultaneously update the INCLUDE-files used for building SMOKE itself.

Back to Contents

Bug-fix to subroutines LAMBERT; enhancements to CKFILE3, and CKDESC3

Eric Giroux of the National Research Council of Canada contributes a bug-fix and code enhancements made by his colleague Dazhong Yin:

• bug-fix to routine LAMBERT, dealing with conversions from polar sterographic to Lat-Lon coordinates.
• enhancements to CKFILE3, and CKDESC3 to provide appropriate "sanity checking" to file descriptions, when the underlying coordinate system is polar sterographic.

Back to Contents

Bug-fix to M3STAT

There were two problems with M3STAT: one serious, the other less so:

• For single-variable boundary files (FTYPE3D=BNDARY3), the program did not set local variable SIZE correctly, leading to segmentation faults when the worker routine tried to use SIZE as a dimension for the input buffer.
  Fixed.

• When run without command-line arguments, the program did not accept NONE as the logical name for the REPORT file, so that the program would generate output to the log file or screen.
  Fixed.

Back to Contents

GDTYP=STEGRD3 checking; Type-portability, I/O API Version 3

Eric Giroux points out that the treatment in CKDESC() for map projection parameter PROJ_GAMMA for polar sterographic grids (GDTYP=STEGRD3) is incorrect: this parameter should have been ignored, instead of being checked against the range [-180,180]. Fixed in the new 2.2 release for Oct. 17.

David Wong points out that ENVREAL does not work correctly on the Cray T3E. A correct fix for this problem involves the use throughout of typedef FREAL and of type-casts back and forth between FREAL and float. Stopgap-fix in the new 2.2 release for Oct. 17; this will be treated more carefully in the upcoming I/O API Version 3 (below)

I/O API Version 3 is now under development. Among its features will be the following:

• More complete WRF support.
• INCLUDE files are now safe for both fixed-form (F77-style) and free-form (F90-style) source code format.
• New script fixed_src.csh and build target make fixed_src to deal with INCLUDE files for nonstandard CMAQ fixed-132 source code form.
• A new "native-binary I/O" (BINFIL3) lower layer, in addition to the current netCDF, BUFFERED, VIRTUAL, and FILE-LIST lower layers.
• A more thorough-going treatment of Fortran/C interface type-matching, to deal with platforms where one or more of the following do not occur:
  • Fortran REAL is not C float
  • Fortran INTEGER is not C int
  • Fortran CHARACTER string length type is not C int

Back to Contents

Makeinclude.OSF1 enhancements

Jinyou Liang, of the California Air Resources Board, suggests the following additions to Makeinclude.OSF1, for making the parallel version of the I/O API on Alpha OSF1 systems:

Add to FOPTFLAGS:

-arch host -tune host

Add to OMPFLAGS:

-omp

Add to OMPLIBS:

-omp

Back to Contents

Compiler-Bug Report: IRIX F90 Version 7.4

Conditional-compilation for the I/O API now takes care of the fact that netCDF open-flag NF_SHARE is not implemented in netCDF for CRAYs. This affects only users who have more than one program simultaneously accessing the files. These users can still share files safely if both reader and writer processes declare the shared file to be "volatile".

In the process of doing the May 19 and June 5 bugfixes and enhancements, an OpenMP critical-section deadlock related to INIT3() was created in various routines. The critical sections related to INIT3() have now been re-organized in a cleaner, more consistent fashion, avoiding this deadlock. Affected routines are:

xtract3.f
write4d.F
write3.F
wrdict3.f
wratt3.f
sync3.f
snoop3.F
read3.F
read4d.F
read3.1.F
rdatt3.f
promptmfile.f
open3.F
m3warn.f
kfwrite.F
kfread.F
kfindx.f
interp3.F
inqatt3.f
getffile.F
getefile.F
getdfile.F
flush3.f
desc3.F
ddtvar3.F
ckgeom.f
check3.F
ll2utm.f
init3.F
m3abort.F


SGI IRIX F90 Version 7.4 does not follow industry-standard practice for ensuring that BLOCK DATA INITBLK3 is correctly linked in, to initialize I/O API internal data structures. Among other things, this causes problems with a user-setLOGFILE; for more details, look here. I have submitted this problem to SGI in an error report.
--CJC Note, Spring 2004: SGI contacted me about this, and have fixed it for the 7.4.1 and later compilers.

Back to Contents

Enhancements for IRIX F90

When standard output is re-directed to a log file, SGI IRIX6 F90 Version 7.4 has anomalies in the way standard output is flushed to a file. Routines GETDBLE, GETMENU, GETNUM, GETREAL, GETSTR, M3MESG, M3MSG2, PROMPTDFILE, PROMPTFFILE, PROMTMFILE, PROMPTGRID have been enhanced to attempt to deal with these anomalies; additionally, new entry point M3FLUSH( INTEGER IUNIT ) for M3MSG2()has been added, for user-code to use for flushing its standard output, in order to similarly avoid these anomalies.

Back to Contents

Minor BugFixes/Enhancements for BUFFERED Files.

There were errors in OPEN3(), XTRACT3(), and RDBUF3() dealing with the return data-types of worker routines when these routines were called on virtual files of type

BUFFERED. There was also a format-mismatch error in a warning message in OPNKF3() These were discovered, and fixes provided by David Wong, US EPA.

Back to Contents

Minor BugFixes/Enhancements

The May 8, 2003 source-distribution tar-file was accidentally constructed using full path-names to specify the directories, rather than relative path names. This was corrected; moreover, make-target gtar was added to the Makefile so that the tar-file building process would be both correct and automated.

Function CRLF(), that had originated in SMOKE and that is used (afaik) only in that model, was modified so that it generates correct C-style line terminators for both UNIX and Windows (instead of only for Windows, as its predecessor did...) [UNIX traditionally uses the ASCII line-feed character as a line terminator; Windows uses a line-feed/carriage-return pair; Mac OS (not yet supported ;-( ) uses a carriage-return/line-feed pair.] Note that the insertion of C-style line terminators into the middle of Fortran strings for carriage control with Fortran I/O is not standard-conforming and in fact will not work on some systems (e.g., VMS, Crays).

Function XTRACT3() was modified to use time-step-check routine RDTFLAG() to bring it into conformity with the rest of the Version 2.2 I/O API input routines (which no longer have in-lined code for time step availability-checking, but use that routine instead).

Back to Contents

Enhancement to allow for OpenMP-parallel INTERP3() and/or KFOPEN() on files not yet opened
New make-target nametest checks linker-compatibility for libioapi.a and libnetcdf.a
New script nm_test.csh for testing linker-name compatibility.

Enhancement allows for more relaxed/less careful model implementation, and removes the possibilities of deadlocks on opening multiple virtual files in parallel.

make nametest assumes that libioapi.a is built, and that libnetcdf.a has been correctly placed in ${OBJDIR}, and then checks that the compilers used to generate these two libraries use compatible name-mangling systems (e.g., the same number of trailing underscores, if any).

make all will run make nametest and report the results to the screen.

nm_test.csh <file1> <file2> <name> may be used to test that the compiler systems used to build file1 and file2 generate the same linker symbol name. Note that file1 and file2 should be object, executable, or archive (".a") files. For testing against libnetcdf.a, a good symbol is nf_open; for testing against libioapi.a, a good symbol is open3 (or possibly the all-caps versions of these names).

Back to Contents

Bugfix for SETTRM(); and new spheroid-selection routines SETSPHERE(), INITSPHERES(), and SPHEREDAT(). Apr. 14, 2003

These routines allow the user to select a non-default sphere or spheroid for use with the I/O  wrappers (LAM2LL(), LL2UTM(), POL2TRM(), etc.) for GCTP(), either by means of environment variables or explicit subroutine calls.

Code for SETTERM() had an incorrect initializations for XCENT and YCENT; this would silently generate corrupt results.

Back to Contents

Bugfix for I/O API Version 2.2 rdatt3c(), wratt3c(), BUFINTX, and BUFINTXD: Feb. 18, 2003

Code for Cray versions of rdatt3c() and wratt3c() was incorrect (backwards test of whether the variable was of type CHARACTER or not, then incorrect Cray-specific treatment of file, variable, and attribute-names.

Code for BUFINTX and BUFINTXD had incorrect pointer-initializations; this would silently generate corrupt results.

Code for the Cray version of public routine SETENVVAR() and private routine lentrim() had syntax bugs, that are now fixed.

Back to Contents

Bugfix for I/O API Version 2.2 program M3TPROC: Feb. 7, 2003

Run-time memory allocation of internal work arrays was handled incorrectly when there were mixed output-lists of M3REAL and M3INT variables with M3DBLE variables. New code uses F90 "auto" work arrays, and completely avoids the problem.

Back to Contents

Bugfix for I/O API Version 2.2 program M3TPROC: Nov. 14, 2002

Back to Contents

Corrected handling of circular-buffer/RESTART files in READ3(), WRITE3(), INTERP3(), and DDTVAR3(): Aug. 29, 2002

(Circular-buffer files keep two time steps, the "even" and the "odd" time step; this allows the maintenance of what would otherwise be huge model-restart files without consuming excessive disk space. Negative values for time-step attribute TSTEP3D are used to indicate to OPEN3() that the requested file to be created is a circular-buffer file...)

Back to Contents

Bugfix in function ISDSTIME() and programs GREGDATE and JULDATE: July 9, 2002

Back to Contents

Bugfix in SETLAM(): June 7, 2002

Back to Contents

Bugfix in SETENVVAR(): May 10, 2002

Back to Contents

Version 2.2: Due May 2002

CMAQ Enhancements

Integration into the main development path of work by Jeff Young and David Wong at EPA ORD:

INTERPX

time-interpolation for a window into the grid defined by column, row, and layer subranges.

SETENV functionality

Allows the setting of environment variables from within model code. This will also wind up supporting NAMELISTs in the follow-on WRFIO implementation using I/O API Version 3

Support for T3E-style parallel I/O

Support for message-passing distributed-memory CMAQ on T3E/Fortran-90 (and similar platforms).

Minor Restructuring

There has been minor restructuring to improve the decoupling between the interface layer and the data storage/retrieval layer, in order more easily to support additional lower layer implementations (such as an MPI coupling mode).

tbd

...

Enhancements for the SMOKE emissions model

Support for input file-lists for READ3() and INTERP3():

setenv foo "LIST:name1,...,namen
setenv name1 <path>
...
setenv namen <path>


indicates that foo will be treated as a concatenation of files name1, ... namen for READ3() and INTERP3(). The files must have the same header structure, except for the time step structure, which must be compatible (identical time step, and all starting dates and times must be valid time steps for the time step sequence of the chronologically-earliest file). READ3() and INTERP3() take their data from the first listed file that has the relevant data (in case the files overlap).

This enhancement will also make it possible to get around the 2GB limits on input file sizes on various 32-bit platforms.

WRF Enhancements

Routines INQATT3(), RDATT3(), and WRATT3() support the specification of additional per-file and per-variable metadata attributes beyond the standard list, found in the FDESC COMMONs and accessed by the DESC3() routine.
Affects INCLUDE-files IODECL3.EXT and iodecl3.h.

The I/O routines has been re-coded so that it is only required that trimmed lengths (i.e., length exclusive of trailing blanks) of logical names and variable-names must not exceed NAMELEN3=16 (e.g, CHARACTER(LEN=24) FOO = 'v23456789012345   ' will be legal since its trimmed length is 15, it number of nonblank characters (which are then followed by 9 blanks to make out its total declared length of 24).

WRF eventually needs a number of additional enhancements that will break source, link, and backwards data compatibility with the current I/O API. These are being considered for an eventual I/O API Version 3, particularly:

• 32-character names
• millisecond-accurate date&time functionality (with accompanying interface changes to READ3(), WRITE3(), etc.
• New WRF-specific vertical coordinate types
• Built-in support for WRF "staggered" variables.
• Support in INTERP3() (and DDTVAR3() for double precision variables.

F9x Enhancements

• INCLUDE-files have been modified so that they are compatible with both the free and the fixed F9x source forms. Among other things, this supports new development for CMAQ, SMOKE, and WRF. Script fix_src.csh is available for those users (e.g., CMAQ) who demand nonstandard 132-column fixed F90 source format; Makefile goal fixed_src constructs nonstandard 132-column INCLUDE-files in a user-configurable directory.

• I/O API Data type PROFIL3 now uses REAL position-variables in each time step record, instead of REAL*8.

Global Climate I/O API Integration

The Global Climate I/O API is a variant of the I/O lAPI intended for use in global clmate modeling, where some modelers work in terms of a 360-day year having twelve months of exactly 30 days each. The library produced "lives" in a directory of its own, that has a _360 suffix (so is called io_360 e.g., Linux2_alpha_360 For Version 2.2 of the I/O API, we will unify the code of this version with the code for the main branch, so that one may obtain the io_360 by #defineing the preprocessor symbol IO_360 as in the Makefile.io_360 and using the usual build system.

LIST-variable Enhancements

Routines INTLIST(), REALIST(), and STRLIST() have been modified so that they accept an optional LIST: prefix for the value-list, so that one may set the list-value as in the following script-example for an INTLIST() list:

setenv MYLIST "LIST:1,24,9876,5"

Bug-fixes

LAMBERT() entries SET*() had a bug introduced when the extra map projections were added in Version 2.1.

Data type PROFIL3 had implementation errors which were corrected. (We finally had our first user of this data type, which was designed to store and retrieve rawinsonde profile data efficiently.)

I/O API Version 3 will support NAMELEN3=32 character variable and logical names, "staggered" variables, and various other things yet to be determined, in support of WRF. It will not be data- nor link-compatible (and may not be fully source-code compatible) with previous versions of the I/O API. We will maintain forwards data compatibility (so that Version 3 will read Version 1 and Version 2 generated files, but possibly not vice versa.

STATUS: under development. Version 2.2 is due out by the end of 2Q 2002; Version 3 should appear during 2Q 2003.

Back to Contents

bugfix in NEXTIME(): Jan. 15, 2002

STATUS: fixed.

Back to Contents

I/O API Version 2.1: December 5, 2001

Intel Fortran and C, and Lahey-Fujitsu Fortran compilers for Linux are now supported by the Makeinclude system.

New routines INTLIST(), REALLIST(), and STRLIST() that read in comma-delimited-list-valued environment variables.

New programs for analysis, model-coupling, grid-to-grid transformation, etc.:

• M3TPROC: compute time period aggregates (e.g., 08:00-16:00 gridded daily maxima) and write them to an output file

• M3CPLE: copy to the same grid, or interpolate to another grid a time sequence of all variables from a source file

• MTXBLEND: Use a sparse-matrix file to interpolate/transform data from an input file to the grid of a "base" file and to merge it with data from the "base" file.

• MTXBUILD: Build a sparse-matrix transform file from user-supplied ASCII coefficient inputs.

• MTXCALC: Build a grid-to-grid sparse-matrix transform file using a sub-sampling algorithm.

• MTXCPLE: Use a sparse-matrix file to interpolate a time sequence of all variables from a source file to a target file.

• PRESTERP: Interpolate from a 3-D sigma-coordinate file to a new 3-D pressure-coordinate file, using coefficients from PRES_CRO_3D from the MCPL I/O API output module for MM5.

• SELMRG2D: select multiple 2D layer/variable combinations from multiple gridded input files, and write result to merged 2D gridded output file

Back to Contents

Bug-fix in  CHECK3(), Makefile

There was also a bug in the way the Makefile was building GCTP(). The problem is that this is a very old Fortran routine from the US Geological Survey, which requires that compile-flags be set so as to SAVE all its local variables (and there turned out to be significant differences how various platforms handled the matter).

Back to Contents

More I/O API map projection support

POLGRD3 = 6

Polar (secant) stereographic

EQMGRD3: = 7

Equatorial Mercator

TRMGRD3 = 8

Transverse Mercator

Back to Contents

New I/O API routine FILCHK3

    ...
    IF ( .NOT.OPEN3( 'MYFILE', FSREAD3, 'Myprogram' ) ) THEN
C              MYFILE could not be opened for read-only
        ...
    ELSE IF ( .NOT. FILCHK3( 'MYFILE',
 &                       GRDDED3, NCOLS, NROWS, NLAYS, NTHIK ) THEN
C            Either the file type for 'MYFILE' was not GRIDDED, or
C            it had unexpected dimensions (see log for more details).
        ...
    END IF
    ...
    

Back to Contents

"Tools" Programs

• mtxblend: Use a sparse-matrix linear-operator transform-file (as from mtxbuild, below) to perform a linear grid-to-grid transform data from an input file to a new output grid (general purpose linear-operator-transform tool), and then perform a masked merge of the result with data from another "base" file on the same grid as the output from the sparse-matrix transform. Supported output file type: GRIDDED.

• mtxbuild: Construct a sparse-matrix linear-operator transform-file (as for mtxcple, below) from a user-supplied ASCII file of coefficients.

• mtxcple: Use a sparse-matrix linear-operator transform-file (as from mtxbuild, above) to perform a linear grid-to-grid transform from an input file to a new output file (general purpose linear-operator-transform tool). Supported output file types are GRIDDED, BOUNDARY, or CUSTOM.

• m3cple: copy to the same grid, or interpolate to another grid a time sequence of all variables from a source file to a target file, under the optional control of an I/O API coupling-mode "synch file".

• selmrg2d: select multiple 2D layer/variable combinations from multiple gridded input files, and write result to merged 2D gridded output file, e.g., for use preparing inputs to various "canned" 2-D graphics programs in automated web-generation, e.g., for NCSC's Numerical Air Quality Prediction project.

Back to Contents

New Release Version

With this version, the source-code-level build-process uses Makefiles rather than the previous Makeit shell-scripts. This allows much faster re-building of the library for small bug-fixes.

With this version, we will be supporting only netCDF 3.x, because of the support it provides for synchronization during model coupling. Please note that the NCAR folks accidentally incorporated a link-incompatibility between versions 2.x and 3.x of netCDF in some of the "magic numbers" used to describe modes of opening files. Note also that versions 3.4 and later use different error numbers than version 3.3.1 and earlier. The I/O API Troubleshooting Page will be updated to reflect both sets of error numbers. (As of 8/20/1999, the current released version of netCDF is 3.4, although we have been testing the new version 3.5b2 in-house recently.)

Also with this version, the default build of the library will require linking with libpvm3.a for coupling mode support. PVM 3.4 or later is required, because of the requirement for PVM mailboxes.

For some vendors, linking of serial programs will also require the explicit specification of an OpenMP run-time library as well -- on SGI's, for example, this requires an additional -lmp in the linking-step command line for serial programs.

Back to Contents

OpenMP Task-Parallel Extensions

We are making the I/O API thread-safe for OpenMP, and have been testing the result in various real-time air quality forecasting and other meteorological projects. The basic principle is that I/O API calls that read or write distinct variables and/or distinct files may be done concurrently from OpenMP programs. The biggest gains for model-users are with INTERP3 and DDTVAR3, where the call involves linear algebra computations (linear interpolation or differencing) in addition to I/O (which is performed sometimes--but not always, because of the internal double-buffering mechanisms. Note that I/O to/from disk is itself sequential in any case.)

Building a parallel program that uses an OpenMP-enabled libioapi.a generally requires compiler-specific parallelization directives in both the compile-lines and the link-lines that make your executable. For examples, ook at your machine/compiler's Makeinclude.${BIN} for variables OMPFLAGS andOMPLIBS used with this build of the library

Back to Contents

Data-Parallel Extensions

As part of the Practical Parallel Project, MCNC has developed an HPF2 implementation of an HPF I/O API module which calls the I/O API and then re-destributes gridded or boundary data according to the data distribution of its caller. Unfortunately, however, the IBM XL HPF and Portland Group HPF compilers we use support only their own extensions to HPF 1.1 at this time, so we have had to develop revised HPF I/O API modules for each which are not fully general but which do fulfill the requirements of our HPF air quality and hydrology models.

Back to Contents

Parallel Extensions for Coupled Models

This has several advantages from the model-engineering point of view:

• The same programs work unchanged both in standalone mode (reading input from files and writing output to files) and in coupled-model mode (reading and writing selected inputs or outputs to/from PVM mailboxes).
• Readers and writers do not need to know about each other in detail. In particular, any reader only needs to know that some writer will put the variables it needs into the mailbox. Writers don't care whether readers even exist or not.
• One writer can supply multiple readers without special programming (and without needing to know who they are). For example, in a coupled system with the MM5/MCIP meteorology model, the SMOKE emissions model, and the MAQSIP air quality model, MM5 produces 5 time-stepped output "virtual files", some variables of two of which are read by SMOKE and all of which are read by MAQSIP; and SMOKE produces one output "virtual files" read by MAQSIP. MAQSIP produces a "synchronization file" read by MM5/MCIP and used to keep MM5/MCIP from running ahead and exhausting available mailbox-buffer space.
• Since data is tagged by variable-name, simulation date, and time, the system is not subject to data scrambling because of implicit programming assumptions about the data ordering in the way that stream-like communications channels are.

Back to Contents

Time Series READ/WRITE

Back to Contents

Additional Supported Platforms

• Feldman-style name mangling (trailing underscores) and CHARACTER string-passing conventions or not; and
• Native length for REAL, INTEGER, and DOUBLE
• Dynamic memory allocation mechanism, whether via Cray-pointers and MALLOC or by automatic arrays

• Compaq fort and gcc on Linux Alpha.
• gcc/g77 (FLDMN=1, REAL8=0, and AUTO_ARRAYS=1)
• Win32 (Windows 3.1/win32s, Windows 95, and Windows NT/x86) using either the Digital or the Absoft Fortran-90 compilers and the Microsoft C compiler (this requires that the Fortran pre-processing be done in advance on a UNIX system; FLDMN=1, REAL8=0, and AUTO_ARRAYS=1);
• Windows NT/Alpha using the Digital compilers (also requires Fortran pre-processing n advance; FLDMN=1, REAL8=0, and AUTO_ARRAYS=1);
• <suggestions? Do you want to donate us a machine (or machine-time) for it?>

Back to Contents

MCPL: an I/O API output module for MM5

• MM5 internal variables (including all those written by OUTTAP);
• all the variables needed for input to the MAQSIP air quality model or the SMOKE emissions model;
• variables needed by the TOPLATS land surface/hydrology model;
• variables interpolated to pressure-surfaces; and
• a variety of other diagnostic or analysis variables

Back to Contents

Go to the I/O API Availability/Download page

Go to the I/O API Home Page

Go to the I/O API User Manual

Send comments to

    $Id: NEWSTUFF.html 277 2025-02-18 16:49:21Z coats $