Index for cjcoats.github.io

Optimizing Environmental Models for Microprocessor Based Systems — The Easy Stuff

Computational efficiency is environmental models is affected greatly by two things: (1) that computer memory is far slower than the processors, and filtered through a "cache" system; and (2) that the processors are deeply pipelined and superscalar, attempting to run many instructions (in various stages of completion) at one time.

This started as a presentation for US EPA's Office of Research and Development in 2002 when the author was at MCNC—North Carolina Supercomputing Center, and has been updated from time to time since. It is amazing just how well the lessons of that time have held up—the only thing is that the tendencies noted in 2002 have become even stronger over time—main memory (and even caches) is even slower relative to processor-cores, for example

The Models-3 I/O API

The Models-3/EDSS Input/Output Applications Programming Interface (I/O API) provides the environmental model developer with an easy-to-learn, easy-to-use programming library for data storage and access, available from both Fortran and C. The I/O API provides a variety of data structure types for organizing the data, and a set of access routines which offer selective direct access to the data in terms meaningful to the modeler, as well as robust error checking and reporting "behind the scenes".

For example,

Read layer 1 of variable 'OZONE' from 'CONCFILE' for 5:00 PM GMT on July 19, 2018 and put the result into array A.

is the direct English translation of a typical I/O API READ3() call.

I/O API Home Page

I/O API Programmers Manual

PAVE-3.0

The Package for Analysis and Visualization of Environmental data (PAVE) is a flexible tool to visualize multivariate gridded environmental datasets. PAVE Version 3.0 is a new release, restructured as a stand-alone application, with major optimizations (after the fashion of Optimizing Environmental Models, above), with Linux "medium memory-model support", so that file size, etc., is essentially limited by available RAM and inherent netCDF limitations,

PAVE Users Guide

PAVE Frequently Asked Questions

SMOKE-m3utilio

This project contains a GitHub Oct. 28, 2023 version of SMOKE (.0.f), a M3UTILIOized, I/O API 3.2-ized (.f) of SMOKE, an findent (.1.f90) free-source-format reference version of SMOKE constructed from the .f, and a cleaned-up .f90 version of SMOKE.

In several places, copyright, attribution, and license notices that had been stripped from the code have been replaced.

A number of additional changes have been made, where error checking was missing, numerics were handled badly, loop-nest orders were inappropriate, and potential bugs and questionable coding-practices were fixed. The code was brought up to I/O API-3.2 standards.

Finally, there is a section describing what further (and larger-scale) changes ought to be made.

This work has been done entirely on my own time. I had no external support for this work whatsoever.
-- Carlie J. Coats, Jr., Ph.D.

M3UTILIO/F90-ized SMOKE

CMAQ singularity container system

This has a virtualized machine based on CentOS-7/x86_64 in which are installed the CMAQ model (see https://www.cmascenter.org/cmaq/) SMOKE (see https://www.cmascenter.org/smoke/), VERDI (see https://www.cmascenter.org/verdi/), The Models-3 I/O API and associated M3Tools (see https://cjcoats.github.io/ioapi/index.html) and various other modeling related programs such as PAVE, ncview, panoply, GrADS, NCAR graphics and gnuplot, so as to create a full environmental modeling set-up in one virtual machine, together with a set of scripts that allow one easily to run the models (etc.) as desired.

Singularity CMAQ Container Document Index

Singularity CMAQ Container Users Manual