This web page is under construction.
As a matter of fact ALL web pages are under construction whether they admit it or not.
--Found on the web page of Do Wong Chu
Best Viewed With Any BrowserThese pages are All Browser Enhancedtm. There are absolutely no <
BLINK>ing text, unnecessary and ill-considered <
TABLE>s nor <
FRAME>s, nor obnoxious <
FACEs at this <
CENTER>, nor are there in-line <
IMAGE>s that take forever to load.
Please report any problems to the authors, Carlie Coats, firstname.lastname@example.org, and John McHenry, email@example.com.
MCPL is an output module for the MM5 meteorology model which produces output via the EDSS/Models-3 I/O API (itself (C) 1992-2002 MCNC, (C) 1997-2002, 2005-2014 Carlie J. Coats, Jr., and 2003-2011 Baron Advanced Meteorological Systems).
This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License (LGPL) as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.MM5 is copyright UCAR 1998. License for MM5 itself is described at http://www.mmm.ucar.edu/mm5/mm5-home.html
This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to theFree Software Foundation, Inc.,
59 Temple Place, Suite 330,
Boston, MA 02111-1307 USA
The LGPL can also be found at URL http://www.gnu.org/copyleft/lesser.html
Back to Contents
MCPL() has a relatively trivial effect upon MM5 performance. Profiling of a 24-hour continental North America 45-kilometer MM5 run on an IBM SP showed a total CPU time of 15,000 seconds, of which 30.5 seconds was spent inside MCPL. Normal run-to-run CPU-time variability on this machine is an order of magnitude larger than the time spent in MCPL, in fact.
When the MM5 Kain-Fritsch cloud package is active, MCPL also can produce (heretofore-nonstandard) files which store the starting conditions for Kain-Fritsch convective cloud events. Data from these files are then used (for example) by air quality models to reproduce exactly the scheduling and behavior of such convective events. This package is not part of the standard release; please contact the MCPL authors for access to it.
For each MM5 domain, the user may select one or more output window
grids. For each requested output window, MCPL produces
any combination of the following twelve
time-independent and time-stepped output
files (which are
EDSS/Models-3 I/O API
selective direct-access files, rather than being sequential
Fortran binary (unformatted) files as are "normal" MM5
outputs). For the time-stepped output files, each output window
has its own starting date, starting time, and time step (with
default values taken from the starting date, starting time, and
TAPFREQ of the MM5 run).
The file list for both the "standard" and the non-standard (KF convective cloud event) MCPL() output files is:
ADDR2MM5-internal variables, and a selection of other variables used by, e.g., the TOPLATS hydrology model or the MAQSIP or Models-3 air quality model.
The MM5 MCPL/IO API Module also produces IO API-style grid description files, if these do not yet exist. These are sequential formatted (ASCII) Fortran files containong grid and coordinate system descriptions in EDSS/Models-3 IO API-compatible terms.
Module operation is as follows: First, this module gets the output
grid names from environment variable
OUTGNAMES (where grid name
means that output for the grid is turned off); then definitions for
the AQM cross and dot point grids either from
environment variables or the
I/O API GRIDDESC
file, checks the defining parameters for the AQM dot-point grid,
and computes the parameters for the boundary structure. If the
GRIDDESC file does not yet exist, it constructs it on the basis
of user-supplied grid-descriptive inputs. It also gets default
starting date, starting time, time step, and duration from MM5.
Then, for each output grid, it gets the starting date, starting time, and time step fron environment variables, constructs headers for the output files requested by the user, and opens them. The variables-list for each of these files is selected by the user from the complete set of potential output variables for that file, and communicated to the routine by way of a text file containing the requested-variables list for that file, formatted with one variable-name per line.
For each relevant time step,
the relevant variables from MM5 internal data structures into
I/O API storage
order in an internal buffer, and writes the result to the
appropriate output file. Additionally if the MCPL-KF module is
activated, it detects the inception of each Kain-Fritsch convective
cloud event, and writes the starting date, time, duration and
starting meteorology data for that cloud event to the KF-files
for all the appropriate output grids.
Back to Contents
CPPFLAGSin configure.user, in order for the compiler to process some of the following flags correctly.
For MM5V3, you should add the preprocessor definition
-DMM5_V3=1 (or its Cray or
IBM SP equivalent) to the
compile line for
mcpl.F, defining the
MM5_V3 so that
MCPL will be able
to access MM5V3 date conventions and file header data structures
instead of the previous versions.
For the Benjamin-Miller sea level pressure algorithm,
you should add the preprocessor definition
-DMCPL_BMP=1 (or equivalent) to the
MM5/mcpl/Makefile compile line for
For the Models-3 air quality extensions, you should
add the preprocessor definition
-DMCPL_ADDMDL3=1 (or equivalent) to
MM5/mcpl/Makefile compile line for
mcpl.F, as well as adding the so-called "EPA
mods" to the relevant land surface and radiation packages.
For OpenMP task-parallel
MCPL operation (probably not necessary unless you're running
either on a system with lots of processors or unless you're generating
output every advection step), you should add the preprocessor
equivalent) to the
MM5/mcpl/Makefile compile line for
NOTE: current versions of IBM's
xlf have an
incomplete OpenMP implementation that does not recognize the OpenMP
SHARED directive for objects which are COMMONs, so this
task-parallel operation should not be turned on for such systems.
(As of 12/20/99, the upcoming next release of IBM's compiler system
promises such support.)
MCPL_GRIDwhich should be called after coarse-grid initialization in the MM5 main program, and the
MCPL_OUT(NEST)output entry, which is most easily called at the end of
SOLVE3(), (allowing high-frequency output if desired, e.g., every advection step). MM5's
driver.Fshould be terminated by a call to the I/O API shutdown routine
M3EXIT()instead of a
STOPstatement, in order to flush all netCDF file headers to disk, etc.:
... C STOP 99999 MM5.237 CALL M3EXIT('MM5', 0, 0, 'Normal completion', 0 ) ...Note that if you want to analyze the output while MM5 is still running, your script must declare the output files to be volatile:
setenv <logical name> "<physical path spec> -v"
MCPL() is called through two or three ENTRY points,
according to whether Kain-Fritsch convective cloud-event data is
MCPL_OUT( NEST )
& NEST, II, JJ, ! what cell?
& P300KF, TCKF, RKF, ICEKF, TADVEC, NIC, CLDTP, ! scalar output variables
& TKF0, QKF0, UKF0, VKF0, WAV0, PKF0,
& DZQKF, DPKF ) ! vertical column output variables
MDATEfrom common block
/VARIOUS/to determine the default starting date and time,
/PARAM2/to determine the default output time step, and
/VARIOUS/to determine the current simulation date and time. Note that it uses environment variable
MCPL_START_DATE, if available, or the current wall clock (if not) to determine the century component for the EDSS/Models-3 dates and times it uses internally. Consequently, MCPL itself is Y2K-compliant if used with user-specified
MCPL_START_DATE, although MM5v2 itself is not. Because of this, there will be problems with default dates when MM5 is used for historical-case runs not for the current century, a difficulty which may be avoided by always explicitly specifying
MCPL()uses a variety of worker routines to extract the requested variables from MM5v2 common blocks (particularly
/ADDR1/for 3-D variables, and
/ADDR2/for 2-D variables), decouple them (MM5 keeps most 3-D variables in the form
PSTAR * <variable>) and perform units conversions where appropriate, and re-order them into standard EDSS/Models-3 I/O API column-row-layer subscript order.
How to modify MCPL to output additional variables using the existing worker routines (or how to create new worker routines in order to output additional derived variables) is described in the section, "Adding New Output Variables to MCPL()"
Back to Contents
DOES NOT WORK WITH DISTRIBUTED-MEMORY
We have looked at the software engineering task of making MCPL work with the distributed-memory version of MM5, but the task of doing a clean implementation of the necessary "gathers" of MM5 data from all of the machines where the data has been distributed into buffers from which MCPL can perform its calculations and write out the results looks to be exceedingly complex. If you would like this capability, please be prepared to come to us with at least two man-years' funding.
OpenMP on SGI: When running with OpenMP parallelism under SGI IRIX6, the default per-thread stack and memory limits are inappropriately tiny. This difficulty may be controlled by setting appropriate values for environment variables that control this behavior. The following values work for us:
MP_SLAVE_STACKSIZE=32000000 >or other reasonable limit< MP_STACK_OVERFLOW=ON
MM5V2, Historical Simulations, and Y2K-Related
Issues If your
script sets all of the date-related environment variables for
MCPL() explicitly, you should not encounter any Y2K-related
problems. However, with MM5V2 (which is itself not Y2K-compliant)
if you use default MCPL starting dates (i.e., you have not done
<date-list, ordered by output window>
), MCPL attempts to fill in the century field by using the
century value obtained from the current real-time wall
clock/calendar. With MM5V2, the default behavior leads
to errors when the simulation century and the wall-clock century
are not the same. For example, if on March 4, 2000,
one is doing a historical MM5V2 simulation for July 17, 1995,
the disparity between the run-time wall clock century and the
simulation century leads MCPL in MM5V2 to think that the requested
output dates will be for year 2095 (= century 2000 from the
wall-clock + year 95 from MM5V2's
For historical cases before 1 AD, there may potentially be problems; the system has not been tested thoroughly for these, although I believe it will behave correctly (Fortran integer division involving negative numbers "does the right thing", unlike C division which is undefined and allowed by the C standard to vary from vendor to vendor).
No MPI/MPP. MCPL follows a
single-output-file model rather than a distributed-I/O model. It
therefore requires that the variables being output are already
gathered into the global arrays pointed to by the pointers in MM5
COMMONs, and that (for KF cloud-event data) the
MCPL_KF sub-module be able to receive all of the KF event data and
serialize it by means of OpenMP critical sections. Note that
since MCPL was first developed for use in coupling MM5 with other
environmental models, its structure is in part dictated by the
requirements of those models. These other models may not be assumed
to have exactly the same data decomposition as MM5 (or even to live
on the same machine(s); it is perfectly reasonable to build a
real-time forecast system coupling an MM5 running in RTP,NC with a
hydrology model running at GA Tech and an air quality model running
No Moving Nests. This is an EDSS/Models-3 I/O API requirement, since each file's grid definition is active for the entire life of the file. Workaround: each domain instantiation should be treated as a distinct domain in MM5, and have its own set of output files, which have starting dates and times matching the time at which the domain is activated. Note that MCPL already turns off output for a domain at the time MM5 turns the domain off.
Now completed: Early versions may not have been tested for all potential output variables, or all output files. Variables needed by MAQSIP, SMOKE, and TOPLATS have been validated. Pressure-level files are currently undergoing test and validation.
If MCPL_KF is used, and MM5 is run in parallel, you must use the thread-safe version of the EDSS/Models-3 I/O API library. This is also true if you use the (currently-being-tested) multithreaded version of MCPL().
Back to Contents
NOTEFor Internet Explorer users:
IE seems to be brain-damaged in some fashion, and refuses to accept an FTP-URL to the current working directory. Try using an alternate browser such as Netscape, Opera, Mozilla, or Konqueror for downloads from this page.
- The version referenced below is current as of 10/08/2002 and includes the following:
- support for three different versions of relative humidity:
- RHWI has liquid-water based RH above CTOK, ice-based RH below CTOK
- RHLW has liquid-water based RH only
- RHKF uses liquid-water based RH above TTFRZ, ice-based RH below TBFRZ, and linear mixture of the two in between (as used in KF cloud parameterization and elsewhere)
PC3now have both relative vorticity variable RVOR and absolute vorticity variable AVOR, instead of just VOR (containing relative vorticity), produced by routines
- Support for up throught 16 output domains.
- If MRF PBL scheme used in MM5, add new
MC2variables QV2, TA2, UX10, VX10, WSPD10
- New entry
MCPL_ATIMEtakes XTIME-like time (minutes since start of the run) as an argument, and returns JDATE:JTIME according to Models-3 conventions.
- new subroutine
MM2IObut rounds to
INTEGER; more variables and validity checks for MM5 physics options with
- IMPHYS > 2,
- ISOIL > 1,
- Pleim-Xiu LSM/PB
- If CCM2 radiation scheme run in MM5, new output
- Support for EPA CMAQ air quality model (many new variables).
- support for additional MM5 map projection types:
- polar stereographic,
- equatorial Mercator
- The version current as of 1/22/2001 includes minor bug-fixes of the code related to grid descriptions, an improved relative humidity calculation which does a linear transition between ice-based relative humidity formulation below 248.16 K and a water-based humidity formulation above 268.16nbsp;K.
- The version current as of 3/21/2000 includes minor bug-fixes, provision for using the Benjamin-Miller sea level pressure algorithm instead of the INTERP algorithm, and provision for putting out wind components (UX,VX) interpolated to cross-points to files MC2 and MC3.
- The version current as of 10/18/1999 includes minor bug-fixes, as well as adaptations for MM5V3, which may be activated by adding a CPP-definition for symbol MM5_V3 to the compilation-flags. As of the current date, we believe the MM5V3 mods work correctly (they are relatively simple, and involve changes to treatment of date-&-time variables, and grid definitions taken from the new file header structure), but they have not been tested, as we at NCSC are not yet running MM5V3 in production mode. With luck, that will happen during January 2000.
- The version current as of 7/29/1999 comments out
IMPLICIT NONEin mcpl.F and includes re-fixes for OpenMP parallelism and for MAQSIP cloud-type information.
- Versions prior to 7/26/1999 still had problems when compiled with OpenMP parallelism turned on. There was also a bug in file-header description of cloud parameterization, when MCPL was run with more than one output window per MM5-domain.
- Versions prior to 7/21/1999 had problems when compiled with OpenMP parallelism turned on.
- This version includes a current snapshot of the EDSS/Models3 I/O API. A new release is expected in September, 1999.
- The initial version is for MM5v2.x. We will re-do for MM5v3 as soon as we can.
- If you get error messages about undeclared variables: remove the
IMPLICIT NONEat the top of mcpl.F: I have had so much trouble dealing with
IMPLICITsource codes that I refuse to deal with them any more. I've written versions of the MM5 include-files which declare everything they define and I use them for my own private development. Then I sometimes forget to delete the
IMPLICIT NONEwhen I prepare a new release.
You may download MCPL as a gzipped tar file from here. This package will include the following:
In addition to the MCPL source, you will also need the EDSS/Models3 I/O API, which is contained downloadable tar-file described above, as well as from here, and the netCDF library from UCAR, available as a UNIX-compressed tar file from the netCDF home page.
Use of I/O coupling mode with MCPL also requires version 3.4 or later of the PVM library, from Oak Ridge National Laboratories available from the PVM home page .
Back to Contents
The authors would also like to thank Dr. Nelson Seaman, Annette Gibbs-Lario, and the Department of Meteorology at the Pennsylvania State University for all their work beta-testing this software during the course of the joint MCNC/Penn State real-time air quality forecasting project, and for all their help and support in developing MCPL.
Back to Contents
Back to the Practical Parallel Computing Project
Send comments to
Carlie J. Coats, Jr.