This document is supposed to help users setup the DIRSIG model to integrate with MODTRAN.
Because DIRSIG and MODTRAN integration is a "black box" to the user, it seems like magic when it works but is very frustrating and obfuscated when it does not. This document is designed to help users in the latter case. This document will guide the user through a step by step process to understanding how DIRSIG and MODTRAN interact and how common problems with that interaction can be resolved.
MODTRAN Software
Versions
At this time, DIRSIG has been tested to work with the following versions of MODTRAN:
-
MODTRAN4 (at least MODTRAN 4v3r1)
-
This is one of the last releases of the MODTRAN4 software. We strongly recommend not using any version prior to this release because of numerous bugs in the software, important improvements to the software, etc. Note, older versions of MODTRAN4 have known issues that can impact how DIRSIG interfaces with it.
-
-
MODTRAN5 (at least MODTRAN 5.2.0)
-
These are the most common releases of MODTRAN5 we encounter in the wild at the time of this writing. Both versions have been used with DIRSIG on all of the supported platforms.
-
-
MODTRAN6 (at least MODTRAN 6.0.2.x)
-
This is the latest version of MODTRAN and it features some changes to the interface. Note, older versions of MODTRAN6 have a bug in the
tape5
support that results in double shadows because it returns the irradiance for the sun rather than the moon.
-
Although MODTRAN 4v3r1 works with DIRSIG, it is strongly recommended that the user employ MODTRAN5 or MODTRAN6 with DIRSIG. For more information, visit http://www.modtran.org. |
These versions of MODTRAN have been found to work on Windows, MacOS and UNIX platforms.
MODTRAN6 vs. Earlier Versions
Prior to MODTRAN6, MODTRAN did not come with a robust, standardized, well documented application programming interface (API) and software like DIRSIG had to integrate with MODTRAN through MODTRAN’s standard file interfaces. For a variety of reasons, this coupling was problematic. For example, different versions of MODTRAN behaved differently when an error occurred or reported that same error differently. In other cases, the format of various input and output files changed and software that reads and writes those files must be updated.
Although the MODTRAN input and output files are documented in the MODTRAN manual, this mechanism falls short at being a robust API for a variety of reasons. The biggest limitation is the lack of a consistent and formalized error reporting mechanism that can be easily interpreted by the calling program. Recent versions of MODTRAN are improving this, but formal APIs have a consistent behavior when errors are produced. |
MODTRAN6 (the latest version of MODTRAN) marks a major milestone in the continued development of one of the most popular and trusted codes for simulating radiative transfer in the atmosphere. In addition to important science related advancements, this latest code also includes significant improvements to the general usability of the software. This includes a new graphical user interface (GUI) and the introduction of a formal application programmer interface (API), which let’s codes like DIRSIG interact with MODTRAN in a far more robust way than previous versions allowed.
Band Models and Spectral Resolution
MODTRAN uses a "band model" approach to store the spectral absorption lines of the native gases in the atmosphere. That means there is a database containing the absorption lines rendered to a specific spectral resolution. For example, MODTRAN6 comes with 4 band model files supporting 4 different spectral resolutions:
-
p1_2013
is the 0.1 cm-1 band model database -
01_2013
is the 1 cm-1 band model database (default) -
05_2013
is the 5 cm-1 band model database -
15_2013
is the 15 cm-1 band model database
In MODTRAN6 (and previously in MODTRAN5), the default
band model is the 1 wavenumber resolution file (i.e.,
01_2013 ). In order to use the 0.1 wavenumber resolution
file the user must specifically request it. This is
achieved by setting the LBMNAM flag to true (i.e., T
or t ) and setting the BMNAM variable to the name of
the band model file to use (i.e., p1_2013 ) in the input
tape5 or JSON file.
|
Many users are more familiar with wavelength units (microns, nanometers, etc.) and find working with wavenumbers awkward. It is important to recognize that there is an inverse relationship between wavenumbers and wavelengths. To compute the equivalent wavenumber for a 0.5 microns, the math is:
\begin{equation*} \frac{10,000}{0.5~\mathrm{microns}} = 20,000 ~\mathrm{cm^{-1}} \end{equation*}
Converting a resolution is not as straight forward since the equivalent wavenumber interval between the converted wavelengths must be computed. For example, a 0.02 micron interval centered at 0.50 microns (0.49 → 0.51 microns) converts to:
\begin{equation*} \frac{10,000}{0.49~\mathrm{microns}} - \frac{10,000}{0.51~\mathrm{microns}} = 20408.163 ~\mathrm{cm^{-1}} - 19607.843 ~\mathrm{cm^{-1}} \approx 800 ~\mathrm{cm^{-1}} \end{equation*}
However, the same 0.02 micron interval centered at 5.0 microns (4.99 → 5.01 microns) converts to:
\begin{equation*} \frac{10000}{4.99~\mathrm{microns}} - \frac{10000}{5.01~\mathrm{microns}} = 2004.008 ~\mathrm{cm^{-1}} - 1996.007 ~\mathrm{cm^{-1}} \approx 8 ~\mathrm{cm^{-1}} \end{equation*}
So the same wavelength interval has a dramatically different equivalent wavenumber interval depending on the wavelength. To make it easier, we provide the following table that includes the equivalent resolution limits for different versions of MODTRAN at different wavelengths:
MODTRAN |
Band Model |
Minimum Sampling Resolution (in microns) |
||||
Version |
Max Resolution |
@ 0.7 um |
@ 1.0 um |
@ 2.5 um |
@ 5.0 um |
@ 14 um |
MODTRAN4 |
1.0 cm-1 |
0.000050 |
0.000100 |
0.0006250 |
0.0025000 |
0.0196000 |
MODTRAN5 |
0.1 cm-1 |
0.000005 |
0.000010 |
0.0000625 |
0.0002500 |
0.0019600 |
MODTRAN6 |
0.1 cm-1 |
0.000005 |
0.000010 |
0.0000625 |
0.0002500 |
0.0019600 |
Testing the MODTRAN Installation
The MODTRAN installation folder contains a folder named TEST
, which
contains 50+ test scenarios for use with MODTRAN. It is best to test
the correct operation of your MODTRAN installation by attempting to
run some of those test cases. Specifically, verifying that those
test cases can be run in a folder outside the MODTRAN installation
folder is important. Consider the following test procedure:
-
Make an empty folder to run the test case in.
-
Copy one of the test case "tape5" files (the file with the
.tp5
file extension) into your empty folder. Change the name of the copied file to be simplytape5
. -
Start up a command-line session (
CMD
,Terminal.app
,xterm
, or whatever is appropriate for your system). -
Change folders (for example, using the
cd
command) to your test folder containing thetape5
file you just copied. -
Attempt to run the MODTRAN executable from the command line using the full path to the installation folder.
-
If the program runs successfully, you should have a
tape6
file that ends without error messages (the last line usually contains a copy of theCARD 5
line from the inputtape5
file).
If one or more of the pre-packaged MODTRAN test cases cannot be run outside of the installation folder as was described above, attempt to resolve the issue before attempting to have DIRSIG utilize MODTRAN. |
If MODTRAN does not run correctly, consider the following steps to understand the nature of the problem:
- Are any MODTRAN output files present?
-
The
tape5
file is the input file, but when MODTRAN is run it should generate a set of output files includingtape6
andtape7
. If you do not see either of these files, then chances are MODTRAN could not be started.-
Are your preferences pointing to the right MODTRAN executable?
-
Can the MODTRAN executable be run from the simulation folder? Sometimes shared/dynamic libraries are not correctly setup for MODTRAN to be run outside the installation folder. See specific notes earlier in this document about this common problem with the executables distributed for the MacOS platform.
-
- Is there a
tape6
file but not atape7
ortape7.scn
file? -
If this is the case, then MODTRAN was successfully started, but could not run correctly with the supplied input.
-
Look at the end of the
tape6
file for error messages -
Look for a
warnings.txt
file for error messages
-
The most common message that a user will receive is that DIRSIG
is that the tape7.scn
(or .7scn
) file could not be read.
Setting up DIRSIG to use MODTRAN
The setup of DIRSIG to use MODTRAN is described in the MODTRAN Setup section of the DIRSIG Installation Guide. The following sections outline some specific aspects of using MODTRAN5 and MODTRAN6 with DIRSIG.
Using MODTRAN6 with DIRSIG
The major development in MODTRAN6 is a shift from the old "tape5" style inputs to a new JSON (JavaScript Object Notation) style input. In addition to improving the general readability of the input, the JSON document format is much easier to read in, modify and write back out. The old tape5 format required software like DIRSIG to know a great deal about the format of the input file in order to correctly read in, modify and write back out a set of MODTRAN inputs. If a new variable was added to an existing input "card", a new input "card" added, etc. then the DIRSIG code needed to be modified to know to read it in and write it back out, even if it never modified those variables. The JSON document style input now employed by MODTRAN6 can be read into a generic structure, the variables of interest can be modified in a straight-forward way, and the document written back out without needing to code extensive logic that understands the meaning and impact of every variable.
Using MODTRAN6 with the DIRSIG5-era atm_builder and fourcurve_builder tools is described in the Atmosphere Model Backends Manual.
Using MODTRAN5 with DIRSIG
This first section discusses the typical installation layout of MODTRAN5. Although the MODTRAN software comes with installation documentation, we will highlight some important aspects here.
Installation Layout
A typical MODTRAN installation is composed of a single file hierarchy that contains the MODTRAN executable, supporting databases, test cases and documentation. The screenshot below shows the typical layout of files and folders in a MODTRAN5 installation (this organization is common to Windows, MacOS and UNIX).
Documentation
The MODTRAN "manual" that focuses on the input file format and how to
enable specific features is usually included as a PDF document (shown
above as MODTRAN5.2.2.pdf
).
Another helpful document is the Frequently Asked Questions (FAQ)
document, which includes lots of useful information about the model
(shown above as FAQmay2012.pdf
).
Databases
The DATA
folder contains database files used by MODTRAN at run-time.
This includes the band model files (databases of the spectral absorption
for atmospheric gases), correlated-K coefficient files and background
reflectance spectrum. If you are compiling MODTRAN from source, then
some of these databases need to be generated. This will be discussed
in another part of this document.
Source Code
The src
file contains the source code to the MODTRAN software. On
some target platforms the user is expected to compiled MODTRAN from
the source code (for example, UNIX). On others, the software comes
pre-compiled for the target platform.
Test Cases
The TEST
folder contains typically 50+ test cases that can be used
by the end-user as examples for specific scenarios, or to verify
correct operation of the model.
Pre-Compiled versus Compile Your Own
As of MODTRAN 5.2.0, both the Windows and MacOS releases had the MODTRAN program, band model files, etc. pre-packaged and ready to use:
-
The Windows release has both 32-bit and 64-bit versions
-
As a reminder, the 32-bit version can be used on a 64-bit system, but not the other way around.
-
-
The MacOS release has only a 32-bit Intel version.
-
The 32-bit version runs fine on modern Macs, which are almost exclusively 64-bit.
-
If you are still using a PowerPC (PPC), then you will need to follow the instructions to compile the software from source, which will require a FORTRAN compiler.
-
On the UNIX platform, the software must be compiled from the source code. Follow the instructions included with MODTRAN on how to perform this task.
Console vs. Window Executables on Windows
A typical Windows installation comes with two executables. One is
for use from a non-interactive console session (see Mod5.2.2cons.exe
in the example file listing above), and the other from interactive
session (see Mod5.2.2qwin.exe
in the example file listing above).
Make sure that DIRSIG is using the "console" executable (the one that
contains cons
in the name). More information about the difference
between these two applications can be found in the MODTRAN documentation.
Dynamic Libraries on MacOS
The MacOS target platform is one where the MODTRAN distribution
comes pre-compiled for the end-user. However, it is pre-compiled
using the gfortran
compiler, which introduces dependencies for
two external, dynamic libraries (they are libgcc_s.1.dylib
and
libgfortran.3.dylib
). These libraries are included with MODTRAN
at the top of the installation folder (at the same level as the
MODTRAN executable). A common problem with the MacOS distribution
is that the MODTRAN executable cannot be run from a folder other
than the installation folder. This is because the MODTRAN executable
does not know where to find these dynamic library files when execution
is started from another folder. This creates a problem for 3rd
party software like DIRSIG that always starts MODTRAN from a folder
other than the installation folder.
To test if this is a problem with your MODTRAN installation, simply
try to run the MODTRAN executable from your home folder via the
command line (for example, from Terminal.app
). If your copy of
MODTRAN is installed in /Applications/Mod5.2.2
and the name
of the MODTRAN5 executable is Mod90_5.2.2.exe
, then try to run
the executable directly with the following syntax:
$ cd $HOME
$ /Applications/Mod5.2.2/Mod90_5.2.2.exe
If you get error messages about missing dynamic libraries for
libgcc
or libgfortran
from the runtime loader (dyld
), then
your installation has this problem. For example:
$ /Applications/Mod5.2.2/Mod90_5.2.2.exe
dyld: Library not loaded: libgfortran.3.dylib
Referenced from: /Applications/Mod5.2.2/Mod90_5.2.2.exe
Reason: image not found
Trace/BPT trap: 5
The solution to this problem is to wrap the MODTRAN executable with
a shell script that tells the executable the absolute path back to
the dynamic library files. This shell script simply needs to add the
correct folder path to the DYLD_LIBRARY_PATH
variable. An example
script is shown below:
#!/bin/sh
# define where MODTRAN lives and the executable name in that location
export MODTRAN_FOLDER=/Applications/Mod5.2.2
export MODTRAN_EXE=Mod90_5.2.2.exe
# check if the dynamic library path variable exists already
if [ -n $DYLD_LIBRARY_PATH ]
then
# make a new variable with the MODTRAN folder in it
export DYLD_LIBRARY_PATH=${MODTRAN_FOLDER}
else
# add the MODTRAN folder to the existing variable
export DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:${MODTRAN_FOLDER}
fi
# run the MODTRAN executable with this environment
${MODTRAN_FOLDER}/${MODTRAN_EXE}
The user made need to modify the MODTRAN_FOLDER and
MODTRAN_EXE variables to reflect the installation folder
and MODTRAN executable on their system.
|
A copy of this script that has been customized with the specifics
of your MODTRAN version and installation location can be placed in
the MODTRAN installation folder with an appropriate name (for example,
run_modtran.sh
). The user’s DIRSIG preferences should then point
to this script rather than the bare MODTRAN executable.
Populating the DATA Folder
The MODTRAN program relies on a set of band model, correlated-K coefficient,
etc. databases to run. These files are stored in the DATA
folder within
the MODTRAN installation folder.
On the Windows and Mac platform, these databases come pre-built and ready for use. If you are deploying MODTRAN on the UNIX platform, or you have compiled MODTRAN from the source code for any reason, then you will need to create the binary database files yourself. Follow the instructions included with MODTRAN on the correct process to perform this task.
Known Issues
Below are a few noted issues related to having DIRSIG and MODTRAN correctly interact on a variety of operating systems. Some of these issues have been resolved in later versions of either DIRSIG or MODTRAN, but they are described here for reference.
Installation Path Length
The MODTRAN input files generated by DIRSIG will include the path to the
MODTRAN DATA
folder containing the band model, correlated-k, etc.
database files. The field for this folder path in MODTRAN is limited
to 80 characters.
Do not install MODTRAN in a folder that results in an
absolute path to the DATA folder that is greater than
80 characters.
|
Installation Path under Windows
If you are using DIRSIG and/or MODTRAN on a Windows machine that employs
the "Universal Naming Convention" (UNC) or what is sometimes called
"Mapped Network Drives", a variety of problems have been reported.
A UNC path in Windows is when a shortcut to a shared drive is
created that does not contain a drive letter. For example, rather than a
user’s home folder being Z:\joeuser
, the mapped name might be
\\home\joeuser
.
Do not use a mapped network drive to store simulation inputs. Consider copying files to a local disk or use the direct path to the files (one that includes the Windows drive letter). |