The make_adb tool is key component of the DIRSIG4 "Classic" atmosphere model (aka "ClassicAtm") and is supported in DIRSIG5 by the BasicAtmosphere plugin). This tool is responsible for creating the atmosopheric database (ADB) from a series of MODTRAN simulations.

Approach

Requirement Collection

When the make_adb tool runs, it is executing MODTRAN simulations that are specific to a given simulation. The specifics of the simulation that are captured by make_adb in the ADB file include:

  • The geographic location of the scene in the simulation,

  • The mean date and time of the simulation,

  • The average altitude of the sensor platform in the simulation,

  • The accumulated field-of-view (FOV) of the sensors in the simulation, and

  • The wavelengths of all the sensor focal planes in the simulation.

The list above includes the words "mean", "average" and "accumulated" because the resulting ADB file contains a single set of tables for the simulation. The general assumption of the ClassicAtm model is that the atmosphere is temporally frozen during the simulation. When the simulation is a single-frame at a point in time, this assumption is easily satisfied. However, when the simulation has a non-zero length collection window and involves a moving platform and/or a scanning sensor(s), then this assumption needs to be accounted for. In these non-instantaneous scenarios, the database is created to span the extents of the simulation in the following ways:

  • The date/time for the database is the mid-point of the collection time window.

  • The altitude for the database is the average altitude of the sensor during the collection time window.

  • The FOV for the database is the accumulated FOV formed by the motion of the platform combined with the platform-relative scanning of the sensor(s) during the collection time window.

Important This frozen time assumption is enforced by a hard-coded time window that spans +/- 15 minutes from the database. If your simulation tasks span more than 30 mins, then you will need to break the simulation into multiple simulations. This is one of the key advantages of the NewAtmosphere toolchain.

During the initialization of the make_adb tool, the tool is gathering the information (date, time, altitude, FOV, wavelengths, etc.) and generating the list of MODTRAN simulations that will need to be run to construct the database required for the simulation.

MODTRAN Interaction

The make_adb tool runs MODTRAN by repeatedly modifying an input MODTRAN "tape5" file, running MODTRAN with that modified input, reading the MODTRAN output and storing it in the atmospheric database. The user-supplied MODTRAN input "tape5" file is referred to as the "template" file since it provides the basis for all the MODTRAN runs executed by the tool. The template "tape5" file is primarily used for it’s card #1 and #2 data (and all the subcards), which describe the optical properties of the atmosphere. In general, the make_adb utility updates the card #3 and #4 data, which contains specifics about the type of path to be modeled, the geometry of that path, the wavelengths, etc. In general, if a tape5 variable describes what the atmosphere looks like, make_adb leaves it alone. If a tape5 variable describes something specific to the DIRSIG simulation, make_adb modifies it. The table below attempts to document which variables are modified. The tool will run 100s of MODTRAN simulations, so different variables are modified at different times depending on the type of MODTRAN simulation being performed.

Note The -save_tapes option will preserve all the MODTRAN input files generated by make_adb and output files generated by MODTRAN if the user wants to verify the modifications to the input "template".
Table 1. MODTRAN "tape5" variables modified by make_adb.
Card Variable Description

Card 1

IEMSCT

Updated as needed to use source irradiance, scattered radiance, etc. modes.

Card 1

ITYPE

Updated as needed to specify the path type.

Card 1A

CDTDIR

Updated to specify the path to the MODTRAN DATA directory.

Card 1A4

DATDIR

Updated to specify the path to the MODTRAN DATA directory.

Card 2

GNDALT

Updated to reflect the ground altitude of the scene.

Card 3

H1

Updated to reflect the path starting altitude.

Card 3

H2

Updated to reflect the path ending altitude.

Card 3

ANGLE

Updated to reflect the path angle.

Card 3

RANGE

Updated to reflect the path range.

Card 3ALT

H1

Updated to reflect the path starting altitude.

Card 3ALT

H2

Updated to reflect the path ending altitude.

Card 3ALT

ANGLE

Updated to reflect the path angle.

Card 3ALT

IDAY

Updated to the day of year (impacts Earth-Sun distance → solar irradiance).

Card 3ALT

RO

Updated to use the default Earth radius (0).

Card 3ALT

ISOURC

Updated to reflect the which exo-source to model (Sun or Moon).

Card 3ALT

ANGLEM

Updated to reflect phase angle of the Moon (from DIRSIG ephemeris model).

Card 3A1

IPARM

Updated to reflect the exo-source geometry method.

Card 3A1

IDAY

Updated to set the day of year (impacts Earth-Sun distance).

Card 3A1

ISOURC

Updated to reflect the which exo-source to model (Sun or Moon).

Card 3A2

PARM[1-4]

Updated to reflect the Sun/Moon position (from DIRSIG ephemeris model).

Card 3A2

PSIPO

Updated to reflect azimuth angle of the path.

Card 3A2

ANGLEM

Updated to reflect phase angle of the Moon (from DIRSIG ephemeris model).

Card 4

V1

Updated to reflect the minimum wavelength (or frequency) of the bandpass.

Card 4

V2

Updated to reflect the maximum wavelength (or frequency) of the bandpass.

Card 4

DV

Updated to reflect the delta wavelength (or frequency) of the bandpass.

Card 4

FWHM

Updated to reflect the delta wavelength (or frequency) of the bandpass.

Card 4

FLAGS[1-8]

Updated to specify the slit shape, disable extra outputs, etc.

Note CARD 3ALT contains redundant variables with CARD 3A1 and CARD 3A2 but is used for directly transmitted irradiance runs versus scattered radiance runs.

MODTRAN Configuration

The setup of DIRSIG to use MODTRAN is described in the MODTRAN Setup section of the DIRSIG Installation Guide. In addition, the MODTRAN Integration Guide contains useful details about using specific versions of MODTRAN with DIRSIG.

The primary configuration task for the tool is the location of the MODTRAN installation to be used by the tool. The configuration captures three elements:

  1. The path to the MODTRAN executable,

  2. The path to the MODTRAN DATA directory, and

  3. The spectral resolution limit for this version of MODTRAN (1.0 wavenumbers for MODTRAN4 and 0.1 for MODTRAN5 and later).

There are two options for specifying these configuration elements.

Older Configuration File Method

In DIRSIG3, the MODTRAN installation specifics were captured in a configuration file that was contained in the software installation (e.g., $DIRSIG_HOME/config/make_adb). By default, the versions of make_adb included in DIRSIG3 would load this file to get information about the MODTRAN installation to be used. An example of this file is shown below:

An example DIRSIG3 era make_adb configuration file.
#
# This is the make_adb configuration file.  This configuration file
# will most likely NOT work for your facility without modifications
# to reflect the specifics of you MODTRAN installation.
#
MAKE_ADB_CFG = 1.0

MODTRAN {
    MODTRAN_EXE = /usr/local/Mod4v3r1/Mod4v3r1.exe
    MODTRAN_DATA_DIR = /usr/local/Mod4v3r1/DATA
    MODTRAN_MIN_DELTA = 1.0
}

BACKGROUND_RADIANCE_FILE = /usr/local/dirsig/lib/data/misc/star_light.dat

In DIRISG4, this mechanism was replaced with a newer mechanism (see below). However, the DIRISG4 version of make_adb still supported a command-line option (see the -use_config option) to specify the path to one of these DIRSIG3 era configuration files. Although not encouraged, this option can still be used to simplify the deployment or execution of make_adb in complex compute environments.

Newer User Profile Method

The primary limitations of the MODTRAN configuration method utilized in DIRSIG3 was that:

  1. The configuration file had to be migrated/updated with every new DIRSIG software update, and

  2. Individual users had to maintain configuration files if they wanted to manually manage or override the centralized configuration in the DIRSIG installation.

This was addressed in DIRSIG4 by relocating the MODTRAN installation specifics to the user’s DIRSIG personal settings. These settings are stored using the common platform specific mechanisms on each host operating system (e.g., user registry entries on Windows, an application PLIST file on MacOS, etc.). Because they are stored in the user’s account, the MODTRAN settings are not lost between upgrades and the user can easily customize their configuration. In addition, the new mechanism allows the user to have multiple MODTRAN installations configured and choose which to use. The MODTRAN installations can be viewed and modified in the DIRSIG graphical user interface Simulation Editor (DIRSIG or DIRSIG.exe) by opening the settings (FileSettings menu item, or DIRSIGPreferences on MacOS) and going to the MODTRAN tab.

MODTRAN6 Integration

The major development in MODTRAN6 is a shift from the old "tape5" style inputs to a new JSON (JavaScript Object Notation) style input. In addition to improving the general readability of the input, the JSON document format is much easier to read in, modify and write back out. However, the DIRSIG4-era ClassicAtm model and make_adb tool will not be updated to use the new JSON input format since feature enhancements on this tool have been suspended in favor of the DIRSIG5-era NewAtm model and atm_builder tool. Instead, the solution for continued use of make_adb is to make a script that wraps around the MODTRAN6 executable and perform some basic file manipulations so that the script makes MODTRAN6 "look like" older versions of MODTRAN. As a result, MODTRAN6 can be used with any existing version of make_adb without any code modifications to DIRSIG or installation updates for the end-user. The example script below assumes DIRSIG wrote to the file tape5, it renames that file to mod6_tmp.tp5 and then launches MODTRAN6 supplying mod6_tmp.tp5 as a command-line argument. Note that this rename approach was selected over the option of converting the input file to JSON in order to avoid the run-time associated with that conversion. Additionally, DIRSIG expects the output files to be named tape6, tape7, etc. so this script also renames the output files after MODTRAN6 has finished execution.

Linux and MacOS

Here is a basic Bourne/Bash script version of the wrapper than can be utilized on Linux or macOS:

#!/bin/sh
#
# Example MODTRAN6 wrapper for Linux and MacOS for DIRSIG's make_adb tool.
#

# This variable points to MODTRAN6 bin folder
BIN_DIR=/usr/local/MODTRAN6.0.0/bin/linux

# This variable points to the MODTRAN6 DATA folder.  Note that this
# is redundant to what is stored in the MODTRAN profile in the DIRSIG
# settings. But it MUST be correctly specified here because it is a
# required command-line option that is not passed in by make_adb.
DATA_DIR=/usr/local/MODTRAN6.0.0/DATA

# This variable defines the basename of the temporary MODTRAN input
# and output files. This makes it easy to change if we find the current
# one is a bad choice.
TMP_BASENAME=mod6_tmp

# Step #1:
# Rename "tape5" to a file with a ".tp5" extension so we
# can provide a it as a command-line argument in Step #2.
mv tape5 ${TMP_BASENAME}.tp5

# Step #2:
# Run the MODTRAN6 "console" executable (not the GUI!) by specifying the
# name of our renamed "tape5" file.  We also provide the path to the DATA
# folder.
${BIN_DIR}/mod6c_cons ${TMP_BASENAME}.tp5 ${DATA_DIR}

# Step #3:
# Rename all the output files to what make_adb expects them to be.
mv ${TMP_BASENAME}.tp5 tape5
mv ${TMP_BASENAME}.tp6 tape6
mv ${TMP_BASENAME}.tp7 tape7
mv ${TMP_BASENAME}.7sc tape7.scn

# Step #4:
# Remove other output files we don't care about.
rm ${TMP_BASENAME}.psc

The user must modify this example script to reflect the location of their MODTRAN6 installation. Specifically, the paths assigned to the BIN_DIR and DATA_DIR variables in the script.

On our systems, we have named this script mod6_wrapper.sh, given it execute permissions and placed it in the Documents/DIRSIG folder. To use the script, you should supply the path to this script (rather than the actual MODTRAN executable) in the applicable MODTRAN profile to your DIRSIG user settings (see the "MODTRAN" tab in the settings/preferences dialog available in the main simulation editor). An example of this file is included in the DIRSIG extras/modtran folder.

Important You must test your script outside of DIRSIG to make sure it functions correctly. This can be accomplished by making a temporary folder that only contains a valid MODTRAN input "tape5" file named tape5 (a set valid files can be found in the MODTRAN TESTS folder). Test the script by running it via the command-line (e.g. bash, CMD, etc.) from that temporary folder. If it functions correctly, it will run without error messages or popups and leave behind files named tape5, tape6, tape7 and tape7.scn.

Windows

The software installation layout for MODTRAN6 on Windows is slightly more complex with the program executables and data files spread across multiple directories. The graphical user interface is typically located in C:\Program Files\SSI\MODTRAN6 as mod6.jar. However, DIRSIG needs to run the command-line executable, which is typically located in C:\Users\Public\Documents\MODTRAN6\Utilities\console\x86_64 as mod6_cons.exe and the location of the standard DATA directory is C:\ProgramData\SSI\MOD6DATA.

Note The directory locations listed here were observed with MODTRAN 6.0.0, but may change with future versions.
Important The C:\ProgramData folder is hidden by default, so do not be confused if you don’t see it in the Windows File Explorer.
Important Paths like C:\Program Files that contain spaces must be wrapped with double-quotes (") so that the interpreter doesn’t think the path is two (or more) separate strings rather than a single one.

Similarly to Linux and MacOS, we need to make a wrapper script to sit between the DIRSIG make_adb program and MODTRAN6. The user should create a Windows .BAT similar to the one shown below:

REM Example MODTRAN6 wrapper for Windows for DIRSIG's make_adb tool.

REM This variable defines the basename of the temporary MODTRAN
REM input and output files. This makes it easy to change if we
REM find the current one is a bad choice.
SET TMP_BASENAME=mod6_tmp

REM Step #1:
REM Rename "tape5" to a file with a ".tp5" extension so we
REM can provide a it as a command-line argument in Step #2.
MOVE "%CD%\tape5" "%CD%\%TMP_BASENAME%.tp5"

REM Step #2:
REM Run the MODTRAN6 "console" executable (not the GUI!) by
REM specifying the name of our renamed "tape5" file (as the 1st option)
REM and the path to the DATA folder (as the 2nd option)
"C:\Users\Public\Documents\MODTRAN6\Utilities\console\x86_64\mod6con.exe" "%CD%\%TMP_BASENAME%.tp5" "C:\ProgramData\SSI\MOD6DATA"

REM Step #3:
REM Rename all the output files to what make_adb expects them to be.
MOVE "%CD%\%TMP_BASENAME%.tp5" "%CD%\tape5"
MOVE "%CD%\%TMP_BASENAME%.tp6" "%CD%\tape6"
MOVE "%CD%\%TMP_BASENAME%.tp7" "%CD%\tape7"
MOVE "%CD%\%TMP_BASENAME%.7sc" "%CD%\tape7.scn"

REM Step #4:
REM Remove other output files we don't care about.
DEL "%CD%\%TMP_BASENAME%.psc"

On our systems, we have named this script mod6_wrapper.bat and placed it in the Documents/DIRSIG folder. To use the script, you should supply the path to this script (rather than the actual MODTRAN executable) in the applicable MODTRAN profile to your DIRSIG user settings (see the "MODTRAN" tab in the settings/preferences dialog available in the main simulation editor). An example of this file is included in the DIRSIG extras/modtran folder.

Important You must test your script outside of DIRSIG to make sure it functions correctly. This can be accomplished by making a temporary folder that only contains a valid MODTRAN input "tape5" file named tape5 (a set valid files can be found in the MODTRAN TESTS folder). Test the script by running it via the command-line (e.g. bash, CMD, etc.) from that temporary folder. If it functions correctly, it will run without error messages or popups and leave behind files named tape5, tape6, tape7 and tape7.scn.

MODTRAN4-P Integration

A small project in the early 2000s produced a prototype version of MODTRAN4 that had a limited set of updated scattering phase functions that enabled the prediction of polarized atmospheric path radiance. If you have access to this version of MODTRAN (referred to as MODTRAN4-P), you can configure it for use in DIRSIG.

Important Work on the MODTRAN4-P prototype ceased nearly 20 years ago. If you do not already have access to this restricted version of MODTRAN, please do not ask RIT and the DIRSIG team how to acquire it.

Simply create a MODTRAN installation profile that points to the MODTRAN4-P installation and use it with the -polarized option:

$ make_adb -polarized demo.sim

If you choose to create an alternate MODTRAN installation configuration file, your command line will look something like this:

$ make_adb -use_config modtran4p.conf -polarized demo.sim

If you receive the following error while make_adb is running in polarized mode then you should verify that you are correctly running MODTRAN4-P.

make_adb: MODTRAN ERROR
    MODTRAN output is not polarized!

Usage

This section outlines the various options for the make_adb tool.

Atmosphere File Options

The <classicradiativetransfer> element in the DIRSIG4 .atm file captures a variety of inputs for the ClassicAtm model. An example of this XML element is included below:

  <classicradiativetransfer>
    <modtrantape5file externalfile="classic_mls_rural_23km.tp5"/>
    <modtranprofilename>MODTRAN6</modtranprofilename>
    <adbfile>classic_mls_rural_23km.adb</adbfile>
    <removesensorpath>false</removesensorpath>
    <trypolarized>false</trypolarized>
    <usefastsky>false</usefastsky>
  </classicradiativetransfer>

The <modtrantape5file> element supplies the MODTRAN template "tape5" file for MODTRAN. If the externalfile attribute is provided, it is assigned the path to the file to be used. Otherwise, this element will contain the contents of the "tape5" enclosed in an XML CDATA block (see below):

  <classicradiativetransfer>
    <modtrantape5file><![CDATA[
M   2    2    2    0    0    0    0    0    0    0    1    1    1   0.000   0.00
    2    0    0    1    0    0     0.000     0.000     0.000     0.000     0.218
    20.218     0.218   180.000     0.000     0.000     0.000    0
    1    2  236    0
    43.000    77.000     0.000     0.000    12.000     0.000     0.000     0.000
       450     35050       100     2
    0
    ]]></modtrantape5file>
    <modtranprofilename>MODTRAN6</modtranprofilename>
    <adbfile>int_mls.adb</adbfile>
    <removesensorpath>false</removesensorpath>
    <trypolarized>false</trypolarized>
    <usefastsky>false</usefastsky>
  </classicradiativetransfer>

The benefit of this "internal" method is that it removes an external file dependency on the .atm file. When you share the .atm file you do not need to also share the "tape5" file referenced in the .atm file.

Important Be careful to avoid perturbing the formatting of the tape5 file when importing the "tape5" file. With the exception of removing empty lines at the start and end of the CDATA the contents are parsed directly as a "tape5" file.

The <modtranprofilename> element specifies which MODTRAN installation profile will be used. In this example, the profile with the name MODTRAN6 fill be used.

Tip If the <modtranprofilename> is not supplied, the tool will use the first (default) profile in the user’s account.

The <adbfile> element contains the name of the ADB file produced by make_adb.

The <removesensorpath> element controls an option to "remove" the path between the sensor and the scene. Setting this option to true will cause the path transmission data to be reset to 1 and the path radiance data to be reset to 0 after reading the database. As a result, the radiance at the sensor is essentially the surface leaving radiance.

The <trypolarized> element mirrors the -polarized command-line option (discussed below). This option will try to run MODTRAN in polarized mode. This mode was only supported by the limited access version of MODTRAN4 that had polarization support (referred to as MODTRAN4-P). Using this option with any other version of MODTRAN will result in an error.

The <usefastsky> element controls an option to populate the angularly sampled sky irradiance data table with a single uplooking sky path from MODTRAN. Originally introduced to facilitate quick database builds when debugging the use of this option is strongly discouraged.

Command-Line Options

DIRSIG4
Release: 2022.05 (03a571b) built on Jan 31 2022 14:22:02
Copyright 2015-2022 Rochester Institute of Technology

usage: make_adb [options] <SIM file>
    where [options] are:
    -help                           Print this help/usage message
    -use_config <file>              Use alternate MODTRAN configuration info
    -show_config                    Print MODTRAN configuration info
    -max_inversion <alt>            Set the max inversion altitude
    -max_range <range>              Set the max range [km] for sensor paths
    -fov_min_deltas <theta,phi>     Set the minimum angle between paths within the sensor FOV
    -polarized                      Create a polarized atmosphere
    -save_tapes                     Save copies of generated tape file
    -interactive <bandpass>         Create Interactive Mode queries atmosphere

The -use_config option can be used to specify the path to a DIRSIG3 era MODTRAN installation configuration file.

The -show_config option will print the MODTRAN configuration(s) available.

The -max_inversion option lets the user specify the inversion altitude used to update the boundary layer data in a MODTRAN user-defined atmosphere (MODTRAN MODEL = 7) from the weather data supplied to the DIRSIG simulation.

The -max_range option is used to specify the maximum range for a simulation where the FOV of the sensor is entirely above the horizon.

The -fov_min_deltas options lets the user override the minimum zenith and azimuth angles between paths within the sensor FOV.

The -polarized option will try to run MODTRAN in polarized mode. This mode was only supported by the limited access version of MODTRAN4 that had polarization support (referred to as MODTRAN4-P). Using this option with any other version of MODTRAN will result in an error.

The -save_tapes option will preserve all the MODTRAN run directories, including the input and outputs.

The -interactive option runs the tool in the "interactive" mode, which supports DIRSIG4’s interactive mode.

Computing Weather Insolations

The weather file used by the THERM weather plugin includes broad-band direct (solar) and diffuse (sky) insolation (broad-band irradiance) values for each time entry. However, those insolation values might not be consistent with the MODTRAN description of the atmosphere. For example, the weather file might have direct and diffuse insolation values indicative of high visibility conditions, but the MODTRAN description of the atmosphere might describe low visibility conditions. The ramifications of this apparent contradiction would manifest itself as predicted scene temperatures that are not correlated with the atmospheric conditions. For example, if the weather file insolations are for high visibility conditions, the temperatures might be higher than expected for the low visibility imaging conditions.

To help address this issue, the make_adb tool has a mode that will compute these weather file insolation values based on the MODTRAN description of the atmosphere. Running the make_adb tool with the -compute_insolations option will take the MODTRAN description of the atmosphere defined in the supplied simulation and compute appropriate broad-band solar and sky irradiance values as a function of time of day. These irradiance values are then converted to the insolation units used in the weather file and a new weather file is generated.

$ make_adb -compute_insolations classic_mls.sim
DIRSIG4
Release: 2022.19 (64fb95f) built on May 13 2022 12:42:08
Copyright 2015-2022 Rochester Institute of Technology

Reading thumbnail
Checking for scene-driven pointing ... done.
    Platform does not require scene geometry

Using MODTRAN profile 'MODTRAN5' (default)

[output deleted for documenation brevity]

Computing weather insolations .................................................
...............................................................................
................................................................. done.

The primary output of this execution is a THERM weather file containing the direct and diffuse insolations computed using the MODTRAN configuration supplied. Currently the new weather file is always named new.wth and it should be appropriately renamed (to indicate its origin and how it was modified) and then referenced from your BasicAtmosphere .atm file.

Troubleshooting

If you are having problems with make_adb that are MODTRAN related, consider running with the -save_tapes option and reviewing the input and output files for the MODTRAN simulation that failed.

In addition, consult the MODTRAN Integration Guide and the Troubleshooting Guide for specific issues.