Summary
The NewAtmosphere plugin was created to provide a new capability supporting for physics-based atmospheric models in DIRSIG5. It attempts to address the limitations of the DIRSIG4-era make_adb and "classic" atmosphere model combination in a variety of ways. Specifically, the goals were:
-
Work with the DIRSIG5 scene HDF file directly (no need to carry around the entire scene directory).
-
Move to a binary database format that is more compact to store and faster to load.
-
Simplify the ability to integrate multiple backend physics-based atmospheric models.
-
Run multiple instances of the backend physics-based atmospheric models to build databases faster.
-
Add support for multiple times in a single database.
-
Add support for multiple altitudes in a single database.
The capability is currently divided into two major components:
-
The atm_builder tool, which interfaces with the externally developed physics-based atmospheric models to construct an HDF database of atmospheric data tables.
-
The NewAtmosphere plugin, which uses the HDF atmospheric database to drive the atmospheric contributions in a DIRSIG5 execution.
Backend Models
One of the key features of the new toolchain is that it is much simpler to implement interfaces to different backend physics models for the atmosphere. We call these different models "backend" models and there are currently 2 backend interfaces that have been created:
-
The "dummy" backend, which doesn’t interface to an external model. Instead, it has analytical models for the atmosphere that are similar to the capabilities of the Simple atmosphere model in DIRSIG4.
-
The "MODTRAN Tape" backend, which interfaces with MODTRAN using the classic "tape-based" input/output files. This should be compatible with MODTRAN4, MODTRAN5 and MODTRAN6.
Which backend model is used is part of the input configuration.
![]() |
A a future date, backends will be developed to (1) use MODTRAN6 via JSON input/output method and (2) interface with the 6SV model. |
Input
The inputs to the plugin is a JSON document that can be either
directly supplied in the JSIM file where the plugin
is requested (via the inputs
section) or in a separate JSON file
(via the input_filename
variable). The details of these two
options are described later. Regardless of the mechanism used to
supply that JSON document to the plugin, this section describes
what that JSON input document looks like.
Below is a generalized example. It consists of an info
section
that describes the atmosphere setup, a backend
section that
describes which backend is to be used (via the name
variable) and
a set of common variables that have a uniform behavior across all
the backends:
"info" : { "name" : "A great atmosphere", "author" : "Joe User", "organization" : "Awesome, Corp", "description" : "This is a great atmosphere setup." }, "backend" : { "name" : "<backend_name>", ... } "max_processes" : 8, "time_delta" : 900, "altitude_delta" : 500, "hdf_filename" : "my_sim.adb.hdf"
Common variables
The maximum number of processes
The max_processes
variable defines the maximum number of processes
that the backend model can use. For example, multiple simultaneous
MODTRAN processes can be used to generate the paths. On a multi-core
computer, you can generally run one process per physical core. On a
multi-core computer that supports hyper-threading, you can usually
run 2 processes per physical core. The default value is set based on
the number of virtual cores available on the host computer.
![]() |
This variable can be overridden with the --max_processes
command-line argument. See below for details.
|
The time sampling
The time_delta
variable defines the time resolution of the database.
Since the atmosphere is not expected to vary rapidly, it does not make
sense to compute new atmospheric values at 10s, 100s or 1000s of Hz if
that is what the readout rate is of the sensor. Instead, the toolchain
makes the assumption that the atmosphere is optically and geometrically
(sun and moon scattering) "frozen" for some period of time. The default
period or time resolution is 900 seconds (15 minutes).
![]() |
This variable can be overridden with the --time_delta
command-line argument. See below for details.
|
The altitude sampling
The altitude_delta
variable defines the altitude resolution of
the database. Since the atmosphere is not assumed to be uniform
in the vertical dimension, it needs to be sampled as a function of
altitude if the scene(s) span significant altitude range. When
the builder loads the scene(s), it looks at the altitudes spanned
by the scene bounding box(es). Using the altitude sampling delta,
it will construct a list of altitudes within each scene and then
merge those into a combined list of altitudes for the database.
The default value for the altitude sampling delta is 500m. In most cases, the number of altitudes used in the database is a scalar multiplier for the number of backend model runs that need to be performed.
-
Decreasing the sampling delta increases the number of altitudes used in the database.
-
Increasing the sampling delta decreases the number of altitudes used in the database.
![]() |
This variable can be overridden with the --altitude_delta
command-line argument. See below for details.
|
The HDF database filename
The hdf_filename
variable defines the name of the HDF file to hold
the resulting atmospheric database. The convention so far has been to
use the compound extension ".adb.hdf
".
Using the Dummy Backend
The "Dummy" backend is built-in (it does not call an external program) and it implements a set simplified calculations to approximate nominal illumination conditions:
-
The sun is modeled with a 5800K blackbody spectral distribution that is magnitude normalized to approximate the Earth reaching irradiance of the sun.
-
The moon is modeled using the same solar spectral distribution, but is magnitude normalized to approximate the Earth reaching irradiance of the moon.
-
The sky is modeled with an exponentially decaying spectral distribution that approximates clear sky scattering conditions in the visible.
-
There is no path radiance or transmission along other paths.
This backed is configured without any additional options:
"backend" : { "name" : "dummy" }
![]() |
The "dummy" backend is primarily for testing purposes and rapid use during simulation setup and debugging. It should not be used for data production simulations. |
Using the MODTRAN Tape Backend
The "MODTRAN Tape" backend is configured with a variety of options that provide details about the external MODTRAN installation and user input files. For the examples included here, the specific paths to the MODTRAN executable, data folder, etc. may differ from your installation.
![]() |
Although the examples below show various configurations for older versions of MODTRAN, the user is strongly encouraged to use the latest version of MODTRAN6. Visit the MODTRAN page at the Spectral Sciences, Inc. website for more information. |
The option to extract atmospheric profile data
The MODTRAN tape backend can optionally extract the atmospheric profile
from the MODTRAN output "tape6" file. When this profile data is extracted,
it is supplied to the radiometry core to drive refraction through the
atmosphere. This option is controlled via the extract_profile
variable,
which is set to either true
or false
(note, boolean values in JSON are
not strings, so do not assign the variable "true"
).
![]() |
This option is disabled by default. |
Using a MODTRAN configuration profile from your settings
The first option for specifying the MODTRAN setup is using the
MODTRAN "profiles" in the user’s DIRSIG settings (or preferences). In
that case the setup includes the modtran_profile
variable:
"backend" : { "name" : "modtran_tape", "modtran_profile" : "MODTRAN5_2_4", "modroot_filename" : "mod5root.in", "tape5_filename" : "./mls.tp5", "extract_profile" : false }
In this example, the profile named "MODTRAN5_2_4" might be mapped to a MODTRAN5 2.4 installation.
![]() |
This backend uses the "modroot" approach to naming the input and
output files associated with the MODTRAN run (see the MODTRAN
documentation for more information about this mechanism). The
modroot_filename variable indicates the name of the file containing
the base name of the MODTRAN input and output files. Since the name
of this file has changed depending on the version of MODTRAN, it is
provided as a user configuration variable.
|
Supplying the installation paths: MODTRAN4
Another option is to specify the path to the executable and data directory directly in the backend description:
"backend" : { "name" : "modtran_tape", "modtran_exe" : "/Users/dirsig/Mod4v3r1/Mod4v3r1.exe", "modtran_data" : "/Users/dirsig/Mod4v3r1/Mod4v3r1/DATA", "modroot_filename" : "modroot.in", "modtran_min_delta" : 1.0, "tape5_filename" : "./mls.tp5", "extract_profile" : false }
In this example, the user is pointing at a MODTRAN4 Version 3, Release
1 installation in their own directory. The modtran_exe
variable
is assigned the full path to the MODTRAN executable, the modtran_data
is assigned the full path to the MODTRAN DATA
directory and the
modtran_min_delta
variable is assigned the limiting frequency
resolution for the band model database available with that version
of MODTRAN (typically 1.0 wavenumbers for MODTRAN4). The name of
the "modroot" file for MODTRAN4 was modroot.in
.
Supplying the installation paths: MODTRAN5
Another option is to specify the path to the executable and data directory directly in the backend description:
"backend" : { "name" : "modtran_tape", "modtran_exe" : "/Users/dirsig/Modtran5.2.4/mod5_cons.exe", "modtran_data" : "/Users/dirsig/Modtran5.2.4/DATA", "modroot_filename" : "mod5root.in", "modtran_min_delta" : 0.1, "tape5_filename" : "./mls.tp5", "extract_profile" : false }
In this example, the user is pointing at a MODTRAN5 2.4 installation
in their own directory. The modtran_exe
variable is assigned the
full path to the MODTRAN executable, the modtran_data
is assigned
the full path to the MODTRAN DATA
directory and the modtran_min_delta
variable is assigned the limiting frequency resolution for the band model
database available with that version of MODTRAN (0.1 wavenumbers for
MODTRAN5 and later). The name of the "modroot" file for MODTRAN5 was
mod5root.in
.
![]() |
The specific paths to the MODTRAN executable will vary based on your installation. |
Supplying the installation paths: MODTRAN6
Although MODTRAN6 introduced a new JSON input/output mechanism, the program
is backward compatible with the traditional "tape" input/output mechanism
utilized by this backend. This input/output option is accessed via the
-modroot
command line option, which is described in the manual and shown
in the usage message of the console executable:
$ ./mod6c_cons -h Usage: ./mod6c_cons input_file [data_path] JSON and TAPE5 input formats are supported. [data_path] specifies an optional path to the MODTRAN data directory. optional flags -modroot Process legacy input formats using mod5root.in or tape5 files. ...
This backend employs the "modroot" approach for all versions of
MODTRAN, where the basename of the input/output files for the MODTRAN
run is stored in a separate file. MODTRAN5 and MODTRAN4 automatically
looked for the mod5root.in
and modroot.in
files, respectively.
MODTRAN6, however, needs the -modroot
option to be specified as
the 1st command-line argument for it to look in the mod5root.in
file. Additionally, the MODTRAN6 executable needs to be supplied
the full path to the MODTRAN6 DATA
folder as the 2nd command-line
argument. Hence, the complete command-line to launch MODTRAN6 needs
to look like:
$ /some/path/MODTRAN6.0.0/bin/mod6c_cons.exe -modroot /some/path/MODTRAN6.0.0/DATA
This backend has an optional configuration variable that allows the
user to customize the command-line arguments that are passed to the
MODTRAN executable. These arguments are defined in the JSON using the
modtran_options
variable, which is a JSON array of strings. The
example configuration below shows how these two additional arguments
are provided to the MODTRAN executable using the options array variable:
"backend" : { "name" : "modtran_tape", "modtran_exe" : "/Users/dirsig/MODTRAN6.0.0/mod6c_cons.exe", "modtran_data" : "/Users/dirsig/MODTRAN6.0.0/DATA", "modroot_filename" : "mod5root.in", "modtran_options" : ["-modroot","/Users/dirsig/MODTRAN6.0.0/DATA"], "modtran_min_delta" : 0.1, "tape5_filename" : "./mls.tp5", "extract_profile" : false }
![]() |
The specific paths to the MODTRAN executable will vary based on your installation. |
The input "tape5" filename
The tape5_filename
variable defines the name of the input "tape5" file
that is used as a template for the MODTRAN runs. In general, the backend
is using the description of the atmosphere in this file (which model,
aerosols, visibility, etc.) and is only updating the key parameters that
are relevant to the specific DIRSIG simulation (the wavelengths, the source
position, the path geometry, etc.).
The user also has the option to store the MODTRAN "tape5" file
within the input JSON description via the tape5_contents
variable.
However, because the "tape5" file is multiple lines and you cannot
embed newlines inside a JSON variable, the user will need to make
the file a single string by replacing the newlines with the \n
character:
"name" : "NewAtmosphere", "inputs" : { "info" : { [deleted for documentation purposed] }, "backend" : { "name" : "modtran_tape", "modtran_profile" : "MODTRAN4", "modtran_exe" : "/Users/scottbrown/pkg/Mod4v3r1/Mod4v3r1.exe", "modtran_options" : [""], "modtran_data" : "/Users/scottbrown/pkg/Mod4v3r1/DATA", "extract_profile" : false, "tape5_contents" : "M 2 2 2 0 0 0 0 0 0 0 1 1 1 0.000 0.00\n 2 0 0 1 0 0 0.000 0.000 0.000 0.000 0.218\n 20.218 0.218 180.000 0.000 0.000 0.000 0\n 1 2 236 0\n 43.000 77.000 0.000 0.000 12.000 0.000 0.000 0.000\n 450 35050 100 2\n0" }, "hdf_filename" : "mls_rural_23km_dis4_scaled.adb.hdf" }
![]() |
The user needs to be very careful when performing this newline replacement operation to otherwise not disrupt the precise format of the MODTRAN "tape5" file. |
Usage
Input description method
As it was mentioned earlier, the plugin and JSIM file support two mechanisms for supplying the JSON document containing the input configuration.
JSIM Using an Internal Description
Below is an example where the plugin description is contained in the JSIM file:
[{ "scene_list" : [ { "inputs" : "./demo.scene" } ], "plugin_list" : [ { "name" : "NewAtmosphere", "inputs" : { "info" : { "name" : "MLS Rural 23km w/ Isaac 2-stream", "author" : "Scott D. Brown", "organization" : "RIT", "description" : "Mid-Latitude Summer with rural aerosols and 23 km visibility, using Isaac 2-stream multiple scattering" }, "backend" : { "name" : "modtran_tape", "modtran_profile" : "MODTRAN5_2_4", "modroot_filename" : "mod5root.in", "tape5_filename" : "./mls_rural_23km_dis4.tp5" }, "hdf_filename" : "mls_rural_23km_dis4.adb.hdf" } }, { "name" : "BasicPlatform", "inputs" : { "platform_filename" : "./demo.platform", "motion_filename" : "./demo.ppd", "tasks_filename" : "./demo.tasks", } } ] }]
JSIM Using an External Description
Alternatively, the JSON can be stored in an external JSON file and
the name of that file specified in the JSIM file. This approach
allows a single copy of a configuration to be stored and shared
among multiple simulations. The example below contains the plugin
configuration using the same JSON structure but in an external file
(named mls_rural_23km_dis4.jatm
in this example):
{ "info" : { "name" : "MLS Rural 23km w/ Isaac 2-stream", "author" : "Scott D. Brown", "organization" : "RIT", "description" : "Mid-Latitude Summer with rural aerosols and 23 km visibility, using Isaac 2-stream multiple scattering" }, "backend" : { "name" : "modtran_tape", "modtran_profile" : "MODTRAN5_2_4", "modroot_filename" : "mod5root.in", "tape5_filename" : "./mls_rural_23km_dis4.tp5" }, "max_processes" : 8, "hdf_filename" : "mls_rural_23km_dis4.adb.hdf" }
and then this external file is referenced in the JSIM file:
[{ "scene_list" : [ { "inputs" : "./demo.scene" } ], "plugin_list" : [ { "name" : "NewAtmosphere", "inputs" : { "input_filename" : "mls_rural_23km_dis4.jatm" } }, { "name" : "BasicPlatform", "inputs" : { "platform_filename" : "./demo.platform", "motion_filename" : "./demo.ppd", "tasks_filename" : "./demo.tasks" } } ] }]
The builder tool
The atm_builder tool is what will build the HDF atmospheric database. It
loads the simulation configuration, determines what backend calculations
need to be performed and then has the backend execute those calculations.
Below is the command-line usage message when either no arguments or the
--help
option is supplied:
$ atm_builder --help usage: atm_builder [options] <sim filename> basic [options]: --version Display version info --help Display this usage message --verbose Enable verbose output of the tool --no_run Do not run the backend atmospheric model ('dry run' mode) --save_files Save input/output files generated by the backend atmospheric model --time_delta=T Set the interval between time steps (default: 900 seconds) --altitude_delta=H Set the interval between altitude steps (default: 500 meters) --fov_min_deltas=X,Y Set the minimum angle between sensor FOV paths (default: 5,5 degrees) --max_processes=N Set the maximum number of backend model processes (default: <virtual cores>)
General operation entails supplying the name of the JSIM file to the tool, along with any options.
$ atm_builder example.jsim
Command-Line Options
The following command-line options provide additional utility:
--no_run
or--no_run=<true,false>
-
This option will tell the backend to configure all the backend runs, but it will not execute them. The input files will be preserved after the program completes. The default value is
false
. --save_files
or--save_files=<true,false>
-
This option will tell the backend to save all the input and output files generated to execute the associated model (overrides the
save_files
option in the JSON configuration). The default value isfalse
. --time_delta=T
-
This option will set the time resolution (in seconds) of the database (overrides the
time_delta
variable in the JSON configuration). The default value is 900 seconds (15 minutes). --altitude_delta=H
-
This option will set the altitude resolution (in meters) of the database (overrides the
altitude_delta
variable in the JSON configuration). The default value is 500 meters (0.5 km). --fov_min_deltas=X,Y
-
This option specifies the minimum angle between path samples within the field-of-view (FOV) of any given capture. The default values are 5 and 5 degrees in zenith and azimuth.
--max_processes=N
-
This option will tell the backed the maximum number of external model processes it is allowed to create (overrides the
process_count
variable in the JSON configuration). The default value is the number of virtual cores on the host computer.
Errors and Debugging
In most situations, the source of errors will be related to the external model executed by the backend. When an error occurs with either the running of the backend model process or in reading the output of the backend model, an error will be generated and the related files will be preserved in the reported path for debugging.
Under the hood of the MODTRAN Tape backend
The "MODTRAN tape" backend creates a folder for all the MODTRAN executions
(named modtran_runs
) within which each model execution is stored in a
sub-folder using a simple numerical numbering scheme from 0
to N. The
backend will purge the contents of the model request folders after the
model has been successfully run and the output ingested. However, these
folders can be preserved by using the --save_files
command-line option
to the builder.
The file listing below shows the contents of a preserved model request folder after execution:
$ ls modtran_runs/0 README.txt modtran_stderr.txt tmp.7sc mod5root.in modtran_stdout.txt tmp.tp5
The README.txt
file contain a "human readable" description of the inputs
to MODTRAN. The mod5root.in
contains the basename of the input/output
files (in the case of this backend, it is always tmp
), the
modtran_stderr.txt
and modtran_stdout.txt
files contain the standard
error and standard output (console) of the MODTRAN program, respectively.
The tmp.tp5
and tmp.7sc
files are the respective input and output
files of the model execution.