The NewAtmosphere plugin and atm_builder tool were created to provide a new capability supporting for physics-based atmospheric models in DIRSIG5.

Summary

The NewAtmosphere and atm_builder toolchain attempts to address the limitations of the DIRSIG4-era make_adb tool and the "Classic" atmosphere model (handled by the BasicAtmosphere plugin in DIRSIG5) combination in a variety of ways.

Improvements

  • Work with the DIRSIG5 scene HDF file directly (no need to carry around the entire scene directory).

  • Move to a binary database format (HDF5) that is more compact to store and faster to load.

  • Simplify the ability to integrate multiple backend physics-based atmospheric models.

  • Run multiple instances of the backend physics-based atmospheric models to build databases faster.

  • Add support for multiple times in a single database.

  • Add support for multiple altitudes in a single database.

An interafce control document (ICD) for the NewAtmosphere HDF5 database can be found here.

Configuration

A detailed description of the atm_builder program is outlined in its respective manual. This section outlines the input configuration for the NewAtmosphere plugin, which is also passed to the atm_builder program to construct the database used by the plugin.

JSIM Setup Options

The inputs to the NewAtmosphere plugin and atm_builder tool are defined in a JSON document that can be either directly supplied in the JSIM file where the plugin is requested (via the inputs section) or in a separate JSON file (via the input_filename variable).

flow
Figure 1. Symbiotic Relationship of NewAtmosphere plugin and atm_builder program.

The JSON description consists of an info section that describes the atmosphere setup, a backend section that describes which backend is to be used (via the name variable) and any backed specific input variables. The role of the "backends" is to provide a unified interface for a variety of different radiative transfer (RT) models that can solve atmospheric modeling requests. For example, "get the spectral irradiance from the sun reaching a given latitude, longitude and altitude at a specific date and time". These backend adaptor modules take these generic requests and translate them into specfic inputs for the respective external atmospheric RT model (e.g. MODTRAN). This common interface means that a new atmospheric RT model can be supported once the request translation logic for that model has been written.

Note These atmospheric radiative transfer model backends are also utilized by the fourcurve_builder toolchain.

Using an Internal Description

Below is an example where the plugin description is fully contained in the JSIM file:

[{
  "scene_list": [
    { "inputs": "./demo.scene" }
  ],
  "plugin_list": [
    {
      "name": "NewAtmosphere",
      "inputs": {
        "info": {
            "name": "A great atmosphere",
            "author": "Joe User",
            "organization": "Awesome, Corp",
            "description": "This is a great atmosphere setup."
        },
        "backend": {
            "name": "<backend_name>"
            ...
        },
        "max_processes": 8,
        "time_delta": 900,
        "altitude_delta": 500,
        "scene_index": 23,
        "remove_sensor_path": false,
        "hdf_filename": "my_sim.adb.hdf"
      }
    },
    {
      "name": "ExampleSensorPlugin",
      "inputs": {
        ...
      }
    }
  ]
}]

Using an External Description

Alternatively, the JSON can be stored in an external JSON file and the name of that file specified in the JSIM file. This approach allows a single copy of a configuration to be stored and shared among multiple simulations. The example below contains the plugin configuration using the same JSON structure but in an external file (named mls_rural_23km_dis4.jatm in this example):

An example external JSON configuration for the plugin.
{
    "info": {
        "name": "A great atmosphere",
        "author": "Joe User",
        "organization": "Awesome, Corp",
        "description": "This is a great atmosphere setup."
    },
    "backend": {
        "name": "modtran_tape",
        "modtran_profile": "MODTRAN6",
        "modroot_filename": "mod5root.in",
        "tape5_filename": "./mls_rural_23km_dis4.tp5"
    },
    "max_processes": 8,
    "time_delta": 900,
    "altitude_delta": 500,
    "hdf_filename": "mls_rural_23km_dis4.adb.hdf"
}

and then this external file is referenced in the JSIM file:

The plugin configuration referencing the external JSON configuration.
[{
  "scene_list": [
    { "inputs": "./demo.scene" }
  ],
  "plugin_list": [
    {
      "name": "NewAtmosphere",
      "inputs": {
        "input_filename": "mls_rural_23km_dis4.jatm"
      }
    },
    {
      "name": "ExampleSensorPlugin",
      "inputs": {
        ...
      }
    }
  ]
}]
Note At this time, we encourage the use of the .jatm file extension for an external atmosphere description file to denote a JSON atmosphere configuration.

Generator Information

The required info section contains information about the generator of the database and is stored in the output database for documentation purposes.

Backend Configuration

The required backend section describes the atmospheric backend model that will be used during the data generation process. The configuration of these models is described in detail in the Atmospheric Model Backends Manual.

Data Generation Variables

This section describes the plugin variables that are primarily used during data generation by the atm_builder program.

The maximum number of processes

The optional max_processes variable defines the maximum number of processes that the backend model can use. For example, multiple simultaneous MODTRAN processes can be used to generate the paths. On a multi-core computer, you can generally run one process per physical core. On a multi-core computer that supports hyper-threading, you can usually run 2 processes per physical core. The default value is set based on the number of virtual cores available on the host computer.

Tip This variable can be overridden with the --max_processes command-line argument.

The time sampling

The optional time_delta variable defines the time resolution of the database. Since the atmosphere is not expected to vary rapidly, it does not make sense to compute new atmospheric values at 10s, 100s or 1000s of Hz if that is what the readout rate is of the sensor. Instead, the toolchain makes the assumption that the atmosphere is optically and geometrically (sun and moon scattering) "frozen" for some period of time. The default period or time resolution is 900 seconds (15 minutes).

Tip This variable can be overridden with the --time_delta command-line argument.

The altitude sampling

Since the atmosphere is not assumed to be uniform in the vertical dimension, it needs to be sampled as a function of altitude if the scene(s) span significant altitude range. When the builder loads the scene(s), it looks at the altitudes spanned by the scene bounding box(es). The atm_builder tool has an algorithm that will determine scene altitudes that will be sampled in the database. The optional altitude_minimum and altitude_delta variables can be used to control the behaviour of this algorithm. The altitude_minimum can be used to define the lowest altitude across the scenes that will be sampled in the database. The altitude_delta variable defines the altitude resolution of the database. Using the altitude sampling delta, it will construct a list of altitudes within each scene and then merge those into a combined list of altitudes for the database.

The default value for the altitude sampling delta is 500 meters. In most cases, the number of altitudes used in the database is a scalar multiplier for the number of backend model runs that need to be performed.

  • Decreasing the sampling delta increases the number of altitudes used in the database.

  • Increasing the sampling delta decreases the number of altitudes used in the database.

Tip These algorithm variables can be overridden with the --altitude_minimum and --altitude_delta command-line argument.

To override the algorithm that determines the altitude samples, the altitude_list variable can be employed. This variable is assigned an array of altitude values in meters.

Explicitly specifying the list of altitudes (meters) to use.
    "altitude_list": [0.0, 200.0, 300.0],
Important When using the altitude_list, the built-in algorithm is not used and the altitude_minimum and altitude_delta are ignored.
Tip At run time, the list of sampled altitudes can be overridden with the --altitude_list command-line argument.

Range control

When the FOV of view of a sensor includes the horizon, the user must specify a maximum range for the primary paths. For a simulation where this is known before hand, it is strongly encouraged to set this in the configuration using the max_range variable:

Specifying the maximum range (meters).
    "max_range": 5000.0,
Tip At run time, the maximum range can be overridden with the --max_range command-line argument.

The scene index

The observation angles for the various exo-atmosphere sources are relative to a specific location on the Earth. Specifically, these angles are assumed to be relative to the origin in the Scene ENU coordinate system. The (optional) scene_index variable can be specified in cases when there are multiple scenes in the simulation. This variable defaults to 0, which corresponds to the first scene loaded. If the user wants to make the scene-relative source angles relative to a different scene, then the scene_index variable should be set to the index for that scene.

Important These angles are primarily used by the atm_builder tool to construct the database used in the simulation. At this time, the database does not support a spatially varying atmosphere across an extended area scene. Hence, this variable is used to define which scene in a multiple scene setup is used to drive the data generation process.

The HDF database filename

The required hdf_filename variable defines the name of the HDF file to store the resulting atmospheric database. The convention so far has been to use the compound extension ".adb.hdf".

Run-Time Variables

This section describes input variables that are used at run-time with the database:

remove_sensor_path (optional)

This boolean controls if the sensor path (the atmospheric path between the sensor and the first thing intersected in the scene(s) should be "removed" during the simulation. Removal means setting the path transmission to 1.0 and the path radiance to 0.0 for all wavelengths. This effectively converts the conventional sensor reaching radiance to a surface leaving radiance. This option is provided as a means to emulate the output of some atmospheric compensation algorithms. Note that this variable does not impact the generation of the database and only impacts the run-time use of the values in that database.