This document is meant to serve as a brief tutorial on how to use DIRSIG5 with existing DIRSIG4 scenes and simulation scenarios.

Introduction

DIRSIG5 represents a rewrite of the over 500,000 lines of code in DIRSIG4 to produce a new DIRSIG model that features higher radiometric and computational performance. This introduction attempts to briefly describe the major differences between the two versions of the software.

Plugin-Based Architecture

The DIRSIG5 software is constructed and deployed differently than DIRSIG4 was. Most features are provided via a series of plugins rather than features to a monolithic compiled code. This helps provide feature modularity and allows the radiometry core to establish stable and well tested APIs through which these plugins communicate to the radiometry engine.

The current DIRSIG5 software includes a set of plugins that provide backward compatibility with existing DIRSIG4 simulation configurations. However, the new DIRSIG5 software does not contain all the features of the mature DIRSIG4 code base that accumulated over the 12 years it was actively developed. As the DIRSIG5 software matures during this beta release period, more and more of the missing features will be introduced until nearly full parity exists between the two codes.

Improved and Simplified Radiometry

In DIRSIG4, the speed and fidelity of the simulation could be controlled through a series of configuration choices and settings spread across the material configurations and sensor modeling parameters. The two primary mechanisms utilized by users were:

Radiometry Solver Configurations

Radiometry solvers are the algorithms that performed the surface (or volume) leaving radiance calculations. The user could select a lower fidelity radiometry solver or dial down the parameters of a higher fidelity solver to decrease fidelity and improve run times.

Sub-Pixel Sampling Configurations

Sub-pixel sampling defines how many sample rays were used in each pixel. The more rays that are used, the better the estimate of the diversity and proportion of radiance signatures that are present within the collection area of the pixel. However, using more sub-pixel rays linearly correlates with run-time.

In general, users were often observed dialing down the fidelity of a DIRSIG4 simulation (using these two mechanisms) in order to meet compute time targets. Furthermore, there wasn’t an easy "single knob" interface to DIRSIG4 to control the fidelity vs. time trade off.

In DIRSIG5 there are no radiometry solvers because it utilizes a new, unified numerical radiometry calculation. DIRSIG5 uses "path tracing", which follows a single ray path from the pixel to either a source of energy (sun, moon, sky, light, etc.) or a termination (absorbed). Since this approach is emulating a Monte-Carlo integral of all the light paths reaching a pixel, the fidelity of the solution is proportional to how many samples (paths) are used. This "number of paths" parameter provides the sought after single knob interface controlling fidelity vs. time for the model. When a small number of paths are used, the resulting radiance estimate will be noisy. As more paths are used, the estimate improves. Controlling the number of paths used per pixel will be discussed below.

Multi-threaded Execution

Unlike DIRSIG4, the DIRSIG5 code is multi-threaded. That means more compute resources can be utilized for micro-scale parallelization on a multi-core computer. In later releases, macro-scale parallelization (using multiple computers in a cluster) will be available. Controlling how many threads are used by the model will be discussed below.

Multi-computer Execution

As of the 5.0.10 release, DIRSIG5 release include a Message Passing Interface (MPI) enabled executable that will allow the software to distribute computations on clustered computing resources. Details about using the MPI-enabled version of DIRSIG is described in a supplementary manual.

GPU Acceleration

Future releases of DIRSIG5 will utilize NVIDIAs CUDA and OptiX frameworks for GPU-accelerated ray tracing, which currently accounts for approximately 80% of the execution time.

Status and Limitations

As of November 2018, DIRISG5 is still only available through limited release to a beta user population. The software will be made available to the general user population over the winter of 2018/2019.

To date, the DIRSIG5 software effort has focused on radiometric and computational performance for passive EO simulations. The following limitations exist at this time.

  • Limited Thermal Capabilties

    • DIRSIG5 does not have a run-time temperature prediction solution at this time. The THERM algorithm used in previous generations of DIRSIG would require a massive rewrite to work in the computational approach and architecture used DIRSIG5 (10s or 100s of samples per pixel, multi-threading, etc.).

    • Scenes that feature user-supplied temperatures can be run. That includes the use of:

      • GDB files with per-facet temperatures,

      • Use of the data-driven temperature solver, and

      • Use of temperature maps

  • No LADAR/LIDAR Capabilties

    • RIT has a white paper outlining the effort to get this active sensing modality supported under the new DIRSIG computational architecture.

  • No SAR Capabilties

    • RIT has a white paper outlining the effort to get this active sensing modality supported under the new DIRSIG computational architecture.

  • No Polarization Capabilties

    • RIT currently has no plans to support this modality. The capability in DIRSIG4 was used by a very small fraction of the user population.

Installation Notes

File and Folder Layout

DIRSIG5 is bundled with a copy of the latest DIRSIG4 development release.

The bin folder

This folder contains a full set of the DIRSIG4 programs plus the DIRSIG5 programs (the main executable, the scene2hdf scene compiler, the SQT utility programs, etc.).

The demos folder

This folder contains the standard DIRSIG4 demos.

The docs folder

This folder contains all of the DIRSIG4 documentation and a handful of documents specific to DIRSIG5, including release notes and this document.

The extra folder

This folder contains various "extras" including the Blender plugins, BIN file reader for Matlab and MODTRAN6 wrapper that can be used with Linux and MacOS to resolve shared library location issues.

The lib folder

This folder (not present on Windows) contains the dynamic (shared) libraries that are used by the executables. On Windows, all dynamic libraries (DLLs) are in the bin folder.

The plugins folder

This folder contains all the DIRSIG5 plugins that provide most of the functional utility to the radiometry core. This currently includes sensor models and atmospheric plugins.

Program Wrappers

On the Linux platform, the bin folder contains a BASH shell script wrapper for the DIRSIG5 executables (see d5_wrapper.sh). Rather than require the user to customize their shell environment to setup shared library paths, it is strongly suggested that these wrappers are used, which allows the user to easily run the programs and avoids potential conflicts of the shared libraries we use with those perhaps installed on the host system. The actual executables are named with the version number appended to their filenames (e.g. the DIRSIG5 executable is named dirsig5-5.0.10) and a symbolic link to the d5_wrapper.sh BASH wrapper is created with the desired base name (e.g. dirsig5). This allows the user to execute the dirsig5 command, have the single script setup the environment and then launch the desired application.

Note On the Windows and Mac OSX platform there is no need for a wrapper script. On Windows, the required DLLs are in the bin folder and will be automatically found and used. On Mac OSX, the path to the dynamic libraries is set in the executables to be relative to the bin folder.

Configuration

There is an optional installation configuration file that allows the installation administrator to control some key parameters for all users. These parameters generally control how compute resources are utilized by individual users.

The INI formatted configuration file is called dirsig.ini and needs to be placed in the top of the installation folder (at the same level that the bin, plugins, etc. folders are located). If the file does not exist or a given parameter is not present, a set of default values are used. An example file is shown below:

[General]
max_threads=8
default_threads=4
max_threads

Defines the maximum number of threads that can be started by an individual simulation. The default value is the number of virtual cores present.

default_threads

Defines the default number of threads started by an individual simulation. The default value is the maximum number of threads. This number can be overridden for a simulation by the --threads command-line option. If the number of requested threads exceeds the maximum, an error is issued.

The idea behind the maximum and default thread count parameters is to help the installation administrator manage a compute resource being shared by many users and/or services. For example, you might have a machine with 32 cores and many users. Without setting the maximum thread parameter, a single user could start up a simulation that spawns a massive number of threads, leaving very few resources for other users. The maximum value is to define the maximum number of threads a single user could ever launch from a single simulation.

Note This parameter does not span processes or replace operating system level resource management. A single user could still launch multiple simulations that use additional thread pools.

The default thread count is the number of threads used by any simulation. This value can be less than the maximum if you want to create an environment where users can request more threads if needed (via the --threads command-line option), but by default they use a modest amount.

General Usage

To use the DIRSIG5 software, you will be running simulations from the command-line. In order to run an existing DIRSIG4 simulation, the only additional step required is to convert the scene into a pre-optimized form that the DIRSIG5 model will ingest. This conversion process is referred to as "scene compilation" and it is described below.

Scene Compilation

The scene compilation process was introduced to facilitate faster start-up times with DIRSIG5. With DIRSIG4, the loading of the scene geometry could be a very slow process if the scene was large and contained a lot of geometry. This DIRSIG4 compute time was spent performing sanity checks on the scene geometry, the association of geometry to materials, etc. The realization was that this work was repeated at the start of every simulation, even if the scene had not changed. With DIRSIG5, we replaced this frequent and compute intensive task with an infrequent pre-simulation task. The scene2hdf program included with DIRSIG5 will "compile" a DIRSIG4 scene into a single, binary Hierarchical Data Format (HDF) file that can be quickly loaded and utilized by the DIRSIG5 radiometry simulation. This compilation step performs all of these checks on the geometry and materials once and then allows the DIRSIG5 model to use it many times.

Important The user does not need to compile the scene before every simulation. This process only needs to be repeated if the contents of the scene have been modified.

To run the scene2hdf scene compiler, you provide the name of the scene file to the program:

$ scene2hdf my_site.scene
Note Please consult the scene2hdf manual to discover and understand all the options to the scene compiler.

The scene compiler will then begin the process of reading and checking all the geometry. It will then produce a file with the same name as the input .scene file but with .hdf appended to it. From our example above, the name of the final scene HDF file would be my_site.scene.hdf.

Important The HDF5 library expects the filesystem to support file locking to avoid corruption introduced by simultaneous read/write commands. If your filesystem doesn’t support file locking, then the HDF5 library will issue an error. The user can disable HDF5 file locking by setting the environment variable HDF5_USE_FILE_LOCKING to FALSE.

When the tool completes, it will print a series of warnings and (potentially) errors. Most warnings are for currently unsupported features that the tool can work around. Errors arise in situations where the tool cannot find a reasonable work around. Please consult the release notes document for each beta release for discussions of currently unsupported features and previously unsupported features that are now supported.

Running a Simulation

To run the simulation, the user simply provides the name of the DIRSIG4 .sim file to the DIRSIG5 program. The usage summary for the DIRSIG5 program is shown below:

DIRSIG5
Release: 5.0.10 (r2733M)
Build Date: Mar 22 2019 16:47:12
Copyright 2015-2019 Rochester Institute of Technology

usage: dirsig5 [options] <sim filename>

basic [options]:
  --version
    Display version info
  --help
    Display this usage message
  --licenses
    Display licenses for third party code used in DIRSIG
  --threads=N
    Set the maximum number of threads to 'N' (default:<max>
  --convergence=A,B,C
    Set min samples to 'A', max samples to 'B' and threshold to 'C' (default:20,100,1e-6)
  --max_nodes=N
    Set the maximum number of nodes per path to 'N' (default:4)
  --disable_earth_core
    Allow rays to pass through the Earth rather than hit the core

BasicPlatform [options]:
  --output_folder=<foldername>
    Specify the folder for output data files
  --output_prefix=<prefix>
    Specify the prefix for output data files
  --split_channels={true,false}
    Use a unique spectral state for each channel

Controlling Multi-Threading

The software will detect the number of virtual cores (Intel Hyperthreading efficiently allows more than 1 thread to run on each physical core) and will automatically try to assign a thread to each. That will yield the highest utilization of the host computer’s resources, but in a shared computing environment this might not be desired. Hence, the user can specify the number of threads to start via the --threads option. For example, to use only 4 threads:

$ dirsig5 --threads=4 example.sim

Output File and Folder Options

To support the output file and folder options available in DIRSIG4, the DIRSIG5 model has the same --output_folder=<foldername> and --output_prefix=<prefix> command-line options.

Controlling Fidelity

In DIRSIG5, the fidelity (quality) of the simulation is driven by how many paths are used in each pixel and the number of nodes (bounces) in those paths. The current algorithm supports an adaptive sampling scheme (similar to that employed in DIRSIG4) to vary the number of samples used in each pixel based on the complexity of the content within that pixel. A pixel with a larger dynamic range, more sub-pixel materials, etc. can use more samples and a pixel imaging a uniform, low variability surface can user fewer samples.

The adaptive pixel sampling algorithm has three parameters: (1) the minimum number of paths, the (2) the maximum number of paths and (3) the threshold for determining if the solution has converged.

  • The minimum number of samples specifies the minimum number of paths used in a pixel.

    • "Why wouldn’t I want this to be 1?"

      • Because the Monte-Carlo integral of a complex function with 1 sample will be poor. This will result in an incredibly very noisy image.

    • "What is the default?"

      • The default value is currently 20, which produces images that are decent quality as long as the illumination is generally high.

    • "Why would I increase the minimum number of samples?"

      • With only 20 samples, you might still get noise in the shadows because 20 wasn’t enough to detect the large dynamic range within a complex illumination environment (for example, shadows under a tree canopy).

  • The maximum number of samples specifies the maximum number of paths used in a pixel.

    • In general, this is the primary "quality" knob.

      • Turn it up to get more samples and drive down noise in the Monte-Carlo integration.

    • "What is the default?"

      • The default value is currently 100, which produces images that are decent quality as long as the illumination is generally high.

    • "Why would I increase the maximum number of samples?"

      • In general, it helps areas that are noisy. For example, shadow regions with complex illumination.

  • The threshold at which additional paths will no longer be add to a pixel.

    • Computed using the average radiance at the reference wavelength

    • This value is in units of radiance - it is not a fraction!

      • This value should be selected in terms of radiometry and not to make something pleasing to the eye.

      • Long term we expect the sensor model to define this value in terms of a minimal detectable radiance.

  • All three parameters are related:

    • At one extreme, if the threshold is very low (for example, 0) then each pixel will use the maximum number of paths.

    • At the other extreme, if the threshold is very high (for example, 1e100) then each pixel will use the minimum number of paths.

The forth key parameter is the number of nodes (bounces) along each path. This is controlled via the --max_nodes option. The default number of nodes is 4, but the user can increase it for highly scattering scenarios.

A summary of suggested convergence and node parameter configurations is listed in the table below.

Table 1. A table of suggested convergence and node count values for different simulations.
Mode Min Paths/Pixel Max Paths/Pixel Radiance Threshold Max Nodes

Default

20

100

1.0e-06

4

Better

30

500

1.0e-06

5

Even better

50

1000

1.0e-06

5

Night

30

500

1.0e-12

5

Preview

3

3

0.0

4

Tip The user can emulate the DIRSIG4 "preview mode" by supplying the -preview option to DIRSIG5. However unlike DIRSIG4, which would skip radiance calculations in this mode, the DIRSIG5 simulation will produce a low-fidelity radiance solution.
Tip At night the radiance values are much lower, so the radiance threshold needs to be decreased in order to match the magnitude range.

Using the Parallel/MPI version

Details about using the MPI-enabled version of DIRSIG is described in a supplementary manual.

DIRSIG4 Compatibility Plugins

DIRSIG5 is able to run DIRSIG4 simulations by leveraging a pair of plugins that provide backward compatibility with DIRSIG4 intput files.

The BasicAtmosphere Plugin

The BasicAtmosphere plugin handles the atmosphere modeling by using the DIRSIG4 era .atm files.

The BasicPlatform Plugin

The BasicPlatform plugin handles the sensor modeling by using the DIRSIG4 era platform, motion (either a GenericMotion or a FlexMotion description) and a tasks file.

Alternative, Plugin-driven Features

The DIRSIG5 model includes several alternative execution modes by leveraging different plugins.