OutputPathGenerator

The OutputPathGenerator module provides tools for preparing the directory structure for your simulation output. This helps you organize your simulation results efficiently and avoid overwriting existing data.

The module offers one function, generate_output_path. The function takes three arguments:

  • output_path: The base directory path for your simulation output.
  • style (Optional): The desired style for output management (defaults to ActiveLinkStyle).
  • context (Optional): the ClimaComms.context. This is required in MPI runs to ensure that all the MPI processes agree on the folder structure.

The function processes the output_path based on the chosen style and returns the final path where you should write your simulation output.

You should use generate_output_path at the beginning of your simulation and use the return value as the base directory where you save all the output your code produces.

Available Styles

The module currently offers two different styles for handling the output directory:

RemovePreexistingStyle (Destructive)

This style directly uses the provided output_path as the final output directory. Important: If a directory already exists at the specified path, it will be removed completely (including any subfolders and files) without confirmation. Use this style cautiously!

ActiveLinkStyle (Non-Destructive)

This style provides a more convenient and non-destructive approach. It manages a sequence of subfolders within the base directory specified by outputpath. It also creates a symbolic link named `outputactive` that points to the current active subfolder. This allows you to easily access the latest simulation results.

Example

Let's assume your output_path is set to data.

  • If data doesn't exist, the module creates it and returns data/output_active. This link points to the newly created subfolder data/output_0000.
  • If data exists and contains an output_active link pointing to data/output_0005, the module creates a new subfolder data/output_0006 and updates output_active to point to it.
  • If data exists with or without an output_active link, the module checks for existing subfolders named data/output_XXXX (with XXXX a number). If none are found, it creates data/output_0000 and a link data/output_active pointing to it.

API

ClimaUtilities.OutputPathGenerator.generate_output_pathFunction
generate_output_path(output_path,
                     context = nothing,
                     style::OutputPathGeneratorStyle = ActiveLinkStyle())

Process the output_path and return a string with the path where to write the output.

The context is a ClimaComms context and is required for MPI runs.

How the output should be structured (in terms of directory tree) is determined by the style.

Styles

  • RemovePreexistingStyle: the output_path provided is the actual output path. If a directory already exists there, remove it without asking for confirmation.

  • ActiveLinkStyle: the output_path provided is output_path/output_active, a link to a subdirectory within output_path. This style looks at the content of output_path and adds new subdirectories. The added directories are named with a counter, with the latest always accessible via the output_path/output_active symlink. This is style is non-destructive.

(Note, "styles" have nothing to do with traits.)

generate_output_path(::RemovePreexistingStyle, output_path, context = nothing)

Documentation for this function is in the RemovePreexistingStyle struct.

generate_output_path(::ActiveLinkStyle, output_path, context = nothing)

Documentation for this function is in the ActiveLinkStyle struct.

ClimaUtilities.OutputPathGenerator.ActiveLinkStyleType
ActiveLinkStyle

This style generates a unique output path within a base directory specified by output_path. It ensures the base directory exists and creates it if necessary. Additionally, it manages a sequence of subfolders and a symbolic link named "output_active" for convenient access to the active output location.

Examples:

Let us assume that output_path = dormouse.

  • dormouse does not exist in the current working directory: ActiveLinkStyle will create it and return dormouse/output_active. dormouse/output_active is a symlink that points to the newly created dormouse/output_0000 directory.
  • dormouse exists and contains a output_active link that points to dormouse/output_0005, ActiveLinkStyle will a new directory dormouse/output_0006 and change the output_active to point to this directory.
  • dormouse exists and does not contain a output_active, ActiveLinkStyle will check if any dormouse/output_XXXX exists. If not, it creates dormouse/output_0000 and a link dormouse/output_active that points to this directory.