NCTiles.jl Package Documentation

NCTiles.jl reads and writes NetCDF files that represent e.g. subdivisions of Earth's surface (tiles). Inter-operability with popular climate model grids and MeshArrays.jl and generation of CF-compliant files are key goals of this package.

Contents

NCTiles.jl derives from the earlier nctiles implementation in gcmfaces (Forget et al. 2015).

Main Features

NCTiles.jl first goes through lazy operations, on data stucture, as it obtains information about variables etc. The second phase calls createfile and then the add* functions to instantiate files. Note: some of the included functions are interfaces to MITgcm output.

Data structures:

  • NCData contains a string or an array of strings (NetCDF file names) + metadata to read files.
  • NCvar contains information needed to write a NetCDF file which can include a list of filenames (see Bindata) if the data is not loaded into memory.
  • BinData is a container for one field.
  • TileData contains a MeshArray or BinData struct in vals, information about the tile layout in tileinfo, and metadata needed for reading/writing tile data.

As an example:

struct TileData{T}
    vals::T
    tileinfo::Dict
    tilesize::Tuple
    precision::Type
    numtiles::Int
end

Use examples

DataStructures/06_nctiles.ipynb in this GlobalOceanNotebooks repo provides a series of examples that overlap somewhat with the ones found in Examples/ex*.jl:

  • ex_1.jl reads a binary file containing one interpolated 2D field on a regular grid. It then writes that array to a NetCDF/NCTiles file.
  • ex_2.jl reads data from a NetCDF file containing one tile of model output. It then writes it to a new NetCDF/NCTiles file. This uses 3D data on a non-regular grid for one ocean subdivision (tile).
  • ex_3.jl is an example of interpolated model output processing in CBIOMES where several variables are included in the same NetCDF/NCTiles file.
  • ex_4.jl generates a tiled netcdf output (i.e., a nctiles output) for a global 2D field on the non-regular LLC90 grid (see MeshArrays.jl). Since the tile width is set to 90, this creates 13 files.

Index

API / Functions

NCTiles.BinData — Type
BinData

Data structure containing a string or an array of strings (NetCDF file names) as well as metadata needed to read a file.

NCTiles.BinData — Method
BinData(fnames::Union{Array{String},String},precision::Type,iosize::Tuple)

Construct a BinData struct for files that contain one field.

NCTiles.NCData — Type
NCData

Data structure containing a string or an array of strings (file names) of NetCDF files as well as information needed to read a file.

NCTiles.NCvar — Type
NCvar

Data structure containing information needed to write a NetCDF file. This includes a list of filenames (see Bindata) if the data is not loaded into memory.

NCTiles.TileData — Type
TileData{T}

Data structure containing either a MeshArray struct or BinData struct (see vals), MeshArray structs describing the tile layout (tileinfo), and other information for reading/writing tile data.

NCTiles.TileData — Method
TileData(vals,tilesize::Tuple)

Construct a TileData struct. First generate the tileinfo, precision, and numtiles attributes.

Base.write — Method
write(myflds::Dict,savename::String;README="",globalattribs=Dict())

Creates NetCDF file and writes myflds and all their dimensions to the file.

Base.write — Method
write(myfld::NCvar,savename::String;README="",globalattribs=Dict())

Creates NetCDF file and writes myfld and all its dimensions to the file.

NCTiles.addData — Method
addData(v::Union{NCDatasets.CFVariable,NetCDF.NcVar},var::NCvar)

Fill variable with data in netcdf file using either NCDatasets.jl or NetCDF.jl

NCTiles.addDim — Method
addDim(ds::NCDatasets.Dataset,dimvar::NCvar) # NCDatasets

Add a dimension to a NCDatasets.Dataset

NCTiles.addDim — Method
addDim(dimvar::NCvar)

Add a dimension to a NetCDF file using NetCDF.jl

NCTiles.addDimData — Method
addDimData(v::Union{NCDatasets.CFVariable,NetCDF.NcVar,Array},var::NCvar)

Add dimension data to predefined dimensions in a NetCDF file.

NCTiles.addVar — Method
addVar(ds::NCDatasets.Dataset,field::NCvar)

Add a variable to a NetCDF file using NCDatasets.jl

NCTiles.addVar — Method
addVar(field::NCvar,dimlist::Array{NetCDF.NcDim})

Add a variable with dimensions dimlist to a NetCDF file using NetCDF.jl

NCTiles.addVar — Method
addVar(field::NCvar})

Add a variable and its dimensions to a NetCDF file using NetCDF.jl

NCTiles.createfile — Function
createfile(filename, field::Union{NCvar,Dict{String,NCvar}}, README;
            fillval=NaN, missval=NaN, itile=1, ntile=1)

Create NetCDF file and add variable + dimension definitions using either NCDatasets.jl or NetCDF.jl

NCTiles.readbin — Function
readbin(fname::String,prec::Type,iosize::Tuple,fldidx=1)

Read in a binary file to an Array.

NCTiles.readbin — Function
readbin(flddata::BinData,tidx=1)

Read in a binary file as an array as specified by BinData

NCTiles.readncfile — Function
readncfile(fname,backend::Module=NCDatasets)

Read in a NetCDF file and return variables/dimensions as NCvar structs, and file attributes as Dict. Large variables/dimensions are not loaded into memory. This can use either NCDatasets.jl or NetCDF.jl