Julia Data Format - JLD2
JLD2 saves and loads Julia data structures in a format comprising a subset of HDF5, without any dependency on the HDF5 C library. It typically outperforms the JLD package (sometimes by multiple orders of magnitude) and often outperforms Julia's built-in serializer. While other HDF5 implementations supporting HDF5 File Format Specification Version 3.0 (i.e. libhdf5 1.10 or later) should be able to read the files that JLD2 produces, JLD2 is likely to be incapable of reading files created or modified by other HDF5 implementations. JLD2 does not aim to be backwards or forwards compatible with the JLD package.
Reading and writing data
@save
and @load
macros
The @save
and @load
macros are the simplest way to interact with a JLD2 file. The @save
macro writes one or more variables from the current scope to the JLD2 file. For example:
using JLD2
hello = "world"
foo = :bar
@save "example.jld2" hello foo
This writes the variables hello
and foo
to datasets in a new JLD2 file named example.jld2
. The @load
macro loads variables out of a JLD2 file:
@load "example.jld2" hello foo
This assigns the contents of the hello
and foo
datasets to variables of the same name in the current scope.
It is best practice to explicitly name the variables to be loaded and saved from a file, so that it is clear from whence these variables arise. However, for convenience, JLD2 also provides variants of @load
and @save
that do not require variables to be named explicitly. When called with no variable arguments, @save <filename>
writes all variables in the global scope of the current module to file <filename>
, while @load <filename>
loads all variables in file <filename>
. When called with no variable arguments, @load
requires that the file name is provided as a string literal, i.e., it is not possible to select the file at runtime.
Additional customization is possible using assignment syntax and option passing:
@save "example.jld2" bye=hello bar=foo
@save "example.jld2" {compress=true} hello bar=foo
JLD2.@save
— Macro@save filename var1 [var2 ...]
@save filename {compress=true} var1 name2=var2
Write one or more variables var1,...
from the current scope to a JLD2 file filename
.
For interactive use you can save all variables in the current module's global scope using @save filename
. More permanent code should prefer the explicit form to avoid saving unwanted variables.
Example
To save the string hello
and array xs
to the JLD2 file example.jld2:
hello = "world"
xs = [1,2,3]
@save "example.jld2" hello xs
For passing options to the saving command use {}
@save "example.jld2" {compress=true} hello xs
For saving variables under a different name use regular assignment syntax
@save "example.jld2" greeting=hello xarray = xs
JLD2.@load
— Macro@load filename var1 [var2 ...]
Load one or more variables var1,...
from JLD2 file filename
into the current scope and return a vector of the loaded variable names.
For interactive use, the form @load "somefile.jld2"
will load all variables from "somefile.jld2"
into the current scope. This form only supports literal file names and should be avoided in more permanent code so that it's clear where the variables come from.
Example
To load the variables hello
and foo
from the file example.jld2, use
@load "example.jld2" hello foo
save
and load
functions
The save
and load
functions, provided by FileIO, provide an alternative mechanism to read and write data from a JLD2 file. To use these functions, you must say using FileIO
; it is not necessary to say using JLD2
since FileIO will determine the correct package automatically.
The save
function accepts an AbstractDict
yielding the key/value pairs, where the key is a string representing the name of the dataset and the value represents its contents:
using FileIO
save("example.jld2", Dict("hello" => "world", "foo" => :bar))
The save
function can also accept the dataset names and contents as arguments:
save("example.jld2", "hello", "world", "foo", :bar)
When using the save
function, the file extension must be .jld2
, since the extension .jld
currently belongs to the previous JLD package.
If called with a filename argument only, the load
function loads all datasets from the given file into a Dict:
load("example.jld2") # -> Dict{String,Any}("hello" => "world", "foo" => :bar)
If called with a single dataset name, load
returns the contents of that dataset from the file:
load("example.jld2", "hello") # -> "world"
If called with multiple dataset names, load
returns the contents of the given datasets as a tuple:
load("example.jld2", "hello", "foo") # -> ("world", :bar)
File interface
It is also possible to interact with JLD2 files using a file-like interface. The jldopen
function accepts a file name and an argument specifying how the file should be opened:
using JLD2
f = jldopen("example.jld2", "r") # open read-only (default)
f = jldopen("example.jld2", "r+") # open read/write, failing if no file exists
f = jldopen("example.jld2", "w") # open read/write, overwriting existing file
f = jldopen("example.jld2", "a+") # open read/write, preserving contents of existing file or creating a new file
Data can be written to the file using write(f, "name", data)
or f["name"] = data
, or read from the file using read(f, "name")
or f["name"]
. When you are done with the file, remember to call close(f)
.
Like open
, jldopen
also accepts a function as the first argument, permitting do
-block syntax:
jldopen("example.jld2", "w") do file
file["bigdata"] = randn(5)
end
Groups
It is possible to construct groups within a JLD2 file, which may or may not be useful for organizing your data. You can create groups explicitly:
jldopen("example.jld2", "w") do file
mygroup = JLD2.Group(file, "mygroup")
mygroup["mystuff"] = 42
end
or implicitly, by saving a variable with a name containing slashes as path delimiters:
jldopen("example.jld2", "w") do file
file["mygroup/mystuff"] = 42
end
# or save("example.jld2", "mygroup/mystuff", 42)
Both of these examples yield the same group structure, which you can see at the REPL:
julia> file = jldopen("example.jld2", "r")
JLDFile /Users/simon/example.jld2 (read-only)
└─📂 mygroup
└─🔢 mystuff
Similarly, you can access groups directly:
jldopen("example.jld2", "r") do file
@assert file["mygroup"]["mystuff"] == 42
end
or using slashes as path delimiters:
@assert load("example.jld2", "mygroup/mystuff") == 42