AbstractToolchain

An AbstractToolchain represents a set of JLLs that should be downloaded to provide some kind of build capability; an example of which is the C toolchain which is used in almost every recipe, but fortran, go, rust, etc.. are all other toolchains which can be included in the build environment.

All toolchains must define the following methods:

• Constructor
  • used to configure tool versions, etc...
• toolchain_sources(toolchain)
  • returns a vector of AbstractSource's representing the dependencies needed to run this toolchain
• toolchainenv(toolchain, deployedprefix::String)
  • returns a dictionary listing the environment variables to be added in to commands that use this toolchain.
• platform(toolchain)
  • returns the platform this toolchain was constructed for. Could be a CrossPlatform (in the case of CToolchain) or a plain Platform (in the case of HostToolsToolchain).

FlagString

Represents a flag that we are going to test against when running the compiler wrapper bash script. Use with the @flag_str macro, like so:

f_asm = flag"-x assembler"

Can use regular expression matching if the r flag macro flag is included:

f_march = flag"-march=.*"r

HostToolsToolchain

This toolchain contains a large number of useful host tools, such as

append_flags(io::IO, flag_type::Symbol, flags::String)

We are often building up the PRE_FLAGS and POST_FLAGS arrays in our bash compiler wrappers; this makes it easy to append a bunch of values to those bash arrays without worrying about quoting, spelling the name of the array right, etc...

Use it like so:

clang_link_only_flags = String["-rtlib=libgcc", "-fuse-ld=x86_64-linux-gnu"]
flagmatch([!flag"-c", !flag"-E", !flag"-M", !flag"-fsyntax-only"]) do io
    append_flags(io, :POST, clang_link_only_flags)
end

To get a bash wrapper script that looks like:

if [[ " ${ARGS[@]} " != *' -c '* ]] && [[ " ${ARGS[@]} " != *' -E '* ]] &&
   [[ " ${ARGS[@]} " != *' -M '* ]] && [[ " ${ARGS[@]} " != *' -fsyntax-only '* ]]; then
    POST+=( '-rtlib=libgcc' 'fuse-ld=x86_64-linux-gnu' )
fi

clang_wrappers(toolchain::CToolchain, dir::String)

Generate wrapper scripts (using compiler_wrapper()) into dir to launch tools like gcc, g++, etc... from GCC_jll with appropriate flags interposed.

compile_flagmatch(f, io)

Convenience function for CToolchain wrappers, using flagmatch() to match only when a compiler invocation is performing compilation. As of this writing, this only excludes the case where clang has been invoked as an assembler via the -x assembler flag.

compiler_wrapper(f::Function, io::IO, prog::String)

This utility function allows the automated construction of bash wrapper scripts for compiler executables. It allows for easy composition of conditionals to insert compiler flags using the helper methods flagmatch and append_flags.

An intermediate example of writing a wrapper for clang for x86_64-linux-gnu:

compiler_wrapper("clang_wrapper.sh", "clang") do io
    append_flags(io, :PRE, [
        # Set the `target` for `clang` so it generates the right kind of code
        "--target=x86_64-linux-gnu",
        # Set the sysroot
        "--sysroot=/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root",
        # Set the GCC toolchain location
        "--gcc-toolchain=/opt/x86_64-linux-gnu",
    ])

    # If linking is involved, ensure that we link against libgcc and use the right linker
    clang_link_only_flags = String["-rtlib=libgcc", "-fuse-ld=x86_64-linux-gnu"]
    flagmatch(io, [!flag"-c", !flag"-E", !flag"-M", !flag"-fsyntax-only"]) do io
        append_flags(io, :POST, clang_link_only_flags)
    end
end

flagmatch(f::Function, flags::Vector{FlagString})

Used to construct strings of bash snippets for use in creation of compiler wrappers; an invocation such as:

io = IOBuffer()
flagmatch(io, [flag"-march=.*"r]) do fio
    println(fio, "die 'Cannot force an architecture via -march'")
end
println(String(take!(io)))

Will result in a string that looks like:

if [[ " ${ARGS[@]} " == *' -march= '* ]]; then
    die 'Cannot force an architecture via -march'
fi

Multiple flags passed into flagmatch() result in multiple conditionals, combined via the && bash operator.

gcc_wrappers(toolchain::CToolchain, dir::String)

Generate wrapper scripts (using compiler_wrapper()) into dir to launch tools like gcc, g++, etc... from GCC_jll with appropriate flags interposed. Typically only generates wrappers with target triplets suffixed, however if toolchain.default_ctoolchain is set, also generates the generic wrapper names cc, gcc, c++, etc...

insert_PATH!(env::Dict, pre_post::Symbol, new_paths::Vector{String})

Given a set of new elements to put onto PATH, append them either at the end of the given path (pre_post == :POST) or at the beginning (pre_post == :PRE).

link_flagmatch(f, io)

Convenience function for CToolchain wrappers, using flagmatch() to match only when a compiler invocation is performing linking. This excludes the cases where the compiler has been invoked to preprocess, compile without linking, act as an assembler, etc...

path_appending_merge(env, others...; appending_keys)

Merges env dictionaries, but intelligently merges entries that correspond to PATH-like lists, such as PATH, LD_LIBRARY_PATH, etc... as denoted in appending_keys. Defaults to default_appending_keys.

resolve_versions!(jlls::Vector{JLLSource}; julia_version)

Given a vector of JLLSource's, resolve any that are not already concretized down to a specific version,

runshell(toolchains::Vector{AbstractToolchain})

Convenience function to launch a shell with the generated wrapper scripts for the given toolchains

update_pkgspec_versions_and_uuids!(pkgs::Vector{PackageSpec}, new_versions::Dict{String,PackageSpec})

Allows updating (in place!) a set of PackageSpecs with new versions and UUIDS. Used to update the internal PackageSpec objects of JLLSources with new results from resolve_pkg_versions().

with_toolchains(f::Function, toolchains::Vector{AbstractToolchain};
                env::Dict{String,String} = ENV,
                deploy_dir::String = mktempdir(),
                verbose::Bool = false)

Call f(prefix, env) with the given toolchains deployed and ready to go within deploy_root. Use this to quickley set up the given toolchains for usage with run() commands as follows:

with_toolchains(toolchains) do prefix, env
    cd(build_path) do
        run(addenv(`make install`, env))
    end
end

The prefix is given in the event that you want to do something with the deployed toolchains, however most users will not need to use it for anything. By default, the commands will inherit the current environment, set the env keyword argument to prevent this. Note that a few known conflicting environment variables are automatically filtered out from the external environment, see filter_env_vars() for more detail.