The Julia module PythonCall

Installation

This package is in the general registry, so to install just type ] in the Julia REPL and run:

pkg> add PythonCall

Getting started

Import the module with:

julia> using PythonCall

By default this will initialize a conda environment in your Julia environment, install Python into it, load the corresponding Python library and initialize an interpreter.

Now you can interact with Python as follows:

julia> re = pyimport("re")
Python module: <module 're' from '[...]/lib/re.py'>

julia> words = re.findall("[a-zA-Z]+", "PythonCall.jl is very useful!")
Python list: ['PythonCall', 'jl', 'is', 'very', 'useful']

julia> sentence = Py(" ").join(words)
Python str: 'PythonCall jl is very useful'

julia> pyconvert(String, sentence)
"PythonCall jl is very useful"

In this example:

  • We used pyimport to import the re module. Equivalently we could have done @py import re (see @py).
  • We called its findall function on a pair of strings, which were automatically converted to Python strings (see Conversion to Python).
  • We called Py to explicitly convert a string to a Python string, so that we could call its join method. All Python objects are of type Py.
  • We called pyconvert to convert the Python string sentence to a Julia string (see Conversion to Julia).

What to read next:

  • The rest of this page details the functions for interacting with Python objects, of type Py.
  • If you need to install Python packages, read here.
  • When you call a Python function, such as re.findall(...) in the above example, its arguments are converted to Python according to this table and its return value is a Py.
  • Python objects can be converted to Julia objects using pyconvert with rules according to this table.
  • Python objects can also be wrapped to provide more Julian semantics. For example, a PyDict wraps a Python dict as a Julia dict, and a PyArray wraps a Python array or buffer as a Julia array. See here.

Py

PythonCall.@pyconst — Macro
@pyconst ex

Equivalent to Py(ex) but always returns the exact same Julia object.

That is, if foo() = @pyconst ex then foo() === foo().

The expression ex is evaluated the first time the code is run.

If ex is a string literal, the string is interned.

Do not use this macro at the top level of a module. Instead, use pynew() and pycopy!().

The object pybuiltins has all the standard Python builtin objects as its properties. Hence you can access pybuiltins.None and pybuiltins.TypeError.

@py

PythonCall.@py — Macro
@py expr

Evaluate the given expression using Pythonic semantics.

For example:

  • f(x, y) is translated to pycall(f, x, y)
  • x + y is translated to pyadd(x, y)
  • x === y is translated to pyis(x, y)
  • x.foo is translated to pygetattr(x, "foo")

Compound statements such as begin, if, while and for are supported.

See the online documentation for more details.

Python functions

Most of the functions in this section are essentially Python builtins with a py prefix. For example pyint(x) converts x to a Python int and is equivalent to int(x) in Python when x is a Python object.

Notable exceptions are:

  • pyconvert to convert a Python object to a Julia object.
  • pyimport to import a Python module.
  • pyjl to directly wrap a Julia object as a Python object.
  • pyclass to construct a new class.
  • pywith to emulate the Python with statement.

If a Julia value is passed as an argument to one of these functions, it is converted to a Python value using the rules documented here.

Constructors

These functions construct Python objects of builtin types from Julia values.

PythonCall.pycomplex — Function
pycomplex(x=0.0)
pycomplex(re, im)

Convert x to a Python complex, or create one from given real and imaginary parts.

PythonCall.pytuple — Function
pytuple(x=())

Convert x to a Python tuple.

If x is a Python object, this is equivalent to tuple(x) in Python. Otherwise x must be iterable.

PythonCall.pylist — Function
pylist(x=())

Convert x to a Python list.

If x is a Python object, this is equivalent to list(x) in Python. Otherwise x must be iterable.

PythonCall.pycollist — Function
pycollist(x::AbstractArray)

Create a nested Python list-of-lists from the elements of x. For matrices, this is a list of columns.

PythonCall.pyrowlist — Function
pyrowlist(x::AbstractArray)

Create a nested Python list-of-lists from the elements of x. For matrices, this is a list of rows.

PythonCall.pyset — Function
pyset(x=())

Convert x to a Python set.

If x is a Python object, this is equivalent to set(x) in Python. Otherwise x must be iterable.

PythonCall.pyfrozenset — Function
pyfrozenset(x=())

Convert x to a Python frozenset.

If x is a Python object, this is equivalent to frozenset(x) in Python. Otherwise x must be iterable.

PythonCall.pydict — Function
pydict(x)
pydict(; x...)

Convert x to a Python dict. In the second form, the keys are strings.

If x is a Python object, this is equivalent to dict(x) in Python. Otherwise x must iterate over key-value pairs.

PythonCall.pyslice — Function
pyslice([start], stop, [step])

Construct a Python slice. Unspecified arguments default to None.

PythonCall.pyrange — Function
pyrange([[start], [stop]], [step])

Construct a Python range. Unspecified arguments default to None.

PythonCall.pyclass — Function
pyclass(name, bases=(); members...)

Construct a new Python type with the given name, bases and members.

Equivalent to pytype(name, bases, members).

Builtins

These functions mimic the Python builtin functions or keywords of the same name.

PythonCall.pyimport — Function
pyimport(m)
pyimport(m => k)
pyimport(m => (k1, k2, ...))
pyimport(m1, m2, ...)

Import a module m, or an attribute k, or a tuple of attributes.

If several arguments are given, return the results of importing each one in a tuple.

PythonCall.pywith — Function
pywith(f, o, d=nothing)

Equivalent to with o as x: f(x) in Python, where x is a Py.

On success, the value of f(x) is returned.

If an exception occurs but is suppressed then d is returned.

PythonCall.pyis — Function
pyis(x, y)

True if x and y are the same Python object. Equivalent to x is y in Python.

PythonCall.pygetattr — Function
pygetattr(x, k, [d])

Equivalent to getattr(x, k) or x.k in Python.

If d is specified, it is returned if the attribute does not exist.

PythonCall.pysetattr — Function
pysetattr(x, k, v)

Equivalent to setattr(x, k, v) or x.k = v in Python.

PythonCall.pycall — Function
pycall(f, args...; kwargs...)

Call the Python object f with the given arguments.

PythonCall.pylen — Function
pylen(x)

The length of x. Equivalent to len(x) in Python, converted to an Integer.

PythonCall.pysetitem — Function
pysetitem(x, k, v)

Equivalent to setitem(x, k, v) or x[k] = v in Python.

PythonCall.pyissubclass — Function
pyissubclass(s, t)

Test if s is a subclass of t. Equivalent to issubclass(s, t) in Python.

PythonCall.pyisinstance — Function
pyisinstance(x, t)

Test if x is of type t. Equivalent to isinstance(x, t) in Python.

PythonCall.pyhash — Function
pyhash(x)

Equivalent to hash(x) in Python, converted to an Integer.

PythonCall.pyeval — Function
pyeval([T=Py], code, globals, locals=nothing)

Evaluate the given Python code, returning the result as a T.

If globals is a Module, then a persistent dict unique to that module is used.

By default the code runs in global scope (i.e. locals===globals). To use a temporary local scope, set locals to (), or to a NamedTuple of variables to include in the scope.

See also @pyeval.

Examples

The following computes 1.1+2.2 in the Main module as a Float64:

pyeval(Float64, "x+y", Main, (x=1.1, y=2.2))  # returns 3.3
PythonCall.pyexec — Function
pyexec([T=Nothing], code, globals, locals=nothing)

Execute the given Python code.

If globals is a Module, then a persistent dict unique to that module is used.

By default the code runs in global scope (i.e. locals===globals). To use a temporary local scope, set locals to (), or to a NamedTuple of variables to include in the scope.

If T==Nothing then returns nothing. Otherwise T must be a concrete NamedTuple type and the corresponding items from locals are extracted and returned.

See also @pyexec.

Examples

The following computes 1.1+2.2 in the Main module as a Float64:

pyexec(@NamedTuple{ans::Float64}, "ans=x+y", Main, (x=1.1, y=2.2))  # returns (ans = 3.3,)

Marking variables as global saves them into the module scope, so that they are available in subsequent invocations:

pyexec("global x; x=12", Main)
pyeval(Int, "x", Main)  # returns 12
PythonCall.@pyeval — Macro
@pyeval [inputs =>] code [=> T]

Evaluate the given code in a new local scope and return the answer as a T.

The global scope is persistent and unique to the current module.

The code must be a literal string or command.

The inputs is a tuple of inputs of the form v=expr to be included in the local scope. Only v is required, expr defaults to v.

Examples

The following computes 1.1+2.2 and returns a Float64:

@pyeval (x=1.1, y=2.2) => `x+y` => Float64  # returns 3.3
PythonCall.@pyexec — Macro
@pyexec [inputs =>] code [=> outputs]

Execute the given code in a new local scope.

The global scope is persistent and unique to the current module.

The code must be a literal string or command.

The inputs is a tuple of inputs of the form v=expr to be included in the local scope. Only v is required, expr defaults to v.

The outputs is a tuple of outputs of the form x::T=v, meaning that v is extracted from locals, converted to T and assigned to x. Only x is required: T defaults to Py and v defaults to x.

Examples

The following computes 1.1+2.2 and assigns its value to ans as a Float64:

@pyexec (x=1.1, y=2.2) => `ans=x+y` => ans::Float64  # returns 3.3

Marking variables as global saves them into the module scope, so that they are available in subsequent invocations:

@pyexec `global x; x=12`
@pyeval `x` => Int  # returns 12

Conversion to Julia

These functions convert Python values to Julia values, using the rules documented here.

PythonCall.pyconvert — Function
pyconvert(T, x, [d])

Convert the Python object x to a T.

If d is specified, it is returned on failure instead of throwing an error.

PythonCall.@pyconvert — Macro
@pyconvert(T, x, [onfail])

Convert the Python object x to a T.

On failure, evaluates to onfail, which defaults to return pyconvert_unconverted() (mainly useful for writing conversion rules).

Wrap Julia values

These functions explicitly wrap Julia values into Python objects, documented here.

As documented here, Julia values are wrapped like this automatically on conversion to Python, unless the value is immutable and has a corresponding Python type.

PythonCall.pyjl — Function
pyjl([t], x)

Create a Python object wrapping the Julia object x.

If x is mutable, then mutating the returned object also mutates x, and vice versa.

Its Python type is normally inferred from the type of x, but can be specified with t.

For example if x is an AbstractVector then the object will have type juliacall.VectorValue. This object will satisfy the Python sequence interface, so for example uses 0-up indexing.

PythonCall.pyjlraw — Function
pyjlraw(v)

Create a Python object wrapping the Julia object x.

It has type juliacall.RawValue. This has a much more rigid "Julian" interface than pyjl(v). For example, accessing attributes or calling this object will always return a RawValue.

PythonCall.pyisjl — Function
pyisjl(x)

Test whether x is a wrapped Julia value, namely an instance of juliacall.ValueBase.

PythonCall.pybinaryio — Function
pybinaryio(io::IO)

Wrap io as a Python binary IO object.

This is the default behaviour of Py(io).

Arithmetic

These functions are equivalent to the corresponding Python arithmetic operators.

Note that the equivalent Julia operators are overloaded to call these when all arguments are Py (or Number). Hence the following are equivalent: Py(1)+Py(2), Py(1)+2, pyadd(1, 2), pyadd(Py(1), Py(2)), etc.

PythonCall.pypow — Function
pypow(x, y, z=None)

Equivalent to x ** y or pow(x, y, z) in Python.

PythonCall.pyiadd — Function
pyiadd(x, y)

In-place add. x = pyiadd(x, y) is equivalent to x += y in Python.

PythonCall.pyisub — Function
pyisub(x, y)

In-place subtract. x = pyisub(x, y) is equivalent to x -= y in Python.

PythonCall.pyimul — Function
pyimul(x, y)

In-place multiply. x = pyimul(x, y) is equivalent to x *= y in Python.

PythonCall.pyimatmul — Function
pyimatmul(x, y)

In-place matrix multiply. x = pyimatmul(x, y) is equivalent to x @= y in Python.

PythonCall.pyipow — Function
pyipow(x, y, z=None)

In-place power. x = pyipow(x, y) is equivalent to x **= y in Python.

PythonCall.pyifloordiv — Function
pyifloordiv(x, y)

In-place floor divide. x = pyifloordiv(x, y) is equivalent to x //= y in Python.

PythonCall.pyitruediv — Function
pyitruediv(x, y)

In-place true division. x = pyitruediv(x, y) is equivalent to x /= y in Python.

PythonCall.pyimod — Function
pyimod(x, y)

In-place subtraction. x = pyimod(x, y) is equivalent to x %= y in Python.

PythonCall.pyilshift — Function
pyilshift(x, y)

In-place left shift. x = pyilshift(x, y) is equivalent to x <<= y in Python.

PythonCall.pyirshift — Function
pyirshift(x, y)

In-place right shift. x = pyirshift(x, y) is equivalent to x >>= y in Python.

PythonCall.pyiand — Function
pyiand(x, y)

In-place and. x = pyiand(x, y) is equivalent to x &= y in Python.

PythonCall.pyixor — Function
pyixor(x, y)

In-place xor. x = pyixor(x, y) is equivalent to x ^= y in Python.

PythonCall.pyior — Function
pyior(x, y)

In-place or. x = pyior(x, y) is equivalent to x |= y in Python.

Logic

These functions are equivalent to the corresponding Python logical operators.

Note that the equivalent Julia operators are overloaded to call these when all arguments are Py (or Number). Hence the following are equivalent: Py(1) < Py(2), Py(1) < 2, pylt(1, 2), pylt(Py(1), Py(2)), etc.

Note that the binary operators by default return Py (not Bool) since comparisons in Python do not necessarily return bool.

PythonCall.pytruth — Function
pytruth(x)

The truthyness of x. Equivalent to bool(x) in Python, converted to a Bool.

PythonCall.pynot — Function
pynot(x)

The falsyness of x. Equivalent to not x in Python, converted to a Bool.

PythonCall.pyeq — Function
pyeq(x, y)
pyeq(Bool, x, y)

Equivalent to x == y in Python. The second form converts to Bool.

PythonCall.pyne — Function
pyne(x, y)
pyne(Bool, x, y)

Equivalent to x != y in Python. The second form converts to Bool.

PythonCall.pyle — Function
pyle(x, y)
pyle(Bool, x, y)

Equivalent to x <= y in Python. The second form converts to Bool.

PythonCall.pylt — Function
pylt(x, y)
pylt(Bool, x, y)

Equivalent to x < y in Python. The second form converts to Bool.

PythonCall.pyge — Function
pyge(x, y)
pyge(Bool, x, y)

Equivalent to x >= y in Python. The second form converts to Bool.

PythonCall.pygt — Function
pygt(x, y)
pygt(Bool, x, y)

Equivalent to x > y in Python. The second form converts to Bool.

Installing Python packages

PythonCall uses CondaPkg.jl to manage its dependencies. Namely, CondaPkg will automatically install Python and any packages required into a Conda environment specific to your current project.

If your project requires any Python packages, add a CondaPkg.toml file to your project specifying what you need. Alternatively use CondaPkg.add() to add dependencies from the REPL.

Writing packages which depend on PythonCall

Example

See https://github.com/cjdoris/Faiss.jl for an example package which wraps the Python FAISS package.

Precompilation

You may not interact with Python during module precompilation. Therefore, instead of

module MyModule
  using PythonCall
  const foo = pyimport("foo")
  bar() = foo.bar() # will crash when called
end

you must do

module MyModule
  using PythonCall
  const foo = PythonCall.pynew() # initially NULL
  function __init__()
    PythonCall.pycopy!(foo, pyimport("foo"))
  end
  bar() = foo.bar() # now ok
end

Dependencies

If your package depends on some Python packages, you must write a CondaPkg.toml file. See Installing Python packages.

Low-level API

The functions here are not exported. They are mostly unsafe in the sense that you can crash Julia by using them incorrectly.

PythonCall.pynew — Function
pynew([ptr])

A new Py representing the Python object at ptr (NULL by default).

If ptr is given and non-NULL, this function steals a reference to the Python object it points at, i.e. the new Py object owns a reference.

Note that NULL Python objects are not safe in the sense that most API functions will probably crash your Julia session if you pass a NULL argument.

PythonCall.pycopy! — Function
pycopy!(dst::Py, src)

Copy the Python object src into dst, so that they both represent the same object.

This function exists to support module-level constant Python objects. It is illegal to call most PythonCall API functions at the top level of a module (i.e. before __init__() has run) so you cannot do const x = pything() at the top level. Instead do const x = pynew() at the top level then pycopy!(x, pything()) inside __init__().

Assumes dst is NULL, otherwise a memory leak will occur.

PythonCall.getptr — Function
getptr(x)

Get the underlying pointer from the Python object x.

PythonCall.pydel! — Function
pydel!(x::Py)

Delete the Python object x.

DANGER! Use this function ONLY IF the Julia object x could have been garbage-collected anyway, i.e. was about to become unreachable. This means you MUST KNOW that no other part of the program has the Julia object x.

This decrements the reference count, sets the pointer to NULL and appends x to a cache of unused objects (PYNULL_CACHE).

This is an optimization to avoid excessive allocation and deallocation in Julia, which can be a significant source of slow-down in code which uses a lot of Python objects. It allows pynew() to pop an item from PYNULL_CACHE instead of allocating one, and avoids calling the relatively slow finalizer on x.

PythonCall.unsafe_pynext — Function
unsafe_pynext(x)

Return the next item in the iterator x. When there are no more items, return NULL.