Supervised-learning-based imitation learning algorithm of deterministic policies.

Refs

[1] F. L. Lewis, D. Vrabie, and K. Vamvoudakis, "Reinforcement Learning and Feedback Control: Using Natural Decision Mehods to Design Optimal Adaptive Controllers," IEEE Control Systems, vol. 32, no. 6, pp.76-105, 2012.

Variables

V̂ ∈ R: the estimate of (state) value function - V̂.basis: Φ - V̂.param: w

d: polynomial degree

Refs

[1] T. Bian and Z.-P. Jiang, “Value Iteration, Adaptive Dynamic Programming, and Optimal Control of Nonlinear Systems,” in 2016 IEEE 55th Conference on Decision and Control (CDC), Las Vegas, NV, USA, Dec. 2016, pp. 3375–3380. doi: 10.1109/CDC.2016.7798777. [2] Forked repo of the code implementation by T. Bian, https://github.com/JinraeKim/nonlinear-VI/blob/main/nonlinear_sys.R

Variables

V̂ ∈ R: the estimate of (state) value function - V̂.basis: Φ - V̂.param: w dV̂ ∈ R: the estimate of time derivative of (state) value function - dV̂.basis: Ψ - V̂.param: c (BE CAREFUL; not used)

n: state dim. m: input dim. d: polynomial degree

References

[1] F. A. Goodarzi, D. Lee, and T. Lee, “Geometric stabilization of a quadrotor UAV with a payload connected by flexible cable,” in 2014 American Control Conference, Jun. 2014, pp. 4925–4930, doi: 10.1109/ACC.2014.6859419.

Variables

p ∈ R^3: inertial position v ∈ R^3: inertial velocity R ∈ so(3): inertial-to-body rotation matrix (R_I2B) ω ∈ R^3: angular rate of body in inertial frame u ∈ R^4: [f, M...] where f ∈ R: total thrust, M ∈ R^3: moment

Variables

pos0 ∈ R^3: initial position dir ∈ S^2 (sphere): a unit vector dir_perp ∈ S^2 (sphere): a unit vector perp. to dir r > 0: radius ω: angular rate θ: phase c: coefficient for forward direction

Notes

The basis vectors of the plane perpendicular to the dir is determined by a keyword argument dir_ref=[0, 0, 1]. For more details, see the source code.

References

[1] M. Islam, M. Okasha, and M. M. Idres, “Dynamics and control of quadcopter using linear model predictive control approach,” IOP Conf. Ser.: Mater. Sci. Eng., vol. 270, p. 012007, Dec. 2017, doi: 10.1088/1757-899X/270/1/012007.

Variables

u ∈ R^4: square of rotor angular speed

Infinite-horizon continuous-time linear quadratic regulator (LQR).

Notes

• [1] is modified for hexa-x configuration; see [2].

References

[1] T. Lee, M. Leok, and N. H. McClamroch, “Geometric Tracking Control of a Quadrotor UAV on SE(3),” in 49th IEEE Conference on Decision and Control (CDC), Atlanta, GA, Dec. 2010, pp. 5420–5425. doi: 10.1109/CDC.2010.5717652. [2] PX4 Airframes Reference, https://docs.px4.io/master/en/airframes/airframe_reference.html.

Variables

u ∈ R^6: rotor forces

n: input dimension d: polynomial degree m: output dimension

Proportional-integral-derivative (PID) controller.

Notes

The derivative term is obtained via second-order filter. For more details, see [1] and singular perturbation theory.

References

[1] J. A. Farrell, M. Polycarpou, M. Sharma, and W. Dong, “Command Filtered Backstepping,” IEEE Transactions on Automatic Control, vol. 54, no. 6, pp. 1391–1395, Jun. 2009, doi: 10.1109/TAC.2009.2015562.

Notes

with_bias=true provides polynomials with the degree of from 0 to d.

with_bias=false provides polynomials with the degree of d.

Variables

n ∈ N: length of array, i.e., x ∈ Rⁿ d ∈ N: degree

Refs.

[1] E. Kaufmann, A. Loquercio, R. Ranftl, M. Müller, V. Koltun, and D. Scaramuzza, “Deep Drone Acrobatics,” arXiv:2006.05768 [cs], Jun. 2020, Accessed: Jun. 02, 2021. [Online]. Available: http://arxiv.org/abs/2006.05768

Variables

d ∈ R^3: displacement r ∈ R^+: radius

Notes

units: [m], [s]

Moore-Penrose inverse control allocator.

Variables

ν: virtual input

Notes

ν = B*u where u: control input

References

• Reference model (e.g., xd, vd, ad, addot, adddot)

[1] S. J. Su, Y. Y. Zhu, H. R. Wang, and C. Yun, “A Method to Construct a Reference Model for Model Reference Adaptive Control,” Adv. Mech. Eng., vol. 11, no. 11, pp. 1–9, 2019.

Notes

d: degree of the reference model autodiff: autodiff = true for tracking time-varying command; otherwise for set-point regulation xcmdfunc(t): function of time

Infer the approximate optimal input.

Notes

NED coordinate system.

• go straight

norm(v0)(t-t0) + 0.5a*(t-t0)^2 = norm(Δp)

• loop

rθ = V(t-t0-tgostraight)

Notes

running_cost: integrand; function of (x, u)

LinearSystemEnv + IntegratorEnv

Minimise J = ∫ (x' Q x + u' R u) from 0 to ∞

Even for scalar integrator, you should put an array of length 1; due to the limitation of in-place method Dynamics!.

Notes

• r denotes integrand of integ.

Common state structure of MulticopterEnv

∫e: integral of the error ê: the estimate of the error ė̂: the estimate of the error rate

x_i denotes ith (time) derivative of x

Variables

State

p ∈ R^3: (inertial) position v ∈ R^3: (inertial) velocity R ∈ so(3): rotation matrix of body frame w.r.t. inertial frame (I to B) ω ∈ R^3: angular rate of body frame w.r.t. inertial frame (I to B)

(Virtual) input

f ∈ R: total thrust M ∈ R^3: moment

Extended to deal with empty Dict.

Infer the approximate optimal input.

c: centre r: radius h: direction of normal vector

Common input to force and moment transformation (default: not applied)

Notes

• The basic concept is borrowed from an MRAC example.
• It is modified to be compatible with SimulationLogger.jl.

Limitations

• Conditional method definition is troublesome; see #16.

Instead of it, I decided to merely define a new method with argument __log_indicator__::__LOG_INDICATOR__, which will provide "empty Dict" in the case of no logging.

Minimise approximate Hamiltonian.

partitionTrainTest(data, at)

Split a dataset into train and test datasets (array).

Common input saturation (default: not applied)

Notes

adp.data contains M+1 data j = 0, 1, ..., M-1

Notes

• Currently, only iip (isinplace) method is supported.

Keyword arguments

Tips

• progress (kwarg): if true, it shows the progress bar (in Juno IDE or supports by TerminalLoggers.jl).

s: states (minibatch) ∈ R^nxN a: actions (minibatch) ∈ R^mxN

Examples

for batch in bc.dataloader FlightSims.update!(bc, batch...) end

render!(multicopter::MulticopterEnv, fig, state)

Variables

fig: figure object, i.e., fig = plot() state: see State(env::MulticopterEnv).

Notes

Default configuration: ENU-coordinate system. For airframe references, see https://docs.px4.io/master/en/airframes/airframe_reference.html.