# Preprocessing API Reference

## Functions

`ChemometricsTools.CORAL`

— Method.`CORAL(X1, X2; lambda = 1.0)`

Performs CORAL to facilitate covariance based transfer from `X1`

to `X2`

with regularization parameter `lambda`

. Returns a CORAL object.

Correlation Alignment for Unsupervised Domain Adaptation. Baochen Sun, Jiashi Feng, Kate Saenko. https://arxiv.org/abs/1612.01939

`ChemometricsTools.CORAL`

— Method.`(C::CORAL)(Z)`

Applies a the transform from a learned `CORAL`

object to new data `Z`

.

`(T::MultiplicativeScatterCorrection)(Z)`

Applies MultiplicativeScatterCorrection from a stored object `T`

to Array `Z`

.

`MultiplicativeScatterCorrection(Z::Array)`

Creates a MultiplicativeScatterCorrection object from the data in Z

Martens, H. Multivariate calibration. Wiley

`OrthogonalSignalCorrection(X, Y; Factors = 1)`

Performs Thomas Fearn's Orthogonal Signal Correction to an endogenous `X`

and exogenous `Y`

. The number of `Factors`

are the number of orthogonal components to be removed from `X`

. This function returns an OSC object.

Tom Fearn. On orthogonal signal correction. Chemometrics and Intelligent Laboratory Systems. Volume 50, Issue 1, 2000, Pages 47-52.

`(OSC::OrthogonalSignalCorrection)(Z; Factors = 2)`

Applies a the transform from a learned orthogonal signal correction object `OSC`

to new data `Z`

.

`TransferByOrthogonalProjection(X1, X2; Factors = 1)`

Performs Thomas Fearns Transfer By Orthogonal Projection to facilitate transfer from `X1`

to `X2`

. Returns a TransferByOrthogonalProjection object.

Anne Andrew, Tom Fearn. Transfer by orthogonal projection: making near-infrared calibrations robust to between-instrument variation. Chemometrics and Intelligent Laboratory Systems. Volume 72, Issue 1, 2004, Pages 51-56,

`(TbOP::TransferByOrthogonalProjection)(X1; Factors = TbOP.Factors)`

Applies a the transform from a learned transfer by orthogonal projection object `TbOP`

to new data `X1`

.

`ChemometricsTools.ALSSmoother`

— Method.`ALSSmoother(X; lambda = 100, p = 0.001, maxiters = 10)`

Applies an assymetric least squares smoothing function to a 2-Array `X`

. The `lambda`

, `p`

, and `maxiters`

parameters control the smoothness. See the reference below for more information.

Paul H. C. Eilers, Hans F.M. Boelens. Baseline Correction with Asymmetric Least Squares Smoothing. 2005

`ChemometricsTools.ConvFilter1DFFT`

— Method.`ConvFilter1DFFT(a, filter)`

Performs a 1D convolution of vector `filter`

onto `a`

via the FFT definition of a convolution. This method implicitly zeropads and truncates the result.

Note: This isn't highly optimized for performance. It was made when DSP.jl broke, and broke the ChemometricsTools.jl package.

`ChemometricsTools.DirectStandardization`

— Method.`DirectStandardization(InstrumentX1, InstrumentX2; Factors = minimum(collect(size(InstrumentX1))) - 1)`

Makes a DirectStandardization object to facilitate the transfer from Instrument #2 to Instrument #1 . The returned object can be used to transfer unseen data to the approximated space of instrument 1. The number of `Factors`

used are those from the internal orthogonal basis.

Yongdong Wang and Bruce R. Kowalski, "Calibration Transfer and Measurement Stability of Near-Infrared Spectrometers," Appl. Spectrosc. 46, 764-771 (1992)

`ChemometricsTools.FirstDerivative`

— Method.`FirstDerivative(X)`

Uses the finite difference method to compute the first derivative for every row in `X`

. *Note: This operation results in the loss of a column dimension.*

`ChemometricsTools.FourierUpsample`

— Method.`FourierUpsample(X, newlength)`

zero pads a vector `X`

in the frequency domain then converts it back to the time domain.

`ChemometricsTools.FractionalDerivative`

— Function.`FractionalDerivative(Y, X = 1 : length(Y); Order = 0.5)`

Calculates the Grunwald-Leitnikov fractional order derivative on every row of Array Y. Array `X`

is a vector that has the spacing between column-wise entries in `Y`

. `X`

can be a scalar if that is constant (common in spectroscopy). `Order`

is the fractional order of the derivative. *Note: This operation results in the loss of a column dimension.*

The Fractional Calculus, by Oldham, K.; and Spanier, J. Hardcover: 234 pages. Publisher: Academic Press, 1974. ISBN 0-12-525550-0

`ChemometricsTools.LinearResample`

— Method.`LinearResample(X, newsize)`

Resamples a vector `X`

to be of size `newsize`

via linear interpolation.

`ChemometricsTools.Noise`

— Method.`Noise(data, level; scale = :percent_max, type = :gaussian )`

Adds noise (uniform, or gaussian) to an array of data. The scale can be made relative to the percent*max, percent*average, or the absolute level values.

Note: for Gaussian noise it is likely convenient to scale by an inverse quantile(say 0.999)

`ChemometricsTools.PerfectSmoother`

— Method.`PerfectSmoother(X; lambda = 100)`

Applies an assymetric least squares smoothing function to a a 2-Array `X`

. The `lambda`

parameter controls the smoothness. See the reference below for more information.

Paul H. C. Eilers. "A Perfect Smoother". Analytical Chemistry, 2003, 75 (14), pp 3631–3636.

`ChemometricsTools.SavitzkyGolay`

— Method.`SavitzkyGolay(X, Delta, PolyOrder, windowsize)`

Performs SavitskyGolay smoothing across every row in an Array `X`

. The `window size`

is the size of the convolution filter, `PolyOrder`

is the order of the polynomial, and `Delta`

is the order of the derivative.

Savitzky, A.; Golay, M.J.E. (1964). "Smoothing and Differentiation of Data by Simplified Least Squares Procedures". Analytical Chemistry. 36 (8): 1627–39. doi:10.1021/ac60214a047.

`ChemometricsTools.Scale1Norm`

— Method.`Scale1Norm(X)`

Scales the columns of `X`

by the 1-Norm of each row. Returns the scaled array.

`ChemometricsTools.Scale2Norm`

— Method.`Scale2Norm(X)`

Scales the columns of `X`

by the 2-Norm of each row. Returns the scaled array.

`ChemometricsTools.ScaleByIntensity`

— Method.`ScaleByIntensity(X, index)`

Scales the columns of `X`

by the value of a peak at a specified `index`

in each row. Returns the scaled array.

`ChemometricsTools.ScaleFNorm`

— Method.`ScaleFNorm(X)`

Scales EACH entry of `X`

by the Frobenius Norm of. Returns the scaled array.

`ChemometricsTools.ScaleInfNorm`

— Method.`ScaleInfNorm(X)`

Scales the columns of `X`

by the Inf-Norm of each row. Returns the scaled array.

`ChemometricsTools.ScaleMinMax`

— Method.`ScaleMinMax(X)`

Scales the columns of `X`

by the Min and Max of each row such that no observation is greater than 1 or less than zero. Returns the scaled array.

`ChemometricsTools.SecondDerivative`

— Method.`FirstDerivative(X)`

Uses the finite difference method to compute the second derivative for every row in `X`

. *Note: This operation results in the loss of two columns.*

`ChemometricsTools.SincInterpolation`

— Method.```
SincInterpolation(Y, S, Up)
Y - vector of a line shape
S - Sampled domain of Y
Up - Upsampled X vector
```

Refactored from: https://gist.github.com/endolith/1297227#file-sinc_interp-m

`ChemometricsTools.StandardNormalVariate`

— Method.`StandardNormalVariate(X)`

Scales the columns of `X`

by the mean and standard deviation of each row. Returns the scaled array.

`ChemometricsTools.boxcar`

— Method.`boxcar(X; windowsize = 3, fn = mean)`

Applies a boxcar function (`fn`

) to each window of size `windowsize`

to every row in `X`

. *Note: the function provided must support a dims argument/parameter.*

`ChemometricsTools.offsetToZero`

— Method.`offsetToZero(X)`

Ensures that no observation(row) of Array `X`

is less than zero, by ensuring the minimum value of each row is zero.

`(DSX::DirectStandardizationXform)(X; Factors = length(DSX.pca.Values))`

Applies a the transform from a learned direct standardization object `DSX`

to new data `X`

.