Function References

const TimeVarType = Union{AbstractVector, AbstractRange}

A supertype for specifying the set of training inputs.

create_Matern3HalfsKernel(λ::T, b::T)::Matern3HalfsKernel{T} where T <: AbstractFloat

Creates a Matern covariance function variable with parameters λ and b. Given two time inputs x and z, the distance between two time inputs, this covariance function has the following formula:

output = b*(1 + λ*norm(x-z))*exp(-λ*norm(x-z))

create_Matern3HalfsKernel(λ::T)::Matern3HalfsKernel{T} where T <: AbstractFloat

Returns create_Matern3HalfsKernel(λ, one(T))

create_Matern3HalfsKernel(::Type{T})::Matern3HalfsKernel{T} where T <: AbstractFloat

Returns a template or dummy variable by calling create_Matern3HalfsKernel(one(T), one(T))

create_sdegp(
    θ_sde::Kernel1D,
    ts::TimeVarType,
    y::Vector{T},
    σ²::T,
) where T <: Real

To run Gaussian process query via the state-space implementation, one needs to do a cache phase then a query phase. This function runs the cache phase.

Inputs:

• θ_sde is a Matern kernel variable
• ts is the set of training inputs.
• y is an array of training outputs.
• σ² is the Gaussian process regression's observation variance.

Returns a variable of SDEGP data type, which is used in the query phase.

query_sdegp(
    S::SDEGP,
    tqs,
    ts,
) where T <: Real

This function allocates the output buffers mqs and vqs, then calls query_sdegp!. See query_sdegp! for details on the inputs.

Returns mqs and vqs.

query_sdegp!(
    mqs::Vector{T}, # mutates, output.
    vqs::Vector{T}, # mutates, output.
    S::SDEGP,
    tqs,
    ts::TimeVarType,
) where T <: Real

To run Gaussian process query via the state-space implementation, one needs to do a cache phase then a query phase. This function runs the query phase.

Inputs:

• mqs Buffer that is mutated and stores the queried predictive means corresponding to entries in tqs.
• vqs Buffer that is mutated and stores the queried predictive variances corresponding to entries in tqs.
• S is the output of the cached phase; see create_sdegp.
• tqs is the set of query inputs.
• ts is the set of training inputs.

The size of mqs and vqs must be the same as tqs.

ts should be the same training inputs used to create the cache S, otherwise S needs to be recomputed via create_sdegp.

Returns nothing.

setup_ml(
    θ_sde::Kernel1D,
    ts::TimeVarType,
    y::Vector{T};
    σ² = one(T),
) where T <: Real

Returns a buffer variable of type MLBuffers for use with eval_ml!, which computes the marginal likelihood. This avoids additional creation and allocation of this buffer variable if the marginal likelihood is to be computed multiple times for different hyperparameters.

eval_ml(
    trait::GainTrait,
    p::Vector{T},
    ts::Union{AbstractRange, AbstractVector},
    y::Vector;
    zero_tol = eps(T)*100,
) where T <: AbstractFloat

Inputs:

• trait is a trait variable that specifies the order of the hyperparameters in p. See trait-based dispatch in the Julia documentation. If typeof(trait) <: InferGain, then the hyperparameters in p are ordered [λ; σ²; b]. If typeof(trait) <: UnityGain, then the hyperparameters in p are ordered [λ; σ²], and b is set to 1.
• p is an ordered set of hyperparameter as an array.
• ts is the set of training inputs.
• y is the set of training outputs.
• zero_tol needs to be a small positive number. It is the lower bound on the covariance function hyperparameters.

Returns the log marginal likelihood over the training set. Some additive constants might be dropped.

eval_ml!(
    B::MLBuffers, # mutates, buffer
    θ_sde::Kernel1D,
    ts::TimeVarType,
    y::Vector,
    σ²::Real,
)

Inputs:

• buffer is the return variable from setup_ml.
• θ_sde is the covariance function variable.
• ts is the set of training inputs.
• y is the set of training outputs.
• σ² is the observation noise variance. Increase this if you experience numerical issues.

Returns the log marginal likelihood over the training set. Additive constants are dropped.

eval_ml!(
    trait::GainTrait,
    buffer::MLBuffers,
    p::Vector{T},
    ts::TimeVarType,
    y::Vector{T};
    zero_tol = eps(T)*100,
) where T <: AbstractFloat

Inputs:

• trait is a trait variable that specifies the order of the hyperparameters in p. See trait-based dispatch in the Julia documentation. If typeof(trait) <: InferGain, then the hyperparameters in p are ordered [λ; σ²; b]. If typeof(trait) <: UnityGain, then the hyperparameters in p are ordered [λ; σ²], and b is set to 1.
• buffer is the return variable from setup_ml.
• p is an ordered set of hyperparameter as an array.
• ts is the set of training inputs.
• y is the set of training outputs.
• zero_tol needs to be a small positive number. It is the lower bound on the covariance function hyperparameters.

Returns the log marginal likelihood over the training set. Some additive constants might be dropped.

eval_ml!(
    buffer::MLBuffers,
    λ::T,
    σ²::T,
    ts::TimeVarType,
    y::Vector{T};
    zero_tol = eps(T)*100,
) where T <: Real

Inputs:

• buffer is the return variable from setup_ml.
• λ and σ² are hyperparameters. The b hyperparameter is set to 1.
• ts is the set of training inputs.
• y is the set of training outputs.
• zero_tol needs to be a small positive number. It is the lower bound on the covariance function hyperparameters.

Returns the log marginal likelihood over the training set. Some additive constants might be dropped.

eval_ml!(
    buffer::MLBuffers,
    λ::T,
    σ²::T,
    b::T,
    ts::TimeVarType,
    y::Vector{T};
    zero_tol = eps(T)*100,
) where T <: Real

Inputs:

• buffer is the return variable from setup_ml.
• λ, σ², and b are hyperparameters.
• ts is the set of training inputs.
• y is the set of training outputs.
• zero_tol needs to be a small positive number. It is the lower bound on the covariance function hyperparameters.

Returns the log marginal likelihood over the training set. Some additive constants might be dropped.

abstract type GainTrait end

Subtypes are InferGain and UnityGain

struct InferGain <: GainTrait end

This trait specifies that the hyperparameter ordering of the parameter array is λ, σ², b.

struct UnityGain <: GainTrait end

This trait specifies that the hyperparameter ordering of the parameter array is λ, σ². The b hyperparameter is fixed at 1.

hp_optim(
    alg_trait::UseMetaheuristics,
    model_trait::GainTrait,
    ts::TimeVarType,
    y::Vector,
    lbs::Vector{T},
    ubs::Vector{T};
    f_calls_limit = 10_000,
    p0s::Vector{Vector{T}} = generate_grid(model_trait, lbs, ubs, 10),
)

Checks if the weak dependencies are loaded in the user's working scope, then checks if the corresponding package extensions are loaded. If so, call the appropriate hyperparameter optimization routine from the package extension.

• alg_trait: see UseMetaheuristics.
• model_trait is a trait variable that specifies the order of the hyperparameters in p. See trait-based dispatch in the Julia documentation. If typeof(trait) <: InferGain, then the hyperparameters in p are ordered [λ; σ²; b]. If typeof(trait) <: UnityGain, then the hyperparameters in p are ordered [λ; σ²], and b is set to 1. p is used internally by hy_optim.
• ts is the set of training inputs.
• y is the set of training outputs.
• lbs and ubs are lower and upper bounds for the hyperparameter vector, p. They follow the ordering as specified by model_trait.
• f_calls_limit is a soft upperbound on the number of marginal likelihood evaluations used during optimization such that the evolutionary algorithm in Metaheuristics.jl tries to not exceed.
• p0s is a nested Vector array that contains hyperparameter states that you want to force the optimization algorithm to evaluate. These states can be interpreted as initial guesses to the solution. The default creates a uniform grid of 10x10x10 (if typeof(model_trait) <: InferGain) or 10x10 (if typeof(model_trait) <: UnityGain) from the lower and upper bounds specified.

See the tutorial for the return type and subsequent usage.

struct UseMetaheuristics <: ExtensionPkgs
    pkg::Module
end

Container for verifying whether the weak dependencies for MetaheuristicsExt.jl is loaded in the user's working scope.

The only dependency required to be loaded in the user's working scope is Metaheuristics.

parse_ml_result(trait::GainTrait, p::Vector{T}) where T <: AbstractFloat

Returns a Matern covariance function variable with the hyperparameters specified in p. The ordering of the hyperparameters in p is specified by trait.

hp_inference(
    alg_trait::UseDynamicHMC,
    gain_trait::GainTrait,
    N_draws::Integer,
    α::Real,
    β::Real,
    ts::TimeVarType,
    y::Vector,
)

Checks if the weak dependencies are loaded in the user's working scope, then checks if the corresponding package extensions are loaded. If so, call the appropriate hyperparameter inference routine from the package extension.

• alg_trait: see UseDynamicHMC.
• gain_trait is a trait variable that specifies the order of the hyperparameters in p. See trait-based dispatch in the Julia documentation. If typeof(trait) <: InferGain, then the hyperparameters in p are ordered [λ; σ²; b]. If typeof(trait) <: UnityGain, then the hyperparameters in p are ordered [λ; σ²], and b is set to 1. p is used internally by hy_optim.
• α and β are the shared hyperparameters of the inverse gamma prior for the hyperparameters.
• ts is the set of training inputs.
• y is the set of training outputs.

See the tutorial for the return type and subsequent usage.

struct UseDynamicHMC <: ExtensionPkgs
    pkg_list::Vector{Module}
end

Container for verifying whether the weak dependencies for DynamicHMCExt.jl is loaded in the user's working scope.

The dependencies are:

• FiniteDiff
• SimpleUnPack
• TransformVariables
• TransformedLogDensities
• LogDensityProblems
• DynamicHMC

simulate_sdegps(
    λ_set::Vector{T},
    σ²_set::Vector{T},
    M::Integer,
    ts::TimeVarType,
    tqs,
    y::Vector,
) where T <: Real

Returns the drawn samples of the ensemble of Gaussian process models that are specified by λ_set and σ²_set. The number of ensemble models is the length of λ_set.

The gain b is set to 1 for the simulation.

Inputs:

• λ_set contain samples of the λ parameter. Same length as σ²_set.
• σ²_set contain samples of the σ² parameter.
• M is the number of samples simulate_sdegps simulates per model.
• ts is the ordered set of training inputs.
• tqs is the ordered set of query inputs.
• y is the ordered set of training outputs.

The output, S, is a M x N x K array, where N is the number of ensemble models, and K is the number of query inputs.

simulate_sdegps(
    λ_set::Vector{T},
    σ²_set::Vector{T},
    b_set::Vector{T},
    M::Integer,
    ts::TimeVarType,
    tqs,
    y::Vector,
) where T <: Real

Returns the drawn samples of the ensemble of Gaussian process models that are specified by λ_set, σ²_set, and b_set. The number of ensemble models is the length of λ_set.

Inputs:

• λ_set contain samples of the λ parameter. Same length as σ²_set.
• σ²_set contain samples of the σ² parameter.
• b_set contain samples of the b parameter.
• M is the number of samples simulate_sdegps simulates per model.
• ts is the ordered set of training inputs.
• tqs is the ordered set of query inputs.
• y is the ordered set of training outputs.

The output, S, is a M x N x K array, where N is the number of ensemble models, and K is the number of query inputs.

compute_mean_var(S::Array{T,3}) where T <: Real

Assumes S is a M x N x K array of drawn samples, where:

• M is the number of samples drawn from a model.
• N is the number of models.
• K is the number of query positions.

compute_mean_var computes the empirical mean and variance for each query position.

Outputs:

• mqs the empirical means. Length K.
• vqs the empirical variances. Length K.