Automatic differentiation

Muscade has its own implementation of forward automatic differentiation for historical reasons: Prototypes automaticdifferentiation of Muscade where developed in parallel with ForwardDiff.jl. While the inner workings of ForwardDiff.jl and Muscade's version are similar ( with ForwardDiff.jl probably having better performance), the API are quite different.

Muscade evaluates second derivative of the Lagrangian, using dneste forward differentiation, which is far from optimal. An ambition is to make use of reverse differentiation (using Zygote.jl, Enzyme.jl or similar).

Muscades automatic differentiation is used as follows:

using SVector
x   = SVector(1.,2.,3.)
N   = length(x)
x1  = variate{1,N}(x)
y1  = f(x1)
y   = value{1}(y1)
yₓ  = ∂{1,N}(y2)    

x must be a SVector. In yₓ, the index over x is post-pended to the indices of y.

Where automatic differentiation is nested, the extractions must be carried out in reverse order:

x   = SVector(1.,2.,3.)
N   = length(x)
x1  = variate{1,N}(x)
x2  = variate{2,N}(x1)
y2  = f(x2)
y   = value{1}(value{2}(y2))
yₓ  = ∂{1,N}(value{2}(y2))  
yₓ  = value{1}(∂{2,N}(y2))
yₓₓ = ∂{1,N}(∂{2,N}(y2))

The first type parameter given to variate, value and ∂ is the precedence P. When automatic differentiation is nested, variate must be called with a value of P that is higher than the precedence of an variable that will influence the output. Inside a function, this precedence can vary depending on the type of arguments provided to the function. To this end, the function constants is provided:

using SVector
...
N   = length(x)
P   = constants(x,a,b,c)
x1  = variate{P,N}(x)
y1  = f(x1,a,b,c)
y   = value{P}(y1)
yₓ  = ∂{P,N}(y2)