How to Write Custom Gradients

To add a custom gradient for a differentiable deterministic computation, define a concrete subtype of CustomGradientGF with the following methods:

• apply

• gradient

• has_argument_grads

For example:

struct MyPlus <: CustomGradientGF{Float64} end

Gen.apply(::MyPlus, args) = args[1] + args[2]
Gen.gradient(::MyPlus, args, retval, retgrad) = (retgrad, retgrad)
Gen.has_argument_grads(::MyPlus) = (true, true)

Optimizing Trainable Parameters

To add support for a new type of gradient-based parameter update, create a new type with the following methods defined for the types of generative functions that are to be supported.

• Gen.init_update_state
• Gen.apply_update!