Example 1 - Old tangent leftovers in forward mode AD

Goal: Learn how side effects can effect the tangent computation.

Prerequisite: Tutorial 1 - Forward mode AD

Function:

Real func(const Real& x, bool updateGlobal) {
  if(updateGlobal) {
    global = x * x;
  }
  return x * global;
}

Full code:

 
#include <codi.hpp>
#include <iostream>
 
using Real = codi::RealForward;
 
Real global = Real(0.0);
 
Real func(const Real& x, bool updateGlobal) {
  if(updateGlobal) {
    global = x * x;
  }
  return x * global;
}
 
int main(int nargs, char** args) {
  Real x = 4.0;
 
  // Step 1: Evaluate function and update global variable
  x.setGradient(1.0);
  Real y = func(x, true);
 
  std::cout << "Update global:" << std::endl;
  std::cout << "f(4.0, true) = " << y << std::endl;
  std::cout << "df/dx(4.0, true) = " << y.getGradient() << std::endl << std::endl;
 
  // Step 2: Evaluate function without updating the global variable
  x.setGradient(1.0);
  y = func(x, false);
 
  std::cout << "No update global:" << std::endl;
  std::cout << "f(4.0, false) = " << y << std::endl;
  std::cout << "df/dx(4.0, false) = " << y.getGradient() << std::endl << std::endl;
 
  // Step 3: Reset tangent on global variable
  global.setGradient(0.0);
  x.setGradient(1.0);
  y = func(x, false);
 
  std::cout << "No update global with reset:" << std::endl;
  std::cout << "f(4.0, false) = " << y << std::endl;
  std::cout << "df/dx(4.0, false) = " << y.getGradient() << std::endl << std::endl;
 
  return 0;
}

The computational path of the function is changed via the parameter updateGlobal. If this parameter is true then x also enters the computation of the global variable. If the parameter is false then the global value is seen as a constant with respect to AD.

The three different calls demonstrate the error in the tangent computation. The first and third call are correct, the second one is wrong. Here, we fix the issue by directly resetting the tangent of the global variable. An other option would be to call func(x, true) again, with a tangent value of zero for x.

Notes

This problem is not specific to CoDiPack. Nearly all tapeless forward AD tools suffer from this problem. One possible final fix is to implement a Gradient type, that tags tangents with an epoch. The user would then need to mange the global valid epoch and the Gradient type can ignore tangents from older epochs.