Lecture 10 Exercise Solutions¶

Exercise 1¶

You will work with the following function for this exercise, \begin{align} f\left(x,y\right) = \exp\left(-\left(\sin\left(x\right) - \cos\left(y\right)\right)^{2}\right). \end{align}

1. Draw the computational graph for the function.
   • Note: This graph will have $2$ inputs.
2. Create the evaluation trace for this function.
3. Use the graph / trace to evaluate $f\left(\dfrac{\pi}{2}, \dfrac{\pi}{3}\right)$.
4. Compute $\dfrac{\partial f}{\partial x}\left(\dfrac{\pi}{2}, \dfrac{\pi}{3}\right)$ and $\dfrac{\partial f}{\partial y}\left(\dfrac{\pi}{2}, \dfrac{\pi}{3}\right)$ using the forward mode of AD.

Solution¶

Trace	Elementary Function	Current Value	Elementary Function Derivative	$\nabla_{x}$ Value	$\nabla_{y}$ Value
$x_{1}$	$x_{1}$	$\dfrac{\pi}{2}$	$\dot{x}_{1}$	$1$	$0$
$x_{2}$	$x_{2}$	$\dfrac{\pi}{3}$	$\dot{x}_{2}$	$0$	$1$
$x_{3}$	$\sin\left(x_{1}\right)$	$1$	$\cos\left(x_{1}\right)\dot{x}_{1}$	$0$	$0$
$x_{4}$	$\cos\left(x_{2}\right)$	$\dfrac{1}{2}$	$-\sin\left(x_{2}\right)\dot{x}_{2}$	$0$	$-\dfrac{\sqrt{3}}{2}$
$x_{5}$	$x_{3} - x_{4}$	$\dfrac{1}{2}$	$\dot{x}_{3} - \dot{x}_{4}$	$0$	$\dfrac{\sqrt{3}}{2}$
$x_{6}$	$x_{5}^{2}$	$\dfrac{1}{4}$	$2x_{5}\dot{x}_{5}$	$0$	$\dfrac{\sqrt{3}}{2}$
$x_{7}$	$-x_{6}$	$-\dfrac{1}{4}$	$-\dot{x}_{6}$	$0$	$-\dfrac{\sqrt{3}}{2}$
$x_{8}$	$\exp\left(x_{7}\right)$	$\exp\left(-1/4\right)$	$\exp\left(x_{7}\right)\dot{x}_{7}$	$0$	$-\dfrac{\sqrt{3}}{2}\exp\left(-1/4\right)$

Exercise 2¶

\begin{align} f\left(x,y\right) = \begin{bmatrix} xy + \sin\left(x\right) \\ x + y + \sin\left(xy\right) \end{bmatrix}. \end{align}

Node	Elementary Function
$x = x_{1}$	$x_{1}$
$y = y_{1}$	$y_{1}$
$x_{3}$	$x_{1}x_{2}$
$x_{4}$	$x_{1} + x_{2}$
$x_{5}$	$\sin\left(x_{1}\right)$
$x_{6}$	$\sin\left(x_{3}\right)$
$x_{7}$	$x_{3} + x_{5}$
$x_{8}$	$x_{4} + x_{6}$

Let's start by computing the gradient of $f_{1}$. Using the directional derivative we have \begin{align} D_{p}x_{7} &= \dfrac{\partial x_{7}}{\partial x_{1}}p_{1} + \dfrac{\partial x_{7}}{\partial x_{2}}p_{2} \\ &= \left(\dfrac{\partial x_{7}}{\partial x_{3}}\dfrac{\partial x_{3}}{\partial x_{1}} + \dfrac{\partial x_{7}}{\partial x_{5}}\dfrac{\partial x_{5}}{\partial x_{1}}\right)p_{1} + \left(\dfrac{\partial x_{7}}{\partial x_{3}}\dfrac{\partial x_{3}}{\partial x_{2}} + \dfrac{\partial x_{7}}{\partial x_{5}}\dfrac{\partial x_{5}}{\partial x_{2}}\right)p_{2} \\ &= \left(x_{2} + \cos\left(x_{1}\right)\right)p_{1} + x_{1}p_{2}. \end{align} Similarly, \begin{align} D_{p}x_{8} &= \left(\frac{\partial x_{8}}{\partial x_{4}}\frac{\partial x_{4}}{\partial x_{1}} + \frac{\partial x_{8}}{\partial x_{6}}\frac{\partial x_{6}}{\partial x_{3}}\frac{\partial x_{3}}{\partial x_{1}}\right)p_{1} + \left(\frac{\partial x_{8}}{\partial x_{4}}\frac{\partial x_{4}}{\partial x_{2}} + \frac{\partial x_{8}}{\partial x_{6}}\frac{\partial x_{6}}{\partial x_{3}}\frac{\partial x_{3}}{\partial x_{2}}\right)p_{2} \\ &= \left(1 + \cos\left(x_{3}\right)x_{2}\right)p_{1} + \left(1 + \cos\left(x_{3}\right)x_{1}\right)p_{2}. \end{align} So, the Jacobian is \begin{align} J = \begin{bmatrix} y + \cos\left(x\right) & x \\ 1 + \cos\left(xy\right)y & 1 + \cos\left(xy\right)x \end{bmatrix} \end{align} Note: You should fill in the details! Choosing $p = (1,0)$ gives $\dfrac{\partial f_{1}}{\partial x}$ and choosing $p = (0,1)$ gives $\dfrac{\partial f_{1}}{\partial y}$. These form the first row of the Jacobian! The second row of the Jacobian can be computed similarly by working with $x_{8}$.