User:Hypercomplex Automatic Differentiation/sandbox

= Complex Taylor Series Expansion (CTSE) = In the realm of numerical design optimization, discrete sensitivity analysis serves as a valuable tool for approximating the derivative of an objective function concerning the design parameters. The merits of steady-state discrete sensitivity analysis lie in its efficiency and numerical precision. Recently, the complex Taylor's series expansion (CTSE) method has emerged as a means to achieve machine-accurate derivatives within the mathematics scene. This approach involves analyzing a complex perturbation of the objective function, enabling precise computation of the derivatives.

Why Use CTSE?
CTSE hosts an extensive array of applications spanning various disciplines and fields of study. Not only is the application of CTSE broad, but also incredibly accurate, preforming derivatives down to machine accuracy. For instance, within mechanical engineering, the calculation of energy release rates concerning strain energy in response to crack perturbations can be achieved through incremental crack extension along the imaginary axis of a complex coordinate system. This methodology yields a solution for the strain energy function with a complex component, wherein the imaginary part encapsulates the energy release rate. In the realm of chemical engineering, the development of atmospheric chemistry-transport models to comprehend emission effects on the atmosphere involved assigning complex values to multivalued functions, thereby deriving equations to quantify the influence of particulate emissions.

What is Required for CTSE?
A optimally small step size is required to preform CTSE.

Example 1
Complex variables can be used to compute accurate first order sensitivities. The derivation starts from the classical Taylor series expansion shown below.

$$f(x+h)\approx f(x)+{df \over dx}h+\frac {1}{2}{d^2f \over dx^2}h^2+\frac {1}{3!}{d^3f \over dx^3}h^3+\frac {1}{4!}{d^4f \over dx^4}h^4+\operatorname{H.O.T.} $$

where h is the step, $${d^nf \over dx^n} $$ is the 𝑛’th derivative at (𝑥), and H.O.T. denotes the higher order terms.

If we replace real variable step ℎ by an imaginary step 𝑖ℎ, we obtain the following

$$\begin{align} f(x+ih) & \approx f(x)+{df \over dx}(x)(ih)+\frac {1}{2}{d^2f \over dx^2}(x)(ih)^2+\frac {1}{3!}{d^3f \over dx^3}(x)(ih)^3+\frac {1}{4!}{d^4f \over dx^4}(x)(ih)^4+\operatorname{H.O.T.} \\ &=f(x)+{df \over dx}(x)(ih)-\frac {1}{2}{d^2f \over dx^2}(x)(h)^2-\frac {1}{3!}{d^3f \over dx^3}(x)(ih)^3+\frac {1}{4!}{d^4f \over dx^4}(x)(h)^4+\operatorname{H.O.T.} \end{align} $$

The end result, separated into real and imaginary terms, is

$$f(x+ih)\approx\left(f(x)-\frac {1}{2}{d^2f \over dx^2}h^2\right)+i\left({df \over dx}h-\frac {1}{3!}{d^3f \over dx^3}ih^3\right)+\operatorname{H.O.T.}$$

The real result is

$$\Re(f(x+ih))=f(x) -\frac {1}{2}{d^2f \over dx^2}+\operatorname{H.O.T.}$$

The imaginary result is

$$\Im(f(x+ih))={df \over dx}h-\frac {1}{3!}{d^3f \over dx^3}ih^3+\operatorname{H.O.T.}$$

For sufficiently small ℎ

Real part unchanged.

$$f \approx \Im(f(x+ih))$$

Imaginary part contains the first derivative.

$${df \over dx} \approx \frac {\Im(f(x+ih))}{h}$$

Example 2
$$\begin{align} f(x)&=\sin(x) \\ (f(x+ih))&=\sin(x+ih) \\ &=\sin(x)\cos(ih)+\cos(x)\sin(ih) \\ &=\sin(x)\cosh(h)+i\cos(x)\sinh(h) \end{align}$$

$$\begin{align} {df \over dx}&=\lim_{h \to 0} \frac {\Im\left((f(x+ih))\right)}{h} \\ &=\lim_{h \to 0}\cos(x)\frac {\sin(h)}{h} \\ &=\cos(x) \end{align} $$

CTSE is exact in the limit as $${h \to 0} $$ since $$\lim_{h \to 0} \frac {\sinh(h)}{h} $$

=1

Example 3
$$\begin{align} f(x)&=x^3 \\ f(x+ih)&=(x+ih)^3 \\ &=(x^3-3h^2x)+i(3hx^2-h^3) \end{align}$$

$$\begin{align} {df \over dx} &=\lim_{h \to 0} \frac {Im(f(x+ih))}{h} \\ &=\lim_{h \to 0} \frac {3hx^2-h^3}{h} \\ &=\lim_{h \to 0}{3x^2-h^2} \\ &=3x^2 \end{align} $$

CTSE is exact in the limit as $${h \to 0} $$