I'm learning the basics about Brownian motion (I know nothing about stochastic calculus), and I've shown that if $W(t)$ is a standard Brownian motion, then $W(t)^2-t$ is a martingale. Now I'm trying to show that $$ f(W(t))-\frac{1}{2}\int_0^t f''(W(s))ds $$ is also a martingale, where $f\in C^2$ and compactly supported.
I've started by showing that the transition density satisfies the diffusion equation. Any guidance or hints would be appreciated!
Denote by
$$p(t,x) := \frac{1}{\sqrt{2\pi t}} \exp \left(- \frac{x^2}{2t} \right), \qquad x \in \mathbb{R},$$
the density of the normal distribution with mean $0$ and variance $t$. As you already noted, this function solves the heat equation, i.e.
$$\frac{\partial}{\partial t} p(t,x) = \frac{1}{2} \frac{\partial^2}{\partial x^2} p(t,x).$$
For $f \in C^2$ compactly supported it follows easily from the integration-by-parts-formula that
$$\int p(t,x) \frac{1}{2} \frac{\partial^2}{\partial x^2} f(x) \, dx = \int f(x) \frac{\partial}{\partial t} p(t,x) \, dx. \tag{1}$$
For $f \in C_b^2$ we set $$M_t^f := f(W_t) - f(W_0) - \int_0^t \frac{1}{2} f''(W_r) \, dr.$$ We have to show that $$\mathbb{E}(M_t^f - M_s^f \mid \mathcal{F}_s) = 0 \qquad \text{for all $s \leq t$}.$$ Since $(W_t)_{t \geq 0}$ has independent and stationary increments, we know that $W_t-W_s$ and $\mathcal{F}_s$ are independent and $W_t-W_s \sim W_{t-s}$. Therefore,
$$\begin{align*} \mathbb{E}(M_t^f - M_s^f \mid \mathcal{F}_s) &= \mathbb{E}(f((W_t-W_s)+W_s) \mid \mathcal{F}_s) - f(W_s) \\ &\quad - \frac{1}{2} \int_s^t \mathbb{E}(f''((W_r-W_s)+W_s) \mid \mathcal{F}_s) \, dr \\ &= \mathbb{E} f(W_{t-s}+z) \bigg|_{z=W_s} - f(W_s) - \frac{1}{2} \int_s^t\mathbb{E}[f''(z+(W_r-W_s))] \big|_{z=W_s} \, dr \\ &= \mathbb{E} f(W_{t-s}+z) \bigg|_{z=W_s} - f(W_s) - \frac{1}{2} \int_0^{t-s} \mathbb{E}[f''(z+W_r)] \big|_{z=W_s} \, dr \\ &= \mathbb{E} \left[ f(z+W_{t-s})-f(z) - \frac{1}{2} \int_0^{t-s} f''(W_r+z) \, dr \right] \bigg|_{z=W_s} \end{align*}$$
(In the last step, we have used that $(W_t)_{t \geq 0}$ has stationary increments, i.e. $W_r-W_s \sim W_{r-s}$, and a change of variables.) Setting $\varphi(x) := f(x+z)$ for fixed $z \in \mathbb{R}$, we therefore conclude that it suffices to show that
$$\mathbb{E}(M_{t-s}^{\varphi} - M_0^{\varphi}) = \mathbb{E} \left[ \varphi(W_{t-s}) - \varphi(0)- \frac{1}{2} \int_0^{t-s} \varphi''(W_r) \, dr \right] = 0. \tag{2}$$
Now fix $0 <\epsilon < u$. Then, by Fubini's theorem ("Fub", for short)
$$\begin{align*} \mathbb{E}(M_u^\varphi-M_\varepsilon^\varphi) &= \mathbb{E}\left(\varphi(W_u) - \varphi(W_{\varepsilon}) - \frac{1}{2} \int_{\varepsilon}^u \varphi''(W_r) \, dr \right) \\ &\stackrel{\text{Fub}}{=} \int_{\mathbb{R}} \varphi(x) \cdot p(u,x) \, dx - \int_{\mathbb{R}} \varphi(x) \cdot p(\varepsilon,x) \, dx - \frac{1}{2} \int_{\varepsilon}^u \int_{\mathbb{R}} \varphi''(x) \cdot p(r,x) \, dx \, dr \\ &\stackrel{(1)}{=} \int_{\mathbb{R}} \varphi(x) \cdot (p(u,x)-p(\varepsilon,x)) \, dx - \int_{\varepsilon}^u \int_{\mathbb{R}} \varphi(x) \cdot \frac{\partial}{\partial r} p(r,x) \, dx \, dr \\ & \overset{\text{Fub}}{\underset{\varepsilon>0}{=}}\int_{\mathbb{R}} \varphi(x) \cdot (p(u,x)-p(\varepsilon,x)) \, dx - \int_{\mathbb{R}} \varphi(x) \cdot \underbrace{\int_{\varepsilon}^u \frac{\partial}{\partial r} p(r,x) \, dr}_{p(u,x)-p(\varepsilon,x)} \, dx \\ &= 0 \tag{3} \end{align*}$$
Since $M_{\epsilon}^{\varphi} \to M_0^{\varphi}$ as $\epsilon \to 0$ (as $f$ is continuous) and $$\|M_{\epsilon}^f\|_{\infty} \leq 2 \|f\|_{\infty} + u \|f''\|_{\infty}, \qquad 0 < \epsilon < u$$ it follows from the dominated convergence theorem that we can let $\epsilon \to 0$ and obtain
$$0 \stackrel{(3)}{=} \lim_{\epsilon \to 0} \mathbb{E}(M_u^\varphi-M_\varepsilon^\varphi) = \mathbb{E}(M_u^\varphi-M_0^\varphi).$$
This proves $(2)$.
Remark: This answer follows the proof presented by René Schilling and Lothar Partzsch in Brownian Motion - An Introduction to Stochastic Processes, Chapter 5.