I know that a derivate of a function can be calculated by multiplication in the frequency domain:
$$f'(x) = \mathscr{F}^{-1}(j \omega \mathscr{F}(f(x))) $$
Since multiplication in frequency is convolution in the time domain I was thinking. How can I represent the derivative using convolution?
$$ f'(x) = f(x) \ast g(x) $$
What's is $g$ above? It should be the inverse of $j \omega$.
This is an interesting question. I will use the convention that
$$\mathscr F f(\xi)=\int_{\mathbb{R}}f(x)e^{-2\pi i\xi x}\,\mathrm{d}x.$$
Make the relevant adjustments for your convention.
So what we start with it observing that we have the relation
$$\mathscr F f'(\omega)=2\pi i \omega\mathscr F f(\omega).$$
In view of the convolution theorem, saying that we have
$$\mathscr F(g*h)=\mathscr F g\cdot\mathscr F h,$$
it is natural to ask whether we can write $2\pi i\omega$ as a Fourier transform of some integrable function and apply this to find $f'$ as a convolution by inverting the Fourier transform. It unfortunately turns out that this is not possible in the normal sense of functions. Indeed if there was such a function $g\in L^1(\mathbb R)$, then by the Riemann-Lebesgue lemma we would have that
$$\lim_{\omega\to\infty}2\pi i\omega=\lim_{\omega\to\infty}\mathscr F g(\omega)=0,$$
which clearly cannot happen. So how can we resolve this? Well we can allow $2\pi i\omega$ to be a Fourier transform of something more general, namely a tempered distribution. I will not go into all the technicalities, but essentially a distribution is a linear functional $\psi$ defined on a space of test functions, meaning that it takes in a test function $\varphi$ and produces a number $\psi(\varphi)$. We often write this as $\psi(\varphi)=\langle\psi,\varphi\rangle$. Now how is this relevant? Well any function $g\in L^1(\mathbb{R})$ defines a tempered distribution $\psi_g$ by defining
$$\langle\psi_g,\varphi\rangle=\int_{\mathbb R}g(x)\varphi(x)\,\mathrm{d}x=\langle g,\varphi\rangle.$$
Tempered distributions also have a lot of properties similar to usual functions. For example you can differentiate them by defining $\psi'$ to be the distribution given by $\langle\psi',\varphi\rangle=-\langle\psi,\varphi'\rangle$. One can then verify by integrating by parts that this distributional derivative acts as one would expect on the distributions defined by nice enough functions: it simply gives us the distribution of the derivative of the function. Similarly one can define a Fourier transform of tempered distributions by $\langle\mathscr F\psi,\varphi\rangle=\langle\psi,\mathscr F\varphi\rangle$. This operation also behaves as one would want when the distribution is given by a function. The final operation we need is the convolution of a distribution by a test function: $\langle\psi*\phi,\varphi\rangle=\langle\psi,\varphi*\tilde\phi\rangle,$ where $\tilde\phi(x)=\phi(-x)$. We mention also that multiplying a tempered distribution by a test function is done in the obvious way.
Looking back at the original question of whether $2\pi i\omega$ is the Fourier transform of a function, which we know is not the case, we can now instead ask whether the distribution it defines the Fourier transform of another distribution. It turns out that this actually is the case. For this, we look first a distribution you've probably seen before: the Dirac delta distribution $\delta$ defined by $\langle\delta,\varphi\rangle=\varphi(0)$. As this is a tempered distribution, we can look at it's distributional derivative $\delta'$. We have that, by definition,
$$\langle\delta',\varphi\rangle=-\langle\delta,\varphi'\rangle=-\varphi'(0).$$
So we have that $\delta'$ is given by $\langle\delta',\varphi\rangle=-\varphi'(0)$. Now what is the Fourier transform of $\delta'$? By definition we have that
$$\langle\mathscr F\delta',\varphi\rangle=\langle\delta',\mathscr F\varphi\rangle=-(\mathscr F\varphi)'(0).$$
But what is $-(\mathscr F\varphi)'(0)$ for some arbitrary test function $\varphi$? Well
$$-(\mathscr F\varphi)'(0)=-\frac{\mathrm{d}}{\mathrm{d}\xi}\int_{\mathbb{R}}\varphi(x)e^{-2\pi i\xi x}\,\mathrm{d}x\biggr|_{\xi=0}=-\int_{\mathbb R}(-2\pi ix)\varphi(x)e^{-2\pi i\xi x}\,\mathrm{d}x\biggr|_{\xi=0}=\int_{\mathbb R}2\pi i x\varphi(x)\,\mathrm{d}x.$$
But this is the formula for the distribution associated with the function $2\pi i x$. What we have found is thus that $2\pi i x$ is the Fourier transform of the distribution $\delta'$. Thus, if we interpret all functions as their associated distributions, we have that
$$\mathscr F f'=\mathscr F\delta'\cdot\mathscr F f=\mathscr F(\delta'*f),$$
and so
$$f'=\delta'*f$$
as distributions. I've left out of a lot of the mathematical details here, but hopefully this gives you a better understanding of what's going on!
My background is from Electrical Engineering, and, based on your notation, I am assuming you come from a related field as well. From what I've seen, any time someone wants to represent differentiation as a convolution (or an LTI system), they use $\delta'$ or $\{ \delta'(t) \}$, in circuits and signals and systems notation, where $\delta$ is the Dirac delta function. Like the Dirac delta, $\delta'$ is a generalized function.
If you are interested in when linear operators can be written as an inner product between functions, you could see the Riesz Representation Theorem, which assumes a functional that is linear (like differentiation) and bounded (unlike differentiation).
