A simple sufficient condition for your claim to be true is $f\ge0$.
Then, $$\Vert f \Vert_{L^1}=\phi(0)\le 2\Vert\phi\Vert_{L^\infty}.$$
In general however, I believe this is false.
Consider the Hilbert transform
$$H\ast \psi(x):=\lim_{\varepsilon\to0}\frac1\pi\int_{\varepsilon<| y| <1/\varepsilon} \frac{\psi(y)}{x-y}\,\mathrm d y.$$
Morally speaking $H= \frac{1}{\pi x}$ and the above is the convolution of $H$ with a function $\psi$. Its a well known fact that $\hat H(\xi)= -i\operatorname{sign}(\xi)$ and more generally, $\mathcal F(H\ast \psi)=-i\operatorname{sign}(\xi)\,\hat \psi.$
All this can be made rigorous in the sense of tempered distributions.
The idea to disproving the claim is to consider a sequence $\psi_\delta\longrightarrow\delta_0$, so that
$$
\Vert H\ast \psi_\delta \Vert_{L^1}\longrightarrow\infty,
\quad\text{while}\quad
\Vert \mathcal{F}(H\ast \psi_\delta)\Vert_{L^\infty}\le C.
$$
This rules out any possible estimate of the form $\Vert f\Vert_{L^1}\le C \Vert \phi \Vert_{L^\infty}$, for a constant $C>0$ to hold for all $f\in L^1$.
To get this more down to earth, let us explicitly choose
$$\psi_{\delta}(x):= \begin{cases}\frac{1}{2\delta} &x\in[-\delta,\delta]\\ 0& \text{else}\end{cases}.$$
You can compute $$H\ast \psi_\delta(x)=\frac{1}{2\pi\delta}\,\log\left(\left\vert\frac{x+\delta}{x-\delta}\right\vert\right)$$
and I am fairly confident that this is an $L^1$ function.
In particular $H\ast \psi_\delta(x)\longrightarrow \frac{1}{\pi x}$ uniformly on every interval $[r,\frac 1 r]$, $r>0$, from which you can deduce $\Vert H\ast \psi_\delta \Vert_{L^1}\longrightarrow\infty$, since $\frac{1}{\pi x}$ is not integrable.
On the Fourier side, $\mathcal F(\psi_\delta) = \frac{\sin(2\pi \delta\xi)}{2\pi \delta\xi}$ and thus
$$\Vert \mathcal{F}(H\ast \psi_\delta)\Vert_{L^\infty} = \left\Vert -i \operatorname{sign}(\xi) \frac{\sin(2\pi \delta\xi)}{2\pi \delta\xi}\right\Vert_{L^\infty}\le 1.$$
This should disprove your conjecture.
As a final remark, there are $L^p$-$L^q$-estimates for the Fourier transform. See e.g. the discussion in this post.