Saz pointed out that Exercise II.3.15 in Revuz and Yor's Brownian motion and continuous martingales gives a negative answer to my question. Here's the counterexample, adapted to a discrete martingale for the sake of variety.
Consider a partition $(A_{k})_{k\geqslant 1}$ of a probability space $\Omega$. Let $(\alpha_{k})$ be any sequence of positive real numbers and set
\begin{equation*}
X_{\infty}=\sum_{k} \alpha_{k}\mathbf{1}_{A_{k}}.
\end{equation*}
This variable is integrable iff the series $S=\sum \alpha_{k}\mathbf{P}(A_{k})$ converges to a finite limit. Assume it is the case, and set $S_{n}$ for its partial sums, also let $P_{n}=\sum_{k=n}^{+\infty}\mathbf{P}(A_{k})$.
If we define $\mathcal{F}_{n}=\sigma(A_{k},k\leqslant n)$ and set $X_{n}=\mathbf{E}(X_{\infty}\mid \mathcal{F}_{n})$ we obtain a martingale for which we have the explicit expression
\begin{equation*}
X_{n}=\sum_{k=0}^{n}\alpha_{k}\mathbf{1}_{A_{k}} + \frac{S-S_{n}}{P_n}\mathbf{1}_{\cup_{k> n}A_{k}}
\end{equation*}
so that
\begin{equation*}
\sup_{n\in\mathbb{N}} X_{n} \geqslant \sum_{k} \frac{S-S_{k}}{P_{k}}\mathbf{1}_{A_{k}}.
\end{equation*}
One checks using the convergence criterion for Bertrand series that if $\mathbf{P}(A_{n})=\frac{1}{n(n+1)}$ and $\alpha_{n}=\frac{n}{\ln^{2}(n)}$ then indeed $S < +\infty$ but $||\sup_{n\in\mathbb{N}}X_{n}||_{1}=+\infty$.