Here's a hint to a possible approach, although it's probably nowhere near the cleanest:
You know that the exponential series converges for all matrices. Consider
$$L^m=(-1)^m\left(\sum_{k=1}^{n-1} \frac{N^k}k\right)^m.$$
You have
$$\left(\sum_{k=1}^{n-1} \frac{N^k}{k}\right)^m=\sum_{\mathbf k\in [n-1]^m}\frac{N^{\sum_{i=1}^m k_i}}{\prod_{i=1}^m k_i},$$
where $\mathbf k$ is a vector of $m$ values of $k$ and $[r]$ denotes the set of integers from $1$ to $r$. You can rewrite this, using $N^n=0$, as
$$\sum_{j=0}^{n-1}N^j\sum_{\substack{\mathbf k\in [n-1]^m\\ \mathbf k\cdot\mathbf 1=j}}\frac1{\prod_{i=1}^m k_i}.$$
Write
$$a_{j,m}=\sum_{\substack{\mathbf k\in [n-1]^m\\ \mathbf k\cdot\mathbf 1=j}}\frac1{\prod_{i=1}^m k_i}.$$
Since the exponential series is always convergent, we have
\begin{align*}
\exp(L)
&=\sum_{m=0}^\infty \frac{L^m}{m!}\\
&=\sum_{m=0}^\infty (-1)^m\frac{1}{m!}\sum_{j=0}^{n-1} N^j a_{j,m}\\
&=\sum_{j=0}^{n-1} N^j\sum_{m=0}^\infty \frac{(-1)^ma_{j,m}}{m!}.
\end{align*}
So, we need to show that
$$\sum_{m=0}^\infty \frac{(-1)^ma_{j,m}}{m!}$$
is $1$ at $j=0$, $-1$ at $j=1$, and zero for larger $j$. See if you can prove this using generating functions and looking at this sum in terms of partitions of $j$.