I came across this nice answer on stack:
However, two things still bother me. First, why do we assume a function that equals its own derivative will take the form of $a_nx^n$? Why should it be an exponential function? I can understand the reason for the sigma a little, it implies a function immune to derivatives should work regardless of the input changing from $0$ to infinity (why not negative infinity as well?)
Second, the condition $f(0) = 1$ also bothers me. So assuming we determine the function will be exponential in some way, I'm guessing this is a necessary condition as all exponential will be $1$ when using $0$ as input. Or is there something more to it?
