Here's an example of such a structure.
$$\begin{array}{c|ccc}
\cdot & e & a & b \\
\hline
e & e & a & b\\
a & a & b & a\\
b & b & b & a\\
\end{array}$$
We see that $e$ is an identity element. However, $(S, \cdot)$ is not associative, since $(a\cdot b)\cdot b = a\cdot b = a$, while $a\cdot (b\cdot b) = a \cdot a = b$.
Also, $(S, \cdot)$ does not have inverses. There is no element $x\in S$ such that $a\cdot x = e$.
Also, though you didn't mention commutativity, $(S, \cdot)$ is not commutative: $a\cdot b = b$, while $b\cdot a = a$. We actually could make it commutative, though:
$$\begin{array}{c|ccc}
\star & e & a & b \\
\hline
e & e & a & b\\
a & a & b & b\\
b & b & b & a\\
\end{array}$$
$(S, \star)$ still has an identity, still lacks inverses, and is now commutative. And $(S, \star)$ is still non-associative, since $(a\star b)\star b = b\star b = a$, while $a \star (b\star b) = a \star a = b$.
I guess one way to make a magma with identity is to start with whatever magma you want, and then add another element, $e$, and extend your binary operation to make $e$ act as an identity. So I think this might explain why the some people suggested in the comments that this type of structure might not be very interesting, and why it doesn't have its own name.