No, that is even implicitly required in the definition you're citing:
An algebra $A$ is a division algebra if given $a,b \in A$ with $ab
= 0$, then either $a = 0$ or $b = 0$. Equivalently, $A$ is a division algebra if the operations of left and right multiplication by any nonzero element are invertible.
Earlier:
For us a vector space will always be a finite-dimensional module over the field of real numbers. An algebra $A$ will be a vector space that is equipped with a bilinear map $m : A \times A \to A$ called 'multiplication' and a nonzero element $1 \in A$ called the 'unit' ...
If $a$ is nonzero, then left multiplication by $a$ is an injective endomorphism of a finite dimensional vector space, and hence it is also surjective. This means there exists a $b$ such that $ab=1$.
The whole point of a division algebra is that you can divide...