In the following, assume that rings are rings with unity.
Here is the definition of $R$-algebra from Wikipedia:
Let $R$ be a commutative ring and $(M,+,\cdot)$ an $R$-module.
Assume $\ast$ is a binary operation on $M$, such that:
- $x\ast (y+z)= x\ast y + x\ast z$
- $\forall x,y,z \in M.(x+y)\ast z = x\ast z + y\ast z$
- $\forall r,s \in R,x,y \in M.(rx)\ast (sy)=(rs)(x\ast y) (r,s\in R, x,y\in M)$
then $(M,+,\cdot,\ast)$ is called an $R$-algebra.
This definition has the same form as my definition for "algebra over a field". Note that this definition does not require $\ast$ to be associative.
Here is an equivalent definition given in Dummit&Foote (original form is given using a ring homomorphism):
Let $(M,+,\cdot)$ be a ring and $R$ a commutive ring.
If $M$ is an $R$-module and the multiplication on $M$ is bilinear, then $M$ is called "an $R$-algebra".
Note that this definition requires $\ast$ to be associative.
What is the usual definition for $R$-algebra?
References
Abstract Algebra, Dummit and Foote, 3rd edition, page 355.