No. It is not true even for principal ideals unless one assumes more about the local ring $R$, e.g., that $R$ is a domain. For a simple example, let $R=k[\![x,y]\!]/(xy)$ and take $I=(x)$. Then, $R/I \cong k[\![y]\!]$ which is Cohen-Macaulay but $R/I^2 \cong k[\![x,y]\!]/(x^2,xy)$ which is famously not Cohen-Macaulay.
For more exotic examples, note there is a short exact sequence $0 \to I/I^2 \to R/I^2 \to R/I \to 0$ which relates Cohen-Macaulayness of $I/I^2$, which is known as the conormal module of $I$, to that of $R/I^2$. The conormal module (and thus $R/I^2$) puts up some resistance to being Cohen-Macaulay. In several situations it is known that Cohen-Macaulayness of the conormal module forces $R/I$ to be Gorenstein. Several of these situations are detailed in this work of Mantero-Xie. One can use these criteria to construct various types of examples.