The starting point strategy (optional)
Besides this integral, there were already two more related integrals posted here before, namely: $\int_0^\frac{\pi}{3}\arccos(2\sin^2 x-\cos x)dx$ and
$\int_{\arccos(1/4)}^{\pi/2}\arccos\left(\cos x\left(2\sin^2x+\sqrt{1+4\sin^4x}\right)\right)$, so it might be confusing to see a different starting point for each of them. However, all share the same target in that we want to arrive at a point where we can utilize:
$$\int_a^b \frac{\arctan\left(f(x)\frac{\sqrt{g(x)}}{\sqrt{h(x)}} \right)}{p(x)\sqrt{g(x)}\sqrt{h(x)}}dx=\int_a^b \frac{1}{p(x)}\int_0^{f(x)} \frac{1}{h(x)+g(x)y^2}dydx$$
The most crucial thing here is to get rid of the square roots. So, those functions can mostly be anything containing polynomials, but from what I've observed, it's best to ensure that $g(x)$ and $h(x)$ have similar "weights" - e.g. don't keep $g(x)=a+bx^2$ and $h(x)=x(m+nx^2)$, either add the $x$ term to $g(x)$ or remove it from $h(x)$. It's also acceptable for one of those to simply be $1$. Additionally, it would be great for $p(x)$ not to contain other square roots. Moreover, there are two useful ways to get from the $\arccos$ to the $\arctan$ form:
$$\arccos x=\arctan\frac{\sqrt{1-x^2}}{x}\ \text{ and } \arccos x = 2\arctan\sqrt{\frac{1-x}{1+x}} $$
So we need the $\arccos$ argument to be acceptable before utilizing one of those two identities. Having $2\sin^2 x-\cos x$ is awesome since it has no square roots so we can try to convert directly to the $\arctan $ form (the second identity behaves nicer in this case). Note that having no square roots doesn't automatically ensure that we will obtain something nice afterwards (but it's a good indicator to try and convert the argument directly).
Having $\cos x\left(2\sin^2x+\sqrt{1+4\sin^4x}\right)$ is horrible due to the fourth power, since we can't substitute out $\sqrt{1+4\sin^4 x}$ like we would do for a simpler $\sqrt{a^2+b^2\sin^2 x}$. Here's where the trick with Vieta's relations comes in handy - as shown in the second linked post.
In our case we don't have fourth powers, only $\sqrt{2-x^2}$, so we don't need the above trick. The argument however is even messier and we still need to remove the square root, or at least to isolate it. Here there were three substitutions that came to my mind: $\sqrt{2-x^2}\to x$, $\frac{\sqrt{2-x^2}}{x}\to x$ and the combination of $x\to \frac{1}{x}$ followed by the Euler substitution $\sqrt{2x^2-1}-\sqrt 2 x\to x$. Luckily, the second one worked nicely (I reversed it to keep the same bounds).
Evaluation of the integral
$$\mathcal I = \int_0^1 \frac{\arccos\left(\frac{3x^3-3x+4x^2\sqrt{2-x^2}}{5x^2-1}\right) }{\sqrt{1-x^2}}dx\overset{\large \frac{x}{\sqrt{2-x^2}}\to x} = \sqrt 2\int_0^1 \frac{\arccos\left(\frac{3+x}{1+3x}\frac{\sqrt{2}\, x}{\sqrt{1+x^2}}\right)}{\sqrt{1-x^2}(1+x^2)}dx$$
$$=\sqrt 2 \int_0^1 \frac{\arctan\left(\small{\frac{\sqrt{1-x^2}}{x}\color{red}{\frac{\sqrt{(1-x)(1+7x)}}{\sqrt 2\, (3+x)}}}\right)}{\sqrt{1-x^2}(1+x^2)}dx =
\sqrt 2\int_0^1 \frac{1}{1+x^2}\int_0^{\color{red}{f(x)}} \frac{x}{x^2+(1-x^2)y^2} dydx$$
$$\overset{\small y\to f(y)}= \sqrt 2 \int_0^1 \frac{1}{1+x^2} \int_x^1 \frac{-xf'(y)}{x^2+(1-x^2) f^2(y)} dydx = \int_0^1 \int_{x}^1 {()} dydx = \int_0^1 \int_0^y {()}dxdy$$
$$=\frac12 \int_0^1 \frac{(1-3y)\ln\left(\frac{(1-y)(1+7y)}{(1+3y)^2}\right)}{(1+y^2)\sqrt{(1-y)(1+7y)}}dy\overset{\large \sqrt{\frac{1-y}{1+7y}}\to y}
= 2\int_0^1 \frac{1-5y^2}{1+6y^2+25y^4}\ln\left(\frac{1+y^2}{2y}\right)dy$$
$$\overset{IBP}=\frac12\int_0^1{{\frac{(1-y^2)\ln\left(\frac{1+2y+5y^2}{1-2y+5y^2}\right)}{y(1+y^2)}}}dy \overset{\large y\to \frac{1-y}{1+y}}= 2\int_0^1 \frac{y\ln\left(\frac{2-2y+y^2}{1-2y+2y^2}\right)}{1-y^4}dy$$
$$\overset{\bigstar}=\int_0^\infty\frac{y\ln\left(\frac{2-2y+y^2}{1-2y+2y^2}\right)}{1-y^4}dy = \int_0^\infty\frac{y\ln\left(\frac{4+y^4}{1+4y^4}\right)+y\ln\left(\frac{1+2y+2y^2}{2+2y+y^2}\right)}{1-y^4}dy$$
$$\overset{\bigstar \bigstar} = 2\int_0^\infty \frac{y\ln\left(\frac{4+y^4}{1+4y^4}\right)+y\ln y}{1-y^4}dy\overset{y^2 \to \frac{1-y}{1+y}}= \frac12\int_{-1}^1 \frac{\ln\left(\frac{1+\frac{6}{5}y+y^2}{1-\frac{6}{5}y+y^2}\right)-\frac14\ln\left(\frac{1+2y+y^2}{1-2y+y^2}\right)}{y}dy$$
$$= \int_{-1}^1 \int_{-\arcsin \frac{3}{5}}^{\arcsin \frac{3}{5}} \frac{\cos t}{(y+\sin t)^2+\cos^2 t} dtdy - \frac14\int_{-1}^1 \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \frac{\cos t}{(y+\sin t)^2+\cos^2 t} dtdy$$
$$= \frac{\pi}{2} \int_{-\arcsin \frac{3}{5}}^{\arcsin \frac{3}{5}} dt -\frac{\pi}{8} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} dt = \pi \arcsin \frac35 - \frac{\pi^2}{8} = \boxed{\frac{3\pi^2}{8}-2\pi \arctan \frac12}$$
Note that above, $\small \bigstar$ stands for the combination of substituting $y \to \frac{1}{y}$ followed by averaging the resulting integral with the original one. Also, to show $\small \bigstar \bigstar$, we can proceed as follows:
$$\int_0^\infty \frac{y\ln\left(\frac{4+y^4}{1+2y+2y^2}\right)}{1-y^4}dy\overset{y\to \frac{1}{y}}=\int_0^\infty \frac{y\ln\left(\frac{1+4y^4}{2+2y+y^2}\right)-y\ln(y^2)}{1-y^4}dy$$
$$\Rightarrow \int_0^\infty \frac{y\ln\left(\frac{1+2y+2y^2}{2+2y+y^2}\right)}{1-y^4}dy=\int_0^\infty \frac{y\ln\left(\frac{4+y^4}{1+4y^4}\right)+y\ln(y^2)}{1-y^4}dy$$
