I’m looking for various ways of either calculating or reducing the integral to simple, related harmonic series, $$\int_0^1 \frac {\arctan^3(x) \log(x)} x \,\mathrm dx $$ $$= \frac {121} {15360} \pi^5 +\log^2(2) \frac {\pi^3} {32} -\frac{\pi}{32} \log^4(2) -\frac{21}{32} \log (2)\pi \zeta(3) -\frac{3}{4} \pi \operatorname{Li}_4\left(\frac{1}{2}\right).$$
Since I didn’t find it on MSE (https://approach0.xyz/ seems to be down), the first question is: Do we have any posts here about this specific integral?
I’ll present two solutions proposed by C.I.Valean, and I would like to see more later if possible.
First solution: Let’s start with the special Fourier series presented in More (Almost) Impossible Integrals, Sums, and Series: A New Collection of Fiendish Problems and Surprising Solutions (2023), page $448$, $$\log^2(\tan(x))=\log^2(\cot(x))$$ $$=\frac{\pi^2}{4}+\sum_{n=1}^{\infty}\left( (1-(-1)^{n-1})\int_0^1 t^{n-1}\left(\frac{1}{n}\frac{1-t}{1+t}-2\log\left(\frac{1-t}{2\sqrt{t}}\right)\right)\,\mathrm dt\right)\cos(2n x)$$ $$=\frac{\pi^2}{4}+2\sum_{n=1}^{\infty} (1-(-1)^{n-1})\left(\frac{H_n}{n}+\frac{\overline{H}_n}{n}\right)\cos(2n x)$$ $$=\frac{\pi^2}{4}+2\sum_{n=1}^{\infty} \left(2\frac{H_{2n}}{n}-\frac{H_n}{n}\right)\cos(4n x) $$ $$=\frac{\pi^2}{4}+8\sum_{n=1}^{\infty}\left(\int_0^1 t^{2n-1}\operatorname{arctanh}(t)\mathrm dt \right)\cos(4n x),\ 0<x<\frac{\pi}{2}, \tag1$$ where $H_n=\sum_{k=1}^n \frac{1}{k}$ is the $n$th harmonic number and $\overline{H}_n=\sum_{k=1}^n(-1)^{k-1}\frac{1}{k}$ is the $n$th skew-harmonic number, and the last line follows immediately from the simple fact that $\displaystyle \int_0^1 t^{2n-1}\operatorname{arctanh}(t)\textrm{d}t=\frac{1}{2}\frac{H_{2n}}{n}-\frac{1}{4}\frac{H_n}{n}$, also found in the sequel, page $28$ (just good to know it, but this last form won't be used in the calculations below).
Combining integration by parts and a simple substitution, and using the Fourier series above, we have $$ \int_0^1 \frac{\arctan^3(x)\log(x)}{x} \textrm{d}x=-\frac{3}{2}\int_0^1 \frac{\arctan^2(x)\log^2(x)}{1+x^2} \,\mathrm dx$$ $$=-\frac{3}{2}\int_0^{\pi/4} x^2 \log^2(\tan(x)) \,\mathrm dx$$ $$=-\frac{3}{2}\int_0^{\pi/4} x^2 \left(\frac{\pi^2}{4}+2\sum_{n=1}^{\infty} \left(2\frac{H_{2n}}{n}-\frac{H_n}{n}\right)\cos(4n x)\right) \,\mathrm dx$$ $$=-\frac{3}{8}\pi^2\int_0^{\pi/4} x^2 \,\mathrm dx-3 \sum_{n=1}^{\infty} \left(2\frac{H_{2n}}{n}-\frac{H_n}{n}\right)\underbrace{\int_0^{\pi/4} x^2 \cos(4nx)\,\mathrm dx}_{\displaystyle (\pi/32) (-1)^n/n^2}$$ $$ =-\frac{\pi^5}{512}-\frac{3}{32}\pi\sum_{n=1}^{\infty} (-1)^{n-1} \frac{H_n}{n^3}+\frac{3}{16}\pi\sum _{n=1}^{\infty} (-1)^{n-1} \frac{H_{2 n}}{n^3}$$ $$=\frac{121}{15360}\pi^5+\log^2(2)\frac{\pi^3}{32}-\frac{\pi }{32}\log^4(2)-\frac{21}{32} \log (2)\pi \zeta(3) -\frac{3}{4} \pi \operatorname{Li}_4\left(\frac{1}{2}\right),$$ where to get that last equality, we exploited that $$\sum_{n=1}^{\infty}(-1)^{n-1} \frac{H_n}{n^3}$$ $$=\frac{11}{360}\pi^4-\frac{7}{4}\log(2)\zeta(3)+\frac{1}{12}\log^2(2)\pi^2-\frac{1}{12}\log^4(2)-2 \operatorname{Li}_4\left(\frac{1}{2}\right), \tag2 $$ shown both in the previous book, page $421$, and in (Almost) Impossible Integrals, Sums, and Series (2019), pages $309$-$310$, and then $$ \sum_{n=1}^{\infty}(-1)^{n-1}\frac{H_{2n}}{n^3}$$ $$=\frac{195}{32}\zeta(4)+\frac{5}{4}\log^2(2)\zeta(2)-\frac{35}{8}\log(2)\zeta(3)-\frac{5}{24}\log^4(2)-5\operatorname{Li}_4\left(\frac{1}{2}\right), \tag 3 $$ which appears in the sequel, pages $449$-$450$, or here How to find $\sum_{n=1}^\infty\frac{(-1)^nH_{2n}}{n^3}$ and $\sum_{n=1}^\infty\frac{(-1)^nH_{2n}^{(2)}}{n^2}$ using real methods?, and the first solution is done.
Second solution: We start with the following special auxiliary integral, $$ \int_0^{\pi/2} \cos(2 n x) \left(\operatorname{arctanh}\left(\tan\left(\frac{x}{2}\right)\right)\right)^2\,\mathrm dx=\frac{\pi}{4}(-1)^n \frac{H_{2n}}{n}-\frac{\pi}{8}(-1)^n \frac{H_n}{n}, \tag 4 $$ which is derived in Another Valuable Elementary Generalized Trigonometric Integral with the Square of the Inverse Hyperbolic Tangent by C.I. Valean.
Therefore, if we multiply both side of $(4)$ by $1/n^2$, and then consider the sum from $n=1$ to $\infty$, where we also use the known Fourier series, $\displaystyle \sum_{n=1}^{\infty} \frac{\cos(2 n x)}{n^2}=\frac{\pi^2}{6}-\pi x+x^2, \ 0\le x\le\pi$, found in a more general form in the sequel, page $442$, we get $$ \frac{\pi}{8}\sum_{n=1}^{\infty}(-1)^{n-1} \frac{H_n}{n^3}-\frac{\pi}{4}\sum_{n=1}^{\infty}(-1)^{n-1} \frac{H_{2n}}{n^3} $$ $$ =\int_0^{\pi/2} \sum_{n=1}^{\infty} \frac{\cos(2 n x)}{n^2} \left(\operatorname{arctanh}\left(\tan\left(\frac{x}{2}\right)\right)\right)^2\,\mathrm dx $$ $$ =\int_0^{\pi/2} \left(\frac{\pi^2}{6}-\pi x+x^2\right) \left(\operatorname{arctanh}\left(\tan\left(\frac{x}{2}\right)\right)\right)^2\,\mathrm dx $$ $$ \overset{(1-\tan(x/2))/(1+\tan(x/2))=y}{=}\underbrace{\int_0^1 \frac{2\arctan^2(y)\log^2(y)}{1+y^2}\,\mathrm dy}_{\displaystyle \text{integrate by parts}}-\frac{\pi^2}{24}\underbrace{\int_0^1 \frac{\log^2(y)}{1+y^2}\,\mathrm dy}_{\displaystyle \pi^3/16} $$ $$ =-\frac{4}{3}\int_0^1 \frac{\arctan^3(y)\log(y)}{y}\,\mathrm dy-\frac{\pi^5}{384}, $$ follows by employing $(2)$ and $(3)$.
Note that $\displaystyle \int_0^1 \frac{\log^2(y)}{1+y^2}\,\mathrm dy=\sum_{n=0}^{\infty} (-1)^n \int_0^1 y^{2n} \log^2(y)\,\mathrm dy=2\sum_{n=0}^{\infty} \frac{(-1)^n}{(2n+1)^3}=\frac{\pi^3}{16}$, and classical ways of evaluating the last series may also be found in Sum : $\sum_{n=0}^\infty \frac{(-1)^n}{(2n+1)^3}$.
How else can we creatively approach or solve this problem? Maybe avoiding Fourier series?
UPDATE I: to @MariuszIwaniuk, related to the below comments, this is an example where the present integral is very helpful: Evaluate$\int_0^1 \frac{\operatorname{arctanh}^2\left(x^2\right) \log^2(x)}{1 +x^2} \mathrm dx$
UPDATE II (a bit longer): Below you may find a short extraction of the series form proposed in comments by Jorge Layja (one of the ways).
Based on the paper Deriving Special Fourier Series Involving the Inverse Tangent Integrals of Order Two and Three, and Other Curious Functions by C.I. Valean we have
$$\operatorname{Ti}_3(\cot(x))$$
$$=\sum_{n=1}^{\infty} \biggr(\frac{\pi^2}{12}\frac{1}{n}+\frac{\pi^2}{12}(-1)^{n-1}\frac{1}{n}+\log(2)\frac{H_n}{n}-\log(2) (-1)^{n-1}\frac{H_n}{n}+\frac{1}{2}\frac{H_n^2}{n}$$
$$+(-1)^{n-1}\frac{H_n \overline{H}_n}{n}+\log(2)\frac{\overline{H}_n}{n}-\log(2)(-1)^{n-1}\frac{\overline{H}_n}{n}-\frac{1}{2}\frac{\overline{H}_n^2}{n}+(-1)^{n-1}\frac{\overline{H}_n^{(2)}}{n}$$
$$-2(-1)^{n-1}\frac{1}{n}\sum_{k=1}^n (-1)^{k-1} \frac{H_k}{k}\biggr)\sin (2 n x)$$
$$=\sum_{n=1}^{\infty}\biggr(\frac{\pi^2}{12}\frac{1}{n}+\frac{\pi^2}{12}(-1)^{n-1}\frac{1}{n}+\log(2)\frac{H_n}{n}-\log(2) (-1)^{n-1}\frac{H_n}{n}+\frac{1}{2}\frac{H_n^2}{n}$$
$$
-(-1)^{n-1}\frac{H_n \overline{H}_n}{n}+\log(2)\frac{\overline{H}_n}{n}-\log(2)(-1)^{n-1}\frac{\overline{H}_n}{n}-\frac{1}{2}\frac{\overline{H}_n^2}{n}-(-1)^{n-1}\frac{\overline{H}_n^{(2)}}{n}$$
$$+2(-1)^{n-1}\frac{1}{n}\sum_{k=1}^n \frac{\overline{H}_k}{k}\biggr)\sin (2 n x);$$
$$\operatorname{Ti}_3(\tan(x))$$
$$=\sum_{n=1}^{\infty} \biggr(\frac{\pi^2}{12}\frac{1}{n}+\frac{\pi^2}{12}(-1)^{n-1}\frac{1}{n}-\log(2) \frac{H_n}{n}+\log(2)(-1)^{n-1}\frac{H_n}{n}+\frac{1}{2}(-1)^{n-1}\frac{H_n^2}{n}$$
$$
+\frac{H_n \overline{H}_n}{n}-\log(2)\frac{\overline{H}_n}{n}+\log(2)(-1)^{n-1}\frac{\overline{H}_n}{n}-\frac{1}{2}(-1)^{n-1}\frac{\overline{H}_n^2}{n}+\frac{\overline{H}_n^{(2)}}{n}$$
$$-2\frac{1}{n}\sum_{k=1}^n (-1)^{k-1} \frac{H_k}{k}\biggr)\sin (2 n x)$$
$$=\sum_{n=1}^{\infty}\biggr(\frac{\pi^2}{12}\frac{1}{n}+\frac{\pi^2}{12}(-1)^{n-1}\frac{1}{n}-\log(2)\frac{H_n}{n}+\log(2)(-1)^{n-1}\frac{H_n}{n}+\frac{1}{2}(-1)^{n-1}\frac{H_n^2}{n}$$
$$-\frac{H_n \overline{H}_n}{n}-\log(2)\frac{\overline{H}_n}{n}+\log(2)(-1)^{n-1}\frac{\overline{H}_n}{n}-\frac{1}{2}(-1)^{n-1}\frac{\overline{H}_n^2}{n}-\frac{\overline{H}_n^{(2)}}{n}$$
$$+2\frac{1}{n}\sum_{k=1}^n \frac{\overline{H}_k}{k}\biggr)\sin (2 n x),$$
where $H_n^{(m)}=1+\frac{1}{2^m}+\cdots+\frac{1}{n^m}, \ m\ge1$, is the $n$th generalized harmonic number of order $m$, $\overline{H}_n^{(m)}=1-\frac{1}{2^m}+\cdots+(-1)^{n-1}\frac{1}{n^m}, \ m\ge1$, represents the $n$th generalized skew-harmonic number of order $m$, $\zeta$ denotes the Riemann zeta function, $\displaystyle \operatorname{Ti}_2(x)=\int_0^x \frac{\arctan(t)}{t}\textrm{d}t$ designates the Inverse tangent integral of order two, and
$\displaystyle \operatorname{Ti}_3(x)=\int_0^x \frac{1}{t}\left(\int_0^t\frac{\arctan(u)}{u}\mathrm du\right) \mathrm dt$ signifies the Inverse tangent integral of order three.
They can be put into simpler forms by using integrals, and then we have
$$\operatorname{Ti}_3(\cot(x))=2\sum_{n=1}^{\infty}\left(\int_0^1 t^{n-1} \operatorname{arctanh}^2(t)\mathrm dt\right)\sin(2nx)$$
and
$$\operatorname{Ti}_3(\tan(x))=2\sum_{n=1}^{\infty}(-1)^{n-1}\left(\int_0^1 t^{n-1} \operatorname{arctanh}^2(t)\mathrm dt\right)\sin(2nx),$$
and the series result follows upon using $$\operatorname{Ti}_3\left(x\right)+\operatorname{Ti}_3\left(\frac{1}{x}\right)=\displaystyle \operatorname{sgn}(x)\left(\frac{\pi^3}{16}+\frac{\pi}{4}\log^2(|x|)\right),$$
proved in the sequel, pages $279$-$280$, and we are done.
Above, I also used that $$ \int_0^1 x^{n-1}\operatorname{arctanh}^2(x)\mathrm dx$$ $$=\frac{\pi^2}{24}\frac{1}{n}+\frac{\pi^2}{24}(-1)^{n-1}\frac{1}{n}+\frac{1}{2}\log(2)\frac{H_n}{n}-\frac{1}{2}\log(2) (-1)^{n-1}\frac{H_n}{n}+\frac{1}{4}\frac{H_n^2}{n}$$ $$ +\frac{1}{2}(-1)^{n-1}\frac{H_n \overline{H}_n}{n}+\frac{1}{2}\log(2)\frac{\overline{H}_n}{n}-\frac{1}{2}\log(2)(-1)^{n-1}\frac{\overline{H}_n}{n}-\frac{1}{4}\frac{\overline{H}_n^2}{n}+\frac{1}{2}(-1)^{n-1}\frac{\overline{H}_n^{(2)}}{n}$$ $$-(-1)^{n-1}\frac{1}{n}\sum_{k=1}^n (-1)^{k-1} \frac{H_k}{k}$$ $$=\frac{\pi^2}{24}\frac{1}{n}+\frac{\pi^2}{24}(-1)^{n-1}\frac{1}{n}+\frac{1}{2}\log(2)\frac{H_n}{n}-\frac{1}{2}\log(2) (-1)^{n-1}\frac{H_n}{n}+\frac{1}{4}\frac{H_n^2}{n}$$ $$-\frac{1}{2}(-1)^{n-1}\frac{H_n \overline{H}_n}{n}+\frac{1}{2}\log(2)\frac{\overline{H}_n}{n}-\frac{1}{2}\log(2)(-1)^{n-1}\frac{\overline{H}_n}{n}-\frac{1}{4}\frac{\overline{H}_n^2}{n}-\frac{1}{2}(-1)^{n-1}\frac{\overline{H}_n^{(2)}}{n}$$ $$+(-1)^{n-1}\frac{1}{n}\sum_{k=1}^n \frac{\overline{H}_k}{k},$$ which is given in the sequel, pages $28$-$29$. In the first place, this is the integral used to get the Fourier series of $\operatorname{Ti}_3(\tan(x))$ and $\operatorname{Ti}_3(\cot(x))$ in the form with harmonic numbers, shown in the paper.
Therefore, we have $$-\frac{\pi ^2}{4}+\frac{16}{\pi }\sum _{n=1}^{\infty }\left(\int _0^1t^{2n-2}\operatorname{arctanh}^2\left(t\right)\mathrm dt\right)\sin \left(2\left(2n-1\right)x\right)=\log ^2\left(\tan \left(x\right)\right),$$ where $ 0<x<\pi/2$.
UPDATE III: A nice fact to note is that combining any of the two solutions in this post and Jorge Layja solution leads to a new extraction of the famous tough harmonic series, $\displaystyle \sum_{n=1}^{\infty}(-1)^{n-1}\frac{H_{2n}}{n^3}$. So, there is hope that we may attack similarly the more advanced series of weight $6$, $\displaystyle \sum_{n=1}^{\infty}(-1)^{n-1}\frac{H_{n}}{n^5}$ and $\displaystyle \sum_{n=1}^{\infty}(-1)^{n-1}\frac{H_{2n}}{n^5}$.