$$ \int_{0}^{\frac{\pi}{2}} (\log \sin x)^2 \, dx = \int_{0}^{\frac{\pi}{2}} (\log \cos x)^2 \, dx = \frac{\pi}{2}\left\{ \frac{\pi^2}{12} + (\log 2)^2 \right\} $$
I tried integration by parts, but I ended up with
$$ \int_{0}^{\frac{\pi}{2}} (\log \sin x)(\log \cos x) \, dx. $$
I tried to use exponential form for integration but I couldn't integrate further.
Since $\log^{n}(1+z)$ is holomorphic on the unit disc $\Bbb{D}$, we have
$$\int_{|z|=r} \frac{\log^{n}(1+z)}{z} \, dz = \log^{n}(1+0) = 0. $$
Plugging $z = r e^{i\theta}$ and taking $r \to 1^{-}$, we obtain
$$\int_{-\pi}^{\pi} \log^{n}(1 + e^{i\theta}) \, d\theta = 0. $$
(Of course this limiting process requires justification, but we omit this part and concentrate on the calculation itself.) Simplifying, we obtain
$$ \sum_{k=0}^{\lfloor n/2 \rfloor} (-1)^{k} \binom{n}{2k} \int_{0}^{\pi/2} \theta^{2k} \log^{n-2k} (2 \cos \theta) \, d\theta = 0. \tag{1} $$
Plugging $n = 1$ to $(1)$, we obtain
$$ \int_{0}^{\pi/2} \log \cos \theta \, d\theta = -\frac{\pi}{2}\log 2. \tag{2} $$
Plugging $n = 2$ to $(1)$,
$$ \int_{0}^{\pi/2} \log^{2}(2\cos\theta) \, d\theta = \int_{0}^{\pi/2} \theta^{2} \, d\theta = \frac{\pi^{3}}{24}. \tag{3} $$
Combining $(2)$ and $(3)$ gives the desired result:
$$ \int_{0}^{\pi/2} \log^{2} \sin\theta \, d\theta = \int_{0}^{\pi/2} \log^{2} \cos\theta \, d\theta = \frac{\pi^{3}}{24} + \frac{\pi}{2}\log^{2} 2. $$
