$$ \lim_{n\to\infty}\sqrt[n]{\int_0^1x^{\frac{n(n+1)}{2}}(1-x)(1-x^2)\cdots(1-x^n)dx} $$
$$ =\lim_{n\to\infty}\sum_{i=1}^n\frac{1}{4i(i+1)} $$
$$ =\lim_{n\to\infty}\sum_{i=1}^n\frac{1}{4}(\frac{1}{i}-\frac{1}{i+1}) $$
$$ =\lim_{n\to\infty}\frac{1}{4}(1-\frac{1}{n+1}) $$
$$ =\lim_{n\to\infty}\frac{1}{4}(\frac{n}{n+1}) $$
$$ =\frac{1}{4} $$
I found an answer, but I never understood where this first step came from.
Using AM $\ge$ GM
$$\sqrt{x^n(1-x^n)}\le\frac{x^n+(1-x^n)}2$$ $$x^n(1-x^n)\le \frac14$$
$$L=\lim_{n \to \infty } \sqrt[n]{\int_0 ^1 x^{\frac{n(n+1)}{2}}(1-x)(1-x^2)\cdots(1-x^n)d x}$$
$$L=\lim_{n \to \infty } \sqrt[n]{\int_0 ^1 x(1-x)x^2(1-x^2)\cdots x^n(1-x^n)d x}$$
$$L=\lim_{n \to \infty } \sqrt[n]{\int_0 ^1 \prod_{k=1}^nx^k(1-x^k)\,dx}\le\lim_{n\to\infty}\sqrt[n]{\int_0^1\left(\frac14\right)^ndx}=\boxed{\frac14}$$
