To get rid of some square roots and simplify, substitute $(a,b)\to (2x, 2y)$. The inequality can be written as:
Let $x,y >0$ and $\frac{1}{x}+\frac{1}{y} = 2$. Prove that:
$$\sqrt{2(x^2+1)}+\sqrt{2(y^2+1)} \leq (x+y)^2$$
In this case, from AM-GM we can see that:
$$\sqrt{2(x^2+1)}+\sqrt{2(y^2+1)} \leq \frac{x^2+1+2}{2}+\frac{y^2+1+2}{2} = \frac{x^2+y^2}{2}+3$$
It will be enough to prove:
$$\frac{x^2+y^2}{2}+3 \leq (x+y)^2$$
or
$$x^2+y^2+4xy \geq 6$$
From AM-GM on the initial condition, we can see that $xy \geq 1$. Therefore
$$x^2+y^2+4xy \geq 6xy \geq 6$$
Equality occurs when $(x,y) = (1,1)$, so when $(a,b)=(2,2)$.