Let $ABC$ be a right triangle with angle $B=\frac{\pi}{2}$. Let $E$ and $F$ be the midpoint of $AB$ and $AC$ respectively. If $I$ the in centre of $ABC$ lies on the circumcircle of $AEF$, find the ratio $BC/AB$.
My guess would be that $AB=BC$ but I am unable to prove it. I tried angle chasing but that doesn't seem to lead me anywhere. Or is this possible by using coordinates?
Without loss of generality you may choose coordinates
$$A=\begin{pmatrix}0\\0\end{pmatrix}\qquad B=\begin{pmatrix}1\\0\end{pmatrix}\qquad C=\begin{pmatrix}1\\\lambda\end{pmatrix}\qquad E=\begin{pmatrix}1/2\\0\end{pmatrix}\qquad F=\begin{pmatrix}1/2\\\lambda/2\end{pmatrix}$$
Now the barycentric coordinates of the incenter are $a:b:c$ and we have $a=\lambda$, $b=\sqrt{1+\lambda^2}$ and $c=1$. So the incenter has coordinates
$$I = \frac{aA+bB+cC}{a+b+c}= \frac{1}{\lambda+\sqrt{1+\lambda^2}+1}\begin{pmatrix} \sqrt{1+\lambda^2}+1\\\lambda \end{pmatrix}$$
Now you want $A,E,F,I$ cocircular. Looking at this answer of mine, you can check that by computing the determinant
$$\begin{vmatrix} x_1^2+y_1^2 & x_1 & y_1 & 1 \\ x_2^2+y_2^2 & x_2 & y_2 & 1 \\ x_3^2+y_3^2 & x_3 & y_3 & 1 \\ x_4^2+y_4^2 & x_4 & y_4 & 1 \end{vmatrix}=0$$
So in this case:
$$\begin{vmatrix} 0 & 0 & 0 & 1 \\ 1/4 & 1/2 & 0 & 1 \\ 1/4+\lambda^2/4 & 1/2 & \lambda/2 & 1 \\ \frac{\left(\sqrt{1+\lambda^2}+1\right)^2+\lambda^2} {\left(\lambda+\sqrt{1+\lambda^2}+1\right)^2} & \frac{\sqrt{1+\lambda^2}+1}{\lambda+\sqrt{1+\lambda^2}+1} & \frac{\lambda}{\lambda+\sqrt{1+\lambda^2}+1} & 1 \end{vmatrix}=0$$
Developing the determinant by the first row simply means restricting things to the lower left $3\times3$ determinant. You can further simplify the equation by multiplying rows by their denominator.
$$\begin{vmatrix} 1 & 2 & 0 \\ 1+\lambda^2 & 2 & 2\lambda \\ \left(\sqrt{1+\lambda^2}+1\right)^2+\lambda^2 & \left(\sqrt{1+\lambda^2}+1\right) \left(\lambda+\sqrt{1+\lambda^2}+1\right) & \lambda\left(\lambda+\sqrt{1+\lambda^2}+1\right) \end{vmatrix}=0$$
Unfortunately the square roots haven't canceled yet, so I'll ask a computer algebra system for help here.
sage: PR1.<x,b> = QQ[]
sage: m = matrix([
... [1,2,0],
... [1+x^2,2,2*x],
... [(b+1)^2+x^2,(b+1)*(x+b+1),x*(x+b+1)]])
sage: d = m.det()
sage: ideal([d, x^2 + 1 - b^2]).variety(QQbar)
[{b: -1, x: 0},
{b: 0, x: -1*I},
{b: 0, x: 1*I},
{b: 1, x: 0},
{b: 1.666666666666667?, x: 1.333333333333334?}]
So there are five algebraic solutions. Two have a complex value for $\lambda$, leading to a zero length $b$ which is not geometrically sensible since it would imply $A=C$. (Speaking about lengths and points at complex coordinates in the same setup would be tricky business, and probably should start with a clear definition of the terms involved.)
Two have $\lambda=0$, so $B=C$, in which case many of the objects involved are not well defined.
The only remaining solution is for $\lambda=\frac43$ as Lucian wrote.
Let $r$ be the inradius of $\triangle ABC$, so that $AI= \frac{r}{\sin \frac{A}2}$. Note that $AEF$ is a right triangle as well, so its circumcenter would be the midpoint of $AF$, call it $F'$. Now as $AF' = IF'$, and $\angle IAF' = \frac{\angle A}2$, we apply the sine rule on $IAF'$ to get:
$$\frac{r}{\sin A \sin \frac{A}2}=\frac{b}{4\sin\frac{A}2}$$ Simplifying: $$4r=b\sin A $$ $$4(\frac{a+c-b}2)=a $$ Thus we get, $a+2c=2b$. Squaring yields: $$a^2+4c^2+4ac=4(a^2+c^2)$$ $$4ac=3a^2$$ $$\therefore \frac{a}{c}= \frac43$$
