You have two correct answers already.
What I would like to do is to put the solution in terms that you can
very easily apply to your problem, that is, I want to give you a simple formula
to directly compute the coordinates of each vertex of each square.
Let's take polar coordinates with the origin at the center of the square, so that the large square in your diagram has vertices at
$(r_0,0)$, $\left(r_0,\frac\pi2\right)$, $(r_0,\pi)$, $\left(r_0,-\frac\pi2\right)$
where $r_0$ is a positive constant determined by how large you want that square to be.
Now we rotate $\delta$ radians to the right and shrink the square so that the new
square's vertices lie on the old square's edges.
(For a $10^\circ$ rotation, you want $\delta = \frac{\pi}{180} \cdot 10.$)
Then the origin, one vertex of the original square, and the corresponding vertex of
the new square make a triangle with angle $\delta$ at the origin, $\frac\pi4$ ($45^\circ$)
at the original square's vertex, and $\frac{3\pi}{4}-\delta$ at the new square's vertex.
Let $r_1$ be the distance from the origin to the new square's vertex.
By the law of sines,
$$ \frac{r_1}{\sin \left(\frac{\pi}{4}\right)}
= \frac{r_0}{\sin \left(\frac{3\pi}{4}-\delta\right)}.$$
Therefore
$$r_1 = \frac{\sin \left(\frac{\pi}{4}\right)}{\sin \left(\frac{3\pi}{4}-\delta\right)} r_0
= \frac{\sin \left(\frac{\pi}{4}\right)}{\sin \left(\frac{\pi}{4}+\delta\right)} r_0.$$
Let
$$\rho = \frac{\sin \left(\frac{\pi}{4}\right)}{\sin \left(\frac{\pi}{4}+\delta\right)}$$
so that $r_1 = \rho\, r_0.$
For an angle of $10^\circ,$ a little computation shows that $\rho \approx 0.86321799\,$
that is,
$$r_1 \approx 0.86321799\, r_0.$$
Checking this against the calculations by @turkeyhundt for the $10$-degree rotation,
we should find that $0.86321799 \approx \frac{\sqrt{a^2+b^2}}{100},$
and indeed for $b = 85.01,$ $a=100-b$ that is what we find.
Now observe that in general, if $r_n$ is the distance from the origin to a vertex
of the $n$th square, then $r_{n+1} = \rho\, r_n.$
Also, if $\alpha$ is the direction from the origin to a vertex of the $n$th square,
then the direction from the origin to a vertex of the $n+1$st square is
$\alpha-\delta$ (because polar coordinates measure angles counterclockwise
and we are rotating clockwise).
Putting these facts together, the vertices of the $n$th square
have polar coordinates
$$
(\rho^n r_0,\; -n\delta),\\
\left(\rho^n r_0,\; \frac\pi2-n\delta\right)\!,\\
(\rho^n r_0,\; \pi-n\delta),\\
\left(\rho^n r_0,\; -\frac\pi2-n\delta\right)\!.
$$
To plot these in Cartesian coordinates, convert the $(r,\theta)$ coordinates
of each vertex of each square to $(x,y)$ coordinate by the usual
transformation, $x=r\cos\theta$ and $y=r\sin\theta.$
For example, one vertex of the $n$th square will have Cartesian coordinates
$(\rho^n r_0 \cos(-n\delta), \rho^n r_0 \sin(-n\delta)),$ and for the other
three you simply add a multiple of $\frac\pi2$ to the angle.