I have a problem where:
$ b = R \left(\theta \hat{u} \right) a $
where $a, b \in \mathbb{R}^3$ are known, and $R\left(\theta \hat{u} \right) \in SO(3)$ is a rotation matrix constructed from angle $\theta$ about axis $\hat{u}$ of unit length. In this particular problem, $\theta$ is known and I wish to solve for $\hat{u}$.
I can brute force this from the Rodrigues rotation formula and solve for the elements of $\hat{u}$ (which turns out to be a three quadric intersection problem) but is there a more elegant way to solve this problem?
Clarifying notes:
Angle $\theta$ is fixed in this problem and is key. I am speculating that this should reduce the family of rotations that would otherwise satisfy the first equation to a unique (or at least finite) number of solutions.
EDIT: Counter-example to the cross-product answer.
Consider the rotation matrix constructed as follows:
uhat = [4; 6; 8] / norm([4; 6; 8])
uhat =
0.37139
0.55709
0.74278
theta = 0.5;
R = RodriguesRotation(uhat, theta)
R =
0.89447 -0.33078 0.30085
0.38144 0.91557 -0.12740
-0.23331 0.22871 0.94512
With vectors $a$ and $b$ being:
a = [1; 2; 3]
b = R * a
b =
1.1355
1.8304
3.0595
Then computing $\hat{u} = \frac{a \times b}{|a\times b|}$:
axb = cross(a, b);
unit_axb = axb / norm(axb)
unit_axb =
0.74582
0.41213
-0.52336
Which is clearly does not recover the original rotation axis.
Edit 2: Widawensen's solution can be verified:
unit_a = a / norm(a);
unit_b = b / norm(b);
cos_theta = cos(theta);
cos_beta = sqrt((cos_alpha - 1) / (cos_theta - 1));
beta = acos(cos_beta_plus);
gamma = pi/2 - beta;
Verifying the dot products:
dot(unit_a, uhat) - cos(gamma)
ans = -8.8818e-16
dot(unit_b, uhat) - cos(gamma)
ans = -8.8818e-16