Y'all no worries! I figured it out – (by a fluke?), I just want someone to confirm this is right :)
Converting to it to original question in my textbook. Let $p= \alpha$ and $q = \beta$
$\alpha$ and $\beta$ are solutions of $a\cos\theta+b\sin\theta=c$
$$a\cos\alpha+b\sin\alpha=c=a\cos\beta+b\sin\beta $$
$$b(\sin\alpha-\sin\beta)=-a(\cos\alpha-\cos\beta)$$
$$ 2b\cos\left(\frac{\alpha+\beta}{2}\right)\sin\left(\frac{\alpha-\beta}{2}\right)=2a\sin\left(\frac{\alpha+\beta}{2}\right)\sin\left(\frac{\alpha-\beta}{2}\right)$$
$$\frac{b}{a}=\frac{\sin\left(\frac{\alpha+\beta}{2}\right)\sin\left(\frac{\alpha-\beta}{2}\right)}{\cos\left(\frac{\alpha+\beta}{2}\right)\sin\left(\frac{\alpha-\beta}{2}\right)}=\tan\left(\frac{\alpha+\beta}{2}\right) $$
$$ \tan^2\left(\frac{\alpha+\beta}{2}\right)=\frac{b^2}{a^2}$$
$$\cos(\alpha+\beta)=\frac{\cos^2\left(\frac{\alpha+\beta}{2}\right)-\sin^2\left(\frac{\alpha+\beta}{2}\right)}{\cos^2\left(\frac{\alpha+\beta}{2}\right)+\sin^2\left(\frac{\alpha+\beta}{2}\right)}$$
Dividing the numerator and denominator by $\cos^2\left(\frac{\alpha+\beta}{2}\right)$ , we get,
$$\cos(\alpha+\beta)=\frac{1-\tan^2\left(\frac{\alpha+\beta}{2}\right)}{1+\tan^2\left(\frac{\alpha+\beta}{2}\right)}=\frac{1-\frac{b^2}{a^2}}{1+\frac{b^2}{a^2}}$$
$$\Rightarrow\qquad \cos(\alpha+\beta)=\frac{a^2-b^2}{a^2+b^2}$$
Thanks to everyone for their efforts :-)