The first formula is correct, subject to confusion that most books (that I use to teach, and that are not my choice) do not clearly explain what are these $f(x)$ and $g(x)$. (Either that, or students try to read too fast and overlook what these $f(x)$ and $g(x)$ are supposed to denote, but at any rate I find the notation $f(x)$ and $g(x)$ unnatural and unfortunate in this context, and no wonder why students would be confused.)
In your question you call these $f(x)$ and $g(x)$ "big function" and "small function" and I suspect you may not quite grasp what these are (though of course I may be wrong). At any rate, I would rather call them "big radius" and "small radius" instead, and denote them say by $R(x)$ (the big radius) and $r(x)$ (the small radius). They depend not only on the functions that describe a bounded region, but also on the axis of revolution. In this notation, the correct formula becomes
$$\pi \int_a^b\big(R(x)\big)^2 - \big(r(x)\big)^2 dx$$ (assuming we use washers and the axis of revolution is horizontal). (BTW, why do you have the bounds $a,b$ reversed in your question?)
Say the region is bounded by the curves $y=x^2$ and $y=3x.$ They intersect when $x^2=3x$, that is $x^2-3x=x(x-3)=0$ with solutions $x=0$ and $x=3$, and these are the $a$ and $b$ that are the bounds of integration. Note that when $0<x<3$ we have $x^2<3x$, so $x^2$ is the smaller function (in this interval $[0,3]$) and $y=3x$ is the bigger function, of the two. (Please sketch a picture yourself.)
Now say the axis of revolution is the $x$-axis, note that it has equation $y=0$. To find the big radius $R(x)$ we subtract the $y=0$ (describing the axis of revolution) from the big $y=3x$ to obtain $R(x)=3x-0=3x$. Similarly, to obtain the small radius $r(x)$ we subtract the $y=0$ (describing the axis of revolution) from the small $y=x^2$ to obtain $r(x)=x^2-0=x^2$. The integral is
$$\pi \int_0^3 (3x)^2 - (x^2)^2\ dx$$
Next say we consider exactly the same region, but with different axis of revolution, the line $y=-1$. This would change the solid and its volume, and the integral. Again,
to find the big radius $R(x)$ we subtract the $y$ describing the axis of revolution from the big $y$,
but this time the $y$ describing the axis of revolution is $y=-1$ (not $y=0$), while the big $y$ is just as before, $y=3x$. So we subtract the $y=-1$ from the $y=3x$ to obtain the big radius $R(x)=3x-(-1)=3x+1$. For the small radius we have $r(x)=x^2-(-1)=x^2+1$. The integral is
$$\pi \int_0^3 (3x+1)^2 - (x^2+1)^2\ dx$$
Finally say (for the same region) the axis of revolution is the horizontal line $y=9$. Note that the graph of this line is higher up than the graphs of $y=3x$ and $y=x^2$ in the interval $[0,3]$. So, this time to obtain the big radius, we subtract the smaller $y$, namely $y=x^2$ from the $y$ describing the axis of revolution, namely $y=9$. That is, $R(x)=9-x^2$. For the smaller radius we subtract the bigger $y$, namely $y=3x$ from the (even bigger) $y$ decribing the axis of revolution $y=9$, to obtain $r(x)=9-3x$. In the end, the integral is
$$\pi \int_0^3 (9-x^2)^2 - (9-3x)^2\ dx$$