
Lots of things going on here. I immediately know that $F(x)$ does exist since $f$ is riemann integrable due to the fact that it is continuous. First I need to show that $F$ is continuous, then find $F'$ and show that it is continuous. What can I do?

Lots of things going on here. I immediately know that $F(x)$ does exist since $f$ is riemann integrable due to the fact that it is continuous. First I need to show that $F$ is continuous, then find $F'$ and show that it is continuous. What can I do?
I can show you how to get the derivative, but I don't know about proving continuity, etc. According to the fundamental theorem of calculus, there exists a function $G(x)$ such that:
$$ F(x) = \int\limits_a^{g(x)} f(x)dx = G\left(g(x)\right) - G(a)\text{, where } G'(x) = f(x) \\ F'(x) = g'(x)G'\left(g(x)\right) - 0 = g'(x)f\left(g(x)\right) $$
I don't really like the following notation, but problems like these seem to warrant it...you could also write it like this:
$$ F'(x) = g'(x) \cdot \left(f\circ g\right)(x) $$
Hint: $F = G\circ g$ where $G(x) = \int_0^x f(t)\,dt$ is a function that you can differentiate.
Since $f$ is continuous in $[a,b]$ then $H(x):=\int_a^x f(t) dt, x\in[a,b]$ is differentiable, by the FTC (see for instance M. Spivak, Calculus). Now, $F$ is nothing but the composition $H\circ g$, this composition being well defined since $Im g\subset Dom H$, is differentiable in $[a,b]$ since it is the composition of two differentiable functions ($H$ is so by construction, and $g$ by hypothesis). Hence, by the chain rule $F'(x)=(H\circ g)'(x)=(H(g(x))'=H'(g(x)) g'(x)=f(g(x)) g'(x)$, using the FTC ($H'(x)=f(x)$). $F'$ is a continuous function since it is the product of two continuous functions on $[a,b]$, $f(g)$ since it is the composition of continuous functions (and $Im g\subset Dom f$) and $g$. Therefore $F\in C^1([a,b])$. QED.