I will give a sketch of the main idea.
We want to show continuity of $b(x)$ at some $x=x_0$. Pick an arbitrary $\epsilon > 0$, and keep it fixed in the following. Next, we evaluate (for arbitrary $\Delta x\in\mathbb R$):
$$b(x_0 + \Delta x) = b(x_0) +\Delta x(2x_0 - 4 +\Delta x).$$
Given our choice of $x_0$ and $\epsilon$, we need to find a $\delta >0$ such that $|\Delta x(2x_0 - 4 +\Delta x)|<\epsilon$ for all $|\Delta x|<\delta$. If we can find such a $\delta$, this would prove continuity at $x_0$ (note that you need to be able to find such a $\delta$ for every choice of $\epsilon > 0$).
The following is a heuristic to convey the main idea of what remains to be done. For small enough $\Delta x$ and $2x_0 - 4$ not close to zero, we would expect something like this:
$$|\Delta x(2x_0 - 4 +\color{red}{\Delta x})|<\epsilon\quad \Leftrightarrow\quad |\Delta x| < \frac{\epsilon}{|2x_0 - 4+\color{red}{\Delta x}|}\approx \frac{\epsilon}{|2x_0 - 4|},$$
so you might choose $\delta \approx \frac{\epsilon}{2x_0 - 4}$ (or a bit smaller, but we are not trying to be rigorous yet). Having found such a $\delta$ for every $\epsilon$, this would complete the proof. However, $\ldots$.
$\ldots$ to be rigorous, you need to solve the inequality $|\Delta x(2x_0 - 4 +\Delta x)|<\epsilon$ for $\Delta x$ and prove that for any choice of $\epsilon > 0 $ and $x_0$, you can find corresponding $\delta_L <0 < \delta_R$ such that the inequality is satisfied for all $\Delta x \in (\delta_L, \delta_R)$. You would pick $\delta=\min(-\delta_L, \delta_R)$, or anything smaller. Often, $\delta_{L,R}$ will be the appropriate roots of the polynomial(s) $\pm y(2x_0 - 4 + y) - \epsilon$, but in general you will need to
- take the absolute value into account (i.e. it's not simply a purely quadratic inequality), and
- be able to argue that all $\Delta x \in (\delta_L, \delta_R)$ satisfy the inequality (and not just $\Delta x =\delta_L$ and $\Delta x = \delta_R$).
I hope this sheds some light on your question. But this should also explain why the theorems we discussed in the comments are so helpful - it is very tedious to show continuity of even a simple thing as a quadratic polynomial rigorously.