Yes, I think that you’re right. In fact, as you already observed, the function $G_x$ is continuous in $(0, \delta)$, and since $(0, \delta) \cap \mathbb Q$ is a dense subset, the two suprema coincide by continuity.
To be more precise: let $X$ be a topological space, let $f \colon X \to \mathbb R$ be continuous and let $D \subset X$ be dense. Then, $\sup_D f = \sup_X f.$
Proof. Since $D \subset X$, we have that $\sup_D f \le \sup_X f$.
If $\sup_X f = +\infty$, for every $M > 0$ there exists $x \in X$ such that $f(x) > M$. Setting $\varepsilon := \frac{f(x)-M}{2} > 0$, it still holds that $f(x) -\varepsilon > M$. Since $f$ is continuous, there exists an open neighbourhood $U \ni x$ such that $\forall y \in U$, $| f(y) - f(x) | < \varepsilon$, so, in particular,$ f(y) - f(x) > -\varepsilon$, hence $f(y) > f(x) - \varepsilon > M$ $\forall y \in U$. Since $D$ is dense, $U \cap D \ne \emptyset$, so there exists $y_D \in D$ such that $f(y_D) > M$. Since this argument holds for every $M > 0$, we conclude that $\sup_D f = +\infty = \sup_X f$.
If $\sup_X f < +\infty$, fix an arbitrary $\eta > 0$. By the definition of supremum, there exists $x \in X$ such that $\sup_X f - \frac{\eta}{2} \le f(x) \le \sup_X f$. By continuity, there exists an open neighbourhood $U \ni x$ such that $|f(y) - f(x)| < \frac{\eta}{2}$ $\forall y \in U$. Arguing as above, by density one is able to find $y_D \in D \cap U$ such that $f(y_D) > f(x) -\frac{\eta}{2} \ge \sup_X -\eta$. Therefore, $\sup_D f \ge f(y_D) > \sup_X f -\eta$. Since $\eta > 0$ is arbitraty, we conclude that $\sup_D f \ge \sup_X f$, and this concludes the proof.
In your case, use this result with $X = (0, \delta)$ and $f = G_x$.