I think the right pedagogical perspective is the other way around: the "right" notion of structure is "that which is preserved by morphisms".
I think your example of topological spaces is instructive. In the beginning, to understand the continuum we used the idea of a metric space: a set of points equipped with a metric. Using the metrics, we can define what it means for a function to be continuous.
We then study metric spaces and continuous functions.
But you quickly run into some oddities; for example, the identity function between the usual Euclidean plane $(\mathbb{R}^2, d_2)$ and taxicab geometry $(\mathbb{R}^2, d_1)$ turns out to be continuous in both directions: a homeomorphism!
So now you have this phenomenon of equivalent metrics. And it turns out that a lot of things you do in geometry respect this equivalence. Making judgement that a function is continuous, or that a set is open/closed/compact, or whether a sequence converges: all of these judgments remain the same if you replace the metrics involved with different but equivalent metrics.
And so, a major component of the practice of geometry eventually becomes working with metric spaces in a metric-independent way, studying what we now call topological properties of spaces.
In other words, in the example of topology, the category came first. In hindsight, the actual, real world history1 is that we first defined the category of metric spaces and continuous functions, and only later did we realize that we were keenly interested in whatever structure is preserved by the morphisms of this category.
In my opinion, you find this sort of thing all over mathematics and physics: you see people first find a way to describe the objects, then how to describe the morphisms, and then later derive results (often viewed as deep) that some aspect of the description of objects is not preserved by the morphisms, and thus is not a relevant feature of what you're studying.
1: I'm relatively sure that this characterization is accurate. But don't take this as definitive since I haven't deliberately studied the history.
P.S. The "category of topological spaces and all functions" is equivalent to Set. And since most of the things you do with categories respect equivalence, the two are effectively interchangeable. So your "useless" example is actually an incredibly important category; it just doesn't tell you much about topology.