Your understand is the "naive understanding" as to why the axiom of choice is needed. But the truth is far more complicated.
First, let's clear something up. What does it mean for a choice function to exist? Well. In set theory something exists if it is a set. And we have tools and machinations for proving that certain sets exists, the axioms of $\sf ZF$ are mainly about "if such set exists, then another one exists", for example the Separation axioms tell you that if $X$ is a set, and $\varphi(x)$ is a formula, then $\{x\in X\mid\varphi(x)\}$ is also a set.
So when we come to prove that a certain family admits a choice function, we are trying to prove the existence of certain sets using whatever axioms we have at our disposal.
Now, let's get back to the argument that you mention. There are two issues here:
The notion of finite might be different between where you live (or rather, the framework in which you conduct your proof), and the model of set theory in which you are interested.
But $\sf ZF$ proves finite choice, and by that we mean internally finite, so this is an even stronger fact than just applying existential quantifiers to the proof. And the way to do this is to use the fact that $\sf ZF$ proves induction, and if we assume that we have a choice function from a family $A$, then we can always add one more non-empty set, and then instantiate an existential quantifier to fix a choice function, and then again to fix an element from the new set.
So this lets you get finite choice even when the notion of "finite" is different from the one in our framework.
Infinite choice can be given by a rule. If I am given a family of singletons, then I can write down a finitary rule for choosing from them. Namely, pick the unique member of the set. If I am given a family of non-empty subsets of the natural numbers I can also write a finitary rule: use the fact that a non-empty set of natural numbers has a least element, and simply pick the least one. I can even make it more complicated if I wanted to: pick the least prime if one exists, otherwise the least even if one exists, otherwise the least member.
Or if you are giving me an infinite family of sets where each set is attached to a given group structure. Then I can always choose the additive neutral element. These choices come from the the structure of the sets given in the family that lets me write a finitary definition for a choice function.
In contrast, there is no way to specify a rule in a finitary way to choose from non-empty sets of real numbers. And we know that, since we know that there are models of $\sf ZF$ in which there is no such choice function.
This is important, since a finitary rule is exactly a rule we can express in the language of set theory and use with the right axioms of $\sf ZF$ in order to prove the existence of a choice function.
The axiom of choice guarantees the existence of a choice function, regardless to whether or not you can specify a rule for it. It simply exists, and by existential instantiation we can fix one. And it is fundamentally different from the rest of the axioms of $\sf ZF$ by not telling you how such a choice axiom looks like, there is no "concrete definition" for the choice function, just assertion of its existence.