I have the belief that first-order logic cannot be recognized by a pushdown automaton because the automaton will have no way to keep track of the variables that are currently in scope. More precisely, I have the belief that the language of well-formed formulas of first-order logic, in its usual formulation, with say finitely many non-logical symbols, cannot be recognized by a pushdown automaton because the automaton will have no way to keep track of the numerals which index all the variables that are currently in scope.
I also have the belief that the difficulty is not essential, and that first-order logic could be reformulated, using harmless tricks, in a format that could be recognized by a pushdown automaton. My difficulty is that I do not know how to define "harmless" correctly.
Obviously, the well-formed formulas of first-order logic can be put recursively into one-to-one correspondence with the unary numerals, and in this new "formulation" the well-formed formulas are very easy to recognize. It seems, however that doing any real work with such well-formed formulas would need to pass through a preliminary translation step by a machine more powerful than I would like.
The logic textbooks I consult (for example, Marker) consider these issues to be fiddly details that be safely mishandled. Where can I go to learn about these fiddly details?