TLDR: It means that they are using a technique to represent Turing machines as numbers in order for the circularity to be possible, hence "arithmetization".
It is not trivial (especially, it was not at the time) that such a thing as code existed, or that things like binary numbers (which are just numbers) could be used to list or index all possible programs.
After Turing introduced the concept of a Turing machine, he specified that there were sets of instructions that would define what particular machine one is talking about. But it is not immediately obvious from this, that everything about those instructions, could, in fact, be described using a finite number.
Look into Gödel numbering also. Gödel showed that numbers can be used also to index all possible proof theories that can be used as foundations of mathematics (rules for what is a valid proof).
It is more evident, however, from Turing's definition, that Turing machines could do operations on numbers. It was just less obvious that they could be described by numbers. Because of these two connections, it is possible to set up circular situations where Turing machines are calculating properties of other Turing machines, including, in particular, themselves.
This allowed Turing to create a machine that "halts (returns a value) only for numbers of (code of) Turing machines that will never halt". If the halting problem were solvable, such a machine would have to exist, because their (own) number would come up in the list of machines. Since this is a contradiction, it proves the halting problem is undecidable by any machine that is equivalent to a Turing machine, i.e. finite, mechanical.