What do we mean by representing ordinals as “relations” on N? Functions of N -> N? A well-ordering on N?
I personally haven't seen functions of $\mathbb N\rightarrow\mathbb N$ used, but for relations on $\mathbb N$, that's equivalent to the "well-ordering on $\mathbb N$" case since the relation in question is a well-ordering. Either way, the point of an ordinal notation system is that it's isomorphic to some relevant ordinal that's used in the analysis.
For example, we can make a simple ordinal notation $(\mathbb N,\prec)$ for $\omega2$ this way: define $\prec$ to be an irreflexive, transitive binary relation on $\mathbb N$ such that $2m\prec 2(m+1)\prec 2n+1$ for any $m,n\in\mathbb N$. This ordering has $0\prec 2\prec 4\prec 6\prec\ldots 1\prec 3\prec 5\prec 7\prec\ldots$, and $(\mathbb N,\prec)$ is isomorphic to $(\omega2,<)$.
Why do we need these functions and what are they doing?
A classic way of analyzing a theory of arithmetic, depicted in Rathjen's "The Realm of Ordinal Analysis", is showing that a weak base theory (e.g. primitive recursive arithmetic) proves that the well-foundedness of some "ordinal" implies consistency of the theory in question. For theories of arithmetic, there's a problem: the von Neumann ordinals don't exist as terms in the domain of discourse. (Even if they did, they're well-founded by definition, so the statement "$\varepsilon_0$ is well-founded" is trivial). Instead, we use an ordinal notation for $\varepsilon_0$ to substitute for it, some relation on $\mathbb N$ with order type $\varepsilon_0$, and some sort of appropriate statement about the relation (e.g. induction along it can be performed up to height $\varepsilon_0$, no infinite decreasing sequences below height $\varepsilon_0$, etc) substitutes for "well-foundedness of $\varepsilon_0$."
For example, the analysis done in "Hydrae and Subsystems of Arithmetic" by Carnielli and Rathjen of some weak theories of arithmetic. Here a pretty natural ordering $\prec$ is defined on $\mathbb N$ based on Cantor normal form, and $(\mathbb N,\prec)$ has order type $\varepsilon_0$. The statement used in place of well-foundedness up to $\alpha$ is "no primitive recursive function picks out a sequence of naturals from the domain of $\prec$ that witness an infinite decreasing $\prec$-chain below height $\alpha$ in the ordering."
At the end, it's remarked (although the proof isn't written in the paper) that when we consider this statement for the full ordering, i.e. "no primitive recursive function picks out values from the domain of $\prec$ that form an infinite decreasing $\prec$-chain," it says from this we can derive $\textrm{Con(PA)}$. Also, it's shown Peano arithmetic can prove this "well-foundedness up to height $\alpha$" statement for any $\alpha<\varepsilon_0$.