Though you said no appeals to Yoneda are allowed, I feel obligated to showcase how much easier the proof is when we allow Yoneda. I'm sure you've seen this, but I will include it all the same:
$$ \text{Hom}(x, a^1) \cong \text{Hom}(x \times 1, a) \cong\text{Hom}(x,a)$$
The first isomorphism follows from the adjunction, the second from the isomorphism you are already aware of ($x \times 1 \cong x$). Since $x$ was arbitary, we can conclude $a^1 \cong a$ by Yoneda.
Now that that's out of my system, we can contrast it with a Yoneda-less proof:
For one direction, we will use the counit $\varepsilon$, which you're calling $\text{ev}$. We can use this to get a map (which you rightfully found):
$$
a^1 \xrightarrow{~\sim~}
a^1 \times 1 \xrightarrow{~\varepsilon~}
a
$$
As is often the case in category theory, if we got half an iso using some construction, we should try to get the other half using the dual of that construction. So let's consult the (much less appreciated) unit $\eta$ of the adjunction. This gives us a map:
$$
a \xrightarrow{~\eta~}
(a \times 1)^1 \xrightarrow{~\sim~}
a^1
$$
I leave it to you to check that these are mutually inverse, and that the obvious iso from $(a \times 1)^1 \cong a^1$ is actually an iso.
Edit:
To address the question in the comments, let's work with only the following statement of Yoneda:
$$\text{Nat}(\text{Hom}(\cdot,a),F) \cong F(a).$$
Then, in particular, $\text{Nat}(\text{Hom}(\cdot,a),\text{Hom}(\cdot,b)) \cong \text{Hom}(a,b)$. That is, every natural transformation from $\text{Hom}(\cdot,a)$ to $\text{Hom}(\cdot,b)$ is of the form $f \circ -$ for some $f \in \text{Hom}(a,b)$.
Now, if $f \circ -$ and $g \circ -$ witness $\text{Hom}(\cdot,a) \cong \text{Hom}(\cdot, b)$ (here $f \in \text{Hom}(a,b)$ and $g \in \text{Hom}(b,a)$) then we see:
$$ \text{Id}(-) = (f \circ -) \circ (g \circ -) = f \circ g \circ - $$
That is, $f \circ g$ is itself the identity, and $a \cong b$!
So when we show that (for arbitrary $x$)
$\text{Hom}(x,a) \cong \text{Hom}(x,a^1)$, we are showing that
$\text{Hom}(\cdot, a) \cong \text{Hom}(\cdot, a^1)$ as natural transformations. From the above discussion, this lets us conclude $a \cong a^1$.
This is a very common mode of argument in category theory, and I recommend Chapter 8 of Awodey's "Category Theory" for a good discussion of this and related techniques.
I hope this helps ^_^