I was wondering if we can construct the category $\mathbf{Grp}$ of groups from the function $G$ which associates to every set $X$ the set $G(X)$ of group structures on $X$, or what we have to add to this function to make it work.
Namely, a group is just a pair $(X,a)$ where $X$ is a set and $a \in G(X)$. The homomorphisms make some trouble, though. A homomorphism $(X,a) \to (Y,b)$ should be a map $f : X \to Y$ such that some property is satisfied, but I don't see how to express it in terms of $G(X)$ and $G(Y)$ alone. If $f$ is bijective, then we have an induced map $f_* : G(X) \to G(Y)$, and the condition is just $f_*(a)=b$. For general $f$, it seems that we need some extra structure on $G(-)$ to make it work. So my question is: What is this extra structure?
Of course, $G(X)$ is just the fiber of the forgetful functor $U : \mathbf{Grp} \to \mathbf{Set}$ at $X$, but I wonder if we can define this structure without "assuming" $\mathbf{Grp}$ is already there. The problem is that $U$ is an iso-fibration, but not a fibration or an opfibration. Let me know if I should clarify the question further.
The analogous problem for $\mathbf{Top}$ can be solved as follows: Consider the functor $T : \mathbf{Set}^{\mathrm{op}} \to \mathbf{Pos}$ which maps a set $X$ to the partially ordered set $T(X)$ of topologies on $X$ and a map $f : X \to Y$ to the map $f^* : T(Y) \to T(X)$, $f^*(\tau) = \{f^{-1}(U) : U \in \tau\}$. Then, a topological space is a pair $(X,\tau)$ with $\tau \in T(X)$, and a continuous map $(X,\tau) \to (Y,\sigma)$ is a map $f : X \to Y$ such that $f^*(\sigma) \subseteq \tau$. In other words, $\mathbf{Top}$ is the Grothendieck construction of the functor $\mathbf{Set}^{\mathrm{op}} \to \mathbf{Pos} \hookrightarrow \mathbf{Cat}$.