Suppose $G$ and $H$ are groups and we have an equivalence of categories between $G\textrm{-}\mathbf{Set}$ and $H\textrm{-}\mathbf{Set}$. (One can think of this as a form of "nonlinear Morita equivalence".) What can be said about $G$ and $H$? I suspect that $G$ and $H$ have to be isomorphic, but I can't prove it. If this is too hard in general, I'm also interested in the case that $G$ and $H$ are assumed to be finite.
Here are some things I've tried:
Since $G$-modules are the abelian group objects in $G\textrm{-}\mathbf{Set}$, we get a Morita equivalence between $\Bbb{Z}[G]$ and $\Bbb{Z}[H]$. So if we tensor with any field $k$, we get a Morita equivalence between $k[G]$ and $k[H]$.
Assume for a moment that $G$ and $H$ are finite. If we take $k = \Bbb{C}$, then this means that $G$ and $H$ have the same number of irreducible representations, so the same number of conjugacy classes. If we take $k=\Bbb{R}, \Bbb{Q}, \overline{\Bbb{F}_p}$, we also get the same number of real, rational and $p$-regular conjugacy classes.
While this approach gives some common properties between $G$ and $H$, it's not possible to conclude that $G$ and $H$ are isomorphic, because there are known examples of non-isomorphic finite groups with isomorphic integral group algebras. (See here) This means that we have to use some of the "nonlinear" information from the category $G\textrm{-}\mathbf{Set}$.
Another thing I considered is that the automorphism group of the forgetful functor $G\textrm{-}\mathbf{Set} \to \mathbf{Set}$ is isomorphic to $G$, by a simple Yoneda-argument, so if we could somehow reconstruct the forgetful functor just from the category $G\textrm{-}\mathbf{Set}$, this would show that $G$ and $H$ must be isomorphic. I haven't been able to do this, but I reconstructed some other functors: The terminal object in $G\textrm{-}\mathbf{Set}$ is a one-point set with a trivial action, denote this $G$-set by $\{*\}$, we have a natural bijection $\operatorname{Hom}_{G\textrm{-}\mathbf{Set}}(\{*\},X) \cong X^G$, where $X^G$ denotes the set of fixed points under the action of $G$. So we can reconstruct the fixed point functor $G\textrm{-}\mathbf{Set} \to \mathbf{Set}$. The left adjoint of that functor is the functor $\mathbf{Set} \to G\textrm{-}\mathbf{Set}$ which gives each set a trivial $G$-action. Denote $X$ with a trivial $G$-action by $X_{triv}$. We have $\operatorname{Hom}_{G\textrm{-}\mathbf{Set}}(X,Y_{triv}) \cong \operatorname{Hom}_{\mathbf{Set}}(X/G,Y)$, so the functor which sends each $G$-set to the orbit space $X/G$ is left adjoint to the functor which gives each set a trivial $G$-action, so we can reconstruct the functor $X \mapsto X/G$ from the category $G\textrm{-}\mathbf{Set}$. Not sure if that's helpful.
Maybe it's even possible that $G$ and $H$ don't have to be isomorphic? I'm looking either for a counterexample or a proof.