First, let's recall the idea of transport of structure. If you have a vector space $V$ over a field $K$ and a set $A$ together with a bijection $f\colon A\rightarrow V$, then there is a unique structure of a $K$-vector space on $A$ such that $f$ becomes an isomorphism of $K$-vector spaces. The addition on $A$ is defined by $a+b:=f^{-1}(f(a)+f(b))$ for $a,b\in A$, where the addition on the RHS is addition in $V$. Similarly, the scalar multiplication is defined by $\lambda a:=f^{-1}(\lambda f(a))$ for $\lambda\in K,a\in A$, where the scalar multiplication on the right is that in $V$. I'll leave it as an exercise to confirm that this makes $A$ into a $K$-vector space and $f$ into an isomorphism.
With this and the fact that there is a bijection between two sets if and only if they have the same cardinality in mind, this leaves us only with the question of which possible cardinalities vector spaces can have. The answer differs depending on the case of finite or infinite cardinality.
First, the finite case. Let's start by considering a finite field $K$. There is a unique ring homomorphism $\mathbb{Z}\rightarrow K$, given by sending $1$ to the multiplicative identity of $K$. Since $\mathbb{Z}$ is infinite and $K$ is finite, it cannot be injective, so its kernel has the form $p\mathbb{Z},p>0$ and the map factors through an injection $\mathbb{Z}/p\mathbb{Z}\rightarrow K$ and since $K$ is a field, so in particular an integral domain, $p$ is a prime. So there is an embedded copy $\mathbb{F}_p$ of the field $\mathbb{Z}/p\mathbb{Z}$ in $K$. Then, restricting the multiplication of $K$ in the first factor to $\mathbb{F}_p$ makes $K$ into an $\mathbb{F}_p$-vector space. Since $K$ is finite, it is a finite-dimensional $\mathbb{F}_p$-vector space, so is isomorphic to $\mathbb{F}_p^n$ for some $n\ge0$ and has cardinality $p^n$. Thus, finite fields always have a prime power cardinality.
Now if $V$ is a vector space over $K$ whose underlying set is finite, the base field $K$ is necessarily finite as well, hence has cardinality $p^k$ for some prime $p$ and $k\ge0$ by the previous argument. Since $V$ is finite, it is finite-dimensional, hence isomorphic to $K^k$ as $K$-vector space for some $k\ge0$. In particular, $V$ has cardinality $(p^n)^k=p^{nk}$. Thus, finite vector spaces always have prime power cardinalities. Conversely, if $p^k$ is some prime power, $(\mathbb{Z}/p\mathbb{Z})^k$ is a vector space over $\mathbb{Z}/p\mathbb{Z}$. In conclusion, the cardinalities of finite vector spaces are precisely the prime powers. It is less trivial, but should still be noted, that in fact, for every prime power $p^k$, there already is a field of that cardinality.
Now, the infinite case. Let $I$ be an infinite set and consider the function field $K:=\mathbb{Q}(x_i,\,i\in I)$ in variables indexed by $I$. Clearly, $|I|\le|K|$. Now, each element of $K$ is a rational function in finitely many of the variables, by definition. If we let $\mathcal{P}_f(I)$ denote the set of finite subsets of $I$, then we thus have the equality $K=\bigcup_{J\in\mathcal{P}_f(I)}\mathbb{Q}(x_j,j\in J)$. Each $\mathbb{Q}(x_j,j\in J)$ with $J\subseteq I$ finite is a countable set, as one shows by induction over $|J|$. Thus, we obtain
$$|K|\le\sum_{J\in\mathcal{P}_f(I)}|\mathbb{Q}(x_j,j\in J)|=|\mathcal{P}_f(I)||\aleph_0|=|I||\aleph_0|=|I|.$$
The second to last step uses the fact that $|\mathcal{P}_f(I)|=|I|$ for $I$ infinite, see here. The last step again uses that $I$ is infinite. By Cantor-Schröder-Bernstein, it follows that $K$ is a field of cardinality $|I|$. Since $I$ was arbitrary, we see that every set of infinite cardinality can be equipped with the structure as a field (and hence with that as a vector space, as every field is of course a vector space over itself). For those who care, yes, the Axiom of Choice has been assumed.
In conclusion, a set can be equipped with the structure of a vector space over some field if and only if it is infinite or finite with the cardinality of a prime power. Furthermore, every such set can already be made into a field itself.
