Let me start by defining spinors (actually, this will take most of the answer). Just from this definition, I hope, it will be clear that the discussion is several levels of complexity higher than the definition of tensors (more precisely, tensor fields) on smooth manifolds. While for tensors on manifolds (e.g. differential forms, which Atiyah compares to spinors) we have a nice local description, which one can explain to an undergraduate student in a Calc-IV class (I am talking about differential forms), I do not believe it is feasible in the case of spinors.
The first step is to define the Clifford algebra $Cl_n$ associated with ${\mathbb R}^n$ equipped with the (standard) positive-definite quadratic form. The definition is not bad but is purely algebraic, lacking the geometric picture of volumes that one has when defining the exterior algebra.
Define Clifford modules. (In most cases, people use just one, the unique, up to an isomorphism, irreducible module,
called $\Delta_n$.) A Clifford module is a vector space equipped with an action of the Clifford algebra $Cl_n$.
As Atiyah says, the algebraic structure of such modules is well-understood, and $\Delta_n$ can be described explicitly but how does one think of these geometrically?
Recall that the group $Spin(n)$ is the 2-fold connected covering group of the orthogonal group $SO(n)$. Each Clifford module $M$ comes with a representation of the spinor group $Spin(n)$, a linear action of $Spin(n)$ on the real vector space underlying $M$, commuting with Clifford multiplication. Again, there is no mystery here, the representation comes from the inclusion $Spin(n)\subset Cl_n$ and this inclusion is frequently used to define what $Spin(n)$ is. Elements of $M$ are frequently called "spinors." They bare the same relation to the true spinors (spinor fields) as elements of the exterior algebra $\Lambda^\bullet {\mathbb R}^n$ to differential forms on a smooth manifold.
At this point, we are done with the "linear algebra" part of the discussion and proceed to spinor fields. First, pick a Riemannian metric $h$ on a smooth oriented $n$-dimensional manifold $X$ (in fact, it suffices to pick a conformal structure). This feels odd (after all, we do not need any metric to define tensor fields), but does have a geometric explanation, due to non-linearity of the 2-fold covering group of $GL_+(n, {\mathbb R})$. In contrast, the 2-fold covering group of the conformal group $CSO(n)={\mathbb R}_+\times SO(n)$ is linear.
Now, comes the least geometric part (or, more precisely, the first truly non-geometric ingredient).
Instead of working with what we want, namely, tensor-like quantities on $X$, we switch to the principal
$SO(n)$-bundle $P_{SO(n)}$ over $X$ associated with the Riemannian metric $h$. This locally-trivial principal bundle can described by a set of transition maps
$$
g_{\alpha\beta}: U_\alpha\cap U_\beta\to SO(n),
$$
where $U_\alpha, U_\beta$ are suitable open subsets of $X$. Why is this non-geometric? Because, to go from our locally defined Riemannian metric $h_{ij}(x)dx^i\otimes dx^j$ to the transition maps $g_{\alpha\beta}$ we need to make one more unnatural choice, local orthonormal frame fields. If you find this satisfactory, good for you, I just cannot share your sense of satisfaction.
Next, one needs to pick a spin-structure on $X$. This can be defined in the form of a collection of lifts
$$
\tilde{g}_{\alpha\beta}: U_\alpha\cap U_\beta\to Spin(n)$$
which are transition maps of a principal $Spin(n)$-bundle $P_{Spin(n)}$ over $X$ covering $P_{SO(n)}$. This lift of the principal bundle does not always exist (and is typically non-unique); manifolds for which it exists are called spin-manifolds. Arguably, this step can be regarded as analogous to choosing an orientation on our manifold.
Going back to step 2, we choose a Clifford module $M$ for $Cl_n$ (or take the standard one, $\Delta_n$) and recall that we have a representation of $Spin(n)$ to the group of automorphisms of $M$. Regarding $M$ as a vector space (now, we are back to vectors!), we take the associated $M$-bundle over $X$
$$
S:= P_{Spin(n)} \times_{Spin(n)} M.
$$
At last, we now can define spinors (spinor fields): These are sections of the bundle $S\to X$.
Once we absorbed Step 4, spinor fields can be described locally as maps
$$
s_\alpha: U_\alpha\to M
$$
which transform under changes of local trivializations
of $P_{Spin(n)}$ via the maps $\tilde{g}_{\alpha\beta}(x)$ (regarded as automorphisms of $M$):
$$
s_\beta(x)= \tilde{g}_{\alpha\beta}(x) s_\alpha(x).
$$
Remark. Once you understand what is hidden in this equation, you can appreciate the following sentiments, taken from a physicist's paper "A Child’s Guide to Spinors" (by William O. Straub):
When I discovered that I was wrong about everything, I quickly learned to hate spinors. They’re non-intuitive, far more complicated than they have any right to be, and come either with two components or four (like the Dirac spinor, which is actually a stack of two spinors called a bispinor), both versions being equally hard to comprehend....
So now you know what a spinor is, and when asked you can confidently reply, like my professor did, that a spinor is a complex, multi-component vector-like quantity that has special transformation properties. No wonder he didn’t elaborate!
Do you really find the above definition (as sections of the bundle $S\to X$) satisfactory? I do not. (I wholeheartedly agree with the sentiment describing spinors as "non-intuitive, far more complicated than they have any right to be.") My interpretation of Atiyah's comments is that he did not find this satisfactory either. One of the most unsatisfactory aspects (in my mind) is the need to go from vector bundles to principal bundles and then back to vector bundles (which requires one, in particular, to choose orthonormal frames). Is there anything more direct? I am unaware of any such construction (in general: in the special case of conformally flat metrics $h$ it is easy to give one). Does it exist? If not, what is a formal or even a heuristic argument? To me, this is the mystery of spinors. Does this agree with what Atiyah had in mind? I do not think we will ever find out.
Why does it matter that our definition of spinors is cumbersome? I agree with Atiyah's analogy regarding differential forms (which "explained" the vector calculus): Having a good and intuitive formalism does matter when working with mathematical objects. In the case of spinors, I think, it would help to guide us through the morass of unintuitive computations.
The situation is better in the case of surfaces since Riemannian metrics on surfaces are conformally-flat and, hence, we have a reasonably geometric description of spinors in the 2-dimensional case. See for instance
Atiyah, Michael F., Riemann surfaces and spin structures, Ann. Sci. Éc. Norm. Supér. (4) 4, 47-62 (1971). ZBL0212.56402.
For instance, holomorphic spinors on Riemann surfaces can be described as "holomorphic half-differentials," locally, $\phi(z)dz^{1/2}$ (holomorphic sections of a half-canonical bundle $K_X^{1/2}$). Making sense of the square roots in transition maps requires some trickery; the nicest explanation I know comes from the Uniformization Theorem:
If a transition map $\gamma(z)=\frac{az+b}{cz+d}$ is a Moebius transformation, then
$$
\gamma'(z)=\frac{1}{(cz+d)^2}.$$
If a lift of $\gamma$ to $SL(2,{\mathbb C})$ is given (this is a choice of a spin structure!) then
$$
\sqrt{\gamma'(z)}= \frac{1}{cz+d}.
$$
Hence, if we realize $X$ as the quotient $H^2/\Gamma$ (which is the case with few exceptions), then holomorphic spinors on $X$ are simply automorphic holomorphic functions
$$
f: H^2\to {\mathbb C},
$$
such that the half-differential $\omega= f(z) dz^{1/2}$ is $\Gamma$-invariant. Here
$$
\gamma^*(\omega)= f(\gamma(z)) (\gamma'(z))^{1/2}dz^{1/2}.
$$
Strangely, Atiyah never mentions this connection in his, otherwise well-written, paper.
I decided not to discuss "why spinors can be regarded as square roots of tensors" or the Dirac operator, since the answer is already long enough. Getting to the Dirac operator will expand the length at least 3-fold.
