The OLP's book spares one and a half pages to some philosophical reflections on the topic. The authors might have thought that it would be useful to touch on several issues that might occur to the reader in order to acknowledge that they are not pointless at all, but without spilling over into philosophical and linguistic areas that are not of significance in mathematics. I shall present an explication of what is at stake within the same limits, hoping to be helpful.
A classical theme of philosophical argumentation is the inquiry into tertium quid (viz., what is the "third thing"?) that unites or mediates two given elements. For an extensive overview and variations of this theme in the context concerning properties and relations, see Katarina Perovic's SEoP entry "Bradley’s Regress" due to Francis H. Bradley's work. We shall focus on the line of reasoning which typically proceeds in the following path:
- We begin with a proposition involving $X$ and $Y$ which are of the same category.
- It is generally untenable to regard $X$ and $Y$ as mere juxtapositions by chance; they are not simply co-present without relation.
- Therefore, there must be some third constituent that connects or relates them. Thus, the structure of the proposition may be represented as $X^{\frown}?^{\frown}Y$.
- Let us call this third constituent $\alpha$, hence $X^{\frown}\alpha^{\frown}Y$.
The task is, then, to adequately characterise $\alpha$.
- If $\alpha$ is of the same category as $X$ and $Y$, the result is either an absurdity, or the need of a third element between $X$ and $\alpha$ and, also between $\alpha$ and $Y$, recurs, which leads to an infinite regress.
- Therefore, $\alpha$ must be of some other category—prompting the need for a proper definition of this mediating term, from which the characterisation problem ensues again.
This issue manifests in set theory as follows: In the same fashion, we have two singular terms here for the set $\langle x, y\rangle$ (which is equal to $\{\{x\},\{x, y\}\}$) and the relation $R$ (which has been defined to be a set). We connect them via $\in$. If $\in$ is a relation, it must count as a set alongside the others, but then the result is nonsensical. Otherwise, we have to put it into another category and characterise it accordingly, but if membership is neither a relation (i.e., that which relates) nor a relatum (i.e., that which is related), then what is it? The question persists.
A few brief remarks about why "this rolls together a simple version of Frege’s 'concept horse' paradox, and a famous objection that Wittgenstein once raised against Russell" may be in order.
Frege held that concepts and objects (these terms are explicated in his writings at length) are fundamentally different: Concepts are unsaturated (they need something to apply to), whereas objects are saturated (complete in themselves). In the language of logic he developed, names must refer to objects. However, Frege's exclusivity thesis—that something is either a concept or an object, never both—produces this result when predicative expressions are nominalised. So, when we speak of 'the concept horse', the phrase behaves like a name and thus refers to an object, not a concept. This leads to the paradoxical result that:
The concept horse is not a concept.
Frege admitted this as a troubling but unavoidable consequence of his system, stemming from the tension between language about concepts and language using concepts predicatively.
A similar incoherence was allegedly laid bare in Russell's analyses by Wittgenstein's views. Wittgenstein emphasised that relations (like set-membership '$\in$') cannot be treated as objects, or sets, without losing their logical role. In his Tractatus Logico-Philosophicus, Wittgenstein writes,
3.1432 Instead of, ‘The complex sign “$aRb$” says that $a$ stands to $b$ in the relation $R$’, we ought to put, ‘That “$a$” stands to “$b$”
in a certain relation says that $aRb$.’
This means the relation ($R$ or $\in$) is not an object but the way in which objects are structured in a proposition. Treating '$\in$' as a set, as Russell’s framework might suggest, reduces it to an object, which Wittgenstein saw as a category mistake akin to Frege’s concept 'horse' paradox.
What emerges of relevance for our present concerns, the question boils down to whether the primitives in formal languages are absolute or relative, as noted in the comments. The mainstream course on this can be described overall as taking a relative (i.e., context-sensitive) stance, in a certain sense. Specifically, $\in$ is taken as a dyadic predicate with a fixed syntax and interpretation in the object language of set theory, allowing alternative treatments in the metalanguage and other contexts. Actually, that is what Benno Kerry, a contemporary critic of Frege, argued: The distinction is relative rather than absolute; a thing may be a concept in one context and an object in another.
Finally, in this connection, I recommend Lewis Carroll's witty article "What the Tortoise Said to Achilles", a dialogue centred on the issue of tertium quid over a deductive inference (its freely available pdf can be found on the Web).
