7

In an invited talk in ICALP Tallinn 2024, Thomas Ehrhard and Laurent Regnier gave an overview of the differential lambda calculus. The talk is interesting and gives nice examples of how differentiation is relevant to lambda calculus. In one of the first slides they introduced the category $\textbf{Rel}_!$:

  • Objects of $\textbf{Rel}_!$ : All sets
  • $\textbf{Rel}_! (E, F) = \mathcal{P} (\mathcal{M}_\text{fin}(E) × F)$ where $\mathcal{M}_\text{fin}(E)$ is the set of all finite multisets $[a_1, \dots , a_n]$ of elements of $E$

As an example, they represented a simple lambda expression: $$λx^\text{Bool} . x ∧ ¬x = \{([t, t],f),\, ([f,f],f), \, ([t,f],f), \, ([t,f], t)\}$$

I don't understand how this relation represents the lambda expression. If there were two variables, one could understand the left part of the tulples as candidates for input and the right part as the answer, but the only bound variable is $x$. It would also be great if someone could provide a link to a simple explanation of the semantics of differential lambda calculus (especially in terms of $\textbf{Rel}_!$ that is not too involved with category theory.

  • “One could understand the left part of the tulples as candidates for input and the right part as the answer”: That's exactly what it is! In some sense the relational model “decorrelates” the different occurrences of variables. – sparusaurata Oct 10 '24 at 09:41
  • Maybe this can help: $\mathbf{Rel}!$ is a CCC, hence you can interpret the simply-typed λ-calculus in it (in the standard way, https://www.irif.fr/~mellies/mpri/mpri-ens/ens-mellies-cours-4.pdf). This is also the construction you perform in $\mathbf{Set}$, in $\omega\mathbf{CPO}$, etc. Now, if you want to interpret the untyped λ-calculus, you have to make something like Scott's $\mathcal{D}{\infty}$ construction; this is done here: https://www.irif.fr/~ehrhard/pub/enough.pdf. See also chap. 3 of https://www.irif.fr/~gmanzone/ManzonettoPhdThesis.pdf. – sparusaurata Oct 10 '24 at 09:52
  • Another way to see it is that $\mathbf{Rel}_!$ is a model of linear logic, and that a λ-term of simple type $A \to B$ can be seen as a proof (through Curry-Howard) of the formula $!A \multimap B$. – sparusaurata Oct 10 '24 at 09:59
  • Are you familiar with the $\lambda$-calculus and $\beta$-reduction? I'm asking that to adapt my answer to your background. – Taroccoesbrocco Oct 10 '24 at 19:29
  • Thanks @Taroccosbrocco , Yes, but I guess the explanation should go in the direction of linear logic, especially in how the relation represents nondeterminism and a resource interpretation. I don't think (but I could be wrong) using intuition for how lambda calculus is represnted in a CCC (using cartesian product to represent the environments etc. is helpful in this case). But I am curious how beta reduction is relevant for understanding the Rel_! interpretation. – Dan Synek Oct 10 '24 at 20:10
  • And @sparusaurata, I think this is an excellent suggestion. Will do it tomorrow! – Dan Synek Oct 10 '24 at 20:14

1 Answers1

4

The $\lambda$-term $\lambda x^\text{Bool}. x \land \lnot x$ represents a function from $\text{Bool}$ to $\text{Bool}$ (incidentally, a constant function). According to the relational semantics $\textbf{Rel}_!$, that $\lambda$-term is interpreted by a morphism $R \in \textbf{Rel}_!(|\text{Bool}|, |\text{Bool}|)$, that is, $R \subseteq \mathcal{M}_\text{fin}(|\text{Bool}|) \times |\text{Bool}|$, where $|\text{Bool}| = \{\textbf{t}, \textbf{f}\}$ is the interpretation of the type $\text{Bool}$ in $\textbf{Rel}_!$. As you correctly said, for every $(m,b) \in R$, intuitively $m \in \mathcal{M}_\text{fin}(|\text{Bool}|)$ represents the interpretation of a possible input to the function $\lambda x^\text{Bool}. x \land \lnot x$, and $b \in |\text{Bool}|$ the interpretation of the corresponding output.

To rigorously compute $R$, I should first define the interpretation of $\land \colon \text{Bool} \times \text{Bool} \to \text{Bool}$ and $\lnot \colon \text{Bool} \to \text{Bool}$ in the relational semantics $\textbf{Rel}_!$, and then define explicitly how $\lambda$-terms are interpreted in $\textbf{Rel}_!$. Detailing all that is quite long and tedious, since it requires that I introduce many auxiliary notions (see [1], [2], and [3] for details, where $\textbf{Rel}_!$ is noted $\textbf{MRel}$). Instead, I try to go straight to the point, keeping an informal approach that is hopefully enough to grasp the intuition.

In the body of the function $\lambda x^\text{Bool}. x \land \lnot x$ there are two occurrences of the variable $x$ of type $\text{Bool}$. This means that when the function $\lambda x^\text{Bool}. x \land \lnot x$ is fed with an input argument of type $\text{Bool}$, this argument is copied (that is, duplicated) and each copy of the argument replaces one of the two occurrences of $x$ in the body of the function. This is exactly what happens with $\beta$-reduction when the function $\lambda x^\text{Bool}. x \land \lnot x$ is applied to an argument $N$ of type $\text{Bool}$:

$$\tag{*} (\lambda x^\text{Bool}. x \land \lnot x)N \ \to_\beta \ N \land \lnot N$$

Relational semantics interprets the two copies of the input argument as if they are independent entities. As a consequence, in the interpretation $R \subseteq \mathcal{M}_\text{fin}(|\text{Bool}|) \times |\text{Bool}|$ of the function $\lambda x^\text{Bool}. x \land \lnot x$, for every $(m, b) \in R $, $m$ must be a multiset over $|\text{Bool}|$ of cardinality $2$, where each of the two elements of $m$ represents the value in $|\text{Bool}|$ of one of the two copies of the input. As $|\text{Bool}| = \{\textbf{t}, \textbf{f}\}$, there are 4 cases:

  1. either $m = [\textbf{t}, \textbf{t}]$ and then the value of the output $b$ is $ \textbf{t} \land \lnot \textbf{t} = \textbf{f}$,

  2. or $m = [\textbf{f}, \textbf{f}]$ and then the value of the output $b$ is $ \textbf{f} \land \lnot \textbf{f} = \textbf{f}$,

  3. or $m = [\textbf{t}, \textbf{f}]$ where $\textbf{t}$ feeds the first occurrence of $x$ and $\textbf{f}$ feeds the second occurrence of $x$, and then the value of the output $b$ is $ \textbf{t} \land \lnot \textbf{f} = \textbf{t}$,

  4. or $m = [\textbf{t}, \textbf{f}]$ where $\textbf{f}$ feeds the first occurrence of $x$ and $\textbf{t}$ feeds the second occurrence of $x$, and then the value of the output $b$ is $ \textbf{f} \land \lnot \textbf{t} = \textbf{f}$.

Summing up, the function $\lambda$-term $\lambda x^\text{Bool}. x \land \lnot x \, \colon \text{Bool} \to \text{Bool}$ is interpreted in relational semantics by

$$ R = \{([\textbf{t}, \textbf{t}], \textbf{f}), \, ([\textbf{f}, \textbf{f}], \textbf{f}), \, ([\textbf{t}, \textbf{f}], \textbf{f}), \, ([\textbf{t}, \textbf{f}], \textbf{t})\} \subseteq \mathcal{M}_\text{fin}(|\text{Bool}|) \times |\text{Bool}| $$


Of course, cases 3 and 4 above sound a bit unnatural or counterintuitive, because they suggest that an input may be interpreted as a "superposition" of two "incoherent" values (the multiset $[\textbf{t}, \textbf{f}]$), and the output is not "deterministic" (both $([\textbf{t}, \textbf{f}], \textbf{f}) \in R$ and $([\textbf{t}, \textbf{f}], \textbf{t}) \in R$). But the idea is that relational semantics takes into account how many copies of the input are demanded by the function and how they are distributed in the body of the function. Perhaps surprisingly, this works perfectly as a denotational model of a perfectly sequential and deterministic model of computation such as the $\lambda$-calculus.

The differential $\lambda$-calculus and its companion the resource $\lambda$-calculus (see [3], as well as Ehrhard and Regnier's talk you cited) are resource-aware variants of the $\lambda$-calculus that internalize the "non-determinism" of the relational semantics into the calculus: inputs cannot be duplicated, hence a function that asks for two copies of an argument must be fed with a bag (represented by a finite multiset) of two independent inputs, and the output non-deterministically depends on which input feds which demand. In the resource $\lambda$-calculus, the $\beta$-reduction $(*)$ above becomes

$$ (\lambda x^\text{Bool}. x \land \lnot x)[N_1, N_2] \ \to \ (N_1 \land \lnot N_2) + (N_2 \land \lnot N_1) $$

References

[1] A. Bucciarelli, T. Ehrhard and G. Manzonetto. Not Enough Points is Enough. 16th EACSL Annual Conference on Computer Science and Logic (CSL'07), volume 4646 of LNCS, pages 268-282, 2007. pdf

[2] A. Bucciarelli, T. Ehrhard and G. Manzonetto. A Relational Semantics for Parallelism and Non-Determinism in a Functional Setting. Annals of Pure and Applied Logic, Vol. 163, issue 7, pp. 918-934, 2012. pdf

[3] G. Manzonetto. What is a Categorical Model of the Differential and the Resource Lambda Calculi? Mathematical Structures in Computer Science, Volume 22(3), pages 451-520, 2012. pdf

  • Thanks, that was an amazing answer. It's on the right level of detail and, incidentally, it did make me excited to dig into the "boring" details in your references. Thanks so much!!! – Dan Synek Oct 12 '24 at 01:43
  • @DanSynek - You're welcome! If you are satisfied with the answer because it solved your problem, you can accept it by clicking on the checkmark on the left, close to the vote area. See here and here. – Taroccoesbrocco Oct 12 '24 at 03:55