Am I right in thinking that differential of $f(x)$ is $dy = {\Delta}y$ when ${\Delta}x$ is infinitesimally small, otherwise $dy$ $\ne$ ${\Delta}y$?

Question

Since ${\Delta}y$ is known to be the $exact$ change in $f(x)$, and $dy$ is the $approximation$ of the change if ${\Delta}x$ is $not$ infinitesimal, then what is the difference between ${\Delta}x$ and $dx$? Or does $dy$ only exist when ${\Delta}x$ is infinitesimally small and is not defined otherwise ($however,$ might be defined when using numerical methods?)

I am new to Maths, correct me if anything is wrong. I need intuitive understanding to learn not to take integration/differentiation as a black box operation.

Answer 1

Given a function $f: \>\Omega\to{\mathbb R}^m$ with domain $\Omega\in{\mathbb R}^n$ (in particular: a function $f:\>J\to{\mathbb R}$ defined on an interval $J\subset {\mathbb R}$) we automatically have the function $$\Delta f:\quad (x,\Delta x)\mapsto\bigl(f(x),f(x+\Delta x)-f(x)\bigr)\tag{1}$$ giving the function value increments $f(x+\Delta x)-f(x)$ at all points $x\in\Omega$ as functions of the increment variable $\Delta x$. Usually one does not exhibit the first variable in $(1)$, i.e., one considers a given point $p\in\Omega$, and writes $$\Delta f=f(p+\Delta x)-f(p)\ .\tag{2}$$ These increments $f(p+\Delta x)-f(p)$ are true function value differences. Their exact values can only be determined by truly computing $f(p+\Delta x)$ and $f(p)$.

If the function $f$ is differentiable at $p$ then there is a linear map, called $df(p)$, approximating $\Delta f$ in $(2)$ when $|\Delta x|$ is "small"; namely $$f(p+\Delta x)-f(p)\approx df(p).\Delta x\qquad(\Delta x\to0)\ ,$$ or precisely: $$f(p+\Delta x)-f(p)= df(p).\Delta x+o\bigl(|\Delta x|\bigr)\qquad(\Delta x\to0)\ .\tag{3}$$ When $f:\>{\mathbb R}\to{\mathbb R}$ then $(3)$ means $$f(p+\Delta x)-f(p)= f'(p)\>\Delta x+o\bigl(|\Delta x|\bigr)\qquad(\Delta x\to0)\ .$$ When one is interested mainly in these linear approximating maps $df(p)$ one denotes the local increment variables $\Delta x$, $\Delta y$, etc., by $dx$ and $dy$. This can be built into an overall encompassing "theory of differentials", but for your everyday use is only a matter of letters.

Answer 2

Let's take a function $f(x)=x^2$ and a point $x_0$. Calculate the difference: $$\Delta y = f(x_0+\Delta x)-f(x_0) = (x_0+\Delta x^2)-x_0^2 = 2x_0\cdot \Delta x + (\Delta x)^2$$

Hmmm, it looks like that $\Delta y = a\cdot \Delta x + (\Delta x)^2$, where $a=2x_0$ is a number. Assuming it is not zero (i.e. $x_0\neq 0$) we see that $|\Delta x| \ll |a|$ the second term is quite negligible, so $\Delta y \approx a\cdot\Delta x$ (and the quality of this approximation depends on the inequality $|\Delta x| \ll |a|$).

Now let's think about infinitesimals. What do we mean that $\Delta x$ is infinitesimally small? We could try to define

$\Delta x$ is infinitesimal if for all real $a\neq 0$ we have $|\Delta x| < |a|$

But... there is no such (non-zero) "infinitesimal": for every $\Delta x$ we can choose $a=\Delta x /2$. We either need to invent new kind of numbers or accept "infinitesimal" to be a loosely defined concept, meaning "for very small $\Delta x$ we have $\Delta y\approx \Delta x$". As you will see, in mathematics usually we don't speak about "infinitesimal" $dy$ and $dx$ – we usually speak about the ratio $dy/dx$ that often happens to be a real number. (Such a ratio doesn't need to exist, for example if your function "jumps", i.e. is not continuous).

I'd suggest thinking about them in a loose way ("$dy/dx \approx \Delta y/\Delta x$ and the approximation becomes better and better if we take $\Delta x$ to be smaller and smaller"), but learning rigorous analysis in which you consider derivatives, not infinitesimal quantities. Eventually you will get used to this way of thinking and to proving rigorous theorems – good luck!

(A small disclaimer: there is also a way to make sense about "infinitesimal" equations like $dy = f'(x)\,dx$ that use the language of differential forms, but both sides are not numbers – they are linear functions. You can read about this staff for example in the books mentioned here).

Answer 3

$y$ is just a different name for $f(x)$, so there is no real meaning in talking about $f(x)$ as $\Delta y$. $dx$ and $dy$ are not numbers, when by their own they might as well be 0, the concept of an infinitesimal quantity is better learnt as in the language of limits.

Since you're new to maths and learning about calculus, I suggest you watch 3b1b(3Blue1Brown) "Essence of Calculus", which is exactly a series on YouTube about taking the black box out of Differentiation and Integration without going too deep on rigorous mathematics proofs.