How to fit parameters to given data of given function?

Question

I am given a function $g(t) = a + b \cdot \exp(-c \cdot t)$ and a set of $(t_i, g(t_i))$ pairs (temperature measurements), and the task is to find values of parameters $a,b,c$ s.t. they fit given data. The task is to do this numerically, and in particular using BFGS method, which is a gradient descent method that uses Rank-2-update to choose descent direction and Wolfe conditions to choose step length. I implemented the algorithm but got stuck in theory.

My approach would be to minimize the sum of the squares of the residuals, like:

$$f(a,b,c) = \sum_i [g(t_i) - (a + b \cdot \exp(-c \cdot t_i))] ^2\rightarrow \min$$

But if we take a look at a gradient we see that:

$$\begin{array}{rcl}\frac{\partial f}{\partial a}(a,b,c) &=& -2\sum_i[g(t_i) - (a + b \cdot \exp(-c \cdot t_i))]\\ \frac{\partial f}{\partial b}(a,b,c) &=& -2\sum_i[g(t_i) - (a + b \cdot \exp(-c \cdot t_i))]\cdot\exp(-c\cdot t_i)\\ \frac{\partial f}{\partial c}(a,b,c) &=& -2\sum_i[g(t_i) - (a + b \cdot \exp(-c \cdot t_i))]\cdot(-bc)\cdot\exp(-c\cdot t_i)\end{array}$$

So we get $\nabla f(a,b,c) = 0$ for all $c=0$ and $a + b = \mu$ – mean value of all $g(t_i)$'s. So this approach doesn't seem to make much sense, since the algorithm guarantees only convergence to stationary points $(\nabla f(x) = 0)$, but then the solution is ambigious and far from optimal.

So my question would be, what function should I minimize to solve the problem?

Note: there were some similar questions to this problem (like this, for instance), but neither of them solved this particular problem, so I hope this question is unique.

Thanks in advance!

Answer 1

First, as NotNotLogical pointed out, you have a typo in the third partial derivative.

Now, more about your problem; you want to fit you data minimizing the sum of the squared errors given by $$f(a,b,c) = \sum_i [g(t_i) -( a + b \cdot \exp(-c \cdot t_i))]^2$$ This model is nonlinear with respect to its own parameters and, whichever method you use for minimization, you need "reasonable" starting values. If you data do not show large errors and cover a sufficeintly large range, you can estimate them using the point close to zero, the value of the asymptote and either the slope at the origin or the point where the function is at the middle of its range. This will not be the most general situation.

If you look at you model, you notice that the model is not linear because of parameter $c$. So, suppose you fix it at an arbitrary value and let us call $y(i)=\exp(-c \cdot t_i)$. Then, for this value of $c$, the sum of squares write $$F(a,b) = \sum_i [g(t_i) - (a + b \cdot y_i)] ^2$$ which corresponds to a linear regression and then, for given $c$, you can easily compute $a$,$b$ and the sum of squares $(SSQ)$. So now, since this problem is extremely simple, you can run diffrent values of $c$ and plot $SSQ$ vs $c$ until you approximately find a minimum. At that point, you then have the "good" estimates of parameters $a,b,c$ and you are ready to start the full nonlinear regression work.

To polish the solution, you have two possibilities :

• minimize $f(a,b,c)$ using a standard minimization method (such as BFGS for example)
• solve the three partials for zero using Newton-Raphson

To summarize, for me, all the problem is to have "good" starting guesses.