0

I'm trying to implement a model for predicting origin location - destination location traffic, from road traffic (if you are interested in the model, it is from this paper: Bell1983). The author defines an objective function to minimize under linear constraints, which I would like to modify the objective function of the model with a loss part for regularization, and look for a good candidate for loss that has similar order.


The author defines the objective function and constraints of the optimization problem, then solves it analytically (with Lagrange multipliers). The objective function originally is $f(\underline{t}) = \frac{(\sum_j t_j)!}{\prod_j (t_j!)}\cdot \prod_j(q_j)^{t_j}$ (nevermind the parameters), but the author takes its logarithm $F(\underline{t}) = \ln(f(\underline{t}))$ for convenience. Therefore: $$F(\underline{t}) = \ln(\sum_j t_j)!-\sum_j\ln(t_j!) + \sum_jt_j\ln(q_j)$$ The paper mentions that the derivative of $(\ln(x!))$ asymptotically equals $\ln(x)$ (this comment shows why), from which one can get the partial derivatives: $\frac{d F(\underline{t})}{d t_k} = \ln(\sum_j t_j) - \ln(t_k) + \ln(q_k)$.


Because the measurements of traffic that I work with are not entirely correct, some of the constraints I have conflict with each other (constraints come in the form: $\underline{v}=\underline{\underline{P}}\cdot\underline{t}$). I decided that the best is to keep only independent constraints, and the conflicting dependent ones I will include as a loss in the objective function, modifying the objective function to be $F(\underline{t})-g(\underline{t})$, where $g(\underline{t})$ is a sum of a function of $|\underline{v_{dep}}-\underline{\underline{P_{dep}}}\cdot\underline{t}|$ errors. Most common loss functions are the L2 (think of L2-distance, Euclidean) and L1 (think of Manhattan-distance). The problem with using these is that L2 grows quadratically, so error grows faster than $F$ which is suboptimal, and L1 doesn't have a continuous derivative which is needed.

I went on to try to design a loss function that grows asymptotically the same way as $F$, but hit difficulties.
Firstly, I might be wrong but I assume for any $t_k$ variable, $F$ grows at the order of $t_k\ln(t_k)$. I made this assumption based on the following argument: Taking $m=\sum_j t_j -t_k$, we can rewrite $F$ as $F(\underline{t}) = \ln((t_k+m)!)-\ln(t_k!) + t_k\ln(q_k) + c$ ($c$ is constant in terms of $t_k$). Modifying the previous formula, if $\frac{d(\ln(x!))}{d x} \sim \ln(x)$, then $\ln(x!) \sim x\ln(x)$. This means $F$ should grow as $(t_k+m)\ln(t_k+m)-x\ln(t_k)+t_k$ does, rewritten as $\ln(t_k)((\frac{\ln(t_k+m)}{\ln(t_k)}-1)t_k+m\frac{\ln(t_k+m)}{\ln(t_k)})+t_k$ should have the order $t_k\ln(t_k)(\frac{\ln(t_k+m)}{\ln(t_k)}-1)$. This, because $m$ is considerably larger than $t_k$, I assume to be asymptotically equal to $t_k\ln(t_k)$ but I might be wrong.
Secondly, now assuming that the order of $F$ in $t_k$ is asymptotically $t_k\ln(t_k)$, I've tried finding functions that'd qualify well for a loss function. We could write the loss function as a function of error. Looking at just one $(v_{dep}, P_{dep})$ loss component, the loss function should have a minimum when $|\underline{v_0}-\underline{\underline{P}}\cdot\underline{t}|$ does, ideally zero, so should have a minimum in $error=0$. The loss should be equal for $error=-\delta$ and $error=+\delta$, therefore it has to be an even function. But I did not manage to find a function satisfying these: I tried for example $(L(\underline{t}))\ln(L(\underline{t})+1)$ where $L(\underline{t}) = \underline{v_{dep}}-\underline{\underline{P_{dep}}}\cdot\underline{t}$, and it has a minimum at error $0$, but is not even: negative loss grows faster. I tried with functions with absolute values on error, but my tries resulted in discontinuous derivatives.

I'm thinking that such function does not exist. My intuition stems from that if such function existed, its derivative should be both an odd function, and asymptotic to $\log(x)$, but no modification will make $\log(x)$ odd.

Is there such a function?

Gary
  • 36,640
me9hanics
  • 115
  • I would also be interested in comments that prove / disprove that $F$ is asymptotically equal to $\underline{t}\ln(\underline{t})$. – me9hanics May 14 '24 at 01:59
  • 1
    $\frac{x}{\pi }\arctan (x)\log (1 + x^2 )$ is an even function that is differentiable everywhere and is asymptotically $|x|\log |x|$ for $x\to\pm \infty$. – Gary May 14 '24 at 02:07

1 Answers1

0

I may have found such a function: $g(\underline{|l|}) = \sum_j |l_j|\ln(|l_j|+1)$, where $l_j = v_j-\sum_k P_{jk}t_k$, the error, $j$ is among the indices of rows that are included in the loss function.

$|l_j|\ln(|l_j|+1)$ has a minimum at $l_j=0$, is even because of the absolute values, has order $\mathcal{O}(x\ln x)$, and its derivative seems to be differentiable everywhere, continuous: formula,plot.

@Gary has also pointed out a function for both cases (asymptotically $x\log x$ and even, and asymptotically $\log x$ and odd): $l\arctan(\underline{l})\log(1+\underline{l}^2)$ (element-wise square) for the first case and $\arctan(\underline{l})\log(1+\underline{l}^2)$ for the second, which is also smooth, unlike the one I gave. This can be important if you need the Hessian or anything requiring second derivatives.

me9hanics
  • 115
  • 1
    I actually modified my comment to cover the $x\log x$ case. The advantage of my example is that it is smooth. The second derivative of $|x|\log(|x|+1)$ does not exist at $x=0$. – Gary May 14 '24 at 02:27