2

On slide 11 here it is claimed that the weighted Frobenius norm leads to a scale-invariant optimization method. Similar claims about this norm can be found throughout the literature see 1,2,3.

In Greenstadt's original paper the Weighted Frobenius norm is proposed without mention of its relation to scale-invariance. It is claimed that this norm is scale-invariant. That I take to mean for any linear transformation $T$, matrix $A$ has the same norm as matrix $TA$. I am not even sure this is is true (plug in the identity matrix for the weight matrix $W$). Perhaps I got the wrong invariance definition.

I have yet to found what exactly is this scale-invariance referring to. It's proof is supposed to be very straightforward because no one seemed to think it necessary to be written down explicitly.

Thank you so much !

Patrick
  • 356

1 Answers1

0

I think they mention "scale invariant" in the context that with the given choice of $W$, you get (see next slide):

$$B_{k+1}=(I-\gamma_k\bar{y}_k\bar{s}_k^T)B_k\cdots$$

where,

$$\gamma_k=\frac{1}{\bar{y}_k^T\bar{s}_k}.$$

Then $\gamma_k \bar{y}_k\bar{s}_k^T$ and related quantities are unitless

Alex R.
  • 33,289
  • What does it have to do with the Weighted Frobenius norm though? What is the intrinsic invariance property of this norm ? – Patrick Jun 28 '16 at 05:15