Outer vs. Inner Products

[roshambo919]

I'm looking at your unconstrained nonlinear optimization code. Specifically the Quasi-Newton methods (DFP, BFGS, ...)

DFP
Q = Q0 + _np.dot(p,p.T)/_np.dot(p.T,q) - ( _np.dot(Q0,q).dot(_np.dot(q.T,Q0)))/(_np.dot(q.T,Q0).dot(q))

When I take this excerpt and run the following (just random numbers)

Q0 = _np.array([[10,-10],[-10,20]])
q = _np.array([3,10])
p = _np.array([10,5])
_np.dot(p,p.T)/_np.dot(p.T,q) - ( _np.dot(Q0,q).dot(_np.dot(q.T,Q0)))/(_np.dot(q.T,Q0).dot(q))

I get a scalar out when I think it should be a matrix of same size. Unless I made some mistake, I think numpy is doing inner products in some places where you want outer products. If so, consider using np.outer to be explicit.

[gusmaogabriels]

@roshambo919 ,

thank you. Can you please tell me what version of python and numpy you have installed?

Thanks

[roshambo919]

Using python 3.8.5 and numpy 1.19.2, the following product equivalent results:

import numpy as np
x = np.array([1, 2])
print(np.dot(x.T, x))
print(np.dot(x, x))
print(np.dot(x, x.T))

Same goes for python 3.7.9. Same goes for python 2.7.18 and numpy 1.16.6

[gusmaogabriels]

@roshambo919,

You need to make sure you follow the column-vector convention used in the package. Maybe I should've been more explicit in that regard.

Please check if the following works for you.

Cheers,

import numpy as np
x = np.array([1, 2]).reshape(-1,1) # or x = np.array([1, 2])[:,np.newaxis]
print(np.dot(x.T, x))
# print(np.dot(x, x)) # <- this should raise an exception
print(np.dot(x, x.T))