gradient for fmin_tnc not working - python-3.x

Question

I am training multiclass logistic regression for handwritting recognition.For function minimization i am using fmin_tnc.
I have implemented gradient function as follows:
def gradient(theta,*args):
X,y,lamda = args;
m = np.size(X,0);
h = X.dot(theta);
grad = (1/m) * X.T.dot( sigmoid(h)-y );
grad[1:np.size(grad),] = grad[1:np.size(grad),] + (lamda/
m)*theta[1:np.size(theta),] ;
return grad.flatten()
#flattened because fmin_tnc accepts list of gradients
This yields correct gradient values for small set example provided below:
theta_t = np.array([[-2],[-1],[1],[2]]);
X_t = np.array([[1,0.1,0.6,1.1],[1,0.2,0.7,1.2],[1,0.3,0.8,1.3],
[1,0.4,0.9,1.4],[1,0.5,1,1.5]])
y_t = np.array([[1],[0],[1],[0],[1]])
lamda_t = 3
But when using checkgrad function from scipy its giving error of 0.6222474393497573
I am not able to trace why this is happening.Because of this may be fmin_tnc is not performing any optimization and always gives optimized parameters equal to initial parameters given.

Answer 1

fmin_tnc function call is as follows:
optimize.fmin_tnc(func=lrcostfunction, x0=initial_theta,fprime = gradient,args=
(X,tmp_y.flatten(),lamda))
As y and theta passed is of form 1-d array having size(n,) it should be converted to 2-d array having size (n,1).This is because 2-d array form is used in gradient function implementation.
Correct implementation is as follow:
def gradient(theta,*args):
#again y and theta reshaped for same reason
X,y,lamda = args;
l = np.size(X,1);
theta = np.reshape(theta,(l,1));
m = np.size(X,0);
y = np.reshape(y,(m,1));
h = sigmoid( X.dot(theta) );
grad = (1/m) * X.T.dot( h-y );
grad[1:np.size(grad),] = grad[1:np.size(grad),] +
(lamda/m)*theta[1:np.size(theta),] ;
return grad.ravel()