python - SciPy Conjugate Gradient Optimisation not invoking callback method after each iteration -

i followed tutorial here in order implement logistic regression using theano. aforementioned tutorial uses scipy's fmin_cg optimisation procedure. among important argument aforementioned function are: f object/cost function minimised, x0 user supplied initial guess of parameters, fprime function provides derivative of function f @ x , callback optional user-supplied function, called after each iteration.

the training function defined follows:

# creates function computes average cost on training set def train_fn(theta_value):     classifier.theta.set_value(theta_value, borrow=true)     train_losses = [batch_cost(i * batch_size)                     in xrange(n_train_batches)]     return numpy.mean(train_losses) 

what above code does, go through minibatches in training dataset, each minibatch calculate average batch cost (i.e. average of cost function applied each of training samples in minibatch) , averages cost on batches. might worth pointing out cost each individual batch calculated batch_cost -- theano function.

to me, seems callback function being called arbitrarily, , not after every iteration documentation in scipy claims.

here output received after modifying train_fn , callback adding "train" , "callback" prints respectively.

... training model train train train callback validation error 29.989583 % train callback validation error 24.437500 % train callback validation error 20.760417 % train callback validation error 16.937500 % train callback validation error 14.270833 % train callback validation error 14.156250 % train callback validation error 13.177083 % train callback validation error 12.270833 % train train callback validation error 11.697917 % train callback validation error 11.531250 % 

my question is, since each call train_fn indeed training epoch, how change behaviour, call callback invoked after train_fn?

each call train_fn not single training epoch. i'm not sure how fmin_cg implemented, in general, conjugate gradient methods may call cost or gradient function more once per minimziation step. (as far understand it) required find conjugate vector relative previous step taken.¹

so callback being called every time fmin_cg takes step. if need function called every time cost or gradient function called, can put call inside relevant function.

^{1. edit: @ least when nonlinear methods, fmin_cg is. wikipedia page suggests vanilla conjugate gradient (cg) methods may not require multiple calls, think aren't suitable optimizing nonlinear functions. cg code i've seen -- guess must have been nonlinear cg -- involved @ least 1 line search per step. call multiple evaluations of gradient function.}