Package omero :: Package util :: Module mpfit
[hide private]
[frames] | no frames]

Source Code for Module omero.util.mpfit

   1  """ 
   2  Perform Levenberg-Marquardt least-squares minimization, based on MINPACK-1. 
   3   
   4                                                                     AUTHORS 
   5    The original version of this software, called LMFIT, was written in FORTRAN 
   6    as part of the MINPACK-1 package by XXX. 
   7   
   8    Craig Markwardt converted the FORTRAN code to IDL.  The information for the 
   9    IDL version is: 
  10           Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770 
  11           craigm@lheamail.gsfc.nasa.gov 
  12           UPDATED VERSIONs can be found on my WEB PAGE: 
  13                  http://cow.physics.wisc.edu/~craigm/idl/idl.html 
  14           
  15    Mark Rivers created this Python version from Craig's IDL version. 
  16          Mark Rivers, University of Chicago 
  17          Building 434A, Argonne National Laboratory 
  18          9700 South Cass Avenue, Argonne, IL 60439 
  19          rivers@cars.uchicago.edu 
  20          Updated versions can be found at http://cars.uchicago.edu/software 
  21    
  22   Sergey Koposov converted the Mark's Python version from Numeric to numpy 
  23          Sergey Koposov, Max Planck Institute for Astronomy 
  24          Heidelberg, Germany, D-69117 
  25          koposov@mpia.de 
  26          Updated versions can be found at http://code.google.com/p/astrolibpy/source/browse/trunk/ 
  27   
  28                                                                   DESCRIPTION 
  29   
  30   MPFIT uses the Levenberg-Marquardt technique to solve the 
  31   least-squares problem.  In its typical use, MPFIT will be used to 
  32   fit a user-supplied function (the "model") to user-supplied data 
  33   points (the "data") by adjusting a set of parameters.  MPFIT is 
  34   based upon MINPACK-1 (LMDIF.F) by More' and collaborators. 
  35   
  36   For example, a researcher may think that a set of observed data 
  37   points is best modelled with a Gaussian curve.  A Gaussian curve is 
  38   parameterized by its mean, standard deviation and normalization. 
  39   MPFIT will, within certain constraints, find the set of parameters 
  40   which best fits the data.  The fit is "best" in the least-squares 
  41   sense; that is, the sum of the weighted squared differences between 
  42   the model and data is minimized. 
  43   
  44   The Levenberg-Marquardt technique is a particular strategy for 
  45   iteratively searching for the best fit.  This particular 
  46   implementation is drawn from MINPACK-1 (see NETLIB), and is much faster 
  47   and more accurate than the version provided in the Scientific Python package 
  48   in Scientific.Functions.LeastSquares. 
  49   This version allows upper and lower bounding constraints to be placed on each 
  50   parameter, or the parameter can be held fixed. 
  51   
  52   The user-supplied Python function should return an array of weighted 
  53   deviations between model and data.  In a typical scientific problem 
  54   the residuals should be weighted so that each deviate has a 
  55   gaussian sigma of 1.0.  If X represents values of the independent 
  56   variable, Y represents a measurement for each value of X, and ERR 
  57   represents the error in the measurements, then the deviates could 
  58   be calculated as follows: 
  59   
  60     DEVIATES = (Y - F(X)) / ERR 
  61   
  62   where F is the analytical function representing the model.  You are 
  63   recommended to use the convenience functions MPFITFUN and 
  64   MPFITEXPR, which are driver functions that calculate the deviates 
  65   for you.  If ERR are the 1-sigma uncertainties in Y, then 
  66   
  67     TOTAL( DEVIATES^2 ) 
  68   
  69   will be the total chi-squared value.  MPFIT will minimize the 
  70   chi-square value.  The values of X, Y and ERR are passed through 
  71   MPFIT to the user-supplied function via the FUNCTKW keyword. 
  72   
  73   Simple constraints can be placed on parameter values by using the 
  74   PARINFO keyword to MPFIT.  See below for a description of this 
  75   keyword. 
  76   
  77   MPFIT does not perform more general optimization tasks.  See TNMIN 
  78   instead.  MPFIT is customized, based on MINPACK-1, to the 
  79   least-squares minimization problem. 
  80   
  81   
  82                                                             USER FUNCTION 
  83   
  84   The user must define a function which returns the appropriate 
  85   values as specified above.  The function should return the weighted 
  86   deviations between the model and the data.  It should also return a status 
  87   flag and an optional partial derivative array.  For applications which 
  88   use finite-difference derivatives -- the default -- the user 
  89   function should be declared in the following way: 
  90   
  91     def myfunct(p, fjac=None, x=None, y=None, err=None) 
  92          # Parameter values are passed in "p" 
  93          # If fjac==None then partial derivatives should not be 
  94          # computed.  It will always be None if MPFIT is called with default 
  95          # flag. 
  96          model = F(x, p) 
  97          # Non-negative status value means MPFIT should continue, negative means 
  98          # stop the calculation. 
  99          status = 0 
 100          return([status, (y-model)/err] 
 101   
 102   See below for applications with analytical derivatives. 
 103   
 104   The keyword parameters X, Y, and ERR in the example above are 
 105   suggestive but not required.  Any parameters can be passed to 
 106   MYFUNCT by using the functkw keyword to MPFIT.  Use MPFITFUN and 
 107   MPFITEXPR if you need ideas on how to do that.  The function *must* 
 108   accept a parameter list, P. 
 109   
 110   In general there are no restrictions on the number of dimensions in 
 111   X, Y or ERR.  However the deviates *must* be returned in a 
 112   one-dimensional Numeric array of type Float. 
 113   
 114   User functions may also indicate a fatal error condition using the 
 115   status return described above. If status is set to a number between 
 116   -15 and -1 then MPFIT will stop the calculation and return to the caller. 
 117   
 118   
 119                                                          ANALYTIC DERIVATIVES 
 120   
 121   In the search for the best-fit solution, MPFIT by default 
 122   calculates derivatives numerically via a finite difference 
 123   approximation.  The user-supplied function need not calculate the 
 124   derivatives explicitly.  However, if you desire to compute them 
 125   analytically, then the AUTODERIVATIVE=0 keyword must be passed to MPFIT. 
 126   As a practical matter, it is often sufficient and even faster to allow 
 127   MPFIT to calculate the derivatives numerically, and so 
 128   AUTODERIVATIVE=0 is not necessary. 
 129   
 130   If AUTODERIVATIVE=0 is used then the user function must check the parameter 
 131   FJAC, and if FJAC!=None then return the partial derivative array in the 
 132   return list. 
 133     def myfunct(p, fjac=None, x=None, y=None, err=None) 
 134          # Parameter values are passed in "p" 
 135          # If FJAC!=None then partial derivatives must be comptuer. 
 136          # FJAC contains an array of len(p), where each entry 
 137          # is 1 if that parameter is free and 0 if it is fixed. 
 138          model = F(x, p) 
 139          Non-negative status value means MPFIT should continue, negative means 
 140          # stop the calculation. 
 141          status = 0 
 142          if (dojac): 
 143             pderiv = zeros([len(x), len(p)], Float) 
 144             for j in range(len(p)): 
 145                   pderiv[:,j] = FGRAD(x, p, j) 
 146          else: 
 147             pderiv = None 
 148          return([status, (y-model)/err, pderiv] 
 149   
 150   where FGRAD(x, p, i) is a user function which must compute the 
 151   derivative of the model with respect to parameter P[i] at X.  When 
 152   finite differencing is used for computing derivatives (ie, when 
 153   AUTODERIVATIVE=1), or when MPFIT needs only the errors but not the 
 154   derivatives the parameter FJAC=None. 
 155   
 156   Derivatives should be returned in the PDERIV array. PDERIV should be an m x 
 157   n array, where m is the number of data points and n is the number 
 158   of parameters.  dp[i,j] is the derivative at the ith point with 
 159   respect to the jth parameter. 
 160   
 161   The derivatives with respect to fixed parameters are ignored; zero 
 162   is an appropriate value to insert for those derivatives.  Upon 
 163   input to the user function, FJAC is set to a vector with the same 
 164   length as P, with a value of 1 for a parameter which is free, and a 
 165   value of zero for a parameter which is fixed (and hence no 
 166   derivative needs to be calculated). 
 167   
 168   If the data is higher than one dimensional, then the *last* 
 169   dimension should be the parameter dimension.  Example: fitting a 
 170   50x50 image, "dp" should be 50x50xNPAR. 
 171   
 172   
 173                     CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD 
 174   
 175   The behavior of MPFIT can be modified with respect to each 
 176   parameter to be fitted.  A parameter value can be fixed; simple 
 177   boundary constraints can be imposed; limitations on the parameter 
 178   changes can be imposed; properties of the automatic derivative can 
 179   be modified; and parameters can be tied to one another. 
 180   
 181   These properties are governed by the PARINFO structure, which is 
 182   passed as a keyword parameter to MPFIT. 
 183   
 184   PARINFO should be a list of dictionaries, one list entry for each parameter. 
 185   Each parameter is associated with one element of the array, in 
 186   numerical order.  The dictionary can have the following keys 
 187   (none are required, keys are case insensitive): 
 188   
 189          'value' - the starting parameter value (but see the START_PARAMS 
 190                           parameter for more information). 
 191   
 192          'fixed' - a boolean value, whether the parameter is to be held 
 193                           fixed or not.  Fixed parameters are not varied by 
 194                           MPFIT, but are passed on to MYFUNCT for evaluation. 
 195   
 196          'limited' - a two-element boolean array.  If the first/second 
 197                             element is set, then the parameter is bounded on the 
 198                             lower/upper side.  A parameter can be bounded on both 
 199                             sides.  Both LIMITED and LIMITS must be given 
 200                             together. 
 201   
 202          'limits' - a two-element float array.  Gives the 
 203                            parameter limits on the lower and upper sides, 
 204                            respectively.  Zero, one or two of these values can be 
 205                            set, depending on the values of LIMITED.  Both LIMITED 
 206                            and LIMITS must be given together. 
 207   
 208          'parname' - a string, giving the name of the parameter.  The 
 209                             fitting code of MPFIT does not use this tag in any 
 210                             way.  However, the default iterfunct will print the 
 211                             parameter name if available. 
 212   
 213          'step' - the step size to be used in calculating the numerical 
 214                          derivatives.  If set to zero, then the step size is 
 215                          computed automatically.  Ignored when AUTODERIVATIVE=0. 
 216   
 217          'mpside' - the sidedness of the finite difference when computing 
 218                            numerical derivatives.  This field can take four 
 219                            values: 
 220   
 221                                   0 - one-sided derivative computed automatically 
 222                                   1 - one-sided derivative (f(x+h) - f(x)  )/h 
 223                                  -1 - one-sided derivative (f(x)   - f(x-h))/h 
 224                                   2 - two-sided derivative (f(x+h) - f(x-h))/(2*h) 
 225   
 226                           Where H is the STEP parameter described above.  The 
 227                           "automatic" one-sided derivative method will chose a 
 228                           direction for the finite difference which does not 
 229                           violate any constraints.  The other methods do not 
 230                           perform this check.  The two-sided method is in 
 231                           principle more precise, but requires twice as many 
 232                           function evaluations.  Default: 0. 
 233   
 234          'mpmaxstep' - the maximum change to be made in the parameter 
 235                                   value.  During the fitting process, the parameter 
 236                                   will never be changed by more than this value in 
 237                                   one iteration. 
 238   
 239                                   A value of 0 indicates no maximum.  Default: 0. 
 240   
 241          'tied' - a string expression which "ties" the parameter to other 
 242                          free or fixed parameters.  Any expression involving 
 243                          constants and the parameter array P are permitted. 
 244                          Example: if parameter 2 is always to be twice parameter 
 245                          1 then use the following: parinfo(2).tied = '2 * p(1)'. 
 246                          Since they are totally constrained, tied parameters are 
 247                          considered to be fixed; no errors are computed for them. 
 248                          [ NOTE: the PARNAME can't be used in expressions. ] 
 249   
 250          'mpprint' - if set to 1, then the default iterfunct will print the 
 251                             parameter value.  If set to 0, the parameter value 
 252                             will not be printed.  This tag can be used to 
 253                             selectively print only a few parameter values out of 
 254                             many.  Default: 1 (all parameters printed) 
 255   
 256   
 257   Future modifications to the PARINFO structure, if any, will involve 
 258   adding dictionary tags beginning with the two letters "MP". 
 259   Therefore programmers are urged to avoid using tags starting with 
 260   the same letters; otherwise they are free to include their own 
 261   fields within the PARINFO structure, and they will be ignored. 
 262   
 263   PARINFO Example: 
 264   parinfo = [{'value':0., 'fixed':0, 'limited':[0,0], 'limits':[0.,0.]}]*5 
 265   parinfo[0]['fixed'] = 1 
 266   parinfo[4]['limited'][0] = 1 
 267   parinfo[4]['limits'][0]  = 50. 
 268   values = [5.7, 2.2, 500., 1.5, 2000.] 
 269   for i in range(5): parinfo[i]['value']=values[i] 
 270   
 271   A total of 5 parameters, with starting values of 5.7, 
 272   2.2, 500, 1.5, and 2000 are given.  The first parameter 
 273   is fixed at a value of 5.7, and the last parameter is 
 274   constrained to be above 50. 
 275   
 276   
 277                                                                     EXAMPLE 
 278   
 279     import mpfit 
 280     import numpy.oldnumeric as Numeric 
 281     x = arange(100, float) 
 282     p0 = [5.7, 2.2, 500., 1.5, 2000.] 
 283     y = ( p[0] + p[1]*[x] + p[2]*[x**2] + p[3]*sqrt(x) + 
 284                   p[4]*log(x)) 
 285     fa = {'x':x, 'y':y, 'err':err} 
 286     m = mpfit('myfunct', p0, functkw=fa) 
 287     print 'status = ', m.status 
 288     if (m.status <= 0): print 'error message = ', m.errmsg 
 289     print 'parameters = ', m.params 
 290   
 291     Minimizes sum of squares of MYFUNCT.  MYFUNCT is called with the X, 
 292     Y, and ERR keyword parameters that are given by FUNCTKW.  The 
 293     results can be obtained from the returned object m. 
 294   
 295   
 296                                                          THEORY OF OPERATION 
 297   
 298     There are many specific strategies for function minimization.  One 
 299     very popular technique is to use function gradient information to 
 300     realize the local structure of the function.  Near a local minimum 
 301     the function value can be taylor expanded about x0 as follows: 
 302   
 303            f(x) = f(x0) + f'(x0) . (x-x0) + (1/2) (x-x0) . f''(x0) . (x-x0) 
 304                           -----   ---------------   -------------------------------  (1) 
 305           Order  0th               1st                                     2nd 
 306   
 307     Here f'(x) is the gradient vector of f at x, and f''(x) is the 
 308     Hessian matrix of second derivatives of f at x.  The vector x is 
 309     the set of function parameters, not the measured data vector.  One 
 310     can find the minimum of f, f(xm) using Newton's method, and 
 311     arrives at the following linear equation: 
 312   
 313            f''(x0) . (xm-x0) = - f'(x0)                                                  (2) 
 314   
 315     If an inverse can be found for f''(x0) then one can solve for 
 316     (xm-x0), the step vector from the current position x0 to the new 
 317     projected minimum.  Here the problem has been linearized (ie, the 
 318     gradient information is known to first order).  f''(x0) is 
 319     symmetric n x n matrix, and should be positive definite. 
 320   
 321     The Levenberg - Marquardt technique is a variation on this theme. 
 322     It adds an additional diagonal term to the equation which may aid the 
 323     convergence properties: 
 324   
 325            (f''(x0) + nu I) . (xm-x0) = -f'(x0)                            (2a) 
 326   
 327     where I is the identity matrix.  When nu is large, the overall 
 328     matrix is diagonally dominant, and the iterations follow steepest 
 329     descent.  When nu is small, the iterations are quadratically 
 330     convergent. 
 331   
 332     In principle, if f''(x0) and f'(x0) are known then xm-x0 can be 
 333     determined.  However the Hessian matrix is often difficult or 
 334     impossible to compute.  The gradient f'(x0) may be easier to 
 335     compute, if even by finite difference techniques.  So-called 
 336     quasi-Newton techniques attempt to successively estimate f''(x0) 
 337     by building up gradient information as the iterations proceed. 
 338   
 339     In the least squares problem there are further simplifications 
 340     which assist in solving eqn (2).  The function to be minimized is 
 341     a sum of squares: 
 342   
 343             f = Sum(hi^2)                                                                                 (3) 
 344   
 345     where hi is the ith residual out of m residuals as described 
 346     above.  This can be substituted back into eqn (2) after computing 
 347     the derivatives: 
 348   
 349             f'  = 2 Sum(hi  hi') 
 350             f'' = 2 Sum(hi' hj') + 2 Sum(hi hi'')                                (4) 
 351   
 352     If one assumes that the parameters are already close enough to a 
 353     minimum, then one typically finds that the second term in f'' is 
 354     negligible [or, in any case, is too difficult to compute].  Thus, 
 355     equation (2) can be solved, at least approximately, using only 
 356     gradient information. 
 357   
 358     In matrix notation, the combination of eqns (2) and (4) becomes: 
 359   
 360                  hT' . h' . dx = - hT' . h                                                 (5) 
 361   
 362     Where h is the residual vector (length m), hT is its transpose, h' 
 363     is the Jacobian matrix (dimensions n x m), and dx is (xm-x0).  The 
 364     user function supplies the residual vector h, and in some cases h' 
 365     when it is not found by finite differences (see MPFIT_FDJAC2, 
 366     which finds h and hT').  Even if dx is not the best absolute step 
 367     to take, it does provide a good estimate of the best *direction*, 
 368     so often a line minimization will occur along the dx vector 
 369     direction. 
 370   
 371     The method of solution employed by MINPACK is to form the Q . R 
 372     factorization of h', where Q is an orthogonal matrix such that QT . 
 373     Q = I, and R is upper right triangular.  Using h' = Q . R and the 
 374     ortogonality of Q, eqn (5) becomes 
 375   
 376                  (RT . QT) . (Q . R) . dx = - (RT . QT) . h 
 377                                           RT . R . dx = - RT . QT . h             (6) 
 378                                                    R . dx = - QT . h 
 379   
 380     where the last statement follows because R is upper triangular. 
 381     Here, R, QT and h are known so this is a matter of solving for dx. 
 382     The routine MPFIT_QRFAC provides the QR factorization of h, with 
 383     pivoting, and MPFIT_QRSOLV provides the solution for dx. 
 384   
 385   
 386                                                                   REFERENCES 
 387   
 388     MINPACK-1, Jorge More', available from netlib (www.netlib.org). 
 389     "Optimization Software Guide," Jorge More' and Stephen Wright, 
 390           SIAM, *Frontiers in Applied Mathematics*, Number 14. 
 391     More', Jorge J., "The Levenberg-Marquardt Algorithm: 
 392           Implementation and Theory," in *Numerical Analysis*, ed. Watson, 
 393           G. A., Lecture Notes in Mathematics 630, Springer-Verlag, 1977. 
 394   
 395   
 396                                                     MODIFICATION HISTORY 
 397   
 398     Translated from MINPACK-1 in FORTRAN, Apr-Jul 1998, CM 
 399   Copyright (C) 1997-2002, Craig Markwardt 
 400   This software is provided as is without any warranty whatsoever. 
 401   Permission to use, copy, modify, and distribute modified or 
 402   unmodified copies is granted, provided this copyright and disclaimer 
 403   are included unchanged. 
 404   
 405     Translated from MPFIT (Craig Markwardt's IDL package) to Python, 
 406     August, 2002.  Mark Rivers 
 407     Converted from Numeric to numpy (Sergey Koposov, July 2008) 
 408  """ 
 409   
 410  from numpy import * 
 411  import numpy 
 412  import types 
 413   
 414  #        Original FORTRAN documentation 
 415  #        ********** 
 416  # 
 417  #        subroutine lmdif 
 418  # 
 419  #        the purpose of lmdif is to minimize the sum of the squares of 
 420  #        m nonlinear functions in n variables by a modification of 
 421  #        the levenberg-marquardt algorithm. the user must provide a 
 422  #        subroutine which calculates the functions. the jacobian is 
 423  #        then calculated by a forward-difference approximation. 
 424  # 
 425  #        the subroutine statement is 
 426  # 
 427  #          subroutine lmdif(fcn,m,n,x,fvec,ftol,xtol,gtol,maxfev,epsfcn, 
 428  #                                               diag,mode,factor,nprint,info,nfev,fjac, 
 429  #                                               ldfjac,ipvt,qtf,wa1,wa2,wa3,wa4) 
 430  # 
 431  #        where 
 432  # 
 433  #          fcn is the name of the user-supplied subroutine which 
 434  #                calculates the functions. fcn must be declared 
 435  #                in an external statement in the user calling 
 436  #                program, and should be written as follows. 
 437  # 
 438  #                subroutine fcn(m,n,x,fvec,iflag) 
 439  #                integer m,n,iflag 
 440  #                double precision x(n),fvec(m) 
 441  #                ---------- 
 442  #                calculate the functions at x and 
 443  #                return this vector in fvec. 
 444  #                ---------- 
 445  #                return 
 446  #                end 
 447  # 
 448  #                the value of iflag should not be changed by fcn unless 
 449  #                the user wants to terminate execution of lmdif. 
 450  #                in this case set iflag to a negative integer. 
 451  # 
 452  #          m is a positive integer input variable set to the number 
 453  #                of functions. 
 454  # 
 455  #          n is a positive integer input variable set to the number 
 456  #                of variables. n must not exceed m. 
 457  # 
 458  #          x is an array of length n. on input x must contain 
 459  #                an initial estimate of the solution vector. on output x 
 460  #                contains the final estimate of the solution vector. 
 461  # 
 462  #          fvec is an output array of length m which contains 
 463  #                the functions evaluated at the output x. 
 464  # 
 465  #          ftol is a nonnegative input variable. termination 
 466  #                occurs when both the actual and predicted relative 
 467  #                reductions in the sum of squares are at most ftol. 
 468  #                therefore, ftol measures the relative error desired 
 469  #                in the sum of squares. 
 470  # 
 471  #          xtol is a nonnegative input variable. termination 
 472  #                occurs when the relative error between two consecutive 
 473  #                iterates is at most xtol. therefore, xtol measures the 
 474  #                relative error desired in the approximate solution. 
 475  # 
 476  #          gtol is a nonnegative input variable. termination 
 477  #                occurs when the cosine of the angle between fvec and 
 478  #                any column of the jacobian is at most gtol in absolute 
 479  #                value. therefore, gtol measures the orthogonality 
 480  #                desired between the function vector and the columns 
 481  #                of the jacobian. 
 482  # 
 483  #          maxfev is a positive integer input variable. termination 
 484  #                occurs when the number of calls to fcn is at least 
 485  #                maxfev by the end of an iteration. 
 486  # 
 487  #          epsfcn is an input variable used in determining a suitable 
 488  #                step length for the forward-difference approximation. this 
 489  #                approximation assumes that the relative errors in the 
 490  #                functions are of the order of epsfcn. if epsfcn is less 
 491  #                than the machine precision, it is assumed that the relative 
 492  #                errors in the functions are of the order of the machine 
 493  #                precision. 
 494  # 
 495  #          diag is an array of length n. if mode = 1 (see 
 496  #                below), diag is internally set. if mode = 2, diag 
 497  #                must contain positive entries that serve as 
 498  #                multiplicative scale factors for the variables. 
 499  # 
 500  #          mode is an integer input variable. if mode = 1, the 
 501  #                variables will be scaled internally. if mode = 2, 
 502  #                the scaling is specified by the input diag. other 
 503  #                values of mode are equivalent to mode = 1. 
 504  # 
 505  #          factor is a positive input variable used in determining the 
 506  #                initial step bound. this bound is set to the product of 
 507  #                factor and the euclidean norm of diag*x if nonzero, or else 
 508  #                to factor itself. in most cases factor should lie in the 
 509  #                interval (.1,100.). 100. is a generally recommended value. 
 510  # 
 511  #          nprint is an integer input variable that enables controlled 
 512  #                printing of iterates if it is positive. in this case, 
 513  #                fcn is called with iflag = 0 at the beginning of the first 
 514  #                iteration and every nprint iterations thereafter and 
 515  #                immediately prior to return, with x and fvec available 
 516  #                for printing. if nprint is not positive, no special calls 
 517  #                of fcn with iflag = 0 are made. 
 518  # 
 519  #          info is an integer output variable. if the user has 
 520  #                terminated execution, info is set to the (negative) 
 521  #                value of iflag. see description of fcn. otherwise, 
 522  #                info is set as follows. 
 523  # 
 524  #                info = 0  improper input parameters. 
 525  # 
 526  #                info = 1  both actual and predicted relative reductions 
 527  #                                  in the sum of squares are at most ftol. 
 528  # 
 529  #                info = 2  relative error between two consecutive iterates 
 530  #                                  is at most xtol. 
 531  # 
 532  #                info = 3  conditions for info = 1 and info = 2 both hold. 
 533  # 
 534  #                info = 4  the cosine of the angle between fvec and any 
 535  #                                  column of the jacobian is at most gtol in 
 536  #                                  absolute value. 
 537  # 
 538  #                info = 5  number of calls to fcn has reached or 
 539  #                                  exceeded maxfev. 
 540  # 
 541  #                info = 6  ftol is too small. no further reduction in 
 542  #                                  the sum of squares is possible. 
 543  # 
 544  #                info = 7  xtol is too small. no further improvement in 
 545  #                                  the approximate solution x is possible. 
 546  # 
 547  #                info = 8  gtol is too small. fvec is orthogonal to the 
 548  #                                  columns of the jacobian to machine precision. 
 549  # 
 550  #          nfev is an integer output variable set to the number of 
 551  #                calls to fcn. 
 552  # 
 553  #          fjac is an output m by n array. the upper n by n submatrix 
 554  #                of fjac contains an upper triangular matrix r with 
 555  #                diagonal elements of nonincreasing magnitude such that 
 556  # 
 557  #                               t        t                 t 
 558  #                          p *(jac *jac)*p = r *r, 
 559  # 
 560  #                where p is a permutation matrix and jac is the final 
 561  #                calculated jacobian. column j of p is column ipvt(j) 
 562  #                (see below) of the identity matrix. the lower trapezoidal 
 563  #                part of fjac contains information generated during 
 564  #                the computation of r. 
 565  # 
 566  #          ldfjac is a positive integer input variable not less than m 
 567  #                which specifies the leading dimension of the array fjac. 
 568  # 
 569  #          ipvt is an integer output array of length n. ipvt 
 570  #                defines a permutation matrix p such that jac*p = q*r, 
 571  #                where jac is the final calculated jacobian, q is 
 572  #                orthogonal (not stored), and r is upper triangular 
 573  #                with diagonal elements of nonincreasing magnitude. 
 574  #                column j of p is column ipvt(j) of the identity matrix. 
 575  # 
 576  #          qtf is an output array of length n which contains 
 577  #                the first n elements of the vector (q transpose)*fvec. 
 578  # 
 579  #          wa1, wa2, and wa3 are work arrays of length n. 
 580  # 
 581  #          wa4 is a work array of length m. 
 582  # 
 583  #        subprograms called 
 584  # 
 585  #          user-supplied ...... fcn 
 586  # 
 587  #          minpack-supplied ... dpmpar,enorm,fdjac2,,qrfac 
 588  # 
 589  #          fortran-supplied ... dabs,dmax1,dmin1,dsqrt,mod 
 590  # 
 591  #        argonne national laboratory. minpack project. march 1980. 
 592  #        burton s. garbow, kenneth e. hillstrom, jorge j. more 
 593  # 
 594  #        ********** 
 595   
596 -class mpfit:
597 - def __init__(self, fcn, xall=None, functkw=None, parinfo=None, 598 ftol=1.e-10, xtol=1.e-10, gtol=1.e-10, 599 damp=0., maxiter=200, factor=100., nprint=1, 600 iterfunct='default', iterkw=None, nocovar=0, 601 fastnorm=0, rescale=0, autoderivative=1, quiet=0, 602 diag=None, epsfcn=None, debug=0):
603 """ 604 Inputs: 605 fcn: 606 The function to be minimized. The function should return the weighted 607 deviations between the model and the data, as described above. 608 609 xall: 610 An array of starting values for each of the parameters of the model. 611 The number of parameters should be fewer than the number of measurements. 612 613 This parameter is optional if the parinfo keyword is used (but see 614 parinfo). The parinfo keyword provides a mechanism to fix or constrain 615 individual parameters. 616 617 Keywords: 618 619 autoderivative: 620 If this is set, derivatives of the function will be computed 621 automatically via a finite differencing procedure. If not set, then 622 fcn must provide the (analytical) derivatives. 623 Default: set (=1) 624 NOTE: to supply your own analytical derivatives, 625 explicitly pass autoderivative=0 626 627 fastnorm: 628 Set this keyword to select a faster algorithm to compute sum-of-square 629 values internally. For systems with large numbers of data points, the 630 standard algorithm can become prohibitively slow because it cannot be 631 vectorized well. By setting this keyword, MPFIT will run faster, but 632 it will be more prone to floating point overflows and underflows. Thus, setting 633 this keyword may sacrifice some stability in the fitting process. 634 Default: clear (=0) 635 636 ftol: 637 A nonnegative input variable. Termination occurs when both the actual 638 and predicted relative reductions in the sum of squares are at most 639 ftol (and status is accordingly set to 1 or 3). Therefore, ftol 640 measures the relative error desired in the sum of squares. 641 Default: 1E-10 642 643 functkw: 644 A dictionary which contains the parameters to be passed to the 645 user-supplied function specified by fcn via the standard Python 646 keyword dictionary mechanism. This is the way you can pass additional 647 data to your user-supplied function without using global variables. 648 649 Consider the following example: 650 if functkw = {'xval':[1.,2.,3.], 'yval':[1.,4.,9.], 651 'errval':[1.,1.,1.] } 652 then the user supplied function should be declared like this: 653 def myfunct(p, fjac=None, xval=None, yval=None, errval=None): 654 655 Default: {} No extra parameters are passed to the user-supplied 656 function. 657 658 gtol: 659 A nonnegative input variable. Termination occurs when the cosine of 660 the angle between fvec and any column of the jacobian is at most gtol 661 in absolute value (and status is accordingly set to 4). Therefore, 662 gtol measures the orthogonality desired between the function vector 663 and the columns of the jacobian. 664 Default: 1e-10 665 666 iterkw: 667 The keyword arguments to be passed to iterfunct via the dictionary 668 keyword mechanism. This should be a dictionary and is similar in 669 operation to FUNCTKW. 670 Default: {} No arguments are passed. 671 672 iterfunct: 673 The name of a function to be called upon each NPRINT iteration of the 674 MPFIT routine. It should be declared in the following way: 675 def iterfunct(myfunct, p, iter, fnorm, functkw=None, 676 parinfo=None, quiet=0, dof=None, [iterkw keywords here]) 677 # perform custom iteration update 678 679 iterfunct must accept all three keyword parameters (FUNCTKW, PARINFO 680 and QUIET). 681 682 myfunct: The user-supplied function to be minimized, 683 p: The current set of model parameters 684 iter: The iteration number 685 functkw: The arguments to be passed to myfunct. 686 fnorm: The chi-squared value. 687 quiet: Set when no textual output should be printed. 688 dof: The number of degrees of freedom, normally the number of points 689 less the number of free parameters. 690 See below for documentation of parinfo. 691 692 In implementation, iterfunct can perform updates to the terminal or 693 graphical user interface, to provide feedback while the fit proceeds. 694 If the fit is to be stopped for any reason, then iterfunct should return a 695 a status value between -15 and -1. Otherwise it should return None 696 (e.g. no return statement) or 0. 697 In principle, iterfunct should probably not modify the parameter values, 698 because it may interfere with the algorithm's stability. In practice it 699 is allowed. 700 701 Default: an internal routine is used to print the parameter values. 702 703 Set iterfunct=None if there is no user-defined routine and you don't 704 want the internal default routine be called. 705 706 maxiter: 707 The maximum number of iterations to perform. If the number is exceeded, 708 then the status value is set to 5 and MPFIT returns. 709 Default: 200 iterations 710 711 nocovar: 712 Set this keyword to prevent the calculation of the covariance matrix 713 before returning (see COVAR) 714 Default: clear (=0) The covariance matrix is returned 715 716 nprint: 717 The frequency with which iterfunct is called. A value of 1 indicates 718 that iterfunct is called with every iteration, while 2 indicates every 719 other iteration, etc. Note that several Levenberg-Marquardt attempts 720 can be made in a single iteration. 721 Default value: 1 722 723 parinfo 724 Provides a mechanism for more sophisticated constraints to be placed on 725 parameter values. When parinfo is not passed, then it is assumed that 726 all parameters are free and unconstrained. Values in parinfo are never 727 modified during a call to MPFIT. 728 729 See description above for the structure of PARINFO. 730 731 Default value: None All parameters are free and unconstrained. 732 733 quiet: 734 Set this keyword when no textual output should be printed by MPFIT 735 736 damp: 737 A scalar number, indicating the cut-off value of residuals where 738 "damping" will occur. Residuals with magnitudes greater than this 739 number will be replaced by their hyperbolic tangent. This partially 740 mitigates the so-called large residual problem inherent in 741 least-squares solvers (as for the test problem CURVI, 742 http://www.maxthis.com/curviex.htm). 743 A value of 0 indicates no damping. 744 Default: 0 745 746 Note: DAMP doesn't work with autoderivative=0 747 748 xtol: 749 A nonnegative input variable. Termination occurs when the relative error 750 between two consecutive iterates is at most xtol (and status is 751 accordingly set to 2 or 3). Therefore, xtol measures the relative error 752 desired in the approximate solution. 753 Default: 1E-10 754 755 Outputs: 756 757 Returns an object of type mpfit. The results are attributes of this class, 758 e.g. mpfit.status, mpfit.errmsg, mpfit.params, npfit.niter, mpfit.covar. 759 760 .status 761 An integer status code is returned. All values greater than zero can 762 represent success (however .status == 5 may indicate failure to 763 converge). It can have one of the following values: 764 765 -16 766 A parameter or function value has become infinite or an undefined 767 number. This is usually a consequence of numerical overflow in the 768 user's model function, which must be avoided. 769 770 -15 to -1 771 These are error codes that either MYFUNCT or iterfunct may return to 772 terminate the fitting process. Values from -15 to -1 are reserved 773 for the user functions and will not clash with MPFIT. 774 775 0 Improper input parameters. 776 777 1 Both actual and predicted relative reductions in the sum of squares 778 are at most ftol. 779 780 2 Relative error between two consecutive iterates is at most xtol 781 782 3 Conditions for status = 1 and status = 2 both hold. 783 784 4 The cosine of the angle between fvec and any column of the jacobian 785 is at most gtol in absolute value. 786 787 5 The maximum number of iterations has been reached. 788 789 6 ftol is too small. No further reduction in the sum of squares is 790 possible. 791 792 7 xtol is too small. No further improvement in the approximate solution 793 x is possible. 794 795 8 gtol is too small. fvec is orthogonal to the columns of the jacobian 796 to machine precision. 797 798 .fnorm 799 The value of the summed squared residuals for the returned parameter 800 values. 801 802 .covar 803 The covariance matrix for the set of parameters returned by MPFIT. 804 The matrix is NxN where N is the number of parameters. The square root 805 of the diagonal elements gives the formal 1-sigma statistical errors on 806 the parameters if errors were treated "properly" in fcn. 807 Parameter errors are also returned in .perror. 808 809 To compute the correlation matrix, pcor, use this example: 810 cov = mpfit.covar 811 pcor = cov * 0. 812 for i in range(n): 813 for j in range(n): 814 pcor[i,j] = cov[i,j]/sqrt(cov[i,i]*cov[j,j]) 815 816 If nocovar is set or MPFIT terminated abnormally, then .covar is set to 817 a scalar with value None. 818 819 .errmsg 820 A string error or warning message is returned. 821 822 .nfev 823 The number of calls to MYFUNCT performed. 824 825 .niter 826 The number of iterations completed. 827 828 .perror 829 The formal 1-sigma errors in each parameter, computed from the 830 covariance matrix. If a parameter is held fixed, or if it touches a 831 boundary, then the error is reported as zero. 832 833 If the fit is unweighted (i.e. no errors were given, or the weights 834 were uniformly set to unity), then .perror will probably not represent 835 the true parameter uncertainties. 836 837 *If* you can assume that the true reduced chi-squared value is unity -- 838 meaning that the fit is implicitly assumed to be of good quality -- 839 then the estimated parameter uncertainties can be computed by scaling 840 .perror by the measured chi-squared value. 841 842 dof = len(x) - len(mpfit.params) # deg of freedom 843 # scaled uncertainties 844 pcerror = mpfit.perror * sqrt(mpfit.fnorm / dof) 845 846 """ 847 848 # See #4749 849 if functkw is None: 850 functkw = {} 851 if iterkw is None: 852 iterkw = {} 853 854 self.niter = 0 855 self.params = None 856 self.covar = None 857 self.perror = None 858 self.status = 0 # Invalid input flag set while we check inputs 859 self.debug = debug 860 self.errmsg = '' 861 self.fastnorm = fastnorm 862 self.nfev = 0 863 self.damp = damp 864 self.machar = machar(double=1) 865 machep = self.machar.machep 866 self.dof=0 867 868 if (fcn==None): 869 self.errmsg = "Usage: parms = mpfit('myfunt', ... )" 870 return 871 872 if (iterfunct == 'default'): iterfunct = self.defiter 873 874 ## Parameter damping doesn't work when user is providing their own 875 ## gradients. 876 if (self.damp != 0) and (autoderivative == 0): 877 self.errmsg = 'ERROR: keywords DAMP and AUTODERIVATIVE are mutually exclusive' 878 return 879 880 ## Parameters can either be stored in parinfo, or x. x takes precedence if it exists 881 if (xall == None) and (parinfo == None): 882 self.errmsg = 'ERROR: must pass parameters in P or PARINFO' 883 return 884 885 ## Be sure that PARINFO is of the right type 886 if (parinfo != None): 887 if (type(parinfo) != types.ListType): 888 self.errmsg = 'ERROR: PARINFO must be a list of dictionaries.' 889 return 890 else: 891 if (type(parinfo[0]) != types.DictionaryType): 892 self.errmsg = 'ERROR: PARINFO must be a list of dictionaries.' 893 return 894 if ((xall != None) and (len(xall) != len(parinfo))): 895 self.errmsg = 'ERROR: number of elements in PARINFO and P must agree' 896 return 897 898 ## If the parameters were not specified at the command line, then 899 ## extract them from PARINFO 900 if (xall == None): 901 xall = self.parinfo(parinfo, 'value') 902 if (xall == None): 903 self.errmsg = 'ERROR: either P or PARINFO(*)["value"] must be supplied.' 904 return 905 906 ## Make sure parameters are Numeric arrays of type Float 907 xall = asarray(xall, float) 908 909 npar = len(xall) 910 self.fnorm = -1. 911 fnorm1 = -1. 912 913 ## TIED parameters? 914 ptied = self.parinfo(parinfo, 'tied', default='', n=npar) 915 self.qanytied = 0 916 for i in range(npar): 917 ptied[i] = ptied[i].strip() 918 if (ptied[i] != ''): self.qanytied = 1 919 self.ptied = ptied 920 921 ## FIXED parameters ? 922 pfixed = self.parinfo(parinfo, 'fixed', default=0, n=npar) 923 pfixed = (pfixed == 1) 924 for i in range(npar): 925 pfixed[i] = pfixed[i] or (ptied[i] != '') ## Tied parameters are also effectively fixed 926 927 ## Finite differencing step, absolute and relative, and sidedness of deriv. 928 step = self.parinfo(parinfo, 'step', default=0., n=npar) 929 dstep = self.parinfo(parinfo, 'relstep', default=0., n=npar) 930 dside = self.parinfo(parinfo, 'mpside', default=0, n=npar) 931 932 ## Maximum and minimum steps allowed to be taken in one iteration 933 maxstep = self.parinfo(parinfo, 'mpmaxstep', default=0., n=npar) 934 minstep = self.parinfo(parinfo, 'mpminstep', default=0., n=npar) 935 qmin = minstep * 0 ## Remove minstep for now!! 936 qmax = maxstep != 0 937 wh = (nonzero(((qmin!=0.) & (qmax!=0.)) & (maxstep < minstep)))[0] 938 if (len(wh) > 0): 939 self.errmsg = 'ERROR: MPMINSTEP is greater than MPMAXSTEP' 940 return 941 wh = (nonzero((qmin!=0.) | (qmax!=0.)))[0] 942 qminmax = len(wh > 0) 943 944 ## Finish up the free parameters 945 ifree = (nonzero(pfixed != 1))[0] 946 nfree = len(ifree) 947 if nfree == 0: 948 self.errmsg = 'ERROR: no free parameters' 949 return 950 951 ## Compose only VARYING parameters 952 self.params = xall ## self.params is the set of parameters to be returned 953 x = take(self.params, ifree) ## x is the set of free parameters 954 955 ## LIMITED parameters ? 956 limited = self.parinfo(parinfo, 'limited', default=[0,0], n=npar) 957 limits = self.parinfo(parinfo, 'limits', default=[0.,0.], n=npar) 958 if (limited != None) and (limits != None): 959 ## Error checking on limits in parinfo 960 wh = (nonzero((limited[:,0] & (xall < limits[:,0])) | 961 (limited[:,1] & (xall > limits[:,1]))))[0] 962 if (len(wh) > 0): 963 self.errmsg = 'ERROR: parameters are not within PARINFO limits' 964 return 965 wh = (nonzero((limited[:,0] & limited[:,1]) & 966 (limits[:,0] >= limits[:,1]) & 967 (pfixed == 0)))[0] 968 if (len(wh) > 0): 969 self.errmsg = 'ERROR: PARINFO parameter limits are not consistent' 970 return 971 972 ## Transfer structure values to local variables 973 qulim = take(limited[:,1], ifree) 974 ulim = take(limits [:,1], ifree) 975 qllim = take(limited[:,0], ifree) 976 llim = take(limits [:,0], ifree) 977 978 wh = (nonzero((qulim!=0.) | (qllim!=0.)))[0] 979 if (len(wh) > 0): qanylim = 1 980 else: qanylim = 0 981 else: 982 ## Fill in local variables with dummy values 983 qulim = zeros(nfree) 984 ulim = x * 0. 985 qllim = qulim 986 llim = x * 0. 987 qanylim = 0 988 989 n = len(x) 990 ## Check input parameters for errors 991 if ((n < 0) or (ftol <= 0) or (xtol <= 0) or (gtol <= 0) 992 or (maxiter < 0) or (factor <= 0)): 993 self.errmsg = 'ERROR: input keywords are inconsistent' 994 return 995 996 if (rescale != 0): 997 self.errmsg = 'ERROR: DIAG parameter scales are inconsistent' 998 if (len(diag) < n): return 999 wh = (nonzero(diag <= 0))[0] 1000 if (len(wh) > 0): return 1001 self.errmsg = '' 1002 1003 # Make sure x is a Numeric array of type Float 1004 x = asarray(x, float) 1005 1006 [self.status, fvec] = self.call(fcn, self.params, functkw) 1007 if (self.status < 0): 1008 self.errmsg = 'ERROR: first call to "'+str(fcn)+'" failed' 1009 return 1010 1011 m = len(fvec) 1012 if (m < n): 1013 self.errmsg = 'ERROR: number of parameters must not exceed data' 1014 return 1015 self.dof = m-nfree 1016 self.fnorm = self.enorm(fvec) 1017 1018 ## Initialize Levelberg-Marquardt parameter and iteration counter 1019 1020 par = 0. 1021 self.niter = 1 1022 qtf = x * 0. 1023 self.status = 0 1024 1025 ## Beginning of the outer loop 1026 1027 while(1): 1028 1029 ## If requested, call fcn to enable printing of iterates 1030 put(self.params, ifree, x) 1031 if (self.qanytied): self.params = self.tie(self.params, ptied) 1032 1033 if (nprint > 0) and (iterfunct != None): 1034 if (((self.niter-1) % nprint) == 0): 1035 mperr = 0 1036 xnew0 = self.params.copy() 1037 1038 dof = numpy.max([len(fvec) - len(x), 0]) 1039 status = iterfunct(fcn, self.params, self.niter, self.fnorm**2, 1040 functkw=functkw, parinfo=parinfo, quiet=quiet, 1041 dof=dof, **iterkw) 1042 if (status != None): self.status = status 1043 1044 ## Check for user termination 1045 if (self.status < 0): 1046 self.errmsg = 'WARNING: premature termination by ' + str(iterfunct) 1047 return 1048 1049 ## If parameters were changed (grrr..) then re-tie 1050 if (numpy.max(abs(xnew0-self.params)) > 0): 1051 if (self.qanytied): self.params = self.tie(self.params, ptied) 1052 x = take(self.params, ifree) 1053 1054 1055 ## Calculate the jacobian matrix 1056 self.status = 2 1057 catch_msg = 'calling MPFIT_FDJAC2' 1058 fjac = self.fdjac2(fcn, x, fvec, step, qulim, ulim, dside, 1059 epsfcn=epsfcn, 1060 autoderivative=autoderivative, dstep=dstep, 1061 functkw=functkw, ifree=ifree, xall=self.params) 1062 if (fjac == None): 1063 self.errmsg = 'WARNING: premature termination by FDJAC2' 1064 return 1065 1066 ## Determine if any of the parameters are pegged at the limits 1067 if (qanylim): 1068 catch_msg = 'zeroing derivatives of pegged parameters' 1069 whlpeg = (nonzero(qllim & (x == llim)))[0] 1070 nlpeg = len(whlpeg) 1071 whupeg = (nonzero(qulim & (x == ulim)))[0] 1072 nupeg = len(whupeg) 1073 ## See if any "pegged" values should keep their derivatives 1074 if (nlpeg > 0): 1075 ## Total derivative of sum wrt lower pegged parameters 1076 for i in range(nlpeg): 1077 sum0 = sum(fvec * fjac[:,whlpeg[i]]) 1078 if (sum0 > 0): fjac[:,whlpeg[i]] = 0 1079 if (nupeg > 0): 1080 ## Total derivative of sum wrt upper pegged parameters 1081 for i in range(nupeg): 1082 sum0 = sum(fvec * fjac[:,whupeg[i]]) 1083 if (sum0 < 0): fjac[:,whupeg[i]] = 0 1084 1085 ## Compute the QR factorization of the jacobian 1086 [fjac, ipvt, wa1, wa2] = self.qrfac(fjac, pivot=1) 1087 1088 ## On the first iteration if "diag" is unspecified, scale 1089 ## according to the norms of the columns of the initial jacobian 1090 catch_msg = 'rescaling diagonal elements' 1091 if (self.niter == 1): 1092 if ((rescale==0) or (len(diag) < n)): 1093 diag = wa2.copy() 1094 wh = (nonzero(diag == 0))[0] 1095 put(diag, wh, 1.) 1096 1097 ## On the first iteration, calculate the norm of the scaled x 1098 ## and initialize the step bound delta 1099 wa3 = diag * x 1100 xnorm = self.enorm(wa3) 1101 delta = factor*xnorm 1102 if (delta == 0.): delta = factor 1103 1104 ## Form (q transpose)*fvec and store the first n components in qtf 1105 catch_msg = 'forming (q transpose)*fvec' 1106 wa4 = fvec.copy() 1107 for j in range(n): 1108 lj = ipvt[j] 1109 temp3 = fjac[j,lj] 1110 if (temp3 != 0): 1111 fj = fjac[j:,lj] 1112 wj = wa4[j:] 1113 ## *** optimization wa4(j:*) 1114 wa4[j:] = wj - fj * sum(fj*wj) / temp3 1115 fjac[j,lj] = wa1[j] 1116 qtf[j] = wa4[j] 1117 ## From this point on, only the square matrix, consisting of the 1118 ## triangle of R, is needed. 1119 fjac = fjac[0:n, 0:n] 1120 fjac.shape = [n, n] 1121 temp = fjac.copy() 1122 for i in range(n): 1123 temp[:,i] = fjac[:, ipvt[i]] 1124 fjac = temp.copy() 1125 1126 ## Check for overflow. This should be a cheap test here since FJAC 1127 ## has been reduced to a (small) square matrix, and the test is 1128 ## O(N^2). 1129 #wh = where(finite(fjac) EQ 0, ct) 1130 #if ct GT 0 then goto, FAIL_OVERFLOW 1131 1132 ## Compute the norm of the scaled gradient 1133 catch_msg = 'computing the scaled gradient' 1134 gnorm = 0. 1135 if (self.fnorm != 0): 1136 for j in range(n): 1137 l = ipvt[j] 1138 if (wa2[l] != 0): 1139 sum0 = sum(fjac[0:j+1,j]*qtf[0:j+1])/self.fnorm 1140 gnorm = numpy.max([gnorm,abs(sum0/wa2[l])]) 1141 1142 ## Test for convergence of the gradient norm 1143 if (gnorm <= gtol): 1144 self.status = 4 1145 break 1146 if maxiter == 0: 1147 self.status = 5 1148 break 1149 1150 ## Rescale if necessary 1151 if (rescale == 0): 1152 diag = choose(diag>wa2, (wa2, diag)) 1153 1154 ## Beginning of the inner loop 1155 while(1): 1156 1157 ## Determine the levenberg-marquardt parameter 1158 catch_msg = 'calculating LM parameter (MPFIT_)' 1159 [fjac, par, wa1, wa2] = self.lmpar(fjac, ipvt, diag, qtf, 1160 delta, wa1, wa2, par=par) 1161 ## Store the direction p and x+p. Calculate the norm of p 1162 wa1 = -wa1 1163 1164 if (qanylim == 0) and (qminmax == 0): 1165 ## No parameter limits, so just move to new position WA2 1166 alpha = 1. 1167 wa2 = x + wa1 1168 1169 else: 1170 1171 ## Respect the limits. If a step were to go out of bounds, then 1172 ## we should take a step in the same direction but shorter distance. 1173 ## The step should take us right to the limit in that case. 1174 alpha = 1. 1175 1176 if (qanylim): 1177 ## Do not allow any steps out of bounds 1178 catch_msg = 'checking for a step out of bounds' 1179 if (nlpeg > 0): 1180 put(wa1, whlpeg, clip( 1181 take(wa1, whlpeg), 0., numpy.max(wa1))) 1182 if (nupeg > 0): 1183 put(wa1, whupeg, clip( 1184 take(wa1, whupeg), numpy.min(wa1), 0.)) 1185 1186 dwa1 = abs(wa1) > machep 1187 whl = (nonzero(((dwa1!=0.) & qllim) & ((x + wa1) < llim)))[0] 1188 if (len(whl) > 0): 1189 t = ((take(llim, whl) - take(x, whl)) / 1190 take(wa1, whl)) 1191 alpha = numpy.min([alpha, numpy.min(t)]) 1192 whu = (nonzero(((dwa1!=0.) & qulim) & ((x + wa1) > ulim)))[0] 1193 if (len(whu) > 0): 1194 t = ((take(ulim, whu) - take(x, whu)) / 1195 take(wa1, whu)) 1196 alpha = numpy.min([alpha, numpy.min(t)]) 1197 1198 ## Obey any max step values. 1199 if (qminmax): 1200 nwa1 = wa1 * alpha 1201 whmax = (nonzero((qmax != 0.) & (maxstep > 0)))[0] 1202 if (len(whmax) > 0): 1203 mrat = numpy.max(numpy.abs(nwa1[whmax]) / 1204 numpy.abs(maxstep[ifree[whmax]])) 1205 if (mrat > 1): alpha = alpha / mrat 1206 1207 ## Scale the resulting vector 1208 wa1 = wa1 * alpha 1209 wa2 = x + wa1 1210 1211 ## Adjust the final output values. If the step put us exactly 1212 ## on a boundary, make sure it is exact. 1213 sgnu = (ulim >= 0) * 2. - 1. 1214 sgnl = (llim >= 0) * 2. - 1. 1215 ## Handles case of 1216 ## ... nonzero *LIM ... ...zero * LIM 1217 ulim1 = ulim * (1 - sgnu * machep) - (ulim == 0) * machep 1218 llim1 = llim * (1 + sgnl * machep) + (llim == 0) * machep 1219 wh = (nonzero((qulim!=0) & (wa2 >= ulim1)))[0] 1220 if (len(wh) > 0): put(wa2, wh, take(ulim, wh)) 1221 wh = (nonzero((qllim!=0.) & (wa2 <= llim1)))[0] 1222 if (len(wh) > 0): put(wa2, wh, take(llim, wh)) 1223 # endelse 1224 wa3 = diag * wa1 1225 pnorm = self.enorm(wa3) 1226 1227 ## On the first iteration, adjust the initial step bound 1228 if (self.niter == 1): delta = numpy.min([delta,pnorm]) 1229 1230 put(self.params, ifree, wa2) 1231 1232 ## Evaluate the function at x+p and calculate its norm 1233 mperr = 0 1234 catch_msg = 'calling '+str(fcn) 1235 [self.status, wa4] = self.call(fcn, self.params, functkw) 1236 if (self.status < 0): 1237 self.errmsg = 'WARNING: premature termination by "'+fcn+'"' 1238 return 1239 fnorm1 = self.enorm(wa4) 1240 1241 ## Compute the scaled actual reduction 1242 catch_msg = 'computing convergence criteria' 1243 actred = -1. 1244 if ((0.1 * fnorm1) < self.fnorm): actred = - (fnorm1/self.fnorm)**2 + 1. 1245 1246 ## Compute the scaled predicted reduction and the scaled directional 1247 ## derivative 1248 for j in range(n): 1249 wa3[j] = 0 1250 wa3[0:j+1] = wa3[0:j+1] + fjac[0:j+1,j]*wa1[ipvt[j]] 1251 1252 ## Remember, alpha is the fraction of the full LM step actually 1253 ## taken 1254 temp1 = self.enorm(alpha*wa3)/self.fnorm 1255 temp2 = (sqrt(alpha*par)*pnorm)/self.fnorm 1256 prered = temp1*temp1 + (temp2*temp2)/0.5 1257 dirder = -(temp1*temp1 + temp2*temp2) 1258 1259 ## Compute the ratio of the actual to the predicted reduction. 1260 ratio = 0. 1261 if (prered != 0): ratio = actred/prered 1262 1263 ## Update the step bound 1264 if (ratio <= 0.25): 1265 if (actred >= 0): temp = .5 1266 else: temp = .5*dirder/(dirder + .5*actred) 1267 if ((0.1*fnorm1) >= self.fnorm) or (temp < 0.1): temp = 0.1 1268 delta = temp*numpy.min([delta,pnorm/0.1]) 1269 par = par/temp 1270 else: 1271 if (par == 0) or (ratio >= 0.75): 1272 delta = pnorm/.5 1273 par = .5*par 1274 1275 ## Test for successful iteration 1276 if (ratio >= 0.0001): 1277 ## Successful iteration. Update x, fvec, and their norms 1278 x = wa2 1279 wa2 = diag * x 1280 fvec = wa4 1281 xnorm = self.enorm(wa2) 1282 self.fnorm = fnorm1 1283 self.niter = self.niter + 1 1284 1285 ## Tests for convergence 1286 if ((abs(actred) <= ftol) and (prered <= ftol) 1287 and (0.5 * ratio <= 1)): self.status = 1 1288 if delta <= xtol*xnorm: self.status = 2 1289 if ((abs(actred) <= ftol) and (prered <= ftol) 1290 and (0.5 * ratio <= 1) and (self.status == 2)): self.status = 3 1291 if (self.status != 0): break 1292 1293 ## Tests for termination and stringent tolerances 1294 if (self.niter >= maxiter): self.status = 5 1295 if ((abs(actred) <= machep) and (prered <= machep) 1296 and (0.5*ratio <= 1)): self.status = 6 1297 if delta <= machep*xnorm: self.status = 7 1298 if gnorm <= machep: self.status = 8 1299 if (self.status != 0): break 1300 1301 ## End of inner loop. Repeat if iteration unsuccessful 1302 if (ratio >= 0.0001): break 1303 1304 ## Check for over/underflow 1305 if ~numpy.all(numpy.isfinite(wa1) & numpy.isfinite(wa2) & 1306 numpy.isfinite(x)) or ~numpy.isfinite(ratio): 1307 errmsg = ('''ERROR: parameter or function value(s) have become 1308 'infinite; check model function for over- 'and underflow''') 1309 self.status = -16 1310 break 1311 ##wh = where(finite(wa1) EQ 0 OR finite(wa2) EQ 0 OR finite(x) EQ 0, ct) 1312 ##if ct GT 0 OR finite(ratio) EQ 0 then begin 1313 1314 if (self.status != 0): break; 1315 ## End of outer loop. 1316 1317 catch_msg = 'in the termination phase' 1318 ## Termination, either normal or user imposed. 1319 if (len(self.params) == 0): 1320 return 1321 #if xnew==None: # failed patch WR 1322 # return 1323 if (nfree == 0): self.params = xall.copy() 1324 else: put(self.params, ifree, x) 1325 if (nprint > 0) and (self.status > 0): 1326 catch_msg = 'calling ' + str(fcn) 1327 [status, fvec] = self.call(fcn, self.params, functkw) 1328 catch_msg = 'in the termination phase' 1329 self.fnorm = self.enorm(fvec) 1330 1331 if ((self.fnorm != None) and (fnorm1 != None)): 1332 self.fnorm = numpy.max([self.fnorm, fnorm1]) 1333 self.fnorm = self.fnorm**2. 1334 1335 self.covar = None 1336 self.perror = None 1337 ## (very carefully) set the covariance matrix COVAR 1338 if ((self.status > 0) and (nocovar==0) and (n != None) 1339 and (fjac != None) and (ipvt != None)): 1340 sz = shape(fjac) 1341 if ((n > 0) and (sz[0] >= n) and (sz[1] >= n) 1342 and (len(ipvt) >= n)): 1343 1344 catch_msg = 'computing the covariance matrix' 1345 cv = self.calc_covar(fjac[0:n,0:n], ipvt[0:n]) 1346 cv.shape = [n, n] 1347 nn = len(xall) 1348 1349 ## Fill in actual covariance matrix, accounting for fixed 1350 ## parameters. 1351 self.covar = zeros([nn, nn], dtype=float) 1352 for i in range(n): 1353 self.covar[ifree,ifree[i]] = cv[:,i] 1354 1355 ## Compute errors in parameters 1356 catch_msg = 'computing parameter errors' 1357 self.perror = zeros(nn, dtype=float) 1358 d = diagonal(self.covar) 1359 wh = (nonzero(d >= 0))[0] 1360 if len(wh) > 0: 1361 put(self.perror, wh, sqrt(take(d, wh))) 1362 return
1363 1364
1365 - def __str__(self):
1366 return {'params': self.params, 1367 'niter': self.niter, 1368 'params': self.params, 1369 'covar': self.covar, 1370 'perror': self.perror, 1371 'status': self.status, 1372 'debug': self.debug, 1373 'errmsg': self.errmsg, 1374 'fastnorm': self.fastnorm, 1375 'nfev': self.nfev, 1376 'damp': self.damp 1377 #,'machar':self.machar 1378 }.__str__()
1379 1380 ## Default procedure to be called every iteration. It simply prints 1381 ## the parameter values.
1382 - def defiter(self, fcn, x, iter, fnorm=None, functkw=None, 1383 quiet=0, iterstop=None, parinfo=None, 1384 format=None, pformat='%.10g', dof=1):
1385 1386 if (self.debug): print 'Entering defiter...' 1387 if (quiet): return 1388 if (fnorm == None): 1389 [status, fvec] = self.call(fcn, x, functkw) 1390 fnorm = self.enorm(fvec)**2 1391 1392 ## Determine which parameters to print 1393 nprint = len(x) 1394 print "Iter ", ('%6i' % iter)," CHI-SQUARE = ",('%.10g' % fnorm)," DOF = ", ('%i' % dof) 1395 for i in range(nprint): 1396 if (parinfo != None) and (parinfo[i].has_key('parname')): 1397 p = ' ' + parinfo[i]['parname'] + ' = ' 1398 else: 1399 p = ' P' + str(i) + ' = ' 1400 if (parinfo != None) and (parinfo[i].has_key('mpprint')): 1401 iprint = parinfo[i]['mpprint'] 1402 else: 1403 iprint = 1 1404 if (iprint): 1405 print p + (pformat % x[i]) + ' ' 1406 return(0)
1407 1408 ## DO_ITERSTOP: 1409 ## if keyword_set(iterstop) then begin 1410 ## k = get_kbrd(0) 1411 ## if k EQ string(byte(7)) then begin 1412 ## message, 'WARNING: minimization not complete', /info 1413 ## print, 'Do you want to terminate this procedure? (y/n)', $ 1414 ## format='(A,$)' 1415 ## k = '' 1416 ## read, k 1417 ## if strupcase(strmid(k,0,1)) EQ 'Y' then begin 1418 ## message, 'WARNING: Procedure is terminating.', /info 1419 ## mperr = -1 1420 ## endif 1421 ## endif 1422 ## endif 1423 1424 1425 ## Procedure to parse the parameter values in PARINFO, which is a list of dictionaries
1426 - def parinfo(self, parinfo=None, key='a', default=None, n=0):
1427 if (self.debug): print 'Entering parinfo...' 1428 if (n == 0) and (parinfo != None): n = len(parinfo) 1429 if (n == 0): 1430 values = default 1431 return(values) 1432 values = [] 1433 for i in range(n): 1434 if ((parinfo != None) and (parinfo[i].has_key(key))): 1435 values.append(parinfo[i][key]) 1436 else: 1437 values.append(default) 1438 1439 # Convert to numeric arrays if possible 1440 test = default 1441 if (type(default) == types.ListType): test=default[0] 1442 if (isinstance(test, types.IntType)): 1443 values = asarray(values, int) 1444 elif (isinstance(test, types.FloatType)): 1445 values = asarray(values, float) 1446 return(values)
1447 1448 ## Call user function or procedure, with _EXTRA or not, with 1449 ## derivatives or not.
1450 - def call(self, fcn, x, functkw, fjac=None):
1451 if (self.debug): print 'Entering call...' 1452 if (self.qanytied): x = self.tie(x, self.ptied) 1453 self.nfev = self.nfev + 1 1454 if (fjac == None): 1455 [status, f] = fcn(x, fjac=fjac, **functkw) 1456 if (self.damp > 0): 1457 ## Apply the damping if requested. This replaces the residuals 1458 ## with their hyperbolic tangent. Thus residuals larger than 1459 ## DAMP are essentially clipped. 1460 f = tanh(f/self.damp) 1461 return([status, f]) 1462 else: 1463 return(fcn(x, fjac=fjac, **functkw))
1464 1465
1466 - def enorm(self, vec):
1467 1468 if (self.debug): print 'Entering enorm...' 1469 ## NOTE: it turns out that, for systems that have a lot of data 1470 ## points, this routine is a big computing bottleneck. The extended 1471 ## computations that need to be done cannot be effectively 1472 ## vectorized. The introduction of the FASTNORM configuration 1473 ## parameter allows the user to select a faster routine, which is 1474 ## based on TOTAL() alone. 1475 1476 # Very simple-minded sum-of-squares 1477 if (self.fastnorm): 1478 ans = sqrt(sum(vec*vec)) 1479 else: 1480 agiant = self.machar.rgiant / len(vec) 1481 adwarf = self.machar.rdwarf * len(vec) 1482 1483 ## This is hopefully a compromise between speed and robustness. 1484 ## Need to do this because of the possibility of over- or underflow. 1485 mx = numpy.max(vec) 1486 mn = numpy.min(vec) 1487 mx = max([abs(mx), abs(mn)]) 1488 if mx == 0: return(vec[0]*0.) 1489 if mx > agiant or mx < adwarf: 1490 ans = mx * sqrt(sum((vec/mx)*(vec/mx))) 1491 else: 1492 ans = sqrt(sum(vec*vec)) 1493 1494 return(ans)
1495 1496
1497 - def fdjac2(self, fcn, x, fvec, step=None, ulimited=None, ulimit=None, dside=None, 1498 epsfcn=None, autoderivative=1, 1499 functkw=None, xall=None, ifree=None, dstep=None):
1500 1501 if (self.debug): print 'Entering fdjac2...' 1502 machep = self.machar.machep 1503 if epsfcn == None: epsfcn = machep 1504 if xall == None: xall = x 1505 if ifree == None: ifree = arange(len(xall)) 1506 if step == None: step = x * 0. 1507 nall = len(xall) 1508 1509 eps = sqrt(numpy.max([epsfcn, machep])) 1510 m = len(fvec) 1511 n = len(x) 1512 1513 ## Compute analytical derivative if requested 1514 if (autoderivative == 0): 1515 mperr = 0 1516 fjac = zeros(nall, dtype=float) 1517 Put(fjac, ifree, 1.0) ## Specify which parameters need derivatives 1518 [status, fp] = self.call(fcn, xall, functkw, fjac=fjac) 1519 1520 if len(fjac) != m*nall: 1521 print 'ERROR: Derivative matrix was not computed properly.' 1522 return(None) 1523 1524 ## This definition is consistent with CURVEFIT 1525 ## Sign error found (thanks Jesus Fernandez <fernande@irm.chu-caen.fr>) 1526 fjac.shape = [m,nall] 1527 fjac = -fjac 1528 1529 ## Select only the free parameters 1530 if len(ifree) < nall: 1531 fjac = fjac[:,ifree] 1532 fjac.shape = [m, n] 1533 return(fjac) 1534 1535 fjac = zeros([m, n], dtype=float) 1536 1537 h = eps * abs(x) 1538 1539 ## if STEP is given, use that 1540 ## STEP includes the fixed parameters 1541 if step != None: 1542 stepi = take(step, ifree) 1543 wh = (nonzero(stepi > 0))[0] 1544 if (len(wh) > 0): put(h, wh, take(stepi, wh)) 1545 1546 ## if relative step is given, use that 1547 ## DSTEP includes the fixed parameters 1548 if (len(dstep) > 0): 1549 dstepi = take(dstep, ifree) 1550 wh = (nonzero(dstepi > 0))[0] 1551 if len(wh) > 0: put(h, wh, abs(take(dstepi,wh)*take(x,wh))) 1552 1553 ## In case any of the step values are zero 1554 wh = (nonzero(h == 0))[0] 1555 if len(wh) > 0: put(h, wh, eps) 1556 1557 ## Reverse the sign of the step if we are up against the parameter 1558 ## limit, or if the user requested it. 1559 ## DSIDE includes the fixed parameters (ULIMITED/ULIMIT have only 1560 ## varying ones) 1561 mask = dside[ifree] == -1 1562 if len(ulimited) > 0 and len(ulimit) > 0: 1563 mask = logical_or((mask!=0), logical_and((ulimited!=0),(x > ulimit-h))) 1564 wh = (nonzero(mask))[0] 1565 if len(wh) > 0: put(h, wh, -take(h, wh)) 1566 ## Loop through parameters, computing the derivative for each 1567 for j in range(n): 1568 xp = xall.copy() 1569 xp[ifree[j]] = xp[ifree[j]] + h[j] 1570 [status, fp] = self.call(fcn, xp, functkw) 1571 if (status < 0): return(None) 1572 1573 if abs(dside[ifree[j]]) <= 1: 1574 ## COMPUTE THE ONE-SIDED DERIVATIVE 1575 ## Note optimization fjac(0:*,j) 1576 fjac[0:,j] = (fp-fvec)/h[j] 1577 1578 else: 1579 ## COMPUTE THE TWO-SIDED DERIVATIVE 1580 xp[ifree[j]] = xall[ifree[j]] - h[j] 1581 1582 mperr = 0 1583 [status, fm] = self.call(fcn, xp, functkw) 1584 if (status < 0): return(None) 1585 1586 ## Note optimization fjac(0:*,j) 1587 fjac[0:,j] = (fp-fm)/(2*h[j]) 1588 return(fjac)
1589 1590 1591 1592 # Original FORTRAN documentation 1593 # ********** 1594 # 1595 # subroutine qrfac 1596 # 1597 # this subroutine uses householder transformations with column 1598 # pivoting (optional) to compute a qr factorization of the 1599 # m by n matrix a. that is, qrfac determines an orthogonal 1600 # matrix q, a permutation matrix p, and an upper trapezoidal 1601 # matrix r with diagonal elements of nonincreasing magnitude, 1602 # such that a*p = q*r. the householder transformation for 1603 # column k, k = 1,2,...,min(m,n), is of the form 1604 # 1605 # t 1606 # i - (1/u(k))*u*u 1607 # 1608 # where u has zeros in the first k-1 positions. the form of 1609 # this transformation and the method of pivoting first 1610 # appeared in the corresponding linpack subroutine. 1611 # 1612 # the subroutine statement is 1613 # 1614 # subroutine qrfac(m,n,a,lda,pivot,ipvt,lipvt,rdiag,acnorm,wa) 1615 # 1616 # where 1617 # 1618 # m is a positive integer input variable set to the number 1619 # of rows of a. 1620 # 1621 # n is a positive integer input variable set to the number 1622 # of columns of a. 1623 # 1624 # a is an m by n array. on input a contains the matrix for 1625 # which the qr factorization is to be computed. on output 1626 # the strict upper trapezoidal part of a contains the strict 1627 # upper trapezoidal part of r, and the lower trapezoidal 1628 # part of a contains a factored form of q (the non-trivial 1629 # elements of the u vectors described above). 1630 # 1631 # lda is a positive integer input variable not less than m 1632 # which specifies the leading dimension of the array a. 1633 # 1634 # pivot is a logical input variable. if pivot is set true, 1635 # then column pivoting is enforced. if pivot is set false, 1636 # then no column pivoting is done. 1637 # 1638 # ipvt is an integer output array of length lipvt. ipvt 1639 # defines the permutation matrix p such that a*p = q*r. 1640 # column j of p is column ipvt(j) of the identity matrix. 1641 # if pivot is false, ipvt is not referenced. 1642 # 1643 # lipvt is a positive integer input variable. if pivot is false, 1644 # then lipvt may be as small as 1. if pivot is true, then 1645 # lipvt must be at least n. 1646 # 1647 # rdiag is an output array of length n which contains the 1648 # diagonal elements of r. 1649 # 1650 # acnorm is an output array of length n which contains the 1651 # norms of the corresponding columns of the input matrix a. 1652 # if this information is not needed, then acnorm can coincide 1653 # with rdiag. 1654 # 1655 # wa is a work array of length n. if pivot is false, then wa 1656 # can coincide with rdiag. 1657 # 1658 # subprograms called 1659 # 1660 # minpack-supplied ... dpmpar,enorm 1661 # 1662 # fortran-supplied ... dmax1,dsqrt,min0 1663 # 1664 # argonne national laboratory. minpack project. march 1980. 1665 # burton s. garbow, kenneth e. hillstrom, jorge j. more 1666 # 1667 # ********** 1668 # 1669 # PIVOTING / PERMUTING: 1670 # 1671 # Upon return, A(*,*) is in standard parameter order, A(*,IPVT) is in 1672 # permuted order. 1673 # 1674 # RDIAG is in permuted order. 1675 # ACNORM is in standard parameter order. 1676 # 1677 # 1678 # NOTE: in IDL the factors appear slightly differently than described 1679 # above. The matrix A is still m x n where m >= n. 1680 # 1681 # The "upper" triangular matrix R is actually stored in the strict 1682 # lower left triangle of A under the standard notation of IDL. 1683 # 1684 # The reflectors that generate Q are in the upper trapezoid of A upon 1685 # output. 1686 # 1687 # EXAMPLE: decompose the matrix [[9.,2.,6.],[4.,8.,7.]] 1688 # aa = [[9.,2.,6.],[4.,8.,7.]] 1689 # mpfit_qrfac, aa, aapvt, rdiag, aanorm 1690 # IDL> print, aa 1691 # 1.81818* 0.181818* 0.545455* 1692 # -8.54545+ 1.90160* 0.432573* 1693 # IDL> print, rdiag 1694 # -11.0000+ -7.48166+ 1695 # 1696 # The components marked with a * are the components of the 1697 # reflectors, and those marked with a + are components of R. 1698 # 1699 # To reconstruct Q and R we proceed as follows. First R. 1700 # r = fltarr(m, n) 1701 # for i = 0, n-1 do r(0:i,i) = aa(0:i,i) # fill in lower diag 1702 # r(lindgen(n)*(m+1)) = rdiag 1703 # 1704 # Next, Q, which are composed from the reflectors. Each reflector v 1705 # is taken from the upper trapezoid of aa, and converted to a matrix 1706 # via (I - 2 vT . v / (v . vT)). 1707 # 1708 # hh = ident ## identity matrix 1709 # for i = 0, n-1 do begin 1710 # v = aa(*,i) & if i GT 0 then v(0:i-1) = 0 ## extract reflector 1711 # hh = hh ## (ident - 2*(v # v)/total(v * v)) ## generate matrix 1712 # endfor 1713 # 1714 # Test the result: 1715 # IDL> print, hh ## transpose(r) 1716 # 9.00000 4.00000 1717 # 2.00000 8.00000 1718 # 6.00000 7.00000 1719 # 1720 # Note that it is usually never necessary to form the Q matrix 1721 # explicitly, and MPFIT does not. 1722 1723
1724 - def qrfac(self, a, pivot=0):
1725 1726 if (self.debug): print 'Entering qrfac...' 1727 machep = self.machar.machep 1728 sz = shape(a) 1729 m = sz[0] 1730 n = sz[1] 1731 1732 ## Compute the initial column norms and initialize arrays 1733 acnorm = zeros(n, dtype=float) 1734 for j in range(n): 1735 acnorm[j] = self.enorm(a[:,j]) 1736 rdiag = acnorm.copy() 1737 wa = rdiag.copy() 1738 ipvt = arange(n) 1739 1740 ## Reduce a to r with householder transformations 1741 minmn = numpy.min([m,n]) 1742 for j in range(minmn): 1743 if (pivot != 0): 1744 ## Bring the column of largest norm into the pivot position 1745 rmax = numpy.max(rdiag[j:]) 1746 kmax = (nonzero(rdiag[j:] == rmax))[0] 1747 ct = len(kmax) 1748 kmax = kmax + j 1749 if ct > 0: 1750 kmax = kmax[0] 1751 1752 ## Exchange rows via the pivot only. Avoid actually exchanging 1753 ## the rows, in case there is lots of memory transfer. The 1754 ## exchange occurs later, within the body of MPFIT, after the 1755 ## extraneous columns of the matrix have been shed. 1756 if kmax != j: 1757 temp = ipvt[j] ; ipvt[j] = ipvt[kmax] ; ipvt[kmax] = temp 1758 rdiag[kmax] = rdiag[j] 1759 wa[kmax] = wa[j] 1760 1761 ## Compute the householder transformation to reduce the jth 1762 ## column of A to a multiple of the jth unit vector 1763 lj = ipvt[j] 1764 ajj = a[j:,lj] 1765 ajnorm = self.enorm(ajj) 1766 if ajnorm == 0: break 1767 if a[j,lj] < 0: ajnorm = -ajnorm 1768 1769 ajj = ajj / ajnorm 1770 ajj[0] = ajj[0] + 1 1771 ## *** Note optimization a(j:*,j) 1772 a[j:,lj] = ajj 1773 1774 ## Apply the transformation to the remaining columns 1775 ## and update the norms 1776 1777 ## NOTE to SELF: tried to optimize this by removing the loop, 1778 ## but it actually got slower. Reverted to "for" loop to keep 1779 ## it simple. 1780 if (j+1 < n): 1781 for k in range(j+1, n): 1782 lk = ipvt[k] 1783 ajk = a[j:,lk] 1784 ## *** Note optimization a(j:*,lk) 1785 ## (corrected 20 Jul 2000) 1786 if a[j,lj] != 0: 1787 a[j:,lk] = ajk - ajj * sum(ajk*ajj)/a[j,lj] 1788 if ((pivot != 0) and (rdiag[k] != 0)): 1789 temp = a[j,lk]/rdiag[k] 1790 rdiag[k] = rdiag[k] * sqrt(numpy.max([(1.-temp**2), 0.])) 1791 temp = rdiag[k]/wa[k] 1792 if ((0.05*temp*temp) <= machep): 1793 rdiag[k] = self.enorm(a[j+1:,lk]) 1794 wa[k] = rdiag[k] 1795 rdiag[j] = -ajnorm 1796 return([a, ipvt, rdiag, acnorm])
1797 1798 1799 # Original FORTRAN documentation 1800 # ********** 1801 # 1802 # subroutine qrsolv 1803 # 1804 # given an m by n matrix a, an n by n diagonal matrix d, 1805 # and an m-vector b, the problem is to determine an x which 1806 # solves the system 1807 # 1808 # a*x = b , d*x = 0 , 1809 # 1810 # in the least squares sense. 1811 # 1812 # this subroutine completes the solution of the problem 1813 # if it is provided with the necessary information from the 1814 # factorization, with column pivoting, of a. that is, if 1815 # a*p = q*r, where p is a permutation matrix, q has orthogonal 1816 # columns, and r is an upper triangular matrix with diagonal 1817 # elements of nonincreasing magnitude, then qrsolv expects 1818 # the full upper triangle of r, the permutation matrix p, 1819 # and the first n components of (q transpose)*b. the system 1820 # a*x = b, d*x = 0, is then equivalent to 1821 # 1822 # t t 1823 # r*z = q *b , p *d*p*z = 0 , 1824 # 1825 # where x = p*z. if this system does not have full rank, 1826 # then a least squares solution is obtained. on output qrsolv 1827 # also provides an upper triangular matrix s such that 1828 # 1829 # t t t 1830 # p *(a *a + d*d)*p = s *s . 1831 # 1832 # s is computed within qrsolv and may be of separate interest. 1833 # 1834 # the subroutine statement is 1835 # 1836 # subroutine qrsolv(n,r,ldr,ipvt,diag,qtb,x,sdiag,wa) 1837 # 1838 # where 1839 # 1840 # n is a positive integer input variable set to the order of r. 1841 # 1842 # r is an n by n array. on input the full upper triangle 1843 # must contain the full upper triangle of the matrix r. 1844 # on output the full upper triangle is unaltered, and the 1845 # strict lower triangle contains the strict upper triangle 1846 # (transposed) of the upper triangular matrix s. 1847 # 1848 # ldr is a positive integer input variable not less than n 1849 # which specifies the leading dimension of the array r. 1850 # 1851 # ipvt is an integer input array of length n which defines the 1852 # permutation matrix p such that a*p = q*r. column j of p 1853 # is column ipvt(j) of the identity matrix. 1854 # 1855 # diag is an input array of length n which must contain the 1856 # diagonal elements of the matrix d. 1857 # 1858 # qtb is an input array of length n which must contain the first 1859 # n elements of the vector (q transpose)*b. 1860 # 1861 # x is an output array of length n which contains the least 1862 # squares solution of the system a*x = b, d*x = 0. 1863 # 1864 # sdiag is an output array of length n which contains the 1865 # diagonal elements of the upper triangular matrix s. 1866 # 1867 # wa is a work array of length n. 1868 # 1869 # subprograms called 1870 # 1871 # fortran-supplied ... dabs,dsqrt 1872 # 1873 # argonne national laboratory. minpack project. march 1980. 1874 # burton s. garbow, kenneth e. hillstrom, jorge j. more 1875 # 1876
1877 - def qrsolv(self, r, ipvt, diag, qtb, sdiag):
1878 if (self.debug): print 'Entering qrsolv...' 1879 sz = shape(r) 1880 m = sz[0] 1881 n = sz[1] 1882 1883 ## copy r and (q transpose)*b to preserve input and initialize s. 1884 ## in particular, save the diagonal elements of r in x. 1885 1886 for j in range(n): 1887 r[j:n,j] = r[j,j:n] 1888 x = diagonal(r) 1889 wa = qtb.copy() 1890 1891 ## Eliminate the diagonal matrix d using a givens rotation 1892 for j in range(n): 1893 l = ipvt[j] 1894 if (diag[l] == 0): break 1895 sdiag[j:] = 0 1896 sdiag[j] = diag[l] 1897 1898 ## The transformations to eliminate the row of d modify only a 1899 ## single element of (q transpose)*b beyond the first n, which 1900 ## is initially zero. 1901 1902 qtbpj = 0. 1903 for k in range(j,n): 1904 if (sdiag[k] == 0): break 1905 if (abs(r[k,k]) < abs(sdiag[k])): 1906 cotan = r[k,k]/sdiag[k] 1907 sine = 0.5/sqrt(.25 + .25*cotan*cotan) 1908 cosine = sine*cotan 1909 else: 1910 tang = sdiag[k]/r[k,k] 1911 cosine = 0.5/sqrt(.25 + .25*tang*tang) 1912 sine = cosine*tang 1913 1914 ## Compute the modified diagonal element of r and the 1915 ## modified element of ((q transpose)*b,0). 1916 r[k,k] = cosine*r[k,k] + sine*sdiag[k] 1917 temp = cosine*wa[k] + sine*qtbpj 1918 qtbpj = -sine*wa[k] + cosine*qtbpj 1919 wa[k] = temp 1920 1921 ## Accumulate the transformation in the row of s 1922 if (n > k+1): 1923 temp = cosine*r[k+1:n,k] + sine*sdiag[k+1:n] 1924 sdiag[k+1:n] = -sine*r[k+1:n,k] + cosine*sdiag[k+1:n] 1925 r[k+1:n,k] = temp 1926 sdiag[j] = r[j,j] 1927 r[j,j] = x[j] 1928 1929 ## Solve the triangular system for z. If the system is singular 1930 ## then obtain a least squares solution 1931 nsing = n 1932 wh = (nonzero(sdiag == 0))[0] 1933 if (len(wh) > 0): 1934 nsing = wh[0] 1935 wa[nsing:] = 0 1936 1937 if (nsing >= 1): 1938 wa[nsing-1] = wa[nsing-1]/sdiag[nsing-1] ## Degenerate case 1939 ## *** Reverse loop *** 1940 for j in range(nsing-2,-1,-1): 1941 sum0 = sum(r[j+1:nsing,j]*wa[j+1:nsing]) 1942 wa[j] = (wa[j]-sum0)/sdiag[j] 1943 1944 ## Permute the components of z back to components of x 1945 put(x, ipvt, wa) 1946 return(r, x, sdiag)
1947 1948 1949 1950 1951 # Original FORTRAN documentation 1952 # 1953 # subroutine lmpar 1954 # 1955 # given an m by n matrix a, an n by n nonsingular diagonal 1956 # matrix d, an m-vector b, and a positive number delta, 1957 # the problem is to determine a value for the parameter 1958 # par such that if x solves the system 1959 # 1960 # a*x = b , sqrt(par)*d*x = 0 , 1961 # 1962 # in the least squares sense, and dxnorm is the euclidean 1963 # norm of d*x, then either par is zero and 1964 # 1965 # (dxnorm-delta) .le. 0.1*delta , 1966 # 1967 # or par is positive and 1968 # 1969 # abs(dxnorm-delta) .le. 0.1*delta . 1970 # 1971 # this subroutine completes the solution of the problem 1972 # if it is provided with the necessary information from the 1973 # qr factorization, with column pivoting, of a. that is, if 1974 # a*p = q*r, where p is a permutation matrix, q has orthogonal 1975 # columns, and r is an upper triangular matrix with diagonal 1976 # elements of nonincreasing magnitude, then lmpar expects 1977 # the full upper triangle of r, the permutation matrix p, 1978 # and the first n components of (q transpose)*b. on output 1979 # lmpar also provides an upper triangular matrix s such that 1980 # 1981 # t t t 1982 # p *(a *a + par*d*d)*p = s *s . 1983 # 1984 # s is employed within lmpar and may be of separate interest. 1985 # 1986 # only a few iterations are generally needed for convergence 1987 # of the algorithm. if, however, the limit of 10 iterations 1988 # is reached, then the output par will contain the best 1989 # value obtained so far. 1990 # 1991 # the subroutine statement is 1992 # 1993 # subroutine lmpar(n,r,ldr,ipvt,diag,qtb,delta,par,x,sdiag, 1994 # wa1,wa2) 1995 # 1996 # where 1997 # 1998 # n is a positive integer input variable set to the order of r. 1999 # 2000 # r is an n by n array. on input the full upper triangle 2001 # must contain the full upper triangle of the matrix r. 2002 # on output the full upper triangle is unaltered, and the 2003 # strict lower triangle contains the strict upper triangle 2004 # (transposed) of the upper triangular matrix s. 2005 # 2006 # ldr is a positive integer input variable not less than n 2007 # which specifies the leading dimension of the array r. 2008 # 2009 # ipvt is an integer input array of length n which defines the 2010 # permutation matrix p such that a*p = q*r. column j of p 2011 # is column ipvt(j) of the identity matrix. 2012 # 2013 # diag is an input array of length n which must contain the 2014 # diagonal elements of the matrix d. 2015 # 2016 # qtb is an input array of length n which must contain the first 2017 # n elements of the vector (q transpose)*b. 2018 # 2019 # delta is a positive input variable which specifies an upper 2020 # bound on the euclidean norm of d*x. 2021 # 2022 # par is a nonnegative variable. on input par contains an 2023 # initial estimate of the levenberg-marquardt parameter. 2024 # on output par contains the final estimate. 2025 # 2026 # x is an output array of length n which contains the least 2027 # squares solution of the system a*x = b, sqrt(par)*d*x = 0, 2028 # for the output par. 2029 # 2030 # sdiag is an output array of length n which contains the 2031 # diagonal elements of the upper triangular matrix s. 2032 # 2033 # wa1 and wa2 are work arrays of length n. 2034 # 2035 # subprograms called 2036 # 2037 # minpack-supplied ... dpmpar,enorm,qrsolv 2038 # 2039 # fortran-supplied ... dabs,dmax1,dmin1,dsqrt 2040 # 2041 # argonne national laboratory. minpack project. march 1980. 2042 # burton s. garbow, kenneth e. hillstrom, jorge j. more 2043 # 2044
2045 - def lmpar(self, r, ipvt, diag, qtb, delta, x, sdiag, par=None):
2046 2047 if (self.debug): print 'Entering lmpar...' 2048 dwarf = self.machar.minnum 2049 machep = self.machar.machep 2050 sz = shape(r) 2051 m = sz[0] 2052 n = sz[1] 2053 2054 ## Compute and store in x the gauss-newton direction. If the 2055 ## jacobian is rank-deficient, obtain a least-squares solution 2056 nsing = n 2057 wa1 = qtb.copy() 2058 rthresh = numpy.max(numpy.abs(diagonal(r))) * machep 2059 wh = (nonzero(numpy.abs(diagonal(r)) < rthresh))[0] #patched from lthresh which doesn't exist WR 2060 if len(wh) > 0: 2061 nsing = wh[0] 2062 wa1[wh[0]:] = 0 2063 if nsing >= 1: 2064 ## *** Reverse loop *** 2065 for j in range(nsing-1,-1,-1): 2066 wa1[j] = wa1[j]/r[j,j] 2067 if (j-1 >= 0): 2068 wa1[0:j] = wa1[0:j] - r[0:j,j]*wa1[j] 2069 2070 ## Note: ipvt here is a permutation array 2071 put(x, ipvt, wa1) 2072 2073 ## Initialize the iteration counter. Evaluate the function at the 2074 ## origin, and test for acceptance of the gauss-newton direction 2075 iter = 0 2076 wa2 = diag * x 2077 dxnorm = self.enorm(wa2) 2078 fp = dxnorm - delta 2079 if (fp <= 0.1*delta): 2080 return[r, 0., x, sdiag] 2081 2082 ## If the jacobian is not rank deficient, the newton step provides a 2083 ## lower bound, parl, for the zero of the function. Otherwise set 2084 ## this bound to zero. 2085 2086 parl = 0. 2087 if nsing >= n: 2088 wa1 = take(diag, ipvt)*take(wa2, ipvt)/dxnorm 2089 wa1[0] = wa1[0] / r[0,0] ## Degenerate case 2090 for j in range(1,n): ## Note "1" here, not zero 2091 sum0 = sum(r[0:j,j]*wa1[0:j]) 2092 wa1[j] = (wa1[j] - sum0)/r[j,j] 2093 2094 temp = self.enorm(wa1) 2095 parl = ((fp/delta)/temp)/temp 2096 2097 ## Calculate an upper bound, paru, for the zero of the function 2098 for j in range(n): 2099 sum0 = sum(r[0:j+1,j]*qtb[0:j+1]) 2100 wa1[j] = sum0/diag[ipvt[j]] 2101 gnorm = self.enorm(wa1) 2102 paru = gnorm/delta 2103 if paru == 0: paru = dwarf/numpy.min([delta,0.1]) 2104 2105 ## If the input par lies outside of the interval (parl,paru), set 2106 ## par to the closer endpoint 2107 2108 par = numpy.max([par,parl]) 2109 par = numpy.min([par,paru]) 2110 if par == 0: par = gnorm/dxnorm 2111 2112 ## Beginning of an interation 2113 while(1): 2114 iter = iter + 1 2115 2116 ## Evaluate the function at the current value of par 2117 if par == 0: par = numpy.max([dwarf, paru*0.001]) 2118 temp = sqrt(par) 2119 wa1 = temp * diag 2120 [r, x, sdiag] = self.qrsolv(r, ipvt, wa1, qtb, sdiag) 2121 wa2 = diag*x 2122 dxnorm = self.enorm(wa2) 2123 temp = fp 2124 fp = dxnorm - delta 2125 2126 if ((abs(fp) <= 0.1*delta) or 2127 ((parl == 0) and (fp <= temp) and (temp < 0)) or 2128 (iter == 10)): break; 2129 2130 ## Compute the newton correction 2131 wa1 = take(diag, ipvt)*take(wa2, ipvt)/dxnorm 2132 2133 for j in range(n-1): 2134 wa1[j] = wa1[j]/sdiag[j] 2135 wa1[j+1:n] = wa1[j+1:n] - r[j+1:n,j]*wa1[j] 2136 wa1[n-1] = wa1[n-1]/sdiag[n-1] ## Degenerate case 2137 2138 temp = self.enorm(wa1) 2139 parc = ((fp/delta)/temp)/temp 2140 2141 ## Depending on the sign of the function, update parl or paru 2142 if fp > 0: parl = numpy.max([parl,par]) 2143 if fp < 0: paru = numpy.min([paru,par]) 2144 2145 ## Compute an improved estimate for par 2146 par = numpy.max([parl, par+parc]) 2147 2148 ## End of an iteration 2149 2150 ## Termination 2151 return[r, par, x, sdiag]
2152 2153 2154 ## Procedure to tie one parameter to another.
2155 - def tie(self, p, ptied=None):
2156 if (self.debug): print 'Entering tie...' 2157 if (ptied == None): return 2158 for i in range(len(ptied)): 2159 if ptied[i] == '': continue 2160 cmd = 'p[' + str(i) + '] = ' + ptied[i] 2161 exec(cmd) 2162 return(p)
2163 2164 2165 # Original FORTRAN documentation 2166 # ********** 2167 # 2168 # subroutine covar 2169 # 2170 # given an m by n matrix a, the problem is to determine 2171 # the covariance matrix corresponding to a, defined as 2172 # 2173 # t 2174 # inverse(a *a) . 2175 # 2176 # this subroutine completes the solution of the problem 2177 # if it is provided with the necessary information from the 2178 # qr factorization, with column pivoting, of a. that is, if 2179 # a*p = q*r, where p is a permutation matrix, q has orthogonal 2180 # columns, and r is an upper triangular matrix with diagonal 2181 # elements of nonincreasing magnitude, then covar expects 2182 # the full upper triangle of r and the permutation matrix p. 2183 # the covariance matrix is then computed as 2184 # 2185 # t t 2186 # p*inverse(r *r)*p . 2187 # 2188 # if a is nearly rank deficient, it may be desirable to compute 2189 # the covariance matrix corresponding to the linearly independent 2190 # columns of a. to define the numerical rank of a, covar uses 2191 # the tolerance tol. if l is the largest integer such that 2192 # 2193 # abs(r(l,l)) .gt. tol*abs(r(1,1)) , 2194 # 2195 # then covar computes the covariance matrix corresponding to 2196 # the first l columns of r. for k greater than l, column 2197 # and row ipvt(k) of the covariance matrix are set to zero. 2198 # 2199 # the subroutine statement is 2200 # 2201 # subroutine covar(n,r,ldr,ipvt,tol,wa) 2202 # 2203 # where 2204 # 2205 # n is a positive integer input variable set to the order of r. 2206 # 2207 # r is an n by n array. on input the full upper triangle must 2208 # contain the full upper triangle of the matrix r. on output 2209 # r contains the square symmetric covariance matrix. 2210 # 2211 # ldr is a positive integer input variable not less than n 2212 # which specifies the leading dimension of the array r. 2213 # 2214 # ipvt is an integer input array of length n which defines the 2215 # permutation matrix p such that a*p = q*r. column j of p 2216 # is column ipvt(j) of the identity matrix. 2217 # 2218 # tol is a nonnegative input variable used to define the 2219 # numerical rank of a in the manner described above. 2220 # 2221 # wa is a work array of length n. 2222 # 2223 # subprograms called 2224 # 2225 # fortran-supplied ... dabs 2226 # 2227 # argonne national laboratory. minpack project. august 1980. 2228 # burton s. garbow, kenneth e. hillstrom, jorge j. more 2229 # 2230 # ********** 2231
2232 - def calc_covar(self, rr, ipvt=None, tol=1.e-14):
2233 2234 if (self.debug): print 'Entering calc_covar...' 2235 if rank(rr) != 2: 2236 print 'ERROR: r must be a two-dimensional matrix' 2237 return(-1) 2238 s = shape(rr) 2239 n = s[0] 2240 if s[0] != s[1]: 2241 print 'ERROR: r must be a square matrix' 2242 return(-1) 2243 2244 if (ipvt == None): ipvt = arange(n) 2245 r = rr.copy() 2246 r.shape = [n,n] 2247 2248 ## For the inverse of r in the full upper triangle of r 2249 l = -1 2250 tolr = tol * abs(r[0,0]) 2251 for k in range(n): 2252 if (abs(r[k,k]) <= tolr): break 2253 r[k,k] = 1./r[k,k] 2254 for j in range(k): 2255 temp = r[k,k] * r[j,k] 2256 r[j,k] = 0. 2257 r[0:j+1,k] = r[0:j+1,k] - temp*r[0:j+1,j] 2258 l = k 2259 2260 ## Form the full upper triangle of the inverse of (r transpose)*r 2261 ## in the full upper triangle of r 2262 if l >= 0: 2263 for k in range(l+1): 2264 for j in range(k): 2265 temp = r[j,k] 2266 r[0:j+1,j] = r[0:j+1,j] + temp*r[0:j+1,k] 2267 temp = r[k,k] 2268 r[0:k+1,k] = temp * r[0:k+1,k] 2269 2270 ## For the full lower triangle of the covariance matrix 2271 ## in the strict lower triangle or and in wa 2272 wa = repeat([r[0,0]], n) 2273 for j in range(n): 2274 jj = ipvt[j] 2275 sing = j > l 2276 for i in range(j+1): 2277 if sing: r[i,j] = 0. 2278 ii = ipvt[i] 2279 if ii > jj: r[ii,jj] = r[i,j] 2280 if ii < jj: r[jj,ii] = r[i,j] 2281 wa[jj] = r[j,j] 2282 2283 ## Symmetrize the covariance matrix in r 2284 for j in range(n): 2285 r[0:j+1,j] = r[j,0:j+1] 2286 r[j,j] = wa[j] 2287 2288 return(r)
2289
2290 -class machar:
2291 - def __init__(self, double=1):
2292 if (double == 0): 2293 self.machep = 1.19209e-007 2294 self.maxnum = 3.40282e+038 2295 self.minnum = 1.17549e-038 2296 self.maxgam = 171.624376956302725 2297 else: 2298 self.machep = 2.2204460e-016 2299 self.maxnum = 1.7976931e+308 2300 self.minnum = 2.2250739e-308 2301 self.maxgam = 171.624376956302725 2302 2303 self.maxlog = log(self.maxnum) 2304 self.minlog = log(self.minnum) 2305 self.rdwarf = sqrt(self.minnum*1.5) * 10 2306 self.rgiant = sqrt(self.maxnum) * 0.1
2307