1 """
2 Perform Levenberg-Marquardt least-squares minimization, based on MINPACK-1.
3
4 AUTHORS
5 The original version of this software, called LMFIT, was written in FORTRAN
6 as part of the MINPACK-1 package by XXX.
7
8 Craig Markwardt converted the FORTRAN code to IDL. The information for the
9 IDL version is:
10 Craig B. Markwardt, NASA/GSFC Code 662, Greenbelt, MD 20770
11 craigm@lheamail.gsfc.nasa.gov
12 UPDATED VERSIONs can be found on my WEB PAGE:
13 http://cow.physics.wisc.edu/~craigm/idl/idl.html
14
15 Mark Rivers created this Python version from Craig's IDL version.
16 Mark Rivers, University of Chicago
17 Building 434A, Argonne National Laboratory
18 9700 South Cass Avenue, Argonne, IL 60439
19 rivers@cars.uchicago.edu
20 Updated versions can be found at http://cars.uchicago.edu/software
21
22 Sergey Koposov converted the Mark's Python version from Numeric to numpy
23 Sergey Koposov, Max Planck Institute for Astronomy
24 Heidelberg, Germany, D-69117
25 koposov@mpia.de
26 Updated versions can be found at http://code.google.com/p/astrolibpy/source/browse/trunk/
27
28 DESCRIPTION
29
30 MPFIT uses the Levenberg-Marquardt technique to solve the
31 least-squares problem. In its typical use, MPFIT will be used to
32 fit a user-supplied function (the "model") to user-supplied data
33 points (the "data") by adjusting a set of parameters. MPFIT is
34 based upon MINPACK-1 (LMDIF.F) by More' and collaborators.
35
36 For example, a researcher may think that a set of observed data
37 points is best modelled with a Gaussian curve. A Gaussian curve is
38 parameterized by its mean, standard deviation and normalization.
39 MPFIT will, within certain constraints, find the set of parameters
40 which best fits the data. The fit is "best" in the least-squares
41 sense; that is, the sum of the weighted squared differences between
42 the model and data is minimized.
43
44 The Levenberg-Marquardt technique is a particular strategy for
45 iteratively searching for the best fit. This particular
46 implementation is drawn from MINPACK-1 (see NETLIB), and is much faster
47 and more accurate than the version provided in the Scientific Python package
48 in Scientific.Functions.LeastSquares.
49 This version allows upper and lower bounding constraints to be placed on each
50 parameter, or the parameter can be held fixed.
51
52 The user-supplied Python function should return an array of weighted
53 deviations between model and data. In a typical scientific problem
54 the residuals should be weighted so that each deviate has a
55 gaussian sigma of 1.0. If X represents values of the independent
56 variable, Y represents a measurement for each value of X, and ERR
57 represents the error in the measurements, then the deviates could
58 be calculated as follows:
59
60 DEVIATES = (Y - F(X)) / ERR
61
62 where F is the analytical function representing the model. You are
63 recommended to use the convenience functions MPFITFUN and
64 MPFITEXPR, which are driver functions that calculate the deviates
65 for you. If ERR are the 1-sigma uncertainties in Y, then
66
67 TOTAL( DEVIATES^2 )
68
69 will be the total chi-squared value. MPFIT will minimize the
70 chi-square value. The values of X, Y and ERR are passed through
71 MPFIT to the user-supplied function via the FUNCTKW keyword.
72
73 Simple constraints can be placed on parameter values by using the
74 PARINFO keyword to MPFIT. See below for a description of this
75 keyword.
76
77 MPFIT does not perform more general optimization tasks. See TNMIN
78 instead. MPFIT is customized, based on MINPACK-1, to the
79 least-squares minimization problem.
80
81
82 USER FUNCTION
83
84 The user must define a function which returns the appropriate
85 values as specified above. The function should return the weighted
86 deviations between the model and the data. It should also return a status
87 flag and an optional partial derivative array. For applications which
88 use finite-difference derivatives -- the default -- the user
89 function should be declared in the following way:
90
91 def myfunct(p, fjac=None, x=None, y=None, err=None)
92 # Parameter values are passed in "p"
93 # If fjac==None then partial derivatives should not be
94 # computed. It will always be None if MPFIT is called with default
95 # flag.
96 model = F(x, p)
97 # Non-negative status value means MPFIT should continue, negative means
98 # stop the calculation.
99 status = 0
100 return([status, (y-model)/err]
101
102 See below for applications with analytical derivatives.
103
104 The keyword parameters X, Y, and ERR in the example above are
105 suggestive but not required. Any parameters can be passed to
106 MYFUNCT by using the functkw keyword to MPFIT. Use MPFITFUN and
107 MPFITEXPR if you need ideas on how to do that. The function *must*
108 accept a parameter list, P.
109
110 In general there are no restrictions on the number of dimensions in
111 X, Y or ERR. However the deviates *must* be returned in a
112 one-dimensional Numeric array of type Float.
113
114 User functions may also indicate a fatal error condition using the
115 status return described above. If status is set to a number between
116 -15 and -1 then MPFIT will stop the calculation and return to the caller.
117
118
119 ANALYTIC DERIVATIVES
120
121 In the search for the best-fit solution, MPFIT by default
122 calculates derivatives numerically via a finite difference
123 approximation. The user-supplied function need not calculate the
124 derivatives explicitly. However, if you desire to compute them
125 analytically, then the AUTODERIVATIVE=0 keyword must be passed to MPFIT.
126 As a practical matter, it is often sufficient and even faster to allow
127 MPFIT to calculate the derivatives numerically, and so
128 AUTODERIVATIVE=0 is not necessary.
129
130 If AUTODERIVATIVE=0 is used then the user function must check the parameter
131 FJAC, and if FJAC!=None then return the partial derivative array in the
132 return list.
133 def myfunct(p, fjac=None, x=None, y=None, err=None)
134 # Parameter values are passed in "p"
135 # If FJAC!=None then partial derivatives must be comptuer.
136 # FJAC contains an array of len(p), where each entry
137 # is 1 if that parameter is free and 0 if it is fixed.
138 model = F(x, p)
139 Non-negative status value means MPFIT should continue, negative means
140 # stop the calculation.
141 status = 0
142 if (dojac):
143 pderiv = zeros([len(x), len(p)], Float)
144 for j in range(len(p)):
145 pderiv[:,j] = FGRAD(x, p, j)
146 else:
147 pderiv = None
148 return([status, (y-model)/err, pderiv]
149
150 where FGRAD(x, p, i) is a user function which must compute the
151 derivative of the model with respect to parameter P[i] at X. When
152 finite differencing is used for computing derivatives (ie, when
153 AUTODERIVATIVE=1), or when MPFIT needs only the errors but not the
154 derivatives the parameter FJAC=None.
155
156 Derivatives should be returned in the PDERIV array. PDERIV should be an m x
157 n array, where m is the number of data points and n is the number
158 of parameters. dp[i,j] is the derivative at the ith point with
159 respect to the jth parameter.
160
161 The derivatives with respect to fixed parameters are ignored; zero
162 is an appropriate value to insert for those derivatives. Upon
163 input to the user function, FJAC is set to a vector with the same
164 length as P, with a value of 1 for a parameter which is free, and a
165 value of zero for a parameter which is fixed (and hence no
166 derivative needs to be calculated).
167
168 If the data is higher than one dimensional, then the *last*
169 dimension should be the parameter dimension. Example: fitting a
170 50x50 image, "dp" should be 50x50xNPAR.
171
172
173 CONSTRAINING PARAMETER VALUES WITH THE PARINFO KEYWORD
174
175 The behavior of MPFIT can be modified with respect to each
176 parameter to be fitted. A parameter value can be fixed; simple
177 boundary constraints can be imposed; limitations on the parameter
178 changes can be imposed; properties of the automatic derivative can
179 be modified; and parameters can be tied to one another.
180
181 These properties are governed by the PARINFO structure, which is
182 passed as a keyword parameter to MPFIT.
183
184 PARINFO should be a list of dictionaries, one list entry for each parameter.
185 Each parameter is associated with one element of the array, in
186 numerical order. The dictionary can have the following keys
187 (none are required, keys are case insensitive):
188
189 'value' - the starting parameter value (but see the START_PARAMS
190 parameter for more information).
191
192 'fixed' - a boolean value, whether the parameter is to be held
193 fixed or not. Fixed parameters are not varied by
194 MPFIT, but are passed on to MYFUNCT for evaluation.
195
196 'limited' - a two-element boolean array. If the first/second
197 element is set, then the parameter is bounded on the
198 lower/upper side. A parameter can be bounded on both
199 sides. Both LIMITED and LIMITS must be given
200 together.
201
202 'limits' - a two-element float array. Gives the
203 parameter limits on the lower and upper sides,
204 respectively. Zero, one or two of these values can be
205 set, depending on the values of LIMITED. Both LIMITED
206 and LIMITS must be given together.
207
208 'parname' - a string, giving the name of the parameter. The
209 fitting code of MPFIT does not use this tag in any
210 way. However, the default iterfunct will print the
211 parameter name if available.
212
213 'step' - the step size to be used in calculating the numerical
214 derivatives. If set to zero, then the step size is
215 computed automatically. Ignored when AUTODERIVATIVE=0.
216
217 'mpside' - the sidedness of the finite difference when computing
218 numerical derivatives. This field can take four
219 values:
220
221 0 - one-sided derivative computed automatically
222 1 - one-sided derivative (f(x+h) - f(x) )/h
223 -1 - one-sided derivative (f(x) - f(x-h))/h
224 2 - two-sided derivative (f(x+h) - f(x-h))/(2*h)
225
226 Where H is the STEP parameter described above. The
227 "automatic" one-sided derivative method will chose a
228 direction for the finite difference which does not
229 violate any constraints. The other methods do not
230 perform this check. The two-sided method is in
231 principle more precise, but requires twice as many
232 function evaluations. Default: 0.
233
234 'mpmaxstep' - the maximum change to be made in the parameter
235 value. During the fitting process, the parameter
236 will never be changed by more than this value in
237 one iteration.
238
239 A value of 0 indicates no maximum. Default: 0.
240
241 'tied' - a string expression which "ties" the parameter to other
242 free or fixed parameters. Any expression involving
243 constants and the parameter array P are permitted.
244 Example: if parameter 2 is always to be twice parameter
245 1 then use the following: parinfo(2).tied = '2 * p(1)'.
246 Since they are totally constrained, tied parameters are
247 considered to be fixed; no errors are computed for them.
248 [ NOTE: the PARNAME can't be used in expressions. ]
249
250 'mpprint' - if set to 1, then the default iterfunct will print the
251 parameter value. If set to 0, the parameter value
252 will not be printed. This tag can be used to
253 selectively print only a few parameter values out of
254 many. Default: 1 (all parameters printed)
255
256
257 Future modifications to the PARINFO structure, if any, will involve
258 adding dictionary tags beginning with the two letters "MP".
259 Therefore programmers are urged to avoid using tags starting with
260 the same letters; otherwise they are free to include their own
261 fields within the PARINFO structure, and they will be ignored.
262
263 PARINFO Example:
264 parinfo = [{'value':0., 'fixed':0, 'limited':[0,0], 'limits':[0.,0.]}]*5
265 parinfo[0]['fixed'] = 1
266 parinfo[4]['limited'][0] = 1
267 parinfo[4]['limits'][0] = 50.
268 values = [5.7, 2.2, 500., 1.5, 2000.]
269 for i in range(5): parinfo[i]['value']=values[i]
270
271 A total of 5 parameters, with starting values of 5.7,
272 2.2, 500, 1.5, and 2000 are given. The first parameter
273 is fixed at a value of 5.7, and the last parameter is
274 constrained to be above 50.
275
276
277 EXAMPLE
278
279 import mpfit
280 import numpy.oldnumeric as Numeric
281 x = arange(100, float)
282 p0 = [5.7, 2.2, 500., 1.5, 2000.]
283 y = ( p[0] + p[1]*[x] + p[2]*[x**2] + p[3]*sqrt(x) +
284 p[4]*log(x))
285 fa = {'x':x, 'y':y, 'err':err}
286 m = mpfit('myfunct', p0, functkw=fa)
287 print 'status = ', m.status
288 if (m.status <= 0): print 'error message = ', m.errmsg
289 print 'parameters = ', m.params
290
291 Minimizes sum of squares of MYFUNCT. MYFUNCT is called with the X,
292 Y, and ERR keyword parameters that are given by FUNCTKW. The
293 results can be obtained from the returned object m.
294
295
296 THEORY OF OPERATION
297
298 There are many specific strategies for function minimization. One
299 very popular technique is to use function gradient information to
300 realize the local structure of the function. Near a local minimum
301 the function value can be taylor expanded about x0 as follows:
302
303 f(x) = f(x0) + f'(x0) . (x-x0) + (1/2) (x-x0) . f''(x0) . (x-x0)
304 ----- --------------- ------------------------------- (1)
305 Order 0th 1st 2nd
306
307 Here f'(x) is the gradient vector of f at x, and f''(x) is the
308 Hessian matrix of second derivatives of f at x. The vector x is
309 the set of function parameters, not the measured data vector. One
310 can find the minimum of f, f(xm) using Newton's method, and
311 arrives at the following linear equation:
312
313 f''(x0) . (xm-x0) = - f'(x0) (2)
314
315 If an inverse can be found for f''(x0) then one can solve for
316 (xm-x0), the step vector from the current position x0 to the new
317 projected minimum. Here the problem has been linearized (ie, the
318 gradient information is known to first order). f''(x0) is
319 symmetric n x n matrix, and should be positive definite.
320
321 The Levenberg - Marquardt technique is a variation on this theme.
322 It adds an additional diagonal term to the equation which may aid the
323 convergence properties:
324
325 (f''(x0) + nu I) . (xm-x0) = -f'(x0) (2a)
326
327 where I is the identity matrix. When nu is large, the overall
328 matrix is diagonally dominant, and the iterations follow steepest
329 descent. When nu is small, the iterations are quadratically
330 convergent.
331
332 In principle, if f''(x0) and f'(x0) are known then xm-x0 can be
333 determined. However the Hessian matrix is often difficult or
334 impossible to compute. The gradient f'(x0) may be easier to
335 compute, if even by finite difference techniques. So-called
336 quasi-Newton techniques attempt to successively estimate f''(x0)
337 by building up gradient information as the iterations proceed.
338
339 In the least squares problem there are further simplifications
340 which assist in solving eqn (2). The function to be minimized is
341 a sum of squares:
342
343 f = Sum(hi^2) (3)
344
345 where hi is the ith residual out of m residuals as described
346 above. This can be substituted back into eqn (2) after computing
347 the derivatives:
348
349 f' = 2 Sum(hi hi')
350 f'' = 2 Sum(hi' hj') + 2 Sum(hi hi'') (4)
351
352 If one assumes that the parameters are already close enough to a
353 minimum, then one typically finds that the second term in f'' is
354 negligible [or, in any case, is too difficult to compute]. Thus,
355 equation (2) can be solved, at least approximately, using only
356 gradient information.
357
358 In matrix notation, the combination of eqns (2) and (4) becomes:
359
360 hT' . h' . dx = - hT' . h (5)
361
362 Where h is the residual vector (length m), hT is its transpose, h'
363 is the Jacobian matrix (dimensions n x m), and dx is (xm-x0). The
364 user function supplies the residual vector h, and in some cases h'
365 when it is not found by finite differences (see MPFIT_FDJAC2,
366 which finds h and hT'). Even if dx is not the best absolute step
367 to take, it does provide a good estimate of the best *direction*,
368 so often a line minimization will occur along the dx vector
369 direction.
370
371 The method of solution employed by MINPACK is to form the Q . R
372 factorization of h', where Q is an orthogonal matrix such that QT .
373 Q = I, and R is upper right triangular. Using h' = Q . R and the
374 ortogonality of Q, eqn (5) becomes
375
376 (RT . QT) . (Q . R) . dx = - (RT . QT) . h
377 RT . R . dx = - RT . QT . h (6)
378 R . dx = - QT . h
379
380 where the last statement follows because R is upper triangular.
381 Here, R, QT and h are known so this is a matter of solving for dx.
382 The routine MPFIT_QRFAC provides the QR factorization of h, with
383 pivoting, and MPFIT_QRSOLV provides the solution for dx.
384
385
386 REFERENCES
387
388 MINPACK-1, Jorge More', available from netlib (www.netlib.org).
389 "Optimization Software Guide," Jorge More' and Stephen Wright,
390 SIAM, *Frontiers in Applied Mathematics*, Number 14.
391 More', Jorge J., "The Levenberg-Marquardt Algorithm:
392 Implementation and Theory," in *Numerical Analysis*, ed. Watson,
393 G. A., Lecture Notes in Mathematics 630, Springer-Verlag, 1977.
394
395
396 MODIFICATION HISTORY
397
398 Translated from MINPACK-1 in FORTRAN, Apr-Jul 1998, CM
399 Copyright (C) 1997-2002, Craig Markwardt
400 This software is provided as is without any warranty whatsoever.
401 Permission to use, copy, modify, and distribute modified or
402 unmodified copies is granted, provided this copyright and disclaimer
403 are included unchanged.
404
405 Translated from MPFIT (Craig Markwardt's IDL package) to Python,
406 August, 2002. Mark Rivers
407 Converted from Numeric to numpy (Sergey Koposov, July 2008)
408 """
409
410 from numpy import *
411 import numpy
412 import types
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
597 - def __init__(self, fcn, xall=None, functkw=None, parinfo=None,
598 ftol=1.e-10, xtol=1.e-10, gtol=1.e-10,
599 damp=0., maxiter=200, factor=100., nprint=1,
600 iterfunct='default', iterkw=None, nocovar=0,
601 fastnorm=0, rescale=0, autoderivative=1, quiet=0,
602 diag=None, epsfcn=None, debug=0):
603 """
604 Inputs:
605 fcn:
606 The function to be minimized. The function should return the weighted
607 deviations between the model and the data, as described above.
608
609 xall:
610 An array of starting values for each of the parameters of the model.
611 The number of parameters should be fewer than the number of measurements.
612
613 This parameter is optional if the parinfo keyword is used (but see
614 parinfo). The parinfo keyword provides a mechanism to fix or constrain
615 individual parameters.
616
617 Keywords:
618
619 autoderivative:
620 If this is set, derivatives of the function will be computed
621 automatically via a finite differencing procedure. If not set, then
622 fcn must provide the (analytical) derivatives.
623 Default: set (=1)
624 NOTE: to supply your own analytical derivatives,
625 explicitly pass autoderivative=0
626
627 fastnorm:
628 Set this keyword to select a faster algorithm to compute sum-of-square
629 values internally. For systems with large numbers of data points, the
630 standard algorithm can become prohibitively slow because it cannot be
631 vectorized well. By setting this keyword, MPFIT will run faster, but
632 it will be more prone to floating point overflows and underflows. Thus, setting
633 this keyword may sacrifice some stability in the fitting process.
634 Default: clear (=0)
635
636 ftol:
637 A nonnegative input variable. Termination occurs when both the actual
638 and predicted relative reductions in the sum of squares are at most
639 ftol (and status is accordingly set to 1 or 3). Therefore, ftol
640 measures the relative error desired in the sum of squares.
641 Default: 1E-10
642
643 functkw:
644 A dictionary which contains the parameters to be passed to the
645 user-supplied function specified by fcn via the standard Python
646 keyword dictionary mechanism. This is the way you can pass additional
647 data to your user-supplied function without using global variables.
648
649 Consider the following example:
650 if functkw = {'xval':[1.,2.,3.], 'yval':[1.,4.,9.],
651 'errval':[1.,1.,1.] }
652 then the user supplied function should be declared like this:
653 def myfunct(p, fjac=None, xval=None, yval=None, errval=None):
654
655 Default: {} No extra parameters are passed to the user-supplied
656 function.
657
658 gtol:
659 A nonnegative input variable. Termination occurs when the cosine of
660 the angle between fvec and any column of the jacobian is at most gtol
661 in absolute value (and status is accordingly set to 4). Therefore,
662 gtol measures the orthogonality desired between the function vector
663 and the columns of the jacobian.
664 Default: 1e-10
665
666 iterkw:
667 The keyword arguments to be passed to iterfunct via the dictionary
668 keyword mechanism. This should be a dictionary and is similar in
669 operation to FUNCTKW.
670 Default: {} No arguments are passed.
671
672 iterfunct:
673 The name of a function to be called upon each NPRINT iteration of the
674 MPFIT routine. It should be declared in the following way:
675 def iterfunct(myfunct, p, iter, fnorm, functkw=None,
676 parinfo=None, quiet=0, dof=None, [iterkw keywords here])
677 # perform custom iteration update
678
679 iterfunct must accept all three keyword parameters (FUNCTKW, PARINFO
680 and QUIET).
681
682 myfunct: The user-supplied function to be minimized,
683 p: The current set of model parameters
684 iter: The iteration number
685 functkw: The arguments to be passed to myfunct.
686 fnorm: The chi-squared value.
687 quiet: Set when no textual output should be printed.
688 dof: The number of degrees of freedom, normally the number of points
689 less the number of free parameters.
690 See below for documentation of parinfo.
691
692 In implementation, iterfunct can perform updates to the terminal or
693 graphical user interface, to provide feedback while the fit proceeds.
694 If the fit is to be stopped for any reason, then iterfunct should return a
695 a status value between -15 and -1. Otherwise it should return None
696 (e.g. no return statement) or 0.
697 In principle, iterfunct should probably not modify the parameter values,
698 because it may interfere with the algorithm's stability. In practice it
699 is allowed.
700
701 Default: an internal routine is used to print the parameter values.
702
703 Set iterfunct=None if there is no user-defined routine and you don't
704 want the internal default routine be called.
705
706 maxiter:
707 The maximum number of iterations to perform. If the number is exceeded,
708 then the status value is set to 5 and MPFIT returns.
709 Default: 200 iterations
710
711 nocovar:
712 Set this keyword to prevent the calculation of the covariance matrix
713 before returning (see COVAR)
714 Default: clear (=0) The covariance matrix is returned
715
716 nprint:
717 The frequency with which iterfunct is called. A value of 1 indicates
718 that iterfunct is called with every iteration, while 2 indicates every
719 other iteration, etc. Note that several Levenberg-Marquardt attempts
720 can be made in a single iteration.
721 Default value: 1
722
723 parinfo
724 Provides a mechanism for more sophisticated constraints to be placed on
725 parameter values. When parinfo is not passed, then it is assumed that
726 all parameters are free and unconstrained. Values in parinfo are never
727 modified during a call to MPFIT.
728
729 See description above for the structure of PARINFO.
730
731 Default value: None All parameters are free and unconstrained.
732
733 quiet:
734 Set this keyword when no textual output should be printed by MPFIT
735
736 damp:
737 A scalar number, indicating the cut-off value of residuals where
738 "damping" will occur. Residuals with magnitudes greater than this
739 number will be replaced by their hyperbolic tangent. This partially
740 mitigates the so-called large residual problem inherent in
741 least-squares solvers (as for the test problem CURVI,
742 http://www.maxthis.com/curviex.htm).
743 A value of 0 indicates no damping.
744 Default: 0
745
746 Note: DAMP doesn't work with autoderivative=0
747
748 xtol:
749 A nonnegative input variable. Termination occurs when the relative error
750 between two consecutive iterates is at most xtol (and status is
751 accordingly set to 2 or 3). Therefore, xtol measures the relative error
752 desired in the approximate solution.
753 Default: 1E-10
754
755 Outputs:
756
757 Returns an object of type mpfit. The results are attributes of this class,
758 e.g. mpfit.status, mpfit.errmsg, mpfit.params, npfit.niter, mpfit.covar.
759
760 .status
761 An integer status code is returned. All values greater than zero can
762 represent success (however .status == 5 may indicate failure to
763 converge). It can have one of the following values:
764
765 -16
766 A parameter or function value has become infinite or an undefined
767 number. This is usually a consequence of numerical overflow in the
768 user's model function, which must be avoided.
769
770 -15 to -1
771 These are error codes that either MYFUNCT or iterfunct may return to
772 terminate the fitting process. Values from -15 to -1 are reserved
773 for the user functions and will not clash with MPFIT.
774
775 0 Improper input parameters.
776
777 1 Both actual and predicted relative reductions in the sum of squares
778 are at most ftol.
779
780 2 Relative error between two consecutive iterates is at most xtol
781
782 3 Conditions for status = 1 and status = 2 both hold.
783
784 4 The cosine of the angle between fvec and any column of the jacobian
785 is at most gtol in absolute value.
786
787 5 The maximum number of iterations has been reached.
788
789 6 ftol is too small. No further reduction in the sum of squares is
790 possible.
791
792 7 xtol is too small. No further improvement in the approximate solution
793 x is possible.
794
795 8 gtol is too small. fvec is orthogonal to the columns of the jacobian
796 to machine precision.
797
798 .fnorm
799 The value of the summed squared residuals for the returned parameter
800 values.
801
802 .covar
803 The covariance matrix for the set of parameters returned by MPFIT.
804 The matrix is NxN where N is the number of parameters. The square root
805 of the diagonal elements gives the formal 1-sigma statistical errors on
806 the parameters if errors were treated "properly" in fcn.
807 Parameter errors are also returned in .perror.
808
809 To compute the correlation matrix, pcor, use this example:
810 cov = mpfit.covar
811 pcor = cov * 0.
812 for i in range(n):
813 for j in range(n):
814 pcor[i,j] = cov[i,j]/sqrt(cov[i,i]*cov[j,j])
815
816 If nocovar is set or MPFIT terminated abnormally, then .covar is set to
817 a scalar with value None.
818
819 .errmsg
820 A string error or warning message is returned.
821
822 .nfev
823 The number of calls to MYFUNCT performed.
824
825 .niter
826 The number of iterations completed.
827
828 .perror
829 The formal 1-sigma errors in each parameter, computed from the
830 covariance matrix. If a parameter is held fixed, or if it touches a
831 boundary, then the error is reported as zero.
832
833 If the fit is unweighted (i.e. no errors were given, or the weights
834 were uniformly set to unity), then .perror will probably not represent
835 the true parameter uncertainties.
836
837 *If* you can assume that the true reduced chi-squared value is unity --
838 meaning that the fit is implicitly assumed to be of good quality --
839 then the estimated parameter uncertainties can be computed by scaling
840 .perror by the measured chi-squared value.
841
842 dof = len(x) - len(mpfit.params) # deg of freedom
843 # scaled uncertainties
844 pcerror = mpfit.perror * sqrt(mpfit.fnorm / dof)
845
846 """
847
848
849 if functkw is None:
850 functkw = {}
851 if iterkw is None:
852 iterkw = {}
853
854 self.niter = 0
855 self.params = None
856 self.covar = None
857 self.perror = None
858 self.status = 0
859 self.debug = debug
860 self.errmsg = ''
861 self.fastnorm = fastnorm
862 self.nfev = 0
863 self.damp = damp
864 self.machar = machar(double=1)
865 machep = self.machar.machep
866 self.dof=0
867
868 if (fcn==None):
869 self.errmsg = "Usage: parms = mpfit('myfunt', ... )"
870 return
871
872 if (iterfunct == 'default'): iterfunct = self.defiter
873
874
875
876 if (self.damp != 0) and (autoderivative == 0):
877 self.errmsg = 'ERROR: keywords DAMP and AUTODERIVATIVE are mutually exclusive'
878 return
879
880
881 if (xall == None) and (parinfo == None):
882 self.errmsg = 'ERROR: must pass parameters in P or PARINFO'
883 return
884
885
886 if (parinfo != None):
887 if (type(parinfo) != types.ListType):
888 self.errmsg = 'ERROR: PARINFO must be a list of dictionaries.'
889 return
890 else:
891 if (type(parinfo[0]) != types.DictionaryType):
892 self.errmsg = 'ERROR: PARINFO must be a list of dictionaries.'
893 return
894 if ((xall != None) and (len(xall) != len(parinfo))):
895 self.errmsg = 'ERROR: number of elements in PARINFO and P must agree'
896 return
897
898
899
900 if (xall == None):
901 xall = self.parinfo(parinfo, 'value')
902 if (xall == None):
903 self.errmsg = 'ERROR: either P or PARINFO(*)["value"] must be supplied.'
904 return
905
906
907 xall = asarray(xall, float)
908
909 npar = len(xall)
910 self.fnorm = -1.
911 fnorm1 = -1.
912
913
914 ptied = self.parinfo(parinfo, 'tied', default='', n=npar)
915 self.qanytied = 0
916 for i in range(npar):
917 ptied[i] = ptied[i].strip()
918 if (ptied[i] != ''): self.qanytied = 1
919 self.ptied = ptied
920
921
922 pfixed = self.parinfo(parinfo, 'fixed', default=0, n=npar)
923 pfixed = (pfixed == 1)
924 for i in range(npar):
925 pfixed[i] = pfixed[i] or (ptied[i] != '')
926
927
928 step = self.parinfo(parinfo, 'step', default=0., n=npar)
929 dstep = self.parinfo(parinfo, 'relstep', default=0., n=npar)
930 dside = self.parinfo(parinfo, 'mpside', default=0, n=npar)
931
932
933 maxstep = self.parinfo(parinfo, 'mpmaxstep', default=0., n=npar)
934 minstep = self.parinfo(parinfo, 'mpminstep', default=0., n=npar)
935 qmin = minstep * 0
936 qmax = maxstep != 0
937 wh = (nonzero(((qmin!=0.) & (qmax!=0.)) & (maxstep < minstep)))[0]
938 if (len(wh) > 0):
939 self.errmsg = 'ERROR: MPMINSTEP is greater than MPMAXSTEP'
940 return
941 wh = (nonzero((qmin!=0.) | (qmax!=0.)))[0]
942 qminmax = len(wh > 0)
943
944
945 ifree = (nonzero(pfixed != 1))[0]
946 nfree = len(ifree)
947 if nfree == 0:
948 self.errmsg = 'ERROR: no free parameters'
949 return
950
951
952 self.params = xall
953 x = take(self.params, ifree)
954
955
956 limited = self.parinfo(parinfo, 'limited', default=[0,0], n=npar)
957 limits = self.parinfo(parinfo, 'limits', default=[0.,0.], n=npar)
958 if (limited != None) and (limits != None):
959
960 wh = (nonzero((limited[:,0] & (xall < limits[:,0])) |
961 (limited[:,1] & (xall > limits[:,1]))))[0]
962 if (len(wh) > 0):
963 self.errmsg = 'ERROR: parameters are not within PARINFO limits'
964 return
965 wh = (nonzero((limited[:,0] & limited[:,1]) &
966 (limits[:,0] >= limits[:,1]) &
967 (pfixed == 0)))[0]
968 if (len(wh) > 0):
969 self.errmsg = 'ERROR: PARINFO parameter limits are not consistent'
970 return
971
972
973 qulim = take(limited[:,1], ifree)
974 ulim = take(limits [:,1], ifree)
975 qllim = take(limited[:,0], ifree)
976 llim = take(limits [:,0], ifree)
977
978 wh = (nonzero((qulim!=0.) | (qllim!=0.)))[0]
979 if (len(wh) > 0): qanylim = 1
980 else: qanylim = 0
981 else:
982
983 qulim = zeros(nfree)
984 ulim = x * 0.
985 qllim = qulim
986 llim = x * 0.
987 qanylim = 0
988
989 n = len(x)
990
991 if ((n < 0) or (ftol <= 0) or (xtol <= 0) or (gtol <= 0)
992 or (maxiter < 0) or (factor <= 0)):
993 self.errmsg = 'ERROR: input keywords are inconsistent'
994 return
995
996 if (rescale != 0):
997 self.errmsg = 'ERROR: DIAG parameter scales are inconsistent'
998 if (len(diag) < n): return
999 wh = (nonzero(diag <= 0))[0]
1000 if (len(wh) > 0): return
1001 self.errmsg = ''
1002
1003
1004 x = asarray(x, float)
1005
1006 [self.status, fvec] = self.call(fcn, self.params, functkw)
1007 if (self.status < 0):
1008 self.errmsg = 'ERROR: first call to "'+str(fcn)+'" failed'
1009 return
1010
1011 m = len(fvec)
1012 if (m < n):
1013 self.errmsg = 'ERROR: number of parameters must not exceed data'
1014 return
1015 self.dof = m-nfree
1016 self.fnorm = self.enorm(fvec)
1017
1018
1019
1020 par = 0.
1021 self.niter = 1
1022 qtf = x * 0.
1023 self.status = 0
1024
1025
1026
1027 while(1):
1028
1029
1030 put(self.params, ifree, x)
1031 if (self.qanytied): self.params = self.tie(self.params, ptied)
1032
1033 if (nprint > 0) and (iterfunct != None):
1034 if (((self.niter-1) % nprint) == 0):
1035 mperr = 0
1036 xnew0 = self.params.copy()
1037
1038 dof = numpy.max([len(fvec) - len(x), 0])
1039 status = iterfunct(fcn, self.params, self.niter, self.fnorm**2,
1040 functkw=functkw, parinfo=parinfo, quiet=quiet,
1041 dof=dof, **iterkw)
1042 if (status != None): self.status = status
1043
1044
1045 if (self.status < 0):
1046 self.errmsg = 'WARNING: premature termination by ' + str(iterfunct)
1047 return
1048
1049
1050 if (numpy.max(abs(xnew0-self.params)) > 0):
1051 if (self.qanytied): self.params = self.tie(self.params, ptied)
1052 x = take(self.params, ifree)
1053
1054
1055
1056 self.status = 2
1057 catch_msg = 'calling MPFIT_FDJAC2'
1058 fjac = self.fdjac2(fcn, x, fvec, step, qulim, ulim, dside,
1059 epsfcn=epsfcn,
1060 autoderivative=autoderivative, dstep=dstep,
1061 functkw=functkw, ifree=ifree, xall=self.params)
1062 if (fjac == None):
1063 self.errmsg = 'WARNING: premature termination by FDJAC2'
1064 return
1065
1066
1067 if (qanylim):
1068 catch_msg = 'zeroing derivatives of pegged parameters'
1069 whlpeg = (nonzero(qllim & (x == llim)))[0]
1070 nlpeg = len(whlpeg)
1071 whupeg = (nonzero(qulim & (x == ulim)))[0]
1072 nupeg = len(whupeg)
1073
1074 if (nlpeg > 0):
1075
1076 for i in range(nlpeg):
1077 sum0 = sum(fvec * fjac[:,whlpeg[i]])
1078 if (sum0 > 0): fjac[:,whlpeg[i]] = 0
1079 if (nupeg > 0):
1080
1081 for i in range(nupeg):
1082 sum0 = sum(fvec * fjac[:,whupeg[i]])
1083 if (sum0 < 0): fjac[:,whupeg[i]] = 0
1084
1085
1086 [fjac, ipvt, wa1, wa2] = self.qrfac(fjac, pivot=1)
1087
1088
1089
1090 catch_msg = 'rescaling diagonal elements'
1091 if (self.niter == 1):
1092 if ((rescale==0) or (len(diag) < n)):
1093 diag = wa2.copy()
1094 wh = (nonzero(diag == 0))[0]
1095 put(diag, wh, 1.)
1096
1097
1098
1099 wa3 = diag * x
1100 xnorm = self.enorm(wa3)
1101 delta = factor*xnorm
1102 if (delta == 0.): delta = factor
1103
1104
1105 catch_msg = 'forming (q transpose)*fvec'
1106 wa4 = fvec.copy()
1107 for j in range(n):
1108 lj = ipvt[j]
1109 temp3 = fjac[j,lj]
1110 if (temp3 != 0):
1111 fj = fjac[j:,lj]
1112 wj = wa4[j:]
1113
1114 wa4[j:] = wj - fj * sum(fj*wj) / temp3
1115 fjac[j,lj] = wa1[j]
1116 qtf[j] = wa4[j]
1117
1118
1119 fjac = fjac[0:n, 0:n]
1120 fjac.shape = [n, n]
1121 temp = fjac.copy()
1122 for i in range(n):
1123 temp[:,i] = fjac[:, ipvt[i]]
1124 fjac = temp.copy()
1125
1126
1127
1128
1129
1130
1131
1132
1133 catch_msg = 'computing the scaled gradient'
1134 gnorm = 0.
1135 if (self.fnorm != 0):
1136 for j in range(n):
1137 l = ipvt[j]
1138 if (wa2[l] != 0):
1139 sum0 = sum(fjac[0:j+1,j]*qtf[0:j+1])/self.fnorm
1140 gnorm = numpy.max([gnorm,abs(sum0/wa2[l])])
1141
1142
1143 if (gnorm <= gtol):
1144 self.status = 4
1145 break
1146 if maxiter == 0:
1147 self.status = 5
1148 break
1149
1150
1151 if (rescale == 0):
1152 diag = choose(diag>wa2, (wa2, diag))
1153
1154
1155 while(1):
1156
1157
1158 catch_msg = 'calculating LM parameter (MPFIT_)'
1159 [fjac, par, wa1, wa2] = self.lmpar(fjac, ipvt, diag, qtf,
1160 delta, wa1, wa2, par=par)
1161
1162 wa1 = -wa1
1163
1164 if (qanylim == 0) and (qminmax == 0):
1165
1166 alpha = 1.
1167 wa2 = x + wa1
1168
1169 else:
1170
1171
1172
1173
1174 alpha = 1.
1175
1176 if (qanylim):
1177
1178 catch_msg = 'checking for a step out of bounds'
1179 if (nlpeg > 0):
1180 put(wa1, whlpeg, clip(
1181 take(wa1, whlpeg), 0., numpy.max(wa1)))
1182 if (nupeg > 0):
1183 put(wa1, whupeg, clip(
1184 take(wa1, whupeg), numpy.min(wa1), 0.))
1185
1186 dwa1 = abs(wa1) > machep
1187 whl = (nonzero(((dwa1!=0.) & qllim) & ((x + wa1) < llim)))[0]
1188 if (len(whl) > 0):
1189 t = ((take(llim, whl) - take(x, whl)) /
1190 take(wa1, whl))
1191 alpha = numpy.min([alpha, numpy.min(t)])
1192 whu = (nonzero(((dwa1!=0.) & qulim) & ((x + wa1) > ulim)))[0]
1193 if (len(whu) > 0):
1194 t = ((take(ulim, whu) - take(x, whu)) /
1195 take(wa1, whu))
1196 alpha = numpy.min([alpha, numpy.min(t)])
1197
1198
1199 if (qminmax):
1200 nwa1 = wa1 * alpha
1201 whmax = (nonzero((qmax != 0.) & (maxstep > 0)))[0]
1202 if (len(whmax) > 0):
1203 mrat = numpy.max(numpy.abs(nwa1[whmax]) /
1204 numpy.abs(maxstep[ifree[whmax]]))
1205 if (mrat > 1): alpha = alpha / mrat
1206
1207
1208 wa1 = wa1 * alpha
1209 wa2 = x + wa1
1210
1211
1212
1213 sgnu = (ulim >= 0) * 2. - 1.
1214 sgnl = (llim >= 0) * 2. - 1.
1215
1216
1217 ulim1 = ulim * (1 - sgnu * machep) - (ulim == 0) * machep
1218 llim1 = llim * (1 + sgnl * machep) + (llim == 0) * machep
1219 wh = (nonzero((qulim!=0) & (wa2 >= ulim1)))[0]
1220 if (len(wh) > 0): put(wa2, wh, take(ulim, wh))
1221 wh = (nonzero((qllim!=0.) & (wa2 <= llim1)))[0]
1222 if (len(wh) > 0): put(wa2, wh, take(llim, wh))
1223
1224 wa3 = diag * wa1
1225 pnorm = self.enorm(wa3)
1226
1227
1228 if (self.niter == 1): delta = numpy.min([delta,pnorm])
1229
1230 put(self.params, ifree, wa2)
1231
1232
1233 mperr = 0
1234 catch_msg = 'calling '+str(fcn)
1235 [self.status, wa4] = self.call(fcn, self.params, functkw)
1236 if (self.status < 0):
1237 self.errmsg = 'WARNING: premature termination by "'+fcn+'"'
1238 return
1239 fnorm1 = self.enorm(wa4)
1240
1241
1242 catch_msg = 'computing convergence criteria'
1243 actred = -1.
1244 if ((0.1 * fnorm1) < self.fnorm): actred = - (fnorm1/self.fnorm)**2 + 1.
1245
1246
1247
1248 for j in range(n):
1249 wa3[j] = 0
1250 wa3[0:j+1] = wa3[0:j+1] + fjac[0:j+1,j]*wa1[ipvt[j]]
1251
1252
1253
1254 temp1 = self.enorm(alpha*wa3)/self.fnorm
1255 temp2 = (sqrt(alpha*par)*pnorm)/self.fnorm
1256 prered = temp1*temp1 + (temp2*temp2)/0.5
1257 dirder = -(temp1*temp1 + temp2*temp2)
1258
1259
1260 ratio = 0.
1261 if (prered != 0): ratio = actred/prered
1262
1263
1264 if (ratio <= 0.25):
1265 if (actred >= 0): temp = .5
1266 else: temp = .5*dirder/(dirder + .5*actred)
1267 if ((0.1*fnorm1) >= self.fnorm) or (temp < 0.1): temp = 0.1
1268 delta = temp*numpy.min([delta,pnorm/0.1])
1269 par = par/temp
1270 else:
1271 if (par == 0) or (ratio >= 0.75):
1272 delta = pnorm/.5
1273 par = .5*par
1274
1275
1276 if (ratio >= 0.0001):
1277
1278 x = wa2
1279 wa2 = diag * x
1280 fvec = wa4
1281 xnorm = self.enorm(wa2)
1282 self.fnorm = fnorm1
1283 self.niter = self.niter + 1
1284
1285
1286 if ((abs(actred) <= ftol) and (prered <= ftol)
1287 and (0.5 * ratio <= 1)): self.status = 1
1288 if delta <= xtol*xnorm: self.status = 2
1289 if ((abs(actred) <= ftol) and (prered <= ftol)
1290 and (0.5 * ratio <= 1) and (self.status == 2)): self.status = 3
1291 if (self.status != 0): break
1292
1293
1294 if (self.niter >= maxiter): self.status = 5
1295 if ((abs(actred) <= machep) and (prered <= machep)
1296 and (0.5*ratio <= 1)): self.status = 6
1297 if delta <= machep*xnorm: self.status = 7
1298 if gnorm <= machep: self.status = 8
1299 if (self.status != 0): break
1300
1301
1302 if (ratio >= 0.0001): break
1303
1304
1305 if ~numpy.all(numpy.isfinite(wa1) & numpy.isfinite(wa2) &
1306 numpy.isfinite(x)) or ~numpy.isfinite(ratio):
1307 errmsg = ('''ERROR: parameter or function value(s) have become
1308 'infinite; check model function for over- 'and underflow''')
1309 self.status = -16
1310 break
1311
1312
1313
1314 if (self.status != 0): break;
1315
1316
1317 catch_msg = 'in the termination phase'
1318
1319 if (len(self.params) == 0):
1320 return
1321
1322
1323 if (nfree == 0): self.params = xall.copy()
1324 else: put(self.params, ifree, x)
1325 if (nprint > 0) and (self.status > 0):
1326 catch_msg = 'calling ' + str(fcn)
1327 [status, fvec] = self.call(fcn, self.params, functkw)
1328 catch_msg = 'in the termination phase'
1329 self.fnorm = self.enorm(fvec)
1330
1331 if ((self.fnorm != None) and (fnorm1 != None)):
1332 self.fnorm = numpy.max([self.fnorm, fnorm1])
1333 self.fnorm = self.fnorm**2.
1334
1335 self.covar = None
1336 self.perror = None
1337
1338 if ((self.status > 0) and (nocovar==0) and (n != None)
1339 and (fjac != None) and (ipvt != None)):
1340 sz = shape(fjac)
1341 if ((n > 0) and (sz[0] >= n) and (sz[1] >= n)
1342 and (len(ipvt) >= n)):
1343
1344 catch_msg = 'computing the covariance matrix'
1345 cv = self.calc_covar(fjac[0:n,0:n], ipvt[0:n])
1346 cv.shape = [n, n]
1347 nn = len(xall)
1348
1349
1350
1351 self.covar = zeros([nn, nn], dtype=float)
1352 for i in range(n):
1353 self.covar[ifree,ifree[i]] = cv[:,i]
1354
1355
1356 catch_msg = 'computing parameter errors'
1357 self.perror = zeros(nn, dtype=float)
1358 d = diagonal(self.covar)
1359 wh = (nonzero(d >= 0))[0]
1360 if len(wh) > 0:
1361 put(self.perror, wh, sqrt(take(d, wh)))
1362 return
1363
1364
1366 return {'params': self.params,
1367 'niter': self.niter,
1368 'params': self.params,
1369 'covar': self.covar,
1370 'perror': self.perror,
1371 'status': self.status,
1372 'debug': self.debug,
1373 'errmsg': self.errmsg,
1374 'fastnorm': self.fastnorm,
1375 'nfev': self.nfev,
1376 'damp': self.damp
1377
1378 }.__str__()
1379
1380
1381
1382 - def defiter(self, fcn, x, iter, fnorm=None, functkw=None,
1383 quiet=0, iterstop=None, parinfo=None,
1384 format=None, pformat='%.10g', dof=1):
1385
1386 if (self.debug): print 'Entering defiter...'
1387 if (quiet): return
1388 if (fnorm == None):
1389 [status, fvec] = self.call(fcn, x, functkw)
1390 fnorm = self.enorm(fvec)**2
1391
1392
1393 nprint = len(x)
1394 print "Iter ", ('%6i' % iter)," CHI-SQUARE = ",('%.10g' % fnorm)," DOF = ", ('%i' % dof)
1395 for i in range(nprint):
1396 if (parinfo != None) and (parinfo[i].has_key('parname')):
1397 p = ' ' + parinfo[i]['parname'] + ' = '
1398 else:
1399 p = ' P' + str(i) + ' = '
1400 if (parinfo != None) and (parinfo[i].has_key('mpprint')):
1401 iprint = parinfo[i]['mpprint']
1402 else:
1403 iprint = 1
1404 if (iprint):
1405 print p + (pformat % x[i]) + ' '
1406 return(0)
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426 - def parinfo(self, parinfo=None, key='a', default=None, n=0):
1427 if (self.debug): print 'Entering parinfo...'
1428 if (n == 0) and (parinfo != None): n = len(parinfo)
1429 if (n == 0):
1430 values = default
1431 return(values)
1432 values = []
1433 for i in range(n):
1434 if ((parinfo != None) and (parinfo[i].has_key(key))):
1435 values.append(parinfo[i][key])
1436 else:
1437 values.append(default)
1438
1439
1440 test = default
1441 if (type(default) == types.ListType): test=default[0]
1442 if (isinstance(test, types.IntType)):
1443 values = asarray(values, int)
1444 elif (isinstance(test, types.FloatType)):
1445 values = asarray(values, float)
1446 return(values)
1447
1448
1449
1450 - def call(self, fcn, x, functkw, fjac=None):
1451 if (self.debug): print 'Entering call...'
1452 if (self.qanytied): x = self.tie(x, self.ptied)
1453 self.nfev = self.nfev + 1
1454 if (fjac == None):
1455 [status, f] = fcn(x, fjac=fjac, **functkw)
1456 if (self.damp > 0):
1457
1458
1459
1460 f = tanh(f/self.damp)
1461 return([status, f])
1462 else:
1463 return(fcn(x, fjac=fjac, **functkw))
1464
1465
1467
1468 if (self.debug): print 'Entering enorm...'
1469
1470
1471
1472
1473
1474
1475
1476
1477 if (self.fastnorm):
1478 ans = sqrt(sum(vec*vec))
1479 else:
1480 agiant = self.machar.rgiant / len(vec)
1481 adwarf = self.machar.rdwarf * len(vec)
1482
1483
1484
1485 mx = numpy.max(vec)
1486 mn = numpy.min(vec)
1487 mx = max([abs(mx), abs(mn)])
1488 if mx == 0: return(vec[0]*0.)
1489 if mx > agiant or mx < adwarf:
1490 ans = mx * sqrt(sum((vec/mx)*(vec/mx)))
1491 else:
1492 ans = sqrt(sum(vec*vec))
1493
1494 return(ans)
1495
1496
1497 - def fdjac2(self, fcn, x, fvec, step=None, ulimited=None, ulimit=None, dside=None,
1498 epsfcn=None, autoderivative=1,
1499 functkw=None, xall=None, ifree=None, dstep=None):
1500
1501 if (self.debug): print 'Entering fdjac2...'
1502 machep = self.machar.machep
1503 if epsfcn == None: epsfcn = machep
1504 if xall == None: xall = x
1505 if ifree == None: ifree = arange(len(xall))
1506 if step == None: step = x * 0.
1507 nall = len(xall)
1508
1509 eps = sqrt(numpy.max([epsfcn, machep]))
1510 m = len(fvec)
1511 n = len(x)
1512
1513
1514 if (autoderivative == 0):
1515 mperr = 0
1516 fjac = zeros(nall, dtype=float)
1517 Put(fjac, ifree, 1.0)
1518 [status, fp] = self.call(fcn, xall, functkw, fjac=fjac)
1519
1520 if len(fjac) != m*nall:
1521 print 'ERROR: Derivative matrix was not computed properly.'
1522 return(None)
1523
1524
1525
1526 fjac.shape = [m,nall]
1527 fjac = -fjac
1528
1529
1530 if len(ifree) < nall:
1531 fjac = fjac[:,ifree]
1532 fjac.shape = [m, n]
1533 return(fjac)
1534
1535 fjac = zeros([m, n], dtype=float)
1536
1537 h = eps * abs(x)
1538
1539
1540
1541 if step != None:
1542 stepi = take(step, ifree)
1543 wh = (nonzero(stepi > 0))[0]
1544 if (len(wh) > 0): put(h, wh, take(stepi, wh))
1545
1546
1547
1548 if (len(dstep) > 0):
1549 dstepi = take(dstep, ifree)
1550 wh = (nonzero(dstepi > 0))[0]
1551 if len(wh) > 0: put(h, wh, abs(take(dstepi,wh)*take(x,wh)))
1552
1553
1554 wh = (nonzero(h == 0))[0]
1555 if len(wh) > 0: put(h, wh, eps)
1556
1557
1558
1559
1560
1561 mask = dside[ifree] == -1
1562 if len(ulimited) > 0 and len(ulimit) > 0:
1563 mask = logical_or((mask!=0), logical_and((ulimited!=0),(x > ulimit-h)))
1564 wh = (nonzero(mask))[0]
1565 if len(wh) > 0: put(h, wh, -take(h, wh))
1566
1567 for j in range(n):
1568 xp = xall.copy()
1569 xp[ifree[j]] = xp[ifree[j]] + h[j]
1570 [status, fp] = self.call(fcn, xp, functkw)
1571 if (status < 0): return(None)
1572
1573 if abs(dside[ifree[j]]) <= 1:
1574
1575
1576 fjac[0:,j] = (fp-fvec)/h[j]
1577
1578 else:
1579
1580 xp[ifree[j]] = xall[ifree[j]] - h[j]
1581
1582 mperr = 0
1583 [status, fm] = self.call(fcn, xp, functkw)
1584 if (status < 0): return(None)
1585
1586
1587 fjac[0:,j] = (fp-fm)/(2*h[j])
1588 return(fjac)
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724 - def qrfac(self, a, pivot=0):
1725
1726 if (self.debug): print 'Entering qrfac...'
1727 machep = self.machar.machep
1728 sz = shape(a)
1729 m = sz[0]
1730 n = sz[1]
1731
1732
1733 acnorm = zeros(n, dtype=float)
1734 for j in range(n):
1735 acnorm[j] = self.enorm(a[:,j])
1736 rdiag = acnorm.copy()
1737 wa = rdiag.copy()
1738 ipvt = arange(n)
1739
1740
1741 minmn = numpy.min([m,n])
1742 for j in range(minmn):
1743 if (pivot != 0):
1744
1745 rmax = numpy.max(rdiag[j:])
1746 kmax = (nonzero(rdiag[j:] == rmax))[0]
1747 ct = len(kmax)
1748 kmax = kmax + j
1749 if ct > 0:
1750 kmax = kmax[0]
1751
1752
1753
1754
1755
1756 if kmax != j:
1757 temp = ipvt[j] ; ipvt[j] = ipvt[kmax] ; ipvt[kmax] = temp
1758 rdiag[kmax] = rdiag[j]
1759 wa[kmax] = wa[j]
1760
1761
1762
1763 lj = ipvt[j]
1764 ajj = a[j:,lj]
1765 ajnorm = self.enorm(ajj)
1766 if ajnorm == 0: break
1767 if a[j,lj] < 0: ajnorm = -ajnorm
1768
1769 ajj = ajj / ajnorm
1770 ajj[0] = ajj[0] + 1
1771
1772 a[j:,lj] = ajj
1773
1774
1775
1776
1777
1778
1779
1780 if (j+1 < n):
1781 for k in range(j+1, n):
1782 lk = ipvt[k]
1783 ajk = a[j:,lk]
1784
1785
1786 if a[j,lj] != 0:
1787 a[j:,lk] = ajk - ajj * sum(ajk*ajj)/a[j,lj]
1788 if ((pivot != 0) and (rdiag[k] != 0)):
1789 temp = a[j,lk]/rdiag[k]
1790 rdiag[k] = rdiag[k] * sqrt(numpy.max([(1.-temp**2), 0.]))
1791 temp = rdiag[k]/wa[k]
1792 if ((0.05*temp*temp) <= machep):
1793 rdiag[k] = self.enorm(a[j+1:,lk])
1794 wa[k] = rdiag[k]
1795 rdiag[j] = -ajnorm
1796 return([a, ipvt, rdiag, acnorm])
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877 - def qrsolv(self, r, ipvt, diag, qtb, sdiag):
1878 if (self.debug): print 'Entering qrsolv...'
1879 sz = shape(r)
1880 m = sz[0]
1881 n = sz[1]
1882
1883
1884
1885
1886 for j in range(n):
1887 r[j:n,j] = r[j,j:n]
1888 x = diagonal(r)
1889 wa = qtb.copy()
1890
1891
1892 for j in range(n):
1893 l = ipvt[j]
1894 if (diag[l] == 0): break
1895 sdiag[j:] = 0
1896 sdiag[j] = diag[l]
1897
1898
1899
1900
1901
1902 qtbpj = 0.
1903 for k in range(j,n):
1904 if (sdiag[k] == 0): break
1905 if (abs(r[k,k]) < abs(sdiag[k])):
1906 cotan = r[k,k]/sdiag[k]
1907 sine = 0.5/sqrt(.25 + .25*cotan*cotan)
1908 cosine = sine*cotan
1909 else:
1910 tang = sdiag[k]/r[k,k]
1911 cosine = 0.5/sqrt(.25 + .25*tang*tang)
1912 sine = cosine*tang
1913
1914
1915
1916 r[k,k] = cosine*r[k,k] + sine*sdiag[k]
1917 temp = cosine*wa[k] + sine*qtbpj
1918 qtbpj = -sine*wa[k] + cosine*qtbpj
1919 wa[k] = temp
1920
1921
1922 if (n > k+1):
1923 temp = cosine*r[k+1:n,k] + sine*sdiag[k+1:n]
1924 sdiag[k+1:n] = -sine*r[k+1:n,k] + cosine*sdiag[k+1:n]
1925 r[k+1:n,k] = temp
1926 sdiag[j] = r[j,j]
1927 r[j,j] = x[j]
1928
1929
1930
1931 nsing = n
1932 wh = (nonzero(sdiag == 0))[0]
1933 if (len(wh) > 0):
1934 nsing = wh[0]
1935 wa[nsing:] = 0
1936
1937 if (nsing >= 1):
1938 wa[nsing-1] = wa[nsing-1]/sdiag[nsing-1]
1939
1940 for j in range(nsing-2,-1,-1):
1941 sum0 = sum(r[j+1:nsing,j]*wa[j+1:nsing])
1942 wa[j] = (wa[j]-sum0)/sdiag[j]
1943
1944
1945 put(x, ipvt, wa)
1946 return(r, x, sdiag)
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045 - def lmpar(self, r, ipvt, diag, qtb, delta, x, sdiag, par=None):
2046
2047 if (self.debug): print 'Entering lmpar...'
2048 dwarf = self.machar.minnum
2049 machep = self.machar.machep
2050 sz = shape(r)
2051 m = sz[0]
2052 n = sz[1]
2053
2054
2055
2056 nsing = n
2057 wa1 = qtb.copy()
2058 rthresh = numpy.max(numpy.abs(diagonal(r))) * machep
2059 wh = (nonzero(numpy.abs(diagonal(r)) < rthresh))[0]
2060 if len(wh) > 0:
2061 nsing = wh[0]
2062 wa1[wh[0]:] = 0
2063 if nsing >= 1:
2064
2065 for j in range(nsing-1,-1,-1):
2066 wa1[j] = wa1[j]/r[j,j]
2067 if (j-1 >= 0):
2068 wa1[0:j] = wa1[0:j] - r[0:j,j]*wa1[j]
2069
2070
2071 put(x, ipvt, wa1)
2072
2073
2074
2075 iter = 0
2076 wa2 = diag * x
2077 dxnorm = self.enorm(wa2)
2078 fp = dxnorm - delta
2079 if (fp <= 0.1*delta):
2080 return[r, 0., x, sdiag]
2081
2082
2083
2084
2085
2086 parl = 0.
2087 if nsing >= n:
2088 wa1 = take(diag, ipvt)*take(wa2, ipvt)/dxnorm
2089 wa1[0] = wa1[0] / r[0,0]
2090 for j in range(1,n):
2091 sum0 = sum(r[0:j,j]*wa1[0:j])
2092 wa1[j] = (wa1[j] - sum0)/r[j,j]
2093
2094 temp = self.enorm(wa1)
2095 parl = ((fp/delta)/temp)/temp
2096
2097
2098 for j in range(n):
2099 sum0 = sum(r[0:j+1,j]*qtb[0:j+1])
2100 wa1[j] = sum0/diag[ipvt[j]]
2101 gnorm = self.enorm(wa1)
2102 paru = gnorm/delta
2103 if paru == 0: paru = dwarf/numpy.min([delta,0.1])
2104
2105
2106
2107
2108 par = numpy.max([par,parl])
2109 par = numpy.min([par,paru])
2110 if par == 0: par = gnorm/dxnorm
2111
2112
2113 while(1):
2114 iter = iter + 1
2115
2116
2117 if par == 0: par = numpy.max([dwarf, paru*0.001])
2118 temp = sqrt(par)
2119 wa1 = temp * diag
2120 [r, x, sdiag] = self.qrsolv(r, ipvt, wa1, qtb, sdiag)
2121 wa2 = diag*x
2122 dxnorm = self.enorm(wa2)
2123 temp = fp
2124 fp = dxnorm - delta
2125
2126 if ((abs(fp) <= 0.1*delta) or
2127 ((parl == 0) and (fp <= temp) and (temp < 0)) or
2128 (iter == 10)): break;
2129
2130
2131 wa1 = take(diag, ipvt)*take(wa2, ipvt)/dxnorm
2132
2133 for j in range(n-1):
2134 wa1[j] = wa1[j]/sdiag[j]
2135 wa1[j+1:n] = wa1[j+1:n] - r[j+1:n,j]*wa1[j]
2136 wa1[n-1] = wa1[n-1]/sdiag[n-1]
2137
2138 temp = self.enorm(wa1)
2139 parc = ((fp/delta)/temp)/temp
2140
2141
2142 if fp > 0: parl = numpy.max([parl,par])
2143 if fp < 0: paru = numpy.min([paru,par])
2144
2145
2146 par = numpy.max([parl, par+parc])
2147
2148
2149
2150
2151 return[r, par, x, sdiag]
2152
2153
2154
2155 - def tie(self, p, ptied=None):
2156 if (self.debug): print 'Entering tie...'
2157 if (ptied == None): return
2158 for i in range(len(ptied)):
2159 if ptied[i] == '': continue
2160 cmd = 'p[' + str(i) + '] = ' + ptied[i]
2161 exec(cmd)
2162 return(p)
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232 - def calc_covar(self, rr, ipvt=None, tol=1.e-14):
2233
2234 if (self.debug): print 'Entering calc_covar...'
2235 if rank(rr) != 2:
2236 print 'ERROR: r must be a two-dimensional matrix'
2237 return(-1)
2238 s = shape(rr)
2239 n = s[0]
2240 if s[0] != s[1]:
2241 print 'ERROR: r must be a square matrix'
2242 return(-1)
2243
2244 if (ipvt == None): ipvt = arange(n)
2245 r = rr.copy()
2246 r.shape = [n,n]
2247
2248
2249 l = -1
2250 tolr = tol * abs(r[0,0])
2251 for k in range(n):
2252 if (abs(r[k,k]) <= tolr): break
2253 r[k,k] = 1./r[k,k]
2254 for j in range(k):
2255 temp = r[k,k] * r[j,k]
2256 r[j,k] = 0.
2257 r[0:j+1,k] = r[0:j+1,k] - temp*r[0:j+1,j]
2258 l = k
2259
2260
2261
2262 if l >= 0:
2263 for k in range(l+1):
2264 for j in range(k):
2265 temp = r[j,k]
2266 r[0:j+1,j] = r[0:j+1,j] + temp*r[0:j+1,k]
2267 temp = r[k,k]
2268 r[0:k+1,k] = temp * r[0:k+1,k]
2269
2270
2271
2272 wa = repeat([r[0,0]], n)
2273 for j in range(n):
2274 jj = ipvt[j]
2275 sing = j > l
2276 for i in range(j+1):
2277 if sing: r[i,j] = 0.
2278 ii = ipvt[i]
2279 if ii > jj: r[ii,jj] = r[i,j]
2280 if ii < jj: r[jj,ii] = r[i,j]
2281 wa[jj] = r[j,j]
2282
2283
2284 for j in range(n):
2285 r[0:j+1,j] = r[j,0:j+1]
2286 r[j,j] = wa[j]
2287
2288 return(r)
2289
2292 if (double == 0):
2293 self.machep = 1.19209e-007
2294 self.maxnum = 3.40282e+038
2295 self.minnum = 1.17549e-038
2296 self.maxgam = 171.624376956302725
2297 else:
2298 self.machep = 2.2204460e-016
2299 self.maxnum = 1.7976931e+308
2300 self.minnum = 2.2250739e-308
2301 self.maxgam = 171.624376956302725
2302
2303 self.maxlog = log(self.maxnum)
2304 self.minlog = log(self.minnum)
2305 self.rdwarf = sqrt(self.minnum*1.5) * 10
2306 self.rgiant = sqrt(self.maxnum) * 0.1
2307