hide long namesshow long names
hide short namesshow short names
Integer type:  int32  int64  nag_int  show int32  show int32  show int64  show int64  show nag_int  show nag_int

PDF version (NAG web site, 64-bit version, 64-bit version)
Chapter Contents
Chapter Introduction
NAG Toolbox

NAG Toolbox: nag_opt_lsq_check_hessian (e04yb)

 Contents

    1  Purpose
    2  Syntax
    7  Accuracy
    9  Example

Purpose

nag_opt_lsq_check_hessian (e04yb) checks that a user-supplied function for evaluating the second derivative term of the Hessian matrix of a sum of squares is consistent with a user-supplied function for calculating the corresponding first derivatives.

Syntax

[fvec, fjac, b, user, ifail] = e04yb(m, lsqfun, lsqhes, x, lb, 'n', n, 'user', user)
[fvec, fjac, b, user, ifail] = nag_opt_lsq_check_hessian(m, lsqfun, lsqhes, x, lb, 'n', n, 'user', user)
Note: the interface to this routine has changed since earlier releases of the toolbox:
At Mark 24: w and iw were removed from the interface; user was added to the interface

Description

Functions for minimizing a sum of squares of m nonlinear functions (or ‘residuals’), fix1,x2,,xn, for i=1,2,,m and mn, may require you to supply a function to evaluate the quantities
bjk=i=1mfi 2fi xjxk  
for j=1,2,,n and k=1,2,,j. nag_opt_lsq_check_hessian (e04yb) is designed to check the bjk calculated by such user-supplied functions. As well as the function to be checked (lsqhes), you must supply a function (lsqfun) to evaluate the fi and their first derivatives, and a point x=x1,x2,,xnT at which the checks will be made. Note that nag_opt_lsq_check_hessian (e04yb) checks functions of the form required by nag_opt_lsq_uncon_mod_deriv2_comp (e04he). nag_opt_lsq_check_hessian (e04yb) is essentially identical to CHKLSH in the NPL Algorithms Library.
nag_opt_lsq_check_hessian (e04yb) first calls user-supplied functions lsqfun and lsqhes to evaluate the first derivatives and the bjk at x. Let J denote the m by n matrix of first derivatives of the residuals. The Hessian matrix of the sum of squares,
G=JTJ+B,  
is calculated and projected onto two orthogonal vectors y and z to give the scalars yTGy and zTGz respectively. The same projections of the Hessian matrix are also estimated by finite differences, giving
p=yTgx+hy-yTgx/h  and q=zTgx+hz-zTgx/h  
respectively, where g denotes the gradient vector of the sum of squares at the point in brackets and h is a small positive scalar. If the relative difference between p and yTGy or between q and zTGz is judged too large, an error indicator is set.

References

None.

Parameters

Compulsory Input Parameters

1:     m int64int32nag_int scalar
The number m of residuals, fix, and the number n of variables, xj.
Constraint: 1nm.
2:     lsqfun – function handle or string containing name of m-file
lsqfun must calculate the vector of values fix and their first derivatives fi xj  at any point x. (nag_opt_lsq_uncon_mod_deriv2_comp (e04he) gives you the option of resetting arguments of lsqfun to cause the minimization process to terminate immediately. nag_opt_lsq_check_hessian (e04yb) will also terminate immediately, without finishing the checking process, if the argument in question is reset.)
[iflag, fvec, fjac, user] = lsqfun(iflag, m, n, xc, ldfjac, user)

Input Parameters

1:     iflag int64int32nag_int scalar
To lsqfun, iflag will be set to 2.
2:     m int64int32nag_int scalar
The numbers m of residuals.
3:     n int64int32nag_int scalar
The numbers n of variables.
4:     xcn – double array
The point x at which the values of the fi and the fi xj  are required.
5:     ldfjac int64int32nag_int scalar
The first dimension of the array fjac.
6:     user – Any MATLAB object
lsqfun is called from nag_opt_lsq_check_hessian (e04yb) with the object supplied to nag_opt_lsq_check_hessian (e04yb).

Output Parameters

1:     iflag int64int32nag_int scalar
If you reset iflag to some negative number in lsqfun and return control to nag_opt_lsq_check_hessian (e04yb), the function will terminate immediately with ifail set to your setting of iflag.
2:     fvecm – double array
Unless iflag is reset to a negative number, fveci must contain the value of fi at the point x, for i=1,2,,m.
3:     fjacldfjacn – double array
Unless iflag is reset to a negative number, fjacij must contain the value of fi xj at the point x, for i=1,2,,m and j=1,2,,n.
4:     user – Any MATLAB object
Note:  nag_opt_lsq_check_deriv (e04ya) should be used to check the first derivatives calculated by lsqfun before nag_opt_lsq_check_hessian (e04yb) is used to check the bjk since nag_opt_lsq_check_hessian (e04yb) assumes that the first derivatives are correct.
3:     lsqhes – function handle or string containing name of m-file
lsqhes must calculate the elements of the symmetric matrix
Bx=i=1mfixGix,  
at any point x, where Gix is the Hessian matrix of fix. (As with lsqfun, a argument can be set to cause immediate termination.)
[iflag, b, user] = lsqhes(iflag, m, n, fvec, xc, lb, user)

Input Parameters

1:     iflag int64int32nag_int scalar
Is set to a non-negative number.
2:     m int64int32nag_int scalar
The numbers m of residuals.
3:     n int64int32nag_int scalar
The numbers n of variables.
4:     fvecm – double array
The value of the residual fi at the point x, for i=1,2,,m, so that the values of the fi can be used in the calculation of the elements of b.
5:     xcn – double array
The point x at which the elements of b are to be evaluated.
6:     lb int64int32nag_int scalar
Gives the length of the array b.
7:     user – Any MATLAB object
lsqhes is called from nag_opt_lsq_check_hessian (e04yb) with the object supplied to nag_opt_lsq_check_hessian (e04yb).

Output Parameters

1:     iflag int64int32nag_int scalar
If lsqhes resets iflag to some negative number, nag_opt_lsq_check_hessian (e04yb) will terminate immediately, with ifail set to your setting of iflag.
2:     blb – double array
Unless iflag is reset to a negative number b must contain the lower triangle of the matrix Bx, evaluated at the point in xc, stored by rows. (The upper triangle is not needed because the matrix is symmetric.) More precisely, bjj-1/2+k must contain i=1mfi 2fi xjxk evaluated at the point x, for j=1,2,,n and k=1,2,,j.
3:     user – Any MATLAB object
4:     xn – double array
xj, for j=1,2,,n, must be set to the coordinates of a suitable point at which to check the bjk calculated by lsqhes. ‘Obvious’ settings, such as 0 or 1, should not be used since, at such particular points, incorrect terms may take correct values (particularly zero), so that errors could go undetected. For a similar reason, it is preferable that no two elements of x should have the same value.
5:     lb int64int32nag_int scalar
The dimension of the array b.
Constraint: lbn+1×n/2.

Optional Input Parameters

1:     n int64int32nag_int scalar
Default: For n, the dimension of the array x.
The number m of residuals, fix, and the number n of variables, xj.
Constraint: 1nm.
2:     user – Any MATLAB object
user is not used by nag_opt_lsq_check_hessian (e04yb), but is passed to lsqfun and lsqhes. Note that for large objects it may be more efficient to use a global variable which is accessible from the m-files than to use user.

Output Parameters

1:     fvecm – double array
Unless you set iflag negative in the first call of lsqfun, fveci contains the value of fi at the point supplied by you in x, for i=1,2,,m.
2:     fjacldfjacn – double array
Unless you set iflag negative in the first call of lsqfun, fjacij contains the value of the first derivative fi xj at the point given in x, as calculated by lsqfun, for i=1,2,,m and j=1,2,,n.
3:     blb – double array
Unless you set iflag negative in lsqhes, bj×j-1/2+k contains the value of bjk at the point given in x as calculated by lsqhes, for j=1,2,,n and k=1,2,,j.
4:     user – Any MATLAB object
5:     ifail int64int32nag_int scalar
ifail=0 unless the function detects an error (see Error Indicators and Warnings).

Error Indicators and Warnings

Note: nag_opt_lsq_check_hessian (e04yb) may return useful information for one or more of the following detected errors or warnings.
Errors or warnings detected by the function:

Cases prefixed with W are classified as warnings and do not generate an error of type NAG:error_n. See nag_issue_warnings.

W  ifail<0
A negative value of ifail indicates an exit from nag_opt_lsq_check_hessian (e04yb) because you have set iflag negative in user-supplied functions lsqfun or lsqhes. The setting of ifail will be the same as your setting of iflag. The check on lsqhes will not have been completed.
   ifail=1
On entry,m<n,
orn<1,
orldfjac<m,
orlb<n+1×n/2,
orliw<1,
orlw<5×n+m+m×n+n×n-1/2, if n>1,
orlw<6+2×m, if n=1.
W  ifail=2
You should check carefully the derivation and programming of expressions for the bjk, because it is very unlikely that lsqhes is calculating them correctly.
   ifail=-99
An unexpected error has been triggered by this routine. Please contact NAG.
   ifail=-399
Your licence key may have expired or may not have been installed correctly.
   ifail=-999
Dynamic memory allocation failed.

Accuracy

ifail is set to 2 if
yTGy-phyTGy+1.0 or zTGz-qhzTGz+1.0  
where h is set equal to ε (ε being the machine precision as given by nag_machine_precision (x02aj)) and other quantities are defined as in Description.

Further Comments

nag_opt_lsq_check_hessian (e04yb) calls lsqhes once and lsqfun three times.

Example

Suppose that it is intended to use nag_opt_lsq_uncon_mod_deriv2_comp (e04he) to find least squares estimates of x1,x2 and x3 in the model
y = x1 + t1 x2 t2+ x3 t3  
using the 15 sets of data given in the following table.
y t1 t2 t3 0.14 1.0 15.0 1.0 0.18 2.0 14.0 2.0 0.22 3.0 13.0 3.0 0.25 4.0 12.0 4.0 0.29 5.0 11.0 5.0 0.32 6.0 10.0 6.0 0.35 7.0 9.0 7.0 0.39 8.0 8.0 8.0 0.37 9.0 7.0 7.0 0.58 10.0 6.0 6.0 0.73 11.0 5.0 5.0 0.96 12.0 4.0 4.0 1.34 13.0 3.0 3.0 2.10 14.0 2.0 2.0 4.39 15.0 1.0 1.0  
This example program could be used to check the bjk calculated by lsqhes required. (The call of nag_opt_lsq_check_hessian (e04yb) is preceded by a call of nag_opt_lsq_check_deriv (e04ya) to check lsqfun which calculates the first derivatives.)
function e04yb_example


fprintf('e04yb example results\n\n');

n = 3;
x = [0.19; -1.34;  0.88];
m = int64(15);
y = [0.14, 0.18, 0.22, 0.25, 0.29, 0.32, 0.35, 0.39, 0.37, ...
     0.58, 0.73, 0.96, 1.34, 2.10, 4.39];

t = [1.0, 15.0, 1.0;
     2.0, 14.0, 2.0;
     3.0, 13.0, 3.0;
     4.0, 12.0, 4.0;
     5.0, 11.0, 5.0;
     6.0, 10.0, 6.0;
     7.0,  9.0, 7.0;
     8.0,  8.0, 8.0;
     9.0,  7.0, 7.0;
     10.0, 6.0, 6.0;
     11.0, 5.0, 5.0;
     12.0, 4.0, 4.0;
     13.0, 3.0, 3.0;
     14.0, 2.0, 2.0;
     15.0, 1.0, 1.0];

lb = int64(6);
user = {y; t};

[fvec, fjac, b, user, ifail] = ...
e04yb( ...
       m, @lsqfun, @lsqhes, x, lb, 'user', user);

fprintf('The test point is:\n');
fprintf('%9.5f',x);
fprintf('\n\n1st derivatives are consistent with residual values\n\n');
fprintf('At the test point, lsqfun gives:\n\n');
fprintf('  Residuals         1st derivatives\n');
fprintf('%10.4f  %10.4f%10.4f%10.4f\n',[fvec fjac]');
fprintf('\nand lsqhes gives the lower triangle of the matrix B\n\n');
k = 1;
for i = 1:n
  fprintf('%15.3e',b(k:(k+i-1)));
  fprintf('\n');
  k = k + i;
end



function [iflag, fvecc, fjacc, user] = lsqfun(iflag, m, n, xc, ljc, user)
  y = user{1};
  t = user{2};

  fvecc = zeros(m, 1);
  fjacc = zeros(ljc, n);

  for i = 1:double(m)
    denom = xc(2)*t(i,2) + xc(3)*t(i,3);
    fvecc(i) = xc(1) + t(i,1)/denom - y(i);
    if (iflag ~= 0)
      fjacc(i,1) = 1;
      dummy = -1/(denom*denom);
      fjacc(i,2) = t(i,1)*t(i,2)*dummy;
      fjacc(i,3) = t(i,1)*t(i,3)*dummy;
    end
  end


function [iflag, b, user] = lsqhes(iflag, m, n, fvecc, xc, lb, user)
  t = user{2};
  b = zeros(lb, 1);

  sum22 = 0;
  sum32 = 0;
  sum33 = 0;
  for i = 1:double(m)
     dummy = 2*t(i,1)/(xc(2)*t(i,2)+xc(3)*t(i,3))^3;
     sum22 = sum22 + fvecc(i)*dummy*t(i,2)^2;
     sum32 = sum32 + fvecc(i)*dummy*t(i,2)*t(i,3);
     sum33 = sum33 + fvecc(i)*dummy*t(i,3)^2;
  end
  b(3) = sum22;
  b(5) = sum32;
  b(6) = sum33;
e04yb example results

The test point is:
  0.19000 -1.34000  0.88000

1st derivatives are consistent with residual values

At the test point, lsqfun gives:

  Residuals         1st derivatives
   -0.0020      1.0000   -0.0406   -0.0027
   -0.1076      1.0000   -0.0969   -0.0138
   -0.2330      1.0000   -0.1785   -0.0412
   -0.3785      1.0000   -0.3043   -0.1014
   -0.5836      1.0000   -0.5144   -0.2338
   -0.8689      1.0000   -0.9100   -0.5460
   -1.3464      1.0000   -1.8098   -1.4076
   -2.3739      1.0000   -4.7259   -4.7259
   -2.9750      1.0000   -6.0762   -6.0762
   -4.0132      1.0000   -7.8765   -7.8765
   -5.3226      1.0000  -10.3970  -10.3970
   -7.2917      1.0000  -14.1777  -14.1777
  -10.5703      1.0000  -20.4789  -20.4789
  -17.1274      1.0000  -33.0813  -33.0813
  -36.8087      1.0000  -70.8885  -70.8885

and lsqhes gives the lower triangle of the matrix B

      0.000e+00
      0.000e+00      1.571e+04
      0.000e+00      1.571e+04      1.571e+04

PDF version (NAG web site, 64-bit version, 64-bit version)
Chapter Contents
Chapter Introduction
NAG Toolbox

© The Numerical Algorithms Group Ltd, Oxford, UK. 2009–2015