NAG Library Routine Document
E05JBF
Note: this routine uses optional parameters to define choices in the problem specification and in the details of the algorithm. If you wish to use default
settings for all of the optional parameters, you need only read Sections 1 to 9 of this document. If, however, you wish to reset some or all of the settings please refer to Section 10 for a detailed description of the algorithm, and to Section 11 for a detailed description of the specification of the optional parameters.
1 Purpose
E05JBF is designed to find the global minimum or maximum of an arbitrary function, subject to simple bound-constraints using a multi-level coordinate search method. Derivatives are not required, but convergence is only guaranteed if the objective function is continuous in a neighbourhood of a global optimum. It is not intended for large problems.
The initialization routine
E05JAF must have been called before calling E05JBF.
2 Specification
| SUBROUTINE E05JBF ( | N, OBJFUN, IBOUND, IINIT, BL, BU, SDLIST, LIST, NUMPTS, INITPT, MONIT, X, OBJ, COMM, LCOMM, IUSER, RUSER, IFAIL) |
| INTEGER | N, IBOUND, IINIT, SDLIST, NUMPTS(N), INITPT(N), LCOMM, IUSER(*), IFAIL |
| double precision | BL(N), BU(N), LIST(N,SDLIST), X(N), OBJ, COMM(LCOMM), RUSER(*) |
| EXTERNAL | OBJFUN, MONIT |
E05JAF must be called before calling E05JBF, or any of the option-setting or option-getting
routines
E05JCF,
E05JDF,
E05JEF,
E05JFF,
E05JGF,
E05JHF,
E05JJF,
E05JKF or
E05JLF.
The contents of
the array
COMM
and the value of the parameter
N must not be altered between calls of the
routines
E05JAF, E05JBF,
E05JCF,
E05JDF,
E05JEF,
E05JFF,
E05JGF,
E05JHF,
E05JJF,
E05JKF or
E05JLF.
3 Description
E05JBF is designed to solve modestly sized global optimization problems having simple bound-constraints only; it finds the global optimum of a nonlinear function subject to a set of bound constraints on the variables. Without loss of generality, the problem is assumed to be stated in the following form:
where
Fx (the
objective function) is a nonlinear scalar function (assumed to be continuous in a neighbourhood of a global minimum), and the bound vectors are elements of
R-n, where
R- denotes the extended reals
R∪-∞,∞. Relational operators between vectors are interpreted elementwise.
The optional parameter Maximize should be set if you wish to solve maximization, rather than minimization, problems.
If certain bounds are not present, the associated elements of l or u can be set to special values that will be treated as -∞ or +∞. See the description of the optional parameter Infinite Bound Size. Phrases in this document containing terms like ‘unbounded values’ should be understood to be taken relative to this optional parameter.
A typical excerpt from a routine calling E05JBF is:
CALL E05JAF (N, COMM, LCOMM, ...)
CALL E05JDF (OPTSTR, COMM, LCOMM, ...)
CALL E05JBF (N, OBJFUN, ...)
where
E05JDF sets the optional parameter and value specified in
OPTSTR.
The initialization routine
E05JAF does not need to be called before each invocation of E05JBF, so long as the problem size,
N, remains unchanged. You should be aware that a call to the initialization routine will reset each optional parameter to its default value, and, if you are using repeatable randomized initialization lists (see the description of the parameter
IINIT), the random seed stored in
the array
COMM
will be destroyed.
You must supply a subroutine that evaluates Fx; derivatives are not required.
The method used by E05JBF is based on MCS, the Multi-level Coordinate Search method described in
Huyer and Neumaier (1999), and the algorithm it uses is described in detail in
Section 10.
4 References
Huyer W and Neumaier A (1999) Global Optimization by Multi-level Coordinate Search
Journal of Global Optimization 14 331–355
5 Parameters
- 1: N – INTEGERInput
-
On entry:
n, the number of variables.
Constraint:
N>0.
- 2: OBJFUN – SUBROUTINE, supplied by the user.External Procedure
-
OBJFUN must evaluate the objective function
Fx for a specified
n-vector
x.
The specification of
OBJFUN is:
| SUBROUTINE OBJFUN ( | N, X, F, NSTATE, IUSER, RUSER, INFORM) |
| INTEGER | N, NSTATE, IUSER(*), INFORM |
| double precision | X(N), F, RUSER(*) |
- 1: N – INTEGERInput
-
On entry: n, the number of variables.
- 2: X(N) – double precision arrayInput
-
On entry: x, the vector at which the objective function is to be evaluated.
- 3: F – double precisionOutput
-
On exit: must be set to the value of the objective function at
x, unless you have specified termination of the current problem using
INFORM.
- 4: NSTATE – INTEGERInput
-
On entry: if
NSTATE=1 then E05JBF is calling
OBJFUN for the first time. This parameter setting allows you to save computation time if certain data must be read or calculated only once.
- 5: IUSER(*) – INTEGER arrayUser Workspace
- 6: RUSER(*) – double precision arrayUser Workspace
-
OBJFUN is called from E05JBF with the parameters
IUSER and
RUSER as supplied to E05JBF. You are free to use the arrays
IUSER and
RUSER to supply information to
OBJFUN.
- 7: INFORM – INTEGEROutput
-
On exit: must be set to a value describing the action to be taken by the solver on return from
OBJFUN. Specifically, if the value is negative the solution of the current problem will terminate immediately; otherwise, computations will continue.
OBJFUN must be declared as EXTERNAL in the (sub)program from which E05JBF is called. Parameters denoted as
Input must
not be changed by this procedure.
- 3: IBOUND – INTEGERInput
-
On entry: indicates whether the facility for dealing with bounds of special forms is to be used.
IBOUND must be set to one of the following values.
- IBOUND=0
- You will supply l and u individually.
- IBOUND=1
- There are no bounds on x.
- IBOUND=2
- There are semi-infinite bounds 0≤x.
- IBOUND=3
- There are constant bounds l=ℓ1 and u=u1.
Constraint:
IBOUND=0, 1, 2 or 3.
- 4: IINIT – INTEGERInput
-
On entry: selects which initialization method to use.
- IINIT=0
- Simple initialization (boundary and midpoint), with
NUMPTSi=3
,
INITPTi=2
and
LISTij
=
BLi,
BLi
+
BUi
/
2
,BUi
,
for
i=1,2,…,N
and j=1,2,3.
- IINIT=1
- Simple initialization (off-boundary and midpoint), with
NUMPTSi=3
,
INITPTi=2
and
LISTij
=
5BLi+
BUi
/
6
,
BLi
+
BUi
/
2
,
BLi
+
5BUi
/
6
,
for
i=1,2,…,N
and j=1,2,3.
- IINIT=2
- Initialization using linesearches.
- IINIT=3
- You are providing your own initialization list.
- IINIT=4
- Generate a random initialization list.
For more information on methods
IINIT=2,
3 or
4 see
Section 10.1.
If ‘infinite’ values (as determined by the value of the optional parameter Infinite Bound Size) are detected by E05JBF when you are using a simple initialization method (IINIT=0 or 1), a safeguarded initialization procedure will be attempted, to avoid overflow.
Suggested value:
IINIT=0
Constraint:
IINIT=0, 1, 2, 3 or 4.
- 5: BL(N) – double precision arrayInput/Output
- 6: BU(N) – double precision arrayInput/Output
-
On entry:
BL is
l, the array of lower bounds.
BU is
u, the array of upper bounds.
If IBOUND=0, you must set BLi to ℓi and BUi to ui, for i=1,2,…,N. If a particular xi is to be unbounded below, the corresponding BLi should be set to -infbnd, where infbnd is the value of the optional parameter Infinite Bound Size. Similarly, if a particular xi is to be unbounded above, the corresponding BUi should be set to infbnd.
If
IBOUND=1 or
2, arrays
BL and
BU need not be set on input.
If
IBOUND=3, you must set
BL1 to
ℓ1 and
BU1 to
u1. The remaining elements of
BL and
BU will then be populated by these initial values.
Fixing a variable (that is, setting BLi=BUi for some i) is not permitted by E05JBF.
On exit: unless
IFAIL=1 or
2 on exit,
BL and
BU are the actual arrays of bounds used by E05JBF.
Constraints:
- if IBOUND=0, BLi<BUi;
- if IBOUND=3, BL1<BU1;
for
i=1,2,…,N.
- 7: SDLIST – INTEGERInput
-
On entry: the second dimension of the array
LIST
SDLIST is,
at least, the maximum over
i of the number of points in coordinate
i at which to split according to the initialization list
LIST; that is,
SDLIST≥maxiNUMPTSi.
Internally, E05JBF uses
LIST to determine sets of points along each coordinate direction to which it fits quadratic interpolants. Since fitting a quadratic requires at least three distinct points, this puts a lower bound on
SDLIST. Furthermore, in the case of initialization by linesearches (
IINIT=2) internal storage considerations require that
SDLIST be at least
192, but not all of this space may be used.
Constraints:
- if IINIT≠2, SDLIST≥3;
- if IINIT=2, SDLIST≥192;
- if IINIT=3, SDLIST≥maxiNUMPTSi.
- 8: LIST(N,SDLIST) – double precision arrayInput/Output
- 9: NUMPTS(N) – INTEGER arrayInput/Output
- 10: INITPT(N) – INTEGER arrayInput/Output
-
On entry: these three parameters need not be set on entry if you wish to use one of the preset initialization methods (
IINIT≠3).
LIST is the ‘initialization list’: whenever a sub-box in the algorithm is split for the first time (either during the
initialization procedure or later), for each coordinate
i the split is done at the values
LISTi1:NUMPTSi, as well as at some adaptively chosen intermediate points.
You must designate a point stored in
LIST that you wish E05JBF to consider as an ‘initial point’ for the purposes of the splitting procedure. Call this initial point
x*. The coordinates of
x* correspond to a set of indices
Ji, for
i=1,2,…,n, such that
xi* is stored in
LISTiJi, for
i=1,2,…,n. You must set
INITPTi=Ji, for
i=1,2,…,n.
The array sections
LISTi1:NUMPTSi, for
i=1,2,…,N, must be in ascending order with each entry being distinct. In this context, ‘distinct’ should be taken to mean relative to the safe-range parameter (see
X02AMF).
On exit: unless
IFAIL=1,
2 or
-999 on exit, the actual initialization data used by E05JBF. If you wish to monitor the contents of
LIST you are advised to do so solely through
MONIT, not through the output value here.
Constraints:
- 1≤INITPTi≤SDLIST, for i=1,2,…,N;
- LISTi1:NUMPTSi is in ascending order with each entry being distinct, for i=1,2,…,N;
- BLi≤LISTij≤BUi, for i=1,2,…,N and j=1,2,…,NUMPTSi;
- NUMPTSi≤SDLIST, for i=1,2,…,N;
- NUMPTSi≥3, for i=1,2,…,N.
- 11: MONIT – SUBROUTINE, supplied by the NAG Library or the user.External Procedure
-
MONIT may be used to monitor the optimization process. It is invoked upon every successful completion of the procedure in which a sub-box is considered for splitting. It will also be called just before E05JBF exits if that splitting procedure was not successful.
If no monitoring is required,
MONIT may be
the dummy monitoring routine E05JBK supplied by the NAG Library.
The specification of
MONIT is:
| SUBROUTINE MONIT ( | N, NCALL, XBEST, ICOUNT, NINIT, LIST, NUMPTS, INITPT, NBASKT, XBASKT, BOXL, BOXU, NSTATE, IUSER, RUSER, INFORM) |
| INTEGER | N, NCALL, ICOUNT(6), NINIT, NUMPTS(N), INITPT(N), NBASKT, NSTATE, IUSER(*), INFORM |
| double precision | XBEST(N), LIST(N,NINIT), XBASKT(N,NBASKT), BOXL(N), BOXU(N), RUSER(*) |
- 1: N – INTEGERInput
-
On entry: n, the number of variables.
- 2: NCALL – INTEGERInput
-
On entry: the cumulative number of calls to
OBJFUN.
- 3: XBEST(N) – double precision arrayInput
-
On entry: the current best point.
- 4: ICOUNT(6) – INTEGER arrayInput
-
On entry: an array of counters.
- ICOUNT1
- nboxes, the current number of sub-boxes.
- ICOUNT2
- ncloc, the cumulative number of calls to OBJFUN made in local searches.
- ICOUNT3
- nloc, the cumulative number of points used as start points for local searches.
- ICOUNT4
- nsweep, the cumulative number of sweeps through levels.
- ICOUNT5
- m, the cumulative number of splits by initialization list.
- ICOUNT6
- s, the current lowest level containing non-split boxes.
- 5: NINIT – INTEGERInput
-
On entry: the maximum over
i of the number of points in coordinate
i at which to split according to the initialization list
LIST. See also the description of the parameter
NUMPTS.
- 6: LIST(N,NINIT) – double precision arrayInput
-
On entry: the initialization list.
- 7: NUMPTS(N) – INTEGER arrayInput
-
On entry: the number of points in each coordinate at which to split according to the initialization list
LIST.
- 8: INITPT(N) – INTEGER arrayInput
-
On entry: a pointer to the ‘initial point’ in
LIST. Element
INITPTi is the column index in
LIST of the
ith coordinate of the initial point.
- 9: NBASKT – INTEGERInput
-
On entry: the number of points in the ‘shopping basket’
XBASKT.
- 10: XBASKT(N,NBASKT) – double precision arrayInput
-
On entry: the ‘shopping basket’ of candidate minima. The jth candidate minimum has its ith coordinate stored in XBASKTj-1×n+i, for i=1,2,…,n and j=1,2,…,NBASKT
- 11: BOXL(N) – double precision arrayInput
-
On entry: the array of lower bounds of the current search box.
- 12: BOXU(N) – double precision arrayInput
-
On entry: the array of upper bounds of the current search box.
- 13: NSTATE – INTEGERInput
-
On entry: is set by E05JBF to indicate at what stage of the minimization
MONIT was called.
- NSTATE=1
- This is the first time that MONIT has been called.
- NSTATE=-1
- This is the last time MONIT will be called.
- NSTATE=0
- This is the first and last time MONIT will be called.
- 14: IUSER(*) – INTEGER arrayUser Workspace
- 15: RUSER(*) – double precision arrayUser Workspace
-
MONIT is called from E05JBF with the parameters
IUSER and
RUSER as supplied to E05JBF. You are free to use the arrays
IUSER and
RUSER to supply information to
MONIT.
- 16: INFORM – INTEGEROutput
-
On exit: must be set to a value describing the action to be taken by the solver on return from
MONIT. Specifically, if the value is negative the solution of the current problem will terminate immediately; otherwise, computations will continue.
MONIT must be declared as EXTERNAL in the (sub)program from which E05JBF is called. Parameters denoted as
Input must
not be changed by this procedure.
- 12: X(N) – double precision arrayOutput
-
On exit: if
IFAIL=0, contains an estimate of the global optimum (see also
Section 7).
- 13: OBJ – double precisionOutput
-
On exit: if
IFAIL=0, contains the function value at
X.
If you request early termination of E05JBF using
INFORM in
OBJFUN or the analogous
INFORM in
MONIT, there is no guarantee that the function value at
X equals
OBJ.
- 14: COMM(LCOMM) – double precision arrayCommunication Array
On exit:
COMM must not be altered between calls to any of the routines E05JBF,
E05JCF,
E05JDF,
E05JEF,
E05JFF,
E05JGF,
E05JHF,
E05JJF,
E05JKF and
E05JLF.
- 15: LCOMM – INTEGERInput
-
On entry: the dimension of the array
COMM as declared in the (sub)program from which E05JBF is called.
Constraint:
LCOMM≥100.
- 16: IUSER(*) – INTEGER arrayUser Workspace
-
Note: the dimension of the array
IUSER
must be at least
1.
IUSER is not used by E05JBF, but is passed directly to the user-supplied subroutines
OBJFUN and
MONIT. It may be used to pass information to and from those routines.
- 17: RUSER(*) – double precision arrayUser Workspace
-
Note: the dimension of the array
RUSER
must be at least
1.
RUSER is not used by E05JBF, but is passed directly to the user-supplied subroutines
OBJFUN and
MONIT. It may be used to pass information to and from those routines.
- 18: IFAIL – INTEGERInput/Output
On entry:
IFAIL must be set to
0,
-1 or 1. If you are unfamiliar with this parameter you should refer to
Section 3.3 in the Essential Introduction for details.
On exit:
IFAIL=0 unless the routine detects an error (see
Section 6).
For environments where it might be inappropriate to halt program execution when an error is detected, the value
-1 or 1 is recommended. If the output of error messages is undesirable, then the value
1 is recommended. Otherwise, if you are not familiar with this parameter, the recommended value is
0.
When the value -1 or 1 is used it is essential to test the value of IFAIL on exit. E05JBF returns with IFAIL=0 if your termination criterion has been met: either a target value has been found to the required relative error (as determined by the values of the optional parameters Target Objective Value, Target Objective Error and Target Objective Safeguard), or the best function value was static for the number of sweeps through levels given by the optional parameter Static Limit. The latter criterion is the default.
6 Error Indicators and Warnings
If on entry
IFAIL=0 or
-1, explanatory error messages are output on the current error message unit (as defined by
X04AAF).
Errors or warnings detected by the routine:
- IFAIL=1
-
Either the initialization routine
E05JAF has not been called,
LCOMM is less than 100, or the value of
N has changed since the last call to
E05JAF.
- IFAIL=2
-
An input parameter is invalid. If IFAIL=0 or -1 on entry, the output message provides more details of the invalid argument.
- IFAIL=3
-
The initialization list contained infinities.
Either the user-supplied initialization list contained infinite values, as determined by the optional parameter Infinite Bound Size, or a finite initialization list could not be computed internally. In the latter case you should consider reformulating the bounds on the problem, try providing your own initialization list, use the randomization option (IINIT=4) or vary the value of Infinite Bound Size.
- IFAIL=4
-
The division procedure completed but your target value could not be reached.
Despite every sub-box being processed smax times (where smax is the value of the optional parameter Splits Limit), the target value you provided via the optional parameter Target Objective Value could not be found to the tolerances given in the optional parameters Target Objective Error and Target Objective Safeguard. You could try increasing Splits Limit or the objective tolerances.
- IFAIL=5
-
The function evaluations limit was exceeded.
Approximately nf function calls (where nf is the value of the optional parameter Function Evaluations Limit) have been made without your chosen termination criterion being satisfied.
- IFAIL=6
-
You terminated the solver.
You indicated that you wished to halt solution of the current problem by setting
INFORM in
OBJFUN or
INFORM in
MONIT to a negative value on exit. If
IFAIL=0 or
-1 on entry to E05JBF, the output message provides more details of where the termination was requested.
- IFAIL=7
-
No further progress could be made on your problem. Try rescaling the objective function, relaxing the bounds, or using a different initialization method.
- IFAIL=-999
-
Internal memory allocation failed.
7 Accuracy
If
IFAIL=0 on exit, then the vector returned in the array
X is an estimate of the solution
x whose function value satisfies your termination criterion: the function value was static for
Static Limit sweeps through levels, or
where
objval is the value of the optional parameter
Target Objective Value,
objerr is the value of the optional parameter
Target Objective Error, and
objsfg is the value of the optional parameter
Target Objective Safeguard.
8 Further Comments
For each invocation of E05JBF, local workspace arrays of fixed length are allocated internally. The total size of these arrays amounts to
10N+smax-1 INTEGER elements, where
smax is the value of the optional parameter
Splits Limit, and
2SDLIST+18N+3N2+1 double precision elements. In addition, if you are using randomized initialization lists (see the description of the parameter
IINIT), a further
21 INTEGER elements are allocated internally.
In order to keep track of the regions of the search space that have been visited while looking for a global optimum, E05JBF internally allocates arrays of increasing sizes depending on the difficulty of the problem. Two of the main factors that govern the amount allocated are the number of sub-boxes (call this quantity
nboxes) and the number of points in the ‘shopping basket’ (the parameter
NBASKT on entry to
MONIT). Safe, pessimistic upper bounds on these two quantities are so large as to be impractical. In fact, the worst-case number of sub-boxes for even the most simple initialization list (when
NINIT=3 on entry to
MONIT) grows like
nn as
n, the number of variables, increases. Thus E05JBF does not attempt to estimate in advance the final values of
nboxes or
NBASKT for a given problem. There are a total of
5 INTEGER arrays and
4+N+NINIT double precision arrays whose lengths depend on
nboxes, and there are a total of
2 INTEGER arrays and
3+N double precision arrays whose lengths depend on
NBASKT. E05JBF makes a fixed initial guess that the maximum number of sub-boxes required will be
10000 and that the maximum number of points in the ‘shopping basket’ will be
1000. If ever a greater amount of sub-boxes or more room in the ‘shopping basket’ is required, E05JBF performs reallocation, usually doubling the size of the inadequately-sized arrays. Clearly this process requires periods where the original array and its extension exist in memory simultaneously, so that the data within can be copied, which compounds the complexity of E05JBF's memory usage. It is possible (although not likely) that if your problem is particularly difficult to solve, or of a large size (hundreds of variables), you may run out of memory.
One array that could be dynamically resized by E05JBF is the ‘shopping basket’ (
XBASKT on entry to
MONIT). If the initial attempt to allocate
1000N double precisions for this array fails,
MONIT will not be called on exit from E05JBF.
E05JBF performs better if your problem is well-scaled. It is worth trying (by guesswork perhaps) to rescale the problem if necessary, as sensible scaling will reduce the difficulty of the optimization problem, so that E05JBF will take less computer time.
9 Example
This example finds the global minimum of the ‘peaks’ function in two dimensions
on the box
-3,3
×
-3,3
.
The function F has several local minima and one global minimum in the given box. The global minimum is approximately located at
0.23,-1.63
, where the function value is approximately -6.55.
We use default values for all the optional parameters, and we instruct E05JBF to use the simple initialization list corresponding to
IINIT=0. In particular, this will set for us the initial point
0,0
(see
Section 9.3).
9.1 Program Text
Program Text (e05jbfe.f)
9.2 Program Data
Program Data (e05jbfe.d)
9.3 Program Results
Program Results (e05jbfe.r)
Note: the remainder of this document is intended for more advanced users. Section 10 contains a detailed description of the algorithm. This information may be needed in order to understand Section 11, which describes the optional parameters that can be set by calls to E05JCF, E05JDF, E05JEF, E05JFF and/or E05JGF.
10 Algorithmic Details
Here we summarize the main features of the MCS algorithm used in E05JBF, and we introduce some terminology used in the description of the subroutine and its arguments. The MCS algorithm is fully described in
Huyer and Neumaier (1999).
10.1 Initialization and Sweeps
Each sub-box is determined by a basepoint x and an opposite point y. We denote such a sub-box by Bx,y. The basepoint is allowed to belong to more than one sub-box, is usually a boundary point, and is often a vertex.
An
initialization procedure produces an initial set of sub-boxes. Whenever a sub-box is split along a coordinate
i for the first time (in the initialization procedure or later), the splitting is done at three or more user-defined values
xijj at which the objective function is sampled, and at some adaptively chosen intermediate points. At least four children are generated. More precisely, we assume that we are given
and a vector
p that, for each
i, locates within
xijj the
ith coordinate of an
initial point x0; that is, if
xi0=xij for some
j=1,2,…,Li, then
pi=j. A good guess for the global optimum can be used as
x0.
The initialization points and the vectors
l and
p are collectively called the
initialization list (and sometimes we will refer to just the initialization points as ‘the initialization list’, whenever this causes no confusion). The initialization data may be input by you, or they can be set to sensible default values by E05JBF: if you provide them yourself,
LISTij should contain
xij,
NUMPTSi should contain
Li, and
INITPTi should contain
pi, for
i=1,2,…,n and
j=1,2,…,Li; if you wish E05JBF to use one of its preset initialization methods, you could choose one of two simple, three-point methods (see
Figure 1). If the list generated by one of these methods contains infinite values, attempts are made to generate a safeguarded list using the function
subintx,y (which is also used during the splitting procedure, and is described in
Section 10.2). If infinite values persist, E05JBF exits with
IFAIL=3. There is also the option to generate an initialization list with the aid of linesearches (by setting
IINIT=2). Starting with the absolutely smallest point in the root box, linesearches are made along each coordinate. For each coordinate, the local minimizers found by the linesearches are put into the initialization list. If there were fewer than three minimizers, they are augmented by close-by values. The final preset initialization option (
IINIT=4) generates a randomized list, so that independent multiple runs may be made if you suspect a global optimum has not been found. Each call to the initialization routine
E05JAF resets the initial-state vector for the Wichmann–Hill base-generator that is used. Depending on whether you set the optional parameter
Repeatability to ‘ON’ or ‘OFF’, the random seed is initialized to give a repeatable or non-repeatable sequence. Then, a random integer between
3 and
SDLIST is selected, which is then used to determine the number of points to be generated in each coordinate; that is,
NUMPTS becomes a constant vector, set to this value. The components of
LIST are then generated, from a uniform distribution on the root box if the box is finite, or else in a safeguarded fashion if any bound is infinite. The array
INITPT is set to point to the best point in
LIST.
Given an initialization list (preset or otherwise), E05JBF evaluates
F at
x0, and sets the initial estimate of the global minimum,
x*, to
x0. Then, for
i=1,2,…,n, the objective function
F is evaluated at
Li-1 points that agree with
x* in all but the
ith coordinate. We obtain pairs
x^
j
,fij
, for
j=1,2,…,Li
, with:
x*=x^j1, say; with, for
j≠j1,
and with
The point having the smallest function value is renamed x* and the procedure is repeated with the next coordinate.
Once E05JBF has a full set of initialization points and function values, it can generate an initial set of sub-boxes. Recall that the
root box is
Bx,y=l,u, having basepoint
x=x0. The opposite point
y is a corner of
l,u farthest away from
x, in some sense. The point
x need not be a vertex of
l,u, and
y is entitled to have infinite coordinates. We loop over each coordinate
i, splitting the current box along coordinate
i into
2Li-2,
2Li-1 or
2Li subintervals with exactly one of the
x^ij as endpoints, depending on whether two, one or none of the
x^ij are on the boundary. Thus, as well as splitting at
x^ij, for
j=1,2,…,Li, we split at additional points
zij, for
j=2,…,Li. These additional
zij are such that
where
q is the golden-section ratio
5-1/2, and the exponent
m takes the value
1 or
2, chosen so that the sub-box with the smaller function value gets the larger fraction of the interval. Each child sub-box gets as basepoint the point obtained from
x* by changing
xi* to the
xij that is a boundary point of the corresponding
ith coordinate interval; this new basepoint therefore has function value
fij. The opposite point is derived from
y by changing
yi to the other end of that interval.
E05JBF can now rank the coordinates based on an estimated variability of F. For each i we compute the union of the ranges of the quadratic interpolant through any three consecutive x^ij, taking the difference between the upper and lower bounds obtained as a measure of the variability of F in coordinate i. A vector π is populated in such a way that coordinate i has the πith highest estimated variability. For tiebreaks, when the x* obtained after splitting coordinate i belongs to two sub-boxes, the one that contains the minimizer of the quadratic models is designated the current sub-box for coordinate i+1.
Boxes are assigned levels in the following manner. The root box is given level 1. When a sub-box of level s is split, the child with the smaller fraction of the golden-section split receives level s+2; all other children receive level s+1. The box with the better function value is given the larger fraction of the splitting interval and the smaller level because then it is more likely to be split again more quickly. We see that after the initialization procedure the first level is empty and the non-split boxes have levels 2,…,n+2, so it is meaningful to choose smax much larger than n. Note that the internal structure of E05JBF demands that smax be at least n+3.
Examples of initializations in two dimensions are given in
Figure 1. In both cases the initial point is
x0=l+u/2; on the left the initialization points are
while on the right the points are
In
Figure 1, basepoints and levels after initialization are displayed. Note that these initialization lists correspond to
IINIT=0 and
IINIT=1, respectively.
Figure 1: Examples of the initialization procedure
After initialization, a series of
sweeps through levels is begun. A sweep is defined by three steps:
| (i) |
scan the list of non-split sub-boxes. Fill a record list b according to bs=0 if there is no box at level s, and with bs pointing to a sub-box with the lowest function value among all sub-boxes with level s otherwise, for 0<s<smax; |
| (ii) |
the sub-box with label bs is a candidate for splitting. If the sub-box is not to be split, according to the rules described in Section 10.2, increase its level by 1 and update bs+1 if necessary. If the sub-box is split, mark it so, insert its children into the list of sub-boxes, and update b if any child with level s′ yields a strict improvement of F over those sub-boxes at level s′; |
| (iii) |
increment s by 1. If s=smax then displaying monitoring information and start a new sweep; else if bs=0 then repeat this step; else display monitoring information and go to the previous step. |
Clearly, each sweep ends after at most smax-1 visits of the third step.
10.2 Splitting
Each sub-box is stored by E05JBF as a set of information about the history of the sub-box: the label of its parent, a label identifying which child of the parent it is, etc. Whenever a sub-box
Bx,y of level
s<smax is a candidate for splitting, as described in
Section 10.1, we recover
x,
y, and the number,
nj, of times coordinate
j has been split in the history of
B. Sub-box
B could be split in one of two ways.
| (i) |
Splitting by rank
If
s
>
2n
minnj+1
, the box is always split. The splitting index is set to a coordinate i such that ni=minnj. |
| (ii) |
Splitting by expected gain
If
s
≤
2n
minnj+1
, the sub-box could be split along a coordinate where a maximal gain in function value is expected. This gain is estimated according to a local separable quadratic model obtained by fitting to 2n+1 function values. If the expected gain is too small the sub-box is not split at all, and its level is increased by 1. |
Eventually, a sub-box that is not eligible for splitting by expected gain will reach level
2n
minnj+1
+1
and then be split by rank, as long as smax is large enough. As smax→∞, the rule for splitting by rank ensures that each coordinate is split arbitrarily often.
Before describing the details of each splitting method, we introduce the procedure for correctly handling splitting at adaptive points and for dealing with unbounded intervals. Suppose we want to split the
ith coordinate interval
▯xi,yi, where we define
▯xi,yi=minxi,yi,maxxi,yi, for
xi∈R and
yi∈R-, and where
x is the basepoint of the sub-box being considered. The descendants of the sub-box should shrink sufficiently fast, so we should not split too close to
xi. Moreover, if
yi is large we want the new
splitting value to
not be too large, so we force it to belong to some smaller interval
▯ξ′,ξ′′, determined by
where the function
subint is defined by
10.2.1 Splitting by rank
Consider a sub-box B with level
s
>
2n
minnj+1
. Although the sub-box has reached a high level, there is at least one coordinate along which it has not been split very often. Among the i such that
ni
=
minnj
for B, select the splitting index to be the coordinate with the lowest πi (and hence highest variability rank). ‘Splitting by rank’ refers to the ranking of the coordinates by ni and πi.
If
ni=0, so that
B has never been split along coordinate
i, the splitting is done according to the initialization list and the adaptively chosen golden-section split points, as described in
Section 10.1. Also as covered there, new basepoints and opposite points are generated. The children having the smaller fraction of the golden-section split (that is, those with larger function values) are given level
mins+2,smax. All other children are given level
s+1.
Otherwise, B ranges between xi and yi in the ith coordinate direction. The splitting value is selected to be zi=xi+2subintxi,yi-xi/3; we are not attempting to split based on a large reduction in function value, merely in order to reduce the size of a large interval, so zi may not be optimal. Sub-box B is split at zi and the golden-section split point, producing three parts and requiring only one additional function evaluation, at the point x′ obtained from x by changing the ith coordinate to zi. The child with the smaller fraction of the golden-section split is given level mins+2,smax, while the other two parts are given level s+1. Basepoints are assigned as follows: the basepoint of the first child is taken to be x, and the basepoint of the second and third children is the point x′. Opposite points are obtained by changing yi to the other end of the ith coordinate-interval of the corresponding child.
10.2.2 Splitting by expected gain
When a sub-box B has level
s
≤
2n
minnj+1
, we compute the optimal splitting index and splitting value from a local separable quadratic used as a simple local approximation of the objective function. To fit this curve, for each coordinate we need two additional points and their function values. Such data may be recoverable from the history of B: whenever the ith coordinate was split in the history of B, we obtained values that can be used for the current quadratic interpolation in coordinate i.
We loop over
i; for each coordinate we pursue the history of
B back to the root box, and we take the first two points and function values we find, since these are expected to be closest to the current basepoint
x. If the current coordinate has not yet been split we use the initialization list. Then we generate a local separable model
eξ for
Fξ by interpolation at
x and the
2n additional points just collected:
We define the
expected gain e^i in function value when we evaluate at a new point obtained by changing coordinate
i in the basepoint, for each
i, based on two cases:
| (i) |
ni=0. We compute the expected gain as
Again, we split according to the initialization list, with the new basepoints and opposite points being as before. |
| (ii) |
ni>0. Now, the ith component of our sub-box ranges from xi to yi. Using the quadratic partial correction function
we can approximate the maximal gain expected when changing xi only. We will choose the splitting value from ▯ξ′,ξ′′. We compute
and call zi the minimizer in ▯ξ′,ξ′′.
If the expected best function value fexp satisfies
where fbest is the current best function value (including those function values obtained by local optimization), we expect the sub-box to contain a better point and so we split it, using as splitting index the component with minimal e^i. Equation (1) prevents wasting function calls by avoiding splitting sub-boxes whose basepoints have bad function values. These sub-boxes will eventually be split by rank anyway. We now have a splitting index and a splitting value zi. The sub-box is split at zi as long as zi≠yi, and at the golden-section split point; two or three children are produced. The larger fraction of the golden-section split receives level s+1, while the smaller fraction receives level mins+2,smax. If it is the case that zi≠yi and the third child is larger than the smaller of the two children from the golden-section split, the third child receives level s+1. Otherwise it is given the level mins+2,smax. The basepoint of the first child is set to x, and the basepoint of the second (and third if it exists) is obtained by changing the ith coordinate of x to zi. The opposite points are again derived by changing yi to the other end of the ith coordinate interval of B.
If equation (1) does not hold, we expect no improvement. We do not split, and we increase the level of B by 1. |
10.3 Local Search
The local optimization algorithm used by E05JBF uses linesearches along directions that are determined by minimizing quadratic models, all subject to bound constraints. Triples of vectors are computed using
coordinate searches based on linesearches. These triples are used in
triple search procedures to build local quadratic models for
F. A trust-region-type approach to minimize these models is then carried out, and more information about the coordinate search and the triple search can be found in
Huyer and Neumaier (1999).
The local search starts by looking for better points without being too local, by making a triple search using points found by a coordinate search. This yields a new point and function value, an approximation of the gradient of the objective, and an approximation of the Hessian of the objective. Then the quadratic model for F is minimized over a small box, with the solution to that minimization problem then being used as a linesearch direction to minimize the objective. A measure r is computed to quantify the predictive quality of the quadratic model.
The third stage is the checking of termination criteria. The local search will stop if more than
loclim visits to this part of the local search have occurred, where
loclim is the value of the optional parameter
Local Searches Limit. If that is not the case, it will stop if the limit on function calls has been exceeded (see the description of the optional parameter
Function Evaluations Limit). The final criterion checks if no improvement can be made to the function value, or whether the approximated gradient
g is small, in the sense that
The vector xold is the best point at the start of the current loop in this iterative local-search procedure, the constant loctol is the value of the optional parameter Local Searches Tolerance, f is the objective value at x, and f0 is the smallest function value found by the initialization procedure.
Next, E05JBF attempts to move away from the boundary, if any components of the current point lie there, using linesearches along the offending coordinates. Local searches are terminated if no improvement could be made.
The fifth stage carries out another triple search, but this time it does not use points from a coordinate search, rather points lying within the trust-region box are taken.
The final stage modifies the trust-region box to be bigger or smaller, depending on the quality of the quadratic model, minimizes the new quadratic model on that box, and does a linesearch in the direction of the minimizer. The value of r is updated using the new data, and then we go back to the third stage (checking of termination criteria).
The Hessians of the quadratic models generated by the local search may not be positive definite, so E05JBF uses the general nonlinear optimizer
E04VHF to minimize the models.
11 Optional Parameters
Several optional parameters in E05JBF define choices in the problem specification or the algorithm logic. In order to reduce the number of formal parameters of E05JBF these optional parameters have associated default values that are appropriate for most problems. Therefore, you need only specify those optional parameters whose values are to be different from their default values.
The remainder of this section can be skipped if you wish to use the default values for all optional parameters. A complete list of optional parameters and their default values is given in
Section 11.1.
Optional parameters may be specified by calling one, or more, of the routines
E05JCF,
E05JDF,
E05JEF,
E05JFF and
E05JGF before a call to E05JBF.
E05JCF reads options from an external options file, with
Begin and
End as the first and last lines respectively, and with each intermediate line defining a single optional parameter. For example,
Begin
Static Limit = 50
End
The call
CALL E05JCF (IOPTS, COMM, LCOMM, IFAIL)
can then be used to read the file on
unit
IOPTS.
IFAIL will be zero
on successful exit.
E05JCF should be consulted for a full description of this method of supplying optional parameters.
E05JDF,
E05JEF,
E05JFF or
E05JGF can be called to supply options directly, one call being necessary for each optional parameter.
E05JDF,
E05JEF,
E05JFF or
E05JGF should be consulted for a full description of this method of supplying optional parameters.
All optional parameters not specified by you are set to their default values. Valid values of optional parameters specified by you are unaltered by E05JBF and so remain in effect for subsequent calls to E05JBF, unless you explicitly change them.
11.1 Optional Parameter Checklist and Default Values
The following list gives the valid options. For each option, we give the keywords, the symbolic name, the default value, and the allowed range. A definition for each option can be found in
Section 11.2, where the letter
a denotes an option that takes an ‘ON’ or ‘OFF’ value and the letters
i and
r denote integer and real values required with certain options, respectively. Option names are case-insensitive and must be provided in full; abbreviations are not recognized. The symbol
ε is a generic notation for
machine precision (see
X02AJF), the symbol
rmax stands for the largest positive model number (see
X02ALF), and the symbol
d stands for the maximum number of decimal digits that can be represented (see
X02BEF).
11.2 Description of the Optional Parameters
This special keyword is used to reset all optional parameters to their default values.
| Function Evaluations Limit | (nf)
i | Default =100n2 |
This puts an approximate limit on the number of function calls allowed. The total number of calls made is checked at the top of an internal iteration loop, so it is possible that a few calls more than nf may be made.
Constraint:
nf>0.
| Infinite Bound Size | (infbnd)
r | Default =rmax14 |
This defines the ‘infinite’ bound infbnd in the definition of the problem constraints. Any upper bound greater than or equal to infbnd will be regarded as ∞ (and similarly any lower bound less than or equal to -infbnd will be regarded as -∞).
Constraint:
rmax14≤infbnd≤rmax12.
| Local Searches | (lcsrch)
a | Default ='ON' |
If you want to try to accelerate convergence of E05JBF by starting local searches from candidate minima, you will require lcsrch to be ‘ON’.
Constraint:
lcsrch='ON' or 'OFF'.
| Local Searches Limit | (loclim)
i | Default =50 |
This defines the maximal number of iterations to be used in the trust-region loop of the local-search procedure.
Constraint:
loclim>0.
| Local Searches Tolerance | (loctol)
r | Default =2ε |
The value of
loctol is the multiplier used during local searches as a stopping criterion for when the approximated gradient is small, in the sense described in
Section 10.3.
Constraint:
loctol≥2ε.
These keywords specify the required direction of optimization.
These options control the echoing of each optional parameter specification as it is supplied.
List turns printing on,
Nolist turns printing off. The output is sent to
the current advisory message unit (as defined by
X04ABF).
| Repeatability | (repeat)
a | Default ='OFF' |
For use with random initialization lists (
IINIT=4). When set to ‘ON’, an internally-initialized random seed is stored in
the array
COMM
for use in subsequent calls to E05JBF.
Constraint:
repeat='ON' or 'OFF'.
| Splits Limit | (smax)
i | Default =dn+2/3 |
Along with the initialization list
LIST, this defines a limit on the number of times the root box will be split along any single coordinate direction. If
Local Searches is ‘OFF’ you may find the default value to be too small.
Constraint:
smax>n+2.
| Static Limit | (stclim)
i | Default =3n |
As the default termination criterion, computation stops when the best function value is static for stclim sweeps through levels. This parameter is ignored if you have specified a target value to reach in Target Objective Value.
Constraint:
stclim>0.
| Target Objective Error | (objerr)
r | Default =ε14 |
If you have given a target objective value to reach in
objval (the value of the optional parameter
Target Objective Value),
objerr sets your desired relative error (from above if
Minimize is set, from below if
Maximize is set) between
OBJ and
objval, as described in
Section 7. See also the description of the optional parameter
Target Objective Safeguard.
Constraint:
objerr≥2ε.
| Target Objective Safeguard | (objsfg)
r | Default =ε12 |
If you have given a target objective value to reach in objval (the value of the optional parameter Target Objective Value), objsfg sets your desired safeguarded termination tolerance, for when objval is close to zero.
Constraint:
objsfg≥2ε.
| Target Objective Value | (objval)
r | |
This parameter may be set if you wish E05JBF to use a specific value as the target function value to reach during the optimization. Setting objval overrides the default termination criterion determined by the optional parameter Static Limit.
