G02 Chapter Contents
G02 Chapter Introduction
NAG Library Manual

NAG Library Routine DocumentG02JCF

Note:  before using this routine, please read the Users' Note for your implementation to check the interpretation of bold italicised terms and other implementation-dependent details.

1  Purpose

G02JCF preprocesses a dataset prior to fitting a linear mixed effects regression model of the following form via either G02JDF or G02JEF.

2  Specification

 SUBROUTINE G02JCF ( WEIGHT, N, NCOL, DAT, LDDAT, LEVELS, Y, WT, FIXED, LFIXED, NRNDM, RNDM, LDRNDM, NFF, NLSV, NRF, RCOMM, LRCOMM, ICOMM, LICOMM, IFAIL)
 INTEGER N, NCOL, LDDAT, LEVELS(NCOL), FIXED(LFIXED), LFIXED, NRNDM, RNDM(LDRNDM,NRNDM), LDRNDM, NFF, NLSV, NRF, LRCOMM, ICOMM(LICOMM), LICOMM, IFAIL REAL (KIND=nag_wp) DAT(LDDAT,NCOL), Y(N), WT(*), RCOMM(LRCOMM) CHARACTER(1) WEIGHT

3  Description

G02JCF must be called prior to fitting a linear mixed effects regression model with either G02JDF or G02JEF.
The model fitting routines G02JDF and G02JEF fit a model of the following form:
 $y=Xβ+Zν+ε$
 where $y$ is a vector of $n$ observations on the dependent variable, $X$ is an $n$ by $p$ design matrix of fixed independent variables, $\beta$ is a vector of $p$ unknown fixed effects, $Z$ is an $n$ by $q$ design matrix of random independent variables, $\nu$ is a vector of length $q$ of unknown random effects, $\epsilon$ is a vector of length $n$ of unknown random errors,
and $\nu$ and $\epsilon$ are Normally distributed with expectation zero and variance/covariance matrix defined by
 $Var ν ε = G 0 0 R$
where $R={\sigma }_{R}^{2}I$, $I$ is the $n×n$ identity matrix and $G$ is a diagonal matrix.
Case weights can be incorporated into the model by replacing $X$ and $Z$ with ${W}_{c}^{1/2}X$ and ${W}_{c}^{1/2}Z$ respectively where ${W}_{c}$ is a diagonal weight matrix.

None.

5  Arguments

1:     $\mathrm{WEIGHT}$ – CHARACTER(1)Input
On entry: indicates if weights are to be used.
${\mathbf{WEIGHT}}=\text{'U'}$
No weights are used.
${\mathbf{WEIGHT}}=\text{'W'}$
Case weights are used and must be supplied in array WT.
Constraint: ${\mathbf{WEIGHT}}=\text{'U'}$ or $\text{'W'}$.
2:     $\mathrm{N}$ – INTEGERInput
On entry: $n$, the number of observations.
The effective number of observations, that is the number of observations with nonzero weight (see WT for more detail), must be greater than the number of fixed effects in the model (as returned in NFF).
Constraint: ${\mathbf{N}}\ge 1$.
3:     $\mathrm{NCOL}$ – INTEGERInput
On entry: the number of columns in the data matrix, DAT.
Constraint: ${\mathbf{NCOL}}\ge 0$.
4:     $\mathrm{DAT}\left({\mathbf{LDDAT}},{\mathbf{NCOL}}\right)$ – REAL (KIND=nag_wp) arrayInput
On entry: a matrix of data, with ${\mathbf{DAT}}\left(i,j\right)$ holding the $i$th observation on the $j$th variable. The two design matrices $X$ and $Z$ are constructed from DAT and the information given in FIXED (for $X$) and RNDM (for $Z$).
Constraint: if ${\mathbf{LEVELS}}\left(j\right)\ne 1,1\le {\mathbf{DAT}}\left(i,j\right)\le {\mathbf{LEVELS}}\left(j\right)$.
5:     $\mathrm{LDDAT}$ – INTEGERInput
On entry: the first dimension of the array DAT as declared in the (sub)program from which G02JCF is called.
Constraint: ${\mathbf{LDDAT}}\ge {\mathbf{N}}$.
6:     $\mathrm{LEVELS}\left({\mathbf{NCOL}}\right)$ – INTEGER arrayInput
On entry: ${\mathbf{LEVELS}}\left(i\right)$ contains the number of levels associated with the $i$th variable held in DAT.
If the $i$th variable is continuous or binary (i.e., only takes the values zero or one) then ${\mathbf{LEVELS}}\left(i\right)$ must be set to $1$. Otherwise the $i$th variable is assumed to take an integer value between $1$ and ${\mathbf{LEVELS}}\left(i\right)$, (i.e., the $i$th variable is discrete with ${\mathbf{LEVELS}}\left(i\right)$ levels).
Constraint: ${\mathbf{LEVELS}}\left(\mathit{i}\right)\ge 1$, for $\mathit{i}=1,2,\dots ,{\mathbf{NCOL}}$.
7:     $\mathrm{Y}\left({\mathbf{N}}\right)$ – REAL (KIND=nag_wp) arrayInput
On entry: $y$, the vector of observations on the dependent variable.
8:     $\mathrm{WT}\left(*\right)$ – REAL (KIND=nag_wp) arrayInput
Note: the dimension of the array WT must be at least ${\mathbf{N}}$ if ${\mathbf{WEIGHT}}=\text{'W'}$.
On entry: if ${\mathbf{WEIGHT}}=\text{'W'}$, WT must contain the diagonal elements of the weight matrix ${W}_{c}$.
If ${\mathbf{WT}}\left(i\right)=0.0$, the $i$th observation is not included in the model and the effective number of observations is the number of observations with nonzero weights.
If ${\mathbf{WEIGHT}}=\text{'U'}$, WT is not referenced and the effective number of observations is $n$.
Constraint: if ${\mathbf{WEIGHT}}=\text{'W'}$, ${\mathbf{WT}}\left(\mathit{i}\right)\ge 0.0$, for $\mathit{i}=1,2,\dots ,n$.
9:     $\mathrm{FIXED}\left({\mathbf{LFIXED}}\right)$ – INTEGER arrayInput
On entry: defines the structure of the fixed effects design matrix, $X$.
${\mathbf{FIXED}}\left(1\right)$
The number of variables, ${N}_{F}$, to include as fixed effects (not including the intercept if present).
${\mathbf{FIXED}}\left(2\right)$
The fixed intercept flag which must contain $1$ if a fixed intercept is to be included and $0$ otherwise.
${\mathbf{FIXED}}\left(2+i\right)$
The column of DAT holding the $\mathit{i}$th fixed variable, for $\mathit{i}=1,2,\dots ,{\mathbf{FIXED}}\left(1\right)$.
See Section 9.1 for more details on the construction of $X$.
Constraints:
• ${\mathbf{FIXED}}\left(1\right)\ge 0$;
• ${\mathbf{FIXED}}\left(2\right)=0\text{​ or ​}1$;
• $1\le {\mathbf{FIXED}}\left(2+\mathit{i}\right)\le {\mathbf{NCOL}}$, for $\mathit{i}=1,2,\dots ,{\mathbf{FIXED}}\left(1\right)$.
10:   $\mathrm{LFIXED}$ – INTEGERInput
On entry: length of the vector FIXED.
Constraint: ${\mathbf{LFIXED}}\ge 2+{\mathbf{FIXED}}\left(1\right)$.
11:   $\mathrm{NRNDM}$ – INTEGERInput
On entry: the second dimension of the array RNDM as declared in the (sub)program from which G02JCF is called.
Constraint: ${\mathbf{NRNDM}}>0$.
12:   $\mathrm{RNDM}\left({\mathbf{LDRNDM}},{\mathbf{NRNDM}}\right)$ – INTEGER arrayInput
On entry: ${\mathbf{RNDM}}\left(i,j\right)$ defines the structure of the random effects design matrix, $Z$. The $b$th column of RNDM defines a block of columns in the design matrix $Z$.
${\mathbf{RNDM}}\left(1,b\right)$
The number of variables, ${N}_{{R}_{b}}$, to include as random effects in the $b$th block (not including the random intercept if present).
${\mathbf{RNDM}}\left(2,b\right)$
The random intercept flag which must contain $1$ if block $b$ includes a random intercept and $0$ otherwise.
${\mathbf{RNDM}}\left(2+i,b\right)$
The column of DAT holding the $\mathit{i}$th random variable in the $b$th block, for $\mathit{i}=1,2,\dots ,{\mathbf{RNDM}}\left(1,b\right)$.
${\mathbf{RNDM}}\left(3+{N}_{{R}_{b}},b\right)$
The number of subject variables, ${N}_{{S}_{b}}$, for the $b$th block. The subject variables define the nesting structure for this block.
${\mathbf{RNDM}}\left(3+{N}_{{R}_{b}}+i,b\right)$
The column of DAT holding the $\mathit{i}$th subject variable in the $b$th block, for $\mathit{i}=1,2,\dots ,{\mathbf{RNDM}}\left(3+{N}_{{R}_{b}},b\right)$.
See Section 9.2 for more details on the construction of $Z$.
Constraints:
• ${\mathbf{RNDM}}\left(1,b\right)\ge 0$;
• ${\mathbf{RNDM}}\left(2,b\right)=0\text{​ or ​}1$;
• at least one random variable or random intercept must be specified in each block, i.e., ${\mathbf{RNDM}}\left(1,b\right)+{\mathbf{RNDM}}\left(2,b\right)>0$;
• the column identifiers associated with the random variables must be in the range $1$ to NCOL, i.e., $1\le {\mathbf{RNDM}}\left(2+\mathit{i},b\right)\le {\mathbf{NCOL}}$, for $\mathit{i}=1,2,\dots ,{\mathbf{RNDM}}\left(1,b\right)$;
• ${\mathbf{RNDM}}\left(3+{N}_{{R}_{b}},b\right)\ge 0$;
• the column identifiers associated with the subject variables must be in the range $1$ to NCOL, i.e., $1\le {\mathbf{RNDM}}\left(3+{N}_{{R}_{b}}+\mathit{i},b\right)\le {\mathbf{NCOL}}$, for $\mathit{i}=1,2,\dots ,{\mathbf{RNDM}}\left(3+{N}_{{R}_{b}},b\right)$.
13:   $\mathrm{LDRNDM}$ – INTEGERInput
On entry: the first dimension of the array RNDM as declared in the (sub)program from which G02JCF is called.
Constraint: ${\mathbf{LDRNDM}}\ge \underset{b}{max}\phantom{\rule{0.25em}{0ex}}\left(3+{N}_{{R}_{b}}+{N}_{{S}_{b}}\right)$.
14:   $\mathrm{NFF}$ – INTEGEROutput
On exit: $p$, the number of fixed effects estimated, i.e., the number of columns in the design matrix $X$.
15:   $\mathrm{NLSV}$ – INTEGEROutput
On exit: the number of levels for the overall subject variable (see Section 9.2 for a description of what this means). If there is no overall subject variable, ${\mathbf{NLSV}}=1$.
16:   $\mathrm{NRF}$ – INTEGEROutput
On exit: the number of random effects estimated in each of the overall subject blocks. The number of columns in the design matrix $Z$ is given by $q={\mathbf{NRF}}×{\mathbf{NLSV}}$.
17:   $\mathrm{RCOMM}\left({\mathbf{LRCOMM}}\right)$ – REAL (KIND=nag_wp) arrayCommunication Array
On exit: communication array as required by the analysis routines G02JDF and G02JEF.
18:   $\mathrm{LRCOMM}$ – INTEGERInput
On entry: the dimension of the array RCOMM as declared in the (sub)program from which G02JCF is called.
Constraint: ${\mathbf{LRCOMM}}\ge {\mathbf{NRF}}×{\mathbf{NLSV}}+{\mathbf{NFF}}+{\mathbf{NFF}}×{\mathbf{NLSV}}+{\mathbf{NRF}}×{\mathbf{NLSV}}+{\mathbf{NFF}}+2$.
19:   $\mathrm{ICOMM}\left({\mathbf{LICOMM}}\right)$ – INTEGER arrayCommunication Array
On exit: if ${\mathbf{LICOMM}}=2$, ${\mathbf{ICOMM}}\left(1\right)$ holds the minimum required value for LICOMM and ${\mathbf{ICOMM}}\left(2\right)$ holds the minimum required value for LRCOMM, otherwise ICOMM is a communication array as required by the analysis routines G02JDF and G02JEF.
20:   $\mathrm{LICOMM}$ – INTEGERInput
On entry: the dimension of the array ICOMM as declared in the (sub)program from which G02JCF is called.
Constraint: ${\mathbf{LICOMM}}=2$ or ${\mathbf{LICOMM}}\ge 34+{N}_{F}×\left(\text{MFL}+1\right)+{\mathbf{NRNDM}}×\text{MNR}×\text{MRL}+\left(\text{LRNDM}+2\right)×{\mathbf{NRNDM}}+{\mathbf{NCOL}}+\text{LDID}×\text{LB,}$
where
• $\text{MNR}=\underset{b}{\mathrm{max}}\phantom{\rule{0.25em}{0ex}}\left({N}_{{R}_{b}}\right)$,
• $\text{MFL}=\underset{i}{\mathrm{max}}\phantom{\rule{0.25em}{0ex}}\left({\mathbf{LEVELS}}\left({\mathbf{FIXED}}\left(2+i\right)\right)\right)$,
• $\text{MRL}=\underset{b,i}{\mathrm{max}}\phantom{\rule{0.25em}{0ex}}\left({\mathbf{LEVELS}}\left({\mathbf{RNDM}}\left(2+i,b\right)\right)\right)$,
• $\text{LDID}=\underset{b}{\mathrm{max}}\phantom{\rule{0.25em}{0ex}}{N}_{{S}_{b}}$,
• $\text{LB}={\mathbf{NFF}}+{\mathbf{NRF}}×{\mathbf{NLSV}}$, and
• $\text{LRNDM}=\underset{b}{max}\phantom{\rule{0.25em}{0ex}}\left(3+{N}_{{R}_{b}}+{N}_{{S}_{b}}\right)$
21:   $\mathrm{IFAIL}$ – INTEGERInput/Output
On entry: IFAIL must be set to $0$, $-1\text{​ or ​}1$. If you are unfamiliar with this argument you should refer to Section 3.4 in How to Use the NAG Library and its Documentation for details.
For environments where it might be inappropriate to halt program execution when an error is detected, the value $-1\text{​ or ​}1$ is recommended. If the output of error messages is undesirable, then the value $1$ is recommended. Otherwise, if you are not familiar with this argument, the recommended value is $0$. When the value $-\mathbf{1}\text{​ or ​}\mathbf{1}$ is used it is essential to test the value of IFAIL on exit.
On exit: ${\mathbf{IFAIL}}={\mathbf{0}}$ unless the routine detects an error or a warning has been flagged (see Section 6).

6  Error Indicators and Warnings

If on entry ${\mathbf{IFAIL}}=0$ or $-1$, explanatory error messages are output on the current error message unit (as defined by X04AAF).
Errors or warnings detected by the routine:
${\mathbf{IFAIL}}=1$
On entry, WEIGHT had an illegal value.
${\mathbf{IFAIL}}=2$
On entry, ${\mathbf{N}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{N}}\ge 1$.
${\mathbf{IFAIL}}=3$
On entry, ${\mathbf{NCOL}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{NCOL}}\ge 0$.
${\mathbf{IFAIL}}=4$
On entry, variable $j$ of observation $i$ is less than $1$ or greater than ${\mathbf{LEVELS}}\left(j\right)$: $i=〈\mathit{\text{value}}〉$, $j=〈\mathit{\text{value}}〉$, value $=〈\mathit{\text{value}}〉$, ${\mathbf{LEVELS}}\left(j\right)=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=5$
On entry, ${\mathbf{LDDAT}}=〈\mathit{\text{value}}〉$ and ${\mathbf{N}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{LDDAT}}\ge {\mathbf{N}}$.
${\mathbf{IFAIL}}=6$
On entry, ${\mathbf{LEVELS}}\left(〈\mathit{\text{value}}〉\right)=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{LEVELS}}\left(i\right)\ge 1$.
${\mathbf{IFAIL}}=8$
On entry, ${\mathbf{WT}}\left(〈\mathit{\text{value}}〉\right)=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{WT}}\left(i\right)\ge 0.0$.
${\mathbf{IFAIL}}=9$
On entry, number of fixed parameters, $〈\mathit{\text{value}}〉$ is less than zero.
${\mathbf{IFAIL}}=10$
On entry, ${\mathbf{LFIXED}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{LFIXED}}\ge 〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=11$
On entry, ${\mathbf{NRNDM}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{NRNDM}}>0$.
${\mathbf{IFAIL}}=12$
On entry, number of random parameters for random statement $i$ is less than $0$: $i=〈\mathit{\text{value}}〉$, number of parameters $\text{}=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=13$
On entry, ${\mathbf{LDRNDM}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{LDRNDM}}\ge 〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=18$
On entry, ${\mathbf{LRCOMM}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{LRCOMM}}\ge 〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=20$
On entry, ${\mathbf{LICOMM}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{LICOMM}}\ge 〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=102$
On entry, more fixed factors than observations, ${\mathbf{N}}=〈\mathit{\text{value}}〉$.
Constraint: ${\mathbf{N}}\ge 〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=108$
On entry, no observations due to zero weights.
${\mathbf{IFAIL}}=109$
On entry, invalid value for fixed intercept flag: value $\text{}=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=112$
On entry, invalid value for random intercept flag for random statement $i$: $i=〈\mathit{\text{value}}〉$, value $\text{}=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=209$
On entry, index of fixed variable $j$ is less than $1$ or greater than ${\mathbf{NCOL}}$: $j=〈\mathit{\text{value}}〉$, index $\text{}=〈\mathit{\text{value}}〉$ and ${\mathbf{NCOL}}=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=212$
On entry, must be at least one parameter, or an intercept in each random statement $i$: $i=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=312$
On entry, index of random variable $j$ in random statement $i$ is less than $1$ or greater than ${\mathbf{NCOL}}$: $i=〈\mathit{\text{value}}〉$, $j=〈\mathit{\text{value}}〉$, index $\text{}=〈\mathit{\text{value}}〉$ and ${\mathbf{NCOL}}=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=412$
On entry, number of subject parameters for random statement $i$ is less than $0$: $i=〈\mathit{\text{value}}〉$, number of parameters $\text{}=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=512$
On entry, nesting variable $j$ in random statement $i$ has one level: $j=〈\mathit{\text{value}}〉$, $i=〈\mathit{\text{value}}〉$.
${\mathbf{IFAIL}}=-99$
See Section 3.9 in How to Use the NAG Library and its Documentation for further information.
${\mathbf{IFAIL}}=-399$
Your licence key may have expired or may not have been installed correctly.
See Section 3.8 in How to Use the NAG Library and its Documentation for further information.
${\mathbf{IFAIL}}=-999$
Dynamic memory allocation failed.
See Section 3.7 in How to Use the NAG Library and its Documentation for further information.

Not applicable.

8  Parallelism and Performance

G02JCF is not threaded in any implementation.

9.1  Construction of the fixed effects design matrix, $X$

Let
• ${N}_{F}$ denote the number of fixed variables, that is ${\mathbf{FIXED}}\left(1\right)={N}_{F}$;
• ${F}_{j}$ denote the $j$th fixed variable, that is the vector of values held in the $k$th column of DAT when ${\mathbf{FIXED}}\left(2+j\right)=k$;
• ${F}_{ij}$ denote the $i$th element of ${F}_{j}$;
• $L\left({F}_{j}\right)$ denote the number of levels for ${F}_{j}$, that is $L\left({F}_{j}\right)={\mathbf{LEVELS}}\left({\mathbf{FIXED}}\left(2+j\right)\right)$;
• ${D}_{v}\left({F}_{j}\right)$ denoted an indicator function that returns a vector of values whose $i$th element is $1$ if ${F}_{ij}=v$ and $0$ otherwise.
The design matrix for the fixed effects, $X$, is constructed as follows:
• set $k$ to one and the flag $\text{done_first}$ to false;
• if a fixed intercept is included, that is ${\mathbf{FIXED}}\left(2\right)=1$,
• set the first column of $X$ to a vector of $1$s;
• set $k=k+1$;
• set $\text{done_first}$ to true;
• loop over each fixed variable, so for each $j=1,2,\dots ,{N}_{F}$,
• if $L\left({F}_{j}\right)=1$,
• set the $k$th column of $X$ to be ${F}_{j}$;
• set $k=k+1$;
• else
• if $\text{done_first}$ is false then
• set the $L\left({F}_{j}\right)$ columns, $k$ to $k+L\left({F}_{j}\right)-1$, of $X$ to ${D}_{\mathit{v}}\left({F}_{j}\right)$, for $\mathit{v}=1,2,\dots ,L\left({F}_{j}\right)$;
• set $k=k+L\left({F}_{j}\right)$;
• set $\text{done_first}$ to true;
• else
• set the $L\left({F}_{j}\right)-1$ columns, $k$ to $k+L\left({F}_{j}\right)-2$, of $X$ to ${D}_{\mathit{v}}\left({F}_{j}\right)$, for $\mathit{v}=2,3,\dots ,L\left({F}_{j}\right)$;
• set $k=k+L\left({F}_{j}\right)-1$.
The number of columns in the design matrix, $X$, is therefore given by
 $p= 1+ ∑ j=1 N F LEVELS FIXED 2+j -1 .$
This quantity is returned in NFF.
In summary, G02JCF converts all non-binary categorical variables (i.e., where $L\left({F}_{j}\right)>1$) to dummy variables. If a fixed intercept is included in the model then the first level of all such variables is dropped. If a fixed intercept is not included in the model then the first level of all such variables, other than the first, is dropped. The variables are added into the model in the order they are specified in FIXED.

9.2  Construction of random effects design matrix, $Z$

Let
• ${N}_{{R}_{b}}$ denote the number of random variables in the $b$th random statement, that is ${N}_{{R}_{b}}={\mathbf{RNDM}}\left(1,b\right)$;
• ${R}_{jb}$ denote the $j$th random variable from the $b$th random statement, that is the vector of values held in the $k$th column of DAT when ${\mathbf{RNDM}}\left(2+j,b\right)=k$;
• ${R}_{ijb}$ denote the $i$th element of ${R}_{jb}$;
• $L\left({R}_{jb}\right)$ denote the number of levels for ${R}_{jb}$, that is $L\left({R}_{jb}\right)={\mathbf{LEVELS}}\left({\mathbf{RNDM}}\left(2+j,b\right)\right)$;
• ${D}_{v}\left({R}_{jb}\right)$ denoted an indicator function that returns a vector of values whose $i$th element is $1$ if ${R}_{ijb}=v$ and $0$ otherwise;
• ${N}_{{S}_{b}}$ denote the number of subject variables in the $b$th random statement, that is ${N}_{{S}_{b}}={\mathbf{RNDM}}\left(3+{N}_{{R}_{b}},b\right)$;
• ${S}_{jb}$ denote the $j$th subject variable from the $b$th random statement, that is the vector of values held in the $k$th column of DAT when ${\mathbf{RNDM}}\left(3+{N}_{{R}_{b}}+j,b\right)=k$;
• ${S}_{ijb}$ denote the $i$th element of ${S}_{jb}$;
• $L\left({S}_{jb}\right)$ denote the number of levels for ${S}_{jb}$, that is $L\left({S}_{jb}\right)={\mathbf{LEVELS}}\left({\mathbf{RNDM}}\left(3+{N}_{{R}_{b}}+j,b\right)\right)$;
• ${I}_{b}\left({s}_{1},{s}_{2},\dots ,{s}_{{N}_{{S}_{b}}}\right)$ denoted an indicator function that returns a vector of values whose $i$th element is $1$ if ${S}_{ijb}={s}_{j}$ for all $j=1,2,\dots ,{N}_{{S}_{b}}$ and $0$ otherwise.
The design matrix for the random effects, $Z$, is constructed as follows:
• set $k$ to one;
• loop over each random statement, so for each $b=1,2,\dots ,{\mathbf{NRNDM}}$,
• loop over each level of the last subject variable, so for each ${s}_{{N}_{{S}_{b}}}=1,2,\dots ,L\left({R}_{{N}_{{S}_{b}}b}\right)$,
• $⋮$
• loop over each level of the second subject variable, so for each ${s}_{2}=1,2,\dots ,L\left({R}_{2b}\right)$,
• loop over each level of the first subject variable, so for each ${s}_{1}=1,2,\dots ,L\left({R}_{1b}\right)$,
• if a random intercept is included, that is ${\mathbf{RNDM}}\left(2,b\right)=1$,
• set the $k$th column of $Z$ to ${I}_{b}\left({s}_{1},{s}_{2},\dots ,{s}_{{N}_{{S}_{b}}}\right)$;
• set $k=k+1$;
• loop over each random variable in the $b$th random statement, so for each $j=1,2,\dots ,{N}_{{R}_{b}}$,
• if $L\left({R}_{jb}\right)=1$,
• set the $k$th column of $Z$ to ${R}_{jb}×{I}_{b}\left({s}_{1},{s}_{2},\dots ,{s}_{{N}_{{S}_{b}}}\right)$ where $×$ indicates an element-wise multiplication between the two vectors, ${R}_{jb}$ and ${I}_{b}\left(\dots \right)$;
• set $k=k+1$;
• else
• set the $L\left({R}_{bj}\right)$ columns, $k$ to $k+L\left({R}_{bj}\right)$, of $Z$ to ${D}_{\mathit{v}}\left({R}_{jb}\right)×{I}_{b}\left({s}_{1},{s}_{2},\dots ,{s}_{{N}_{{S}_{b}}}\right)$, for $\mathit{v}=1,2,\dots ,L\left({R}_{jb}\right)$. As before, $×$ indicates an element-wise multiplication between the two vectors, ${D}_{v}\left(\dots \right)$ and ${I}_{b}\left(\dots \right)$;
• set $k=k+L\left({R}_{jb}\right)$.
In summary, each column of RNDM defines a block of consecutive columns in $Z$. G02JCF converts all non-binary categorical variables (i.e., where $L\left({R}_{jb}\right)$ or $L\left({S}_{jb}\right)>1$) to dummy variables. All random variables defined within a column of RNDM are nested within all subject variables defined in the same column of RNDM. In addition each of the subject variables are nested within each other, starting with the first (i.e., each of the ${R}_{jb},j=1,2,\dots ,{N}_{{R}_{b}}$ are nested within ${S}_{1b}$ which in turn is nested within ${S}_{2b}$, which in turn is nested within ${S}_{3b}$, etc.).
If the last subject variable in each column of RNDM are the same (i.e., ${S}_{{N}_{{S}_{1}}1}={S}_{{N}_{{S}_{2}}2}=\dots ={S}_{{N}_{{S}_{b}}b}$) then all random effects in the model are nested within this variable. In such instances the last subject variable (${S}_{{N}_{{S}_{1}}1}$) is called the overall subject variable. The fact that all of the random effects in the model are nested within the overall subject variable means that ${Z}^{\mathrm{T}}Z$ is block diagonal in structure. This fact can be utilised to improve the efficiency of the underlying computation and reduce the amount of internal storage required. The number of levels in the overall subject variable is returned in ${\mathbf{NLSV}}=L\left({S}_{{N}_{{S}_{1}}1}\right)$.
If the last $k$ subject variables in each column of RNDM are the same, for $k>1$ then the overall subject variable is defined as the interaction of these $k$ variables and
 $NLSV= ∏ j=NS1-k+1 NS1 LSj1 .$
If there is no overall subject variable then ${\mathbf{NLSV}}=1$.
The number of columns in the design matrix $Z$ is given by $q={\mathbf{NRF}}×{\mathbf{NLSV}}$.

9.3  The RNDM argument

To illustrate some additional points about the RNDM argument, we assume that we have a dataset with three discrete variables, ${V}_{1}$, ${V}_{2}$ and ${V}_{3}$, with $2,4$ and $3$ levels respectively, and that ${V}_{1}$ is in the first column of DAT, ${V}_{2}$ in the second and ${V}_{3}$ the third. Also assume that we wish to fit a model containing ${V}_{1}$ along with ${V}_{2}$ nested within ${V}_{3}$, as random effects. In order to do this the RNDM matrix requires two columns:
 $RNDM= 1 1 0 0 1 2 0 1 0 3$
The first column, $\left(1,0,1,0,0\right)$, indicates one random variable (${\mathbf{RNDM}}\left(1,1\right)=1$), no intercept (${\mathbf{RNDM}}\left(2,1\right)=0$), the random variable is in the first column of DAT (${\mathbf{RNDM}}\left(3,1\right)=1$), there are no subject variables; as no nesting is required for ${V}_{1}$ (${\mathbf{RNDM}}\left(4,1\right)=0$). The last element in this column is ignored.
The second column, $\left(1,0,2,1,3\right)$, indicates one random variable (${\mathbf{RNDM}}\left(1,2\right)=1$), no intercept (${\mathbf{RNDM}}\left(2,2\right)=0$), the random variable is in the second column of DAT $\left({\mathbf{RNDM}}\left(3,2\right)=2\right)$, there is one subject variable (${\mathbf{RNDM}}\left(4,2\right)=1$), and the subject variable is in the third column of DAT $\left({\mathbf{RNDM}}\left(5,2\right)=3\right)$.
The corresponding $Z$ matrix would have $14$ columns, with $2$ coming from ${V}_{1}$ and $12$ ($4×3$) from ${V}_{2}$ nested within ${V}_{3}$. The, symmetric, ${Z}^{\mathrm{T}}Z$ matrix has the form
 $- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - -$
where $0$ indicates a structural zero, i.e., it always takes the value $0$, irrespective of the data, and $-$ a value that is not a structural zero. The first two rows and columns of ${Z}^{\mathrm{T}}Z$ correspond to ${V}_{1}$. The block diagonal matrix in the 12 rows and columns in the bottom right correspond to ${V}_{2}$ nested within ${V}_{3}$. With the $4×4$ blocks corresponding to the levels of ${V}_{2}$. There are three blocks as the subject variable (${V}_{3}$) has three levels.
The model fitting routines, G02JDF and G02JEF, use the sweep algorithm to calculate the log-likelihood function for a given set of variance components. This algorithm consists of moving down the diagonal elements (called pivots) of a matrix which is similar in structure to ${Z}^{\mathrm{T}}Z$, and updating each element in that matrix. When using the $k$ diagonal element of a matrix $A$, an element ${a}_{ij},i\ne k,j\ne k$, is adjusted by an amount equal to ${a}_{ik}{a}_{ij}/{a}_{kk}$. This process can be referred to as sweeping on the $k$th pivot. As there are no structural zeros in the first row or column of the above ${Z}^{\mathrm{T}}Z$, sweeping on the first pivot of ${Z}^{\mathrm{T}}Z$ would alter each element of the matrix and therefore destroy the structural zeros, i.e., we could no longer guarantee they would be zero.
Reordering the RNDM matrix to
 $RNDM= 1 1 0 0 2 1 1 0 3 0$
i.e., the swapping the two columns, results in a ${Z}^{\mathrm{T}}Z$ matrix of the form
 $- - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 - - - - 0 0 0 0 - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - 0 0 0 0 0 0 0 0 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -$
This matrix is identical to the previous one, except the first two rows and columns have become the last two rows and columns. Sweeping a matrix, $A=\left\{{a}_{ij}\right\}$, of this form on the first pivot will only affect those elements ${a}_{ij}$, where ${a}_{i1}\ne 0\text{​ and ​}{a}_{1j}\ne 0$, which is only the $13$th and $14$th row and columns, and the top left hand block of $4$ rows and columns. The block diagonal nature of the first $12$ rows and columns therefore greatly reduces the amount of work the algorithm needs to perform.
G02JCF constructs the ${Z}^{\mathrm{T}}Z$ as specified by the RNDM matrix, and does not attempt to reorder it to improve performance. Therefore for best performance some thought is required on what ordering to use. In general it is more efficient to structure RNDM in such a way that the first row relates to the deepest level of nesting, the second to the next level, etc..

10  Example

See Section 10 in G02JDF and G02JEF.