naginterfaces.library.correg.ssqmat_​combine

naginterfaces.library.correg.ssqmat_combine(xsw, xmean, xc, ysw, ymean, yc, mean='M')

ssqmat_combine combines two sets of sample means and sums of squares and cross-products matrices. It is designed to be used in conjunction with ssqmat() to allow large datasets to be summarised.

For full information please refer to the NAG Library document for g02bz

https://support.nag.com/numeric/nl/nagdoc_30/flhtml/g02/g02bzf.html

Parameters

xswfloat

$W_x$, the sum of weights, from the first set of data, $X$. If the data is unweighted then this will be the number of observations in the first dataset.

xmeanfloat, array-like, shape $\left(m\right)$

${\mu }_x$, the sample means for the first set of data, $X$.

xcfloat, array-like, shape $\left((m\times m+m)/2\right)$

$C_x$, the sums of squares and cross-products matrix for the first set of data, $X$, as returned by ssqmat().

ssqmat(), returns this matrix packed by columns, i.e., the cross-product between the $j$th and $k$th variable, $k\ge j$, is stored in $xc[k\times (k-1)/2+j-1]$.

No check is made that $C_x$ is a valid cross-products matrix.

yswfloat

$W_y$, the sum of weights, from the second set of data, $Y$. If the data is unweighted then this will be the number of observations in the second dataset.

ymeanfloat, array-like, shape $\left(m\right)$

${\mu }_y$, the sample means for the second set of data, $Y$.

ycfloat, array-like, shape $\left((m\times m+m)/2\right)$

$C_y$, the sums of squares and cross-products matrix for the second set of data, $Y$, as returned by ssqmat().

ssqmat(), returns this matrix packed by columns, i.e., the cross-product between the $j$th and $k$th variable, $k\ge j$, is stored in $yc[k\times (k-1)/2+j-1]$.

No check is made that $C_y$ is a valid cross-products matrix.

meanstr, length 1, optional

Indicates whether the matrices supplied in $xc$ and $yc$ are sums of squares and cross-products, or sums of squares and cross-products of deviations about the mean.

$mean=\text{‘M'}$

Sums of squares and cross-products of deviations about the mean have been supplied.

$mean=\text{‘Z'}$

Sums of squares and cross-products have been supplied.

Returns

xswfloat

$W_z$, the sum of weights, from the combined dataset, $Z$. If both datasets are unweighted then this will be the number of observations in the combined dataset.

xmeanfloat, ndarray, shape $\left(m\right)$

${\mu }_z$, the sample means for the combined data, $Z$.

xcfloat, ndarray, shape $\left((m\times m+m)/2\right)$

$C_z$, the sums of squares and cross-products matrix for the combined dataset, $Z$.

This matrix is again stored packed by columns.

Raises

NagValueError

(errno $11$)

On entry, $mean=⟨value⟩$.

Constraint: $mean=\text{‘M'}\text{or}\text{‘Z'}$.

(errno $21$)

On entry, $m=⟨value⟩$.

Constraint: $m\ge 1$.

(errno $31$)

On entry, $xsw=⟨value⟩$.

Constraint: $xsw\ge 0.0$.

(errno $61$)

On entry, $ysw=⟨value⟩$.

Constraint: $ysw\ge 0.0$.

Notes

Let $X$ and $Y$ denote two sets of data, each with $m$ variables and $n_x$ and $n_y$ observations respectively. Let ${\mu }_x$ denote the (optionally weighted) vector of $m$ means for the first dataset and $C_x$ denote either the sums of squares and cross-products of deviations from ${\mu }_x$

$$C_x={(X-e{\mu }_x^T)}^T{D}_x(X-e{\mu }_x^T)$$

or the sums of squares and cross-products, in which case

$$C_x=X^T{D}_{x}X$$

where $e$ is a vector of $n_x$ ones and $D_x$ is a diagonal matrix of (optional) weights and $W_x$ is defined as the sum of the diagonal elements of $D$. Similarly, let ${\mu }_y$, $C_y$ and $W_y$ denote the same quantities for the second dataset.

Given ${\mu }_x,{\mu }_y,C_x,C_y,W_x$ and $W_y$ ssqmat_combine calculates ${\mu }_z$, $C_z$ and $W_z$ as if a dataset $Z$, with $m$ variables and $n_x+n_y$ observations were supplied to ssqmat(), with $Z$ constructed as

$$\begin{array}{c}Z=\left(\begin{array}{c}X\\ Y\end{array}\right)\text{.}\end{array}$$

ssqmat_combine has been designed to combine the results from two calls to ssqmat() allowing large datasets, or cases where all the data is not available at the same time, to be summarised.

References

Bennett, J, Pebay, P, Roe, D and Thompson, D, 2009, Numerically stable, single-pass, parallel statistics algorithms, Proceedings of IEEE International Conference on Cluster Computing