Methods
As shown in Chen et al. (2014), we formulate the group analysis under a multivariate GLM or MVM platform that is expressed from a subject-wise perspective, βiT=xiTA+δiT, or through the variable-wise pivot, bj = Xaj + dj, or in the following concise form, (1) Bn×m=Xn×q   Aq×m+Dn×m.
The n rows of the response matrix B=(βij)n×m=(β1T,β2T,...,βnT)T=(b1,b2,...,bm) represent the data from the n subjects while the m columns correspond to the levels of within-subject factor(s). For example, the effect estimates from the multiple basis functions under ESM or ASM can be considered the response values associated with the levels of a within-subject or repeated-measures factor (termed Component hereafter). When multiple within-subject factors occur, all their level combinations for each subject are flattened from a multi-dimensional space onto a one-dimensional row of B. It is noteworthy that the within-subject factors are expressed as columns in B on the left-hand side of the model (1), and only between-subjects variables such as subjects-grouping factors (e.g., sex, genotypes), subject-specific measures (e.g., age, IQ) and their interactions are treated as q explanatory variables on the right-hand side. The same linear system is assumed for all the m response variables, which share the same design matrix X=(xih)=(x1,x2,...,xn)T. Without loss of generality, X is assumed to have full column-rank q. Each column of the regression coefficient matrix A = (αhj) corresponds to a response variable, and each row is associated with an explanatory variable. Lastly, the error matrix D=(δij)n×m=(δ1,δ2,...,δn)T=(d1,d2,...,dm) is assumed nm-dimensional Gaussian: vec(D) ~N(0, In ⊗ Σ), where vec and ⊗ are column stacking and direct (or Kronecker) product operators respectively. As in univariate modeling (UVM), the assumptions for model (1) are linearity, Gaussianity and homogeneity of variance-covariance structure (same Σ across all the between-subjects effects). When only one group of subjects is involved (q = 1), the parameter matrix A becomes a row vector (α1, α2, …, αm) that is associated with the m levels of a within-subject factor.
As demonstrated in Chen et al. (2014), MVM has a few advantages over its univariate counterpart. When the data are essentially multidimensional like the multiple effect estimates from ESM or ASM, MVM has a crucial role in formulating hypothesis testing. In addition, it characterizes and quantifies the intercorrelations among the variables based on the data rather than a presumed variance-covariance structure as in UVM. Furthermore, MVM in general provides a better control for false positives than UVM. Lastly, the conventional univariate testing (UVT) under GLM can be easily performed under the MVM framework with a few extra advantages. Here we discuss one aspect by which the group analysis of neuroimaging data will benefit from the MVM facility when the HDR profile is estimated from multiple basis functions instead of being presumed to have a fixed shape. Then in the section Simulations and Real Experiment Results, we elaborate and compare a few testing alternatives in terms of power and false positives, using simulations and in terms of performance with real data.

Different testing strategies
Here we exemplify two simple and prototypical cases with the HDR profile modeled by m basis functions at the individual subject level: a) one group of subjects with the associated effects at the group level expressed as α1, α2, …, αm under (1), and b) either two groups or two conditions and the two sets of effect estimates for HDR are α1j and α2j respectively, j = 1, 2, …, m. To simplify geometric representations, we assume equal number of subjects across groups in the case of group comparison, but the assumption is not required from the modeling perspective. The various modeling strategies discussed below for these two cases can be easily extended to situations with more explanatory variables, including factors and quantitative covariates.

Multivariate testing (MVT)
As the analogs of one- and two-sample or paired t-tests under UVT, the two prototypes can be expressed with the following null hypotheses, (2a)  H01MVT:α1=0,α2=0,...,αm=0, (2b) H02MVT:α11=α21,α12=α22,...,α1m=α2m.
In other words, the m regression coefficients associated with the m basis functions from each subject are brought to the group level and treated as the instantiated values of m simultaneous variables. When the effect estimates associated with the basis functions of ESM or ASM are treated as the values of m simultaneous response variables, the hypothesis (2a) or (2b) can be analyzed through MVT under the model (1). Geometrically, the data for H01MVT represent the group centroid (α1, α2, …, αm) in the m-dimensional real coordinate space ℝm (Table 1), and the associated one-sample Hotelling T2-test is performed to reveal whether the group centroid lies in the rejection region (outside of an m-dimensional ellipse centering around the origin in the case of H01MVT). Similarly, the data for H02MVT are expressed as two group centroids, (α11, α12, …, α1m) and (α21, α22, …, α2m), and the corresponding two-sample Hotelling T2-test is conducted to see if the hypothesis (2b) about the two centroids can be rejected. The hypothesis (2b) can be easily generalized to the situation with more than two groups of subjects (e.g., three genotypes) as well as more than one subject-grouping variable (e.g., sex, genotypes, and handedness) through the formulation of general linear testing (Chen et al., 2014). One noteworthy feature of MVT is that it allows those simultaneous effects to have different scales or units, unlike the traditional AN(C)OVA or univariate GLM in which all the levels of a factor are usually of the same dimension.
Table 1  Schematic comparisons among various testing methods.
One-sample
Methoda   MVT/LME   AUC   L2D   EXC (XUV and XMV)
H 0  α1 = … = αm = 0  α1 + … + αm = 0  ( α  1  2   + … + α  m  2    )   1 ∕ 2   = 0  α1 = … = αm
Dimensions in ℝm  0  m−1  m−1  1
DFs for F-statisticb  m, n−m−q+1  1, n−q  1, n−q  m−1, (m−1)(n−q)
Geometric representationc of H0 and H1 (m = 2)
Geometric representationd of HDR when detection failure occurs due to improper H0 formulation  no   no
Two-sample or paired
Method   MVT   AUC   L2D   EXC (XUV and XMV)
H 0  α11 = α21, …, α1m = α2m  ∑  j = 1  m   α  1 j   = ∑  j = 1  m   α  2 j  ( ∑  j = 1  m   α  1 j  2    )   1 ∕ 2   = ( ∑  j = 1  m   α  2 j  2    )   1 ∕ 2  α11 − α21 = … = α1m − α2m
Dimensions in ℝm  0  m−1  m−1  1
DFs for F-statistic  m, n−m−q+1  1, n−q  1, n−q  m−1, (m−1)(n−q)
Geometric representatione of H0 and H1
Geometric representationf of HDR when detection failure occurs due to improper H0 formulation  no
a  The table is meant to show the dimensions of each null hypothesis and an instantiation in the rejection domain while the whole rejection domain is not represented here. For example, the reject region of one-sample Hotelling T2-test for MVT (2a) is outside of an m-dimensional ellipse.
b  An interesting fact is that the numerator degrees of freedom for the F-statistic under MVT and UVT are the dimensions of the complementary space to the associated null hypothesis H0, or the dimensions of the alternative hypothesis H1.
c  The two axes represent the two weights associated with the two basis functions. The whole rejection regions are not shown here, and the shaded (gray) and solid (black) areas correspond respectively to the null hypothesis H0 space and an instantiation (and its dimension) in the alternative hypothesis H1 space. Detection failure occurs when the group centroid falls on the diagonal line other than the origin under AUC and EXC.
d  The horizontal and vertical axes represent time and the amplitude of HDR curve (dashed line).
e  The two axes represent the two weights associated with the two basis functions. The whole rejection regions are not shown here, and the shaded and sold areas correspond respectively to the null hypothesis H0 space and an instantiation (and its dimension) in the alternative hypothesis H1 space. The two types of line thickness (or dot size) differentiate the two groups (or conditions).
f  The horizontal and vertical axes represent time and the amplitude of HDR curves. The two line types, dashed and dotted, differentiate the two groups or conditions.

Linear mixed-effects modeling (LME)
As demonstrated in Chen et al. (2013), linear mixed-effects modeling (LME) can be adopted for group analysis when the HDR is estimated through multiple basis functions. Specifically, the m regression coefficients from each subject associated with the m basis functions are modeled as values corresponding to m levels of a within-subject factor under the LME framework. When no other explanatory variables are present in the model, the LME methodology can be formulated by (2a) with an intercept of 0. That is, the m effects are coded by m indicator variables instead of any conventional contrast coding. Suppose that the m effect estimates associated with the m basis functions from the ith subject are βi1, βi2, …, βim, the LME model can be specified as, βij=αjxij+δi+ϵij,i=1,2,...,n,j=1,2,...,m. where the random effect δi characterizes the deviation or shift of the ith subject's HDR from the overall group HDR, the residual term ϵij indicates the deviation of each effect estimate βij from the ith subject's HDR, and the indicator variables xij take the cell mean coding, xij={1,if ith subject is at jth level,0,otherwise. so that the parameters αj, j = 1, 2, …, m capture the overall group HDR. The significance of the overall HDR at the group level can be tested through LME on the same hypothesis as (2a), (3) H0LME:α1=0,α2=0,...,αm=0.
It is of note that the LME approach does not work when other explanatory variables (multiple groups, conditions, or quantitative covariates) are involved because (2a) or (2b) cannot be formulated due to the parameterization constraint through dummy coding. For instance, when there are two groups involved, the typical contrast coding for the two groups renders one dummy variable (e.g., the contrast of one group vs. the other when effect coding is adopted); however, such a coding strategy relies on the existence of an intercept in the model. If the two groups are coded by two indicator variables, the model matrix would become overparameterized.

Area-under-the-curve (AUC)
The multiple estimates associated with the multiple basis functions can be reduced to a single value, which is the area under the curve of the estimated response function. The AUC hypotheses for the two prototypes (2a) and (2b) become (4a) H01AUC:∑j=1mαj = 0, (4b) H02AUC:∑j=1mα1j = ∑j=1mα2j.
That is, the sum of the m coefficients (or area under the HDR curve) is used to summarize the overall response amplitude per subject in one- or two-sample t-test at the group level. The AUC hypotheses (4a) and (4b) are essentially a zero-way interaction (or intercept) and a one-way interaction (or the main effect of Group or Condition) respectively and can be performed under the AN(C)OVA, GLM, or MVM framework. Their geometrical interpretations are as follows (cf. Table 1). The data for H01AUC lie on an ℝm−1 isosurface (or hyperplane) α1 + … + αm = c, and the associated test for AUC (4a) is executed on the distance between the data isosurface and the null isosurface α1 + … + αm = 0. As the correct null hypothesis for MVT (2a) is only a subset of AUC (4a), the rejection domain of AUC (4a) is only a subset of the rejection domain for MVT (2a), leading to a misrepresentation in (4a) and a detection failure when a data point lies on α1 + … + αm = 0 but not at the origin (i.e., the HDR curve has roughly equal area below and above the x-axis, e.g., a large undershoot). Similarly for H02AUC.

Euclidean distance (L2D)
As an alternate dimension reduction approach, the null hypotheses associated with the Euclidean or L2 distance (L2D) for ESM can be formulated respectively as (5a) H01L2D:(∑j=1mαj2)1/2=0, (5b) H02L2D:(∑j=1mα1j2)1/2=(∑j=1mα2j2)1/2.
In other words, one captures the overall magnitude for each subject using the L2-distance of the m regression coefficients from no response, and then performs one- or two-sample t-test on the distances.
For ASM, the null hypotheses with the focus on the canonical basis are (6a) H0CAN:α1=0, (6b) H0CAN:α11=α21.
And the null hypotheses for L2D (Calhoun et al., 2004; Steffener et al., 2010) are tested with the first two bases, (7a) H0L2D:sgn(α1)(α12+α22)1/2=0, (7b) H0L2D:sgn(α11)(α112+α122)1/2=sgn(α21)(α212+α222)1/2 or with all the three bases, (8a) H0L2D:sgn(α1)(α12+α22+α32)1/2=0, (8b) H0L2D:sgn(α11)(α112+α122+α132)1/2=sgn(α21)                                                                        (α212+α222+α232)1/2, where sgn is the sign function. That is, the L2D for ASM is similar to the L2D for ESM, but using the two or three weights associated with the two or three basis functions in ASM and assigning the sign of the canonical response to the resultant L2-distance.
Their geometrical interpretations are as follows (Table 1). The data for H01L2D lie on an ℝm−1 iso-sphere, and the associated test for (5a) is executed on the radius of the ℝm−1 iso-sphere, leading to no geometrical distortion (but not necessarily true statistically). On the other hand, the data for H02L2D are on two ℝm−1 iso-sphere surfaces, and the associated test for (5b) acts on the radius difference between the two ℝm−1 iso-spheres, resulting a detection failure when the two HDR curves have roughly the same radius.

Effect-by-component interaction (EXC: XUV and XMV)
By treating the m effect estimates from ESM as m levels of a within-subject factor Component, one can test the hypothesis for the effect-by-component interaction (EXC); that is, the m regression coefficients associated the m basis functions are taken to the group level without any condensation: (9a) H01EXC:α1=α2=...=αm, (9b) H02EXC:α11−α21=α12−α22=...=α1m−α2m.
As discussed in Chen et al. (2014), EXC (9) can be tested through two methods, one univariate testing for the interaction (XUV), and one multivariate testing for the interaction (XMV). More specifically, with XUV one tests the equality among the m components in (9) by treating them as the m levels of a within-subject factor in an AN(C)OVA or univariate GLM platform. In contrast, the equality among the m components in (9) is tested in XMV as m simultaneous variables in an MAN(C)OVA or multivariate GLM (Appendix B).
The geometrical interpretations of the hypotheses are the following (Table 1). EXC (9a) tests the main effect (or first-way interaction) of Component, representing a straight line in ℝm. The associated test for (9a) is executed on the distance between the data line and the null line (a diagonal line through the origin). As the correct null hypothesis (2a) is only a subset of H01EXC, its rejection domain is only a subset of the rejection domain for MVT (2a), leading to a misrepresentation in (9a) and a detection failure when the group centroid lies on the null line but not at the origin (i.e., the HDR curve is roughly a flat line). Similarly, EXC (9b) as a two-way interaction between Group/Condition and Component is represented by two lines, and the corresponding test acts on the distance between the two lines: are the HDR profiles parallel with each other between the two groups or conditions? As the correct null hypothesis (2b) is only a subset of EXC (9b), the rejection domain of EXC (9b) is only a subset of MVT (2b), resulting in a misrepresentation in (9b) and a detection failure when the two HDR curves are roughly parallel with each other (Table 1).