PMC:4979053 / 25649-34062 JSONTXT

{"target":"https://pubannotation.org/docs/sourcedb/PMC/sourceid/4979053","sourcedb":"PMC","sourceid":"4979053","source_url":"https://www.ncbi.nlm.nih.gov/pmc/4979053","text":"3.3. Between-Platform Agreement\nTo perform agreement evaluation, miRNA intensities were averaged across technical replicates for each array and both samples. Then, pairwise array agreements were evaluated in terms of miRNA lying within the modified agreement interval described in the Experimental Section. Estimates of the measurement error model for error-variance ratio λ equal to one, presented in Table 4, show that the relationship between Agilent and Illumina was the one that is closest to the agreement line with intercept zero and slope one for both samples. On the other hand, models that include the Affymetrix platform for line A498 showed a very negatively large intercept (−12.4128 and −17.4064), which possibly reflected the technical bias already highlighted in the previous section. However, if line hREF is considered, Affymetrix was confirmed to be the array deviating most from the line of perfect agreement with both Illumina and Agilent, whereas these two showed patterns very close to concordance (slope = 1.0925, CI 95%: 1.0371–1.1479). Since the confidence intervals for the intercept and the slope suggested an intercept different from zero and a slope different from one for all comparisons in both samples, the agreement intervals following Formula 11 were built.\nmicroarrays-03-00302-t004_Table 4 Table 4 Estimates of the linear measurement error model, λ = 1. In Figure 3 and Figure 4, graphical results for lines A498 and hREF, respectively, were reported. When a value k of tolerance equal to 0.05×813 ≃ 41 was set, thus considering the platforms to be in agreement if no more than k measurements lay outside these intervals, the pair Illumina-Agilent was the only one that could be said to be in agreement for both samples. In fact, 97.79% (CI 95%: 96.52–98.68) of miRNAs for hREF and 95.45% (CI 95%: 93.78–96.78) of miRNAs for A498 were concordant between the two platforms (795 and 776 miRNAs in agreement for cell lines hREF and A498, respectively). On the other hand, the remaining pairs showed poor patterns of concordance. The Affymetrix platform on line A498 was poorly concordant with both Agilent and Affymetrix, possibly because of the issues previously described, but also for line hREF, it showed a poor degree of agreement. In particular, only 82.53% (CI 95%: 79.75–85.08) and 82.78% (CI 95%: 80.01–85.31) of miRNAs, corresponding to 671 and 673, were found to lie within the agreement interval for comparison to Agilent and Illumina, respectively.\nFigure 3 Agreement intervals for line hREF, λ=1. (A) The comparison Agilent-Affymetrix; (B) the comparison Illumina-Affymetrix; and (C) Illumina-Agilent. The samples/miRNAS have been plotted in ascending order according to their value on the second platform in the y-axis label, so that the x-axis only contains a progressive value from one to 813 according to such ordering.\nFigure 4 Agreement intervals for line A498, λ=1. (A) The comparison Agilent-Affymetrix; (B) the comparison Illumina-Affymetrix; and (C) Illumina-Agilent. The samples/miRNAs have been plotted in ascending order according to their value on the second platform in the y-axis label, so that the x-axis only contains a progressive value from one to 813 according to such ordering. These results relied on the assumption that the ratio of the error variances from the two methods being compared was equal to one. The main advantage of this assumption was that previous knowledge of the error variances was not required; however, it was likely to be violated when the methods to be compared had very different analytical properties, such as microarray platforms (see, for instance, Figure 2). The technical replicates were used to fit a random effects model for each combination of cell line and array, estimated the error variance as the residual error of the model itself and computed the parameter λ as the ratio of the error variance of Y and X (see Table 5). The confidence intervals for λ did not include one in any comparison, thus suggesting a different measurement error for each platform.\nmicroarrays-03-00302-t005_Table 5 Table 5 Estimates of λ and CI 95%. Values were obtained as the ratio of σϵ2 (error variance of Y) and σδ2 (error variance of X), estimated via random effects models. Model parameters estimates when λ is estimated according to a random effects model are reported in Table 6. The estimates of the slopes and of the intercepts suggested that also in this case, the interval to be preferred should be the one described in Formula 11.\nmicroarrays-03-00302-t006_Table 6 Table 6 Estimates of the linear measurement error model, λ estimated. Illumina and Agilent showed the best patterns of concordance for both samples, resulting in a percentage of miRNAs in agreement exceeding the 95% threshold previously discussed: 97.54% (CI 95%: 96.23–98.49) for hREF and 96.3% (CI 95%: 94.77–97.50) for A498, corresponding to 793 and 783 miRNA (see Figure 5 and Figure 6).\nIn general, estimating the value of λ led to an increased number of miRNAs within the determined agreement interval, in particular for the Affymetrix-related comparisons (see Table 7).\nPerforming the same analysis on detection call filtered data (thus, only on 347 miRNAs and comparing Agilent and Illumina platforms only) led to similar results, though with some improvement for line A498. In particular, if λ was set to one, the proportion of concordant miRNAs was equal to 97.69% (CI 95%: 95.51–99.00) for line hREF and to 98.56% (CI 95%: 96.67–99.53) for line A498. Estimating the error variance ratio (0.629, CI 95%: 0.557–0.710 for hREF; 0943, CI 95%: 0.835–1.065 for A498) led to slightly improved results, the proportion of concordant miRNAs being equal to 99.14% for hREF (CI 95%: 96.67–99.53) and to 98.56% (CI 95%: 97.49–99.82) for A498.\nFigure 5 Agreement intervals for line hREF, λ estimated. (A) The comparison Agilent-Affymetrix; (B) the comparison Illumina-Affymetrix; and (C) Illumina-Agilent. The samples/miRNAs have been plotted in ascending order according to their value on the second platform in the y-axis label, so that the x-axis only contains a progressive value from one to 813 according to such ordering.\nFigure 6 Agreement intervals for line A498, λ estimated. (A) The comparison Agilent-Affymetrix; (B) the comparison Illumina-Affymetrix; and (C) Illumina-Agilent. The samples/miRNAs have been plotted in ascending order according to their value on the second platform in the y-axis label, so that the x-axis only contains a progressive value from one to 813 according to such ordering. The same analysis was performed also on quantile and loess normalized data, with contrasting results. For sample A498, there was a general increase in the performance of all platforms, yet only Agilent and Illumina could be considered to be concordant (net of bias correction), whereas for sample hREF, there was a general decrease, at least when λ = 1 was considered, in the proportion of concordant miRNAs. This difference was substantially relevant when Affymetrix and Illumina were compared (see Tables S8 and S12 in the Supplementary Material). The fact that differences between normalized and un-normalized data were more relevant when λ was assumed to be one could be due to the effect of the normalization procedure on the error variance ratio: in particular, when normalization is performed, there is a use of information carried by the data that can lead to a reduction in the residual error variance and in the ratio λ and, eventually, to more stable results and, thus, more concordant measurements. On the other hand, differences between platforms are already taken into account when λ is estimated, so that a normalization procedure could only limitedly improve results.\nmicroarrays-03-00302-t007_Table 7 Table 7 miRNA in agreement between arrays. Number (n) and proportion (%) of miRNAs lying in the different agreement intervals, estimated according to the measurement error model parameters estimated by setting λ = 1 and by estimating it via random effects models. Confidence intervals for the proportions were computed using the Clopper–Pearson exact method [39]. †: the platform pair is in agreement. Results for model parameter and λ estimation on normalized data are available in the Supplementary Material, from Tables S5 to S7 for quantile normalization and from S9 to S11 for loess normalization.","divisions":[{"label":"Title","span":{"begin":0,"end":31}},{"label":"Table caption","span":{"begin":1293,"end":1393}},{"label":"Figure caption","span":{"begin":2499,"end":2877}},{"label":"Figure caption","span":{"begin":2876,"end":3254}},{"label":"Table caption","span":{"begin":4071,"end":4273}},{"label":"Table caption","span":{"begin":4538,"end":4644}},{"label":"Figure caption","span":{"begin":5816,"end":6202}},{"label":"Figure caption","span":{"begin":6201,"end":6587}},{"label":"Table caption","span":{"begin":7772,"end":8212}}],"tracks":[]}