Results and discussion

ceRNA depletion on PTEN-3’UTR luciferase reporter activity
We independently replicated an experiment to test if putative PTEN ceRNAs modulate the 3’UTR of PTEN. This experiment used a chimeric luciferase construct tagged with the PTEN 3’UTR (Luc-PTEN-3’UTR) to uncouple regulation of PTEN via 3’UTR-targeting microRNAs from PTEN mRNA transcription and protein stability. This is similar to what was reported in Figure 3C and Supplemental Figure S3A of Tay et al., 2011 and described in Protocol 1 in the Registered Report (Phelps et al., 2016). DU145 cells were co-transfected with Luc-PTEN-3’UTR and siRNAs targeting the same putative PTEN ceRNAs as the original study. Knockdown efficiency was examined by reverse transcription-quantitative polymerase chain reaction (RT-qPCR). The average reduction in gene expression relative to control siRNA was 65% when SERINC1, VAPA, CNOT6L, or PTEN were targeted, but was only 21% for ZNF460 (Figure 1—figure supplement 1C), despite transfection efficiency being at least 90% based on a fluorescent transfection indicator (Figure 1—figure supplement 1A). Luciferase activity was decreased in PTEN depleted cells (average RLU = 12%) relative to control siRNA (average RLU = 100%); however, luciferase activity when the putative PTEN ceRNAs were targeted for depletion were similar to control siRNA (Figure 1, Figure 1—figure supplement 1B). All planned comparisons were not statistically significant (see Figure 1 legend). The original study reported statistically significant decreased luciferase activity with siRNA-mediated depletion of SERINC1 (average RLU = 70%), VAPA (average RLU = 48%), CNOT6L (average RLU = 70%), or PTEN (average RLU = 20%), but not for knockdown of ZNF460 (average RLU = 109%), compared to control siRNA (average RLU = 100%) (Tay et al., 2011). The range of luciferase values reported in the original study had relative standard deviations (RSDs) (control = 9%; SERINC1 = 10%; VAPA = 6%; CNOT6L = 5%; ZNF460 = 8%; PTEN = 5%) that were much smaller than the RSDs observed in this replication attempt (control = 36%; SERINC1 = 57%; VAPA = 36%; CNOT6L = 50%; ZNF460 = 33%; PTEN = 19%), which is one of the factors that could influence if statistical significance is reached, particularly since the sample size of this replication attempt was determined a priori to detect the effect based on the originally reported data. The original study also reported an achieved knockdown of 90% or greater when SERINC1, VAPA, CNOT6L, or PTEN were targeted, but was 65% for ZNF460 (Tay et al., 2011). The difference in achieved knockdown between the original study and this replication attempt is a possible reason for the differences in Luc-PTEN-3’UTR outcomes. A higher level of knockdown might be required to observe an effect with this experimental design. Although, unlike experiments that evaluate protein function where a higher level of knockdown or a longer period of time is usually needed to observe a phenotype (Curtis and Nardulli, 2009; O’Keefe, 2013), the putative ceRNA function of these mRNAs should correspond to the level of knockdown. Thus, a 65% knockdown would have been expected to capture ~72% of the effect observed in the original study that reported a 90% knockdown. To summarize, for this experiment, we found results that were not statistically significant where predicted, varied in direction relative to the original study for the putative PTEN ceRNAs, and in the same direction as the original study for cells transfected with siPTEN.
Figure 1.  Luciferase activity in DU145 cells co-transfected with siRNA against PTEN ceRNAs and a luciferase-PTEN 3’UTR reporter construct.
DU145 cells were transfected with a luciferase reporter with a fragment of the 3’UTR of PTEN. Cells were also co-transfected with non-targeting control siRNA (siNC) or siRNA plasmids targeting SERINC1 (siSER), ZNF460 (siZNF), VAPA (siVAPA), CNOT6L (siCNO), or PTEN (siPTEN). Cells were harvested 72 hr later for luciferase activity. Relative luminescence unit (RLU) is presented for each condition relative to the siNC condition. Means reported and error bars represent SD from four independent biological repeats. Two-sample t-test of RLU values between siNC and siSER: t(6) = 0.177, uncorrected p=0.866 with a priori Bonferroni adjusted significance threshold of 0.01, Bonferroni corrected p>0.99; siNC and siZNF: t(6) = 0.899, uncorrected p=0.403, Bonferroni corrected p>0.99; siNC and siVAPA: t(6) = 0.225, uncorrected p=0.829, Bonferroni corrected p>0.99; siNC and siCNO: t(6) = 0.426, uncorrected p=0.685, Bonferroni corrected p>0.99; Wilcoxon-Mann-Whitney test of RLU values between siNC and siPTEN: U = 16, uncorrected p=0.029, Bonferroni corrected p=0.143. Additional details for this experiment can be found at https://osf.io/spv4f/.
Figure 1—figure supplement 1.  Knockdown efficiency and individual repeats of luciferase-PTEN 3’UTR reporter assay in DU145 cells co-transfected with siRNA against PTEN ceRNAs.
This is the same experiment as Figure 1. (A) Representative microscopy images (10X magnification) of DY-547-labeled siGLO RISC-Free control transfected DU145 cells 48 hr after transfection. Transfection efficiency was estimated to be >90%. (B) Independent biological repeats of luciferase reporter assay. Relative luminescence unit (RLU) is presented for each condition relative to the siNC condition. Means reported and error bars represent SD from three technical replicates. (C) Independent biological repeats of RT-qPCR analysis. Expression of each transcript after transfection of its respective siRNA relative to negative control transfection (siNC) is presented. Transcripts listed on y-axis. Means reported and error bars represent SD from three technical replicates. One-sample t-tests of transcript expression data after transfection of respective siRNA to a constant of 1 (relative value of siNC). SERINC1: t(3) = 45.8, uncorrected p=2.30×10−5, Bonferroni corrected p=1.15×10−4; ZNF460: t(3) = 2.98, uncorrected p=0.059, Bonferroni corrected p=0.294; VAPA: t(3) = 8.94, uncorrected p=0.0030, Bonferroni corrected p=0.015; CNOT6L: t(3) = 11.3, uncorrected p=0.0015, Bonferroni corrected p=0.0074; PTEN: t(3) = 11.2, uncorrected p=0.0015, Bonferroni corrected p=0.0077. Additional details for this experiment can be found at https://osf.io/spv4f/.

ceRNA overexpression on PTEN-3’UTR luciferase reporter activity
To test if sequestration of the putative PTEN ceRNAs impacted PTEN expression, we ectopically overexpressed the 3’UTR of the same putative PTEN ceRNAs as the original study in DU145 cells along with the Luc-PTEN-3’UTR plasmid. This is similar to what was reported in Figure 3D of Tay et al., 2011 and described in Protocol 2 in the Registered Report (Phelps et al., 2016). We used the same plasmids as the original study, which cloned the 3’UTRs of VAPA and CNOT6L as two separate fragments due to their large size with the fragments subdivided based on location of predicted MREs (Tay et al., 2011). We found that compared to cells transfected with empty vector control, cells transfected with 3’UTR of the putative PTEN ceRNA plasmids or the 3’UTR of PTEN had decreased luciferase activity (Figure 2, Figure 2—figure supplement 1). The planned comparisons were statistically significant for SERINC1 3’UTR, VAPA 3’UTR2, CNOT6L 3’UTR1, CNOT6L 3’UTR2, and PTEN 3’UTR1, but not for VAPA 3’UTR1 (see Figure 2 legend). The original study reported statistically significant increased luciferase activity with SERINC1 3’UTR (average RLU = 128%), VAPA 3’UTR1 (average RLU = 141%), VAPA 3’UTR2 (average RLU = 150%), CNOT6L 3’UTR1 (average RLU = 143%), CNOT6L 3’UTR2 (average RLU = 146%), or PTEN 3’UTR (average RLU = 153%) compared to empty vector control (average RLU = 100%) (Tay et al., 2011). The range of luciferase values reported in the original study had RSDs (control = 9%; SERINC1 3’UTR = 9%; VAPA 3’UTR1 = 13%; VAPA 3’UTR2 = 6%; CNOT6L 3’UTR1 = 7%; CNOT6L 3’UTR2 = 7%; PTEN 3’UTR = 1%) that were smaller than the RSDs observed in this replication attempt (control = 17%; SERINC1 3’UTR = 12%; VAPA 3’UTR1 = 12%; VAPA 3’UTR2 = 15%; CNOT6L 3’UTR1 = 15%; CNOT6L 3’UTR2 = 14%; PTEN 3’UTR = 10%). To summarize, we found results that were statistically significant (with the exception of VAPA 3’UTR1) and in the opposite direction as the original study.
Figure 2.  Luciferase activity in DU145 cells co-transfected with 3’UTR of PTEN ceRNAs and a luciferase-PTEN 3’UTR reporter construct.
DU145 cells were transfected with a luciferase reporter with a fragment of the 3’UTR of PTEN. Cells were also co-transfected with empty vector (EV) or plasmids that express the 3’UTR of SERINC1 (SER 3’U), VAPA (VAPA 3’U1 and VAPA 3’U2), CNOT6L (CNOT 3’U1 and CNOT 3’U2), or PTEN (PTEN 3’U). Cells were harvested 72 hr later for luciferase activity. Relative luminescence unit (RLU) is presented for each condition relative to the EV condition. Means reported and error bars represent SD from six independent biological repeats. Two-sample t-test of RLU values between SER 3’U and EV: t(10) = 3.32, uncorrected p=0.0077 with a priori Bonferroni adjusted significance threshold of 0.0083, Bonferroni corrected p=0.046; VAPA 3’U1 and EV: t(10) = 3.13, uncorrected p=0.011, Bonferroni corrected p=0.064; VAPA 3’U2 and EV: t(10) = 4.83, uncorrected p=6.90×10−4, Bonferroni corrected p=0.0041; CNOT 3’U1 and EV: t(10) = 4.42, uncorrected p=0.0013, Bonferroni corrected p=0.0078; CNOT 3’U2 and EV: t(7.1) = 5.09, uncorrected p=0.0014, Bonferroni corrected p=0.0082; PTEN 3’U and EV: t(5.6) = 7.50, uncorrected p=3.99×10−4, Bonferroni corrected p=0.0024. Additional details for this experiment can be found at https://osf.io/mryvq/.
Figure 2—figure supplement 1.  Individual repeats of luciferase-PTEN 3’UTR reporter assay in DU145 cells co-transfected with 3’UTR of PTEN ceRNAs.
This is the same experiment as Figure 2. Independent biological repeats of luciferase reporter assay. Relative luminescence unit (RLU) is presented for each condition relative to the EV condition. Means reported and error bars represent SD from three technical replicates. Additional details for this experiment can be found at https://osf.io/mryvq/.

ceRNA depletion on PTEN expression
We replicated an experiment to test the microRNA dependency of the putative PTEN ceRNAs. This experiment used the same isogenic wild-type and DICER mutant (DicerEx5) HCT116 colon carcinoma cells as the original study. The DicerEx5 cell line, which was engineered to disrupt a well-conserved segment of the N-terminal helicase domain in exon 5 of DICER, while leaving the RNase III domains intact, displays a hypomorphic phenotype in the processing of mature microRNAs (Cummins et al., 2006). This experiment is similar to what was reported in Figure 3G–H and Supplemental Figure S3B of Tay et al., 2011 and described in Protocol 3 in the Registered Report (Phelps et al., 2016). Wild-type and DicerEx5 HCT116 cells were transfected with siRNAs targeting the same putative PTEN ceRNAs as the original study. Knockdown efficiency, measured by RT-qPCR, revealed the average reduction in gene expression relative to control siRNA was 81% in both cell lines for all putative PTEN ceRNAs, with the greatest biological variability in DicerEx5 HCT116 cells when targeting CNOT6L (Figure 3—figure supplement 1B).
Depletion of the putative PTEN ceRNAs resulted in downregulation of PTEN protein in wild-type HCT116 cells to an average of 80%, 43%, or 61% for siRNA-mediated depletion of SERINC1, VAPA, or CNOT6L, respectively, relative to control siRNA (average PTEN expression = 100%) (Figure 3A–B, Figure 3—figure supplement 1A). As a control, siRNAs targeting PTEN reduced PTEN protein levels to an average of 1.6%. To compare the relative PTEN expression among the various conditions, we planned to conduct four comparisons using the Bonferroni correction to adjust for multiple comparisons. The comparison of PTEN protein levels between control siRNA and siRNA targeting VAPA, CNOT6L, or PTEN were statistically significant, while the comparison of control siRNA and siRNA targeting SERINC1 were not (see Figure 3 legend). The original study reported statistically significant decreases in PTEN protein levels with siRNA-mediated depletion of SERINC1 (average PTEN expression = 53%), VAPA (average PTEN expression = 52%), CNOT6L (average PTEN expression = 59%), or PTEN (average PTEN expression = 1.9%) compared to control siRNA (average PTEN expression = 100%) in wild-type HCT116 cells (Tay et al., 2011).
Figure 3.  PTEN protein expression in wild-type and DICER mutant HCT116 cells depleted of PTEN ceRNAs.
Wild-type (WT) and DICER mutant (DicerEx5) HCT116 cells were transfected with non-targeting control siRNA (siNC) or siRNA plasmids targeting SERINC1 (siSER), VAPA (siVAPA), CNOT6L (siCNO), or PTEN (siPTEN). Cells were harvested 72 hr later for Western blot analysis. (A) Relative protein expression (PTEN/HSP90) are presented for each condition. Western blot bands were quantified, PTEN levels were normalized to HSP90, with protein expression presented relative to siNC. Means reported and error bars represent SD from three independent biological repeats for wild-type HCT116 cells and four repeats for DicerEx5 HCT116 cells. Analysis of wild-type HCT116 cells: one-way ANOVA (equal variance) on PTEN/HSP90 expression: F(4,10) = 25.4, I=3.18×10−5. Planned contrasts between siNC and siSER: t(10) = 1.94, uncorrected I=0.082 with a priori Bonferroni adjusted significance threshold of 0.0125, Bonferroni corrected p=0.326; siNC and siVAPA: t(10) = 5.44, uncorrected p=2.85×10−4, Bonferroni corrected p=0.0011; siNC and siCNOT: t(10) = 3.69, uncorrected p=0.0042, Bonferroni corrected p=0.017; siNC and siPTEN: t(10) = 9.34, uncorrected p=2.97×10−6, Bonferroni corrected p=1.19×10−5. Analysis of DicerEx5 HCT116 cells: one-way ANOVA (unequal variance) on PTEN/HSP90 expression: F(4,6.0) = 19.3, p=0.0014. Planned comparisons: siNC and siSER: two-sample t-test, t(6) = 3.96, uncorrected p=0.0074 with a priori Bonferroni adjusted significance threshold of 0.0125, Bonferroni corrected p=0.030; siNC and siVAPA: two-sample t-test, t(6) = 0.896, uncorrected p=0.405, Bonferroni corrected p>0.99; siNC and siCNOT: Welch’s t-test, t(4.36) = 2.92, uncorrected p=0.039, Bonferroni corrected p=0.156; siNC and siPTEN: two-sample t-test, t(6) = 4.15, uncorrected p=0.0060, Bonferroni corrected p=0.024. (B) Representative Western blots probed with an anti-PTEN antibody and anti-HSP90 antibody. Additional details for this experiment can be found at https://osf.io/drcbw/.
Figure 3—figure supplement 1.  Knockdown efficiency and individual repeats of PTEN protein expression in wild-type and DICER mutant HCT116 cells transfected with siRNA against PTEN ceRNAs.
This is the same experiment as Figure 1. (A) Independent biological repeats of Western blot assay. PTEN/HSP90 protein expression is presented for each condition relative to the siNC condition. (B) Independent biological repeats of RT-qPCR analysis. Expression of each transcript after transfection of its respective siRNA relative to negative control transfection (siNC) is presented. Transcripts listed on y-axis. Means reported and error bars represent SD from three technical replicates. One-sample t-tests of transcript expression data after transfection of respective siRNA to a constant of 1 (relative value of siNC). Wild-type HCT116 cells: SERINC1: t(2) = 8.41, uncorrected p=0.014, Bonferroni corrected p=0.055; VAPA: t(2) = 120, uncorrected p=6.98×10−5, Bonferroni corrected p=2.79×10−4; CNOT6L: t(2) = 10.4, uncorrected p=0.0091, Bonferroni corrected p=0.036; PTEN: t(2) = 30.6, uncorrected p=0.0011, Bonferroni corrected p=0.0043. DicerEx5 HCT116 cells: SERINC1: t(2) = 28.5, uncorrected p=0.0012, Bonferroni corrected p=0.0049; VAPA: t(2) = 48.3, uncorrected p=4.28×10−4, Bonferroni corrected p=0.0017; CNOT6L: t(2) = 3.23, uncorrected p=0.084, Bonferroni corrected p=0.336; PTEN: t(2) = 242, uncorrected p=1.71×10−5, Bonferroni corrected p=6.83×10−5. Additional details for this experiment can be found at https://osf.io/drcbw/. For DicerEx5 HCT116 cells, we found depletion of the putative PTEN ceRNAs resulted in higher PTEN protein levels (SERINC1: 290%; VAPA: 144%; CNOT6L: 256%) relative to control siRNA (average PTEN expression = 100%), while targeting PTEN reduced PTEN protein levels to an average of 2.6%. (Figure 3A–B, Figure 3—figure supplement 1A). To compare the relative PTEN expression among the various conditions, a similar analysis as described above for wild-type HCT116 cells was performed for DicerEx5 HCT116 cells. We found that PTEN protein levels between control siRNA and siRNA targeting SERINC1 or PTEN were statistically significant, while the comparisons between control siRNA and siRNA targeting VAPA or CNOT6L were not (see Figure 3 legend). The original study reported PTEN downregulation by ceRNA depletion was attenuated in DicerEx5 HCT116 cells with average PTEN expression around the same as control siRNA (control siRNA: 100%; SERINC1: 117%; VAPA: 108%; CNOT6L: 113%), while the average PTEN expression in cells transfected with siRNA-mediated depletion of PTEN was 1.3% (Tay et al., 2011). Similar to the siRNA-mediated depletion of putative PTEN ceRNA in DU145 cells described above, the original study reported a knockdown of greater than 90% for most conditions (Tay et al., 2011). The level of knockdown required to yield a given phenotype varies because it is system-dependent (Bailoo et al., 2014), thus the difference in achieved knockdown between the original study and this replication attempt should be considered when interpreting these results. Further, the original study reported lower RSDs for PTEN protein levels across all the siRNA conditions in the DicerEx5 HCT116 cells compared to the wild-type HCT116 cells (DicerEx5: 0.1–9% vs wild-type: 8–17%), while this replication attempt observed larger RSDs compared to the original study, especially for DicerEx5 HCT116 cells (DicerEx5: 29–61% vs wild-type: 12–46%). Importantly, the individual biological repeats were largely consistent relative to the control siRNA condition (Figure 3—figure supplement 1A). This difference in variance between the original study and this replication attempt could influence if statistical significance is reached. To summarize, for this experiment, we found results that were generally in the same direction as the original study, varied in terms of statistical significance, and in DicerEx5 HCT116 cells effects that were of a larger magnitude than the original study. This absence of an attenuated ceRNA effect in this replication attempt suggests the null hypothesis that there is no difference in PTEN protein expression when the microRNA machinery is disrupted can be rejected.

ceRNA depletion on cell proliferation
We replicated an experiment to evaluate cell proliferation of DU145, wild-type HCT116, and DicerEx5 HCT116 cells in response to siRNA-mediated silencing of the putative PTEN ceRNAs. This is similar to what was reported in Figure 5B of Tay et al., 2011 and described in Protocol 4 in the Registered Report (Phelps et al., 2016). Cells were transfected with siRNAs targeting the same putative PTEN ceRNAs as the original study. Knockdown efficiency, measured by RT-qPCR, revealed an average reduction in gene expression relative to control siRNA was 79% when considering all cell lines (Figure 4—figure supplement 1B). Proliferation activity was determined using the crystal violet assay starting the day after transfection with results presented as the difference in the values at the start of the timecourse for each condition (i.e. for each condition the value at the start of the timecourse was set to 0), similar to the original study. For DU145 cells, we found that siRNA-mediated depletion of VAPA or PTEN resulted in increased cell proliferation compared to cells transfected with control siRNA, while depletion of CNOT6L resulted in decreased cell proliferation (Figure 4, Figure 4—figure supplement 1A). The area under the curve (AUC) during the timecourse for each biological repeat was used to compare each condition to the control siRNA, which were not statistically significant (see Figure 4 legend). The original study reported siRNA-mediated targeting of VAPA, CNOT6L, or PTEN in DU145 cells resulted in a statistically significant increase in proliferation compared to control siRNA (Tay et al., 2011). The range of AUC values reported in the original study had a RSD for the control condition (26%) similar to this replication study (20%); however, the RSDs for the other conditions were much lower in the original study (VAPA = 4%; CNOT6L = 9%; PTEN = 7%) then this replication attempt (VAPA = 26%; CNOT6L = 20%; PTEN = 10%). As stated above this difference in variance between the original study and this replication attempt is a factor that could influence if statistical significance is reached.
Figure 4.  Growth of cells depleted of PTEN ceRNAs.
DU145, wild-type (WT) and DICER mutant (DicerEx5) HCT116 cells were transfected with either a non-targeting control siRNA (siNC) or siRNA plasmids targeting VAPA (siVAPA), CNOT6L (siCNO), or PTEN (siPTEN). Crystal violet proliferation assays were performed each day as indicated starting the day after transfection. Relative OD590 was calculated relative to the average Day 0 values for each condition. Means reported and error bars represent SD from five independent biological repeats for DU145 cells and four times for HCT116 WT and DicerEx5 cells. Analysis on the area under the curve (AUC) for each condition of each biological repeat (reported as dot plot in Figure 4—figure supplement 1A). Analysis results for DU145 cells: one-way ANOVA (equal variance): F(3,16) = 3.27, p=0.049. Planned contrasts between siNC and siVAPA: t(16) = 0.648, uncorrected p=0.526 with a priori Bonferroni adjusted significance threshold of 0.0167, Bonferroni corrected p>0.99; siNC and siCNOT6L: t(16) = 0.950, uncorrected p=0.356, Bonferroni corrected p>0.99; siNC and siPTEN: t(16) = 2.09, uncorrected p=0.053, Bonferroni corrected p=0.158. Analysis of HCT116 cells: two-way ANOVA interaction between DICER status (wild-type or Ex5) and siRNA target: F(3,24) = 0.734, p=0.542; main effect of DICER status: F(1,24) = 1.81, p=0.191; main effect of siRNA target: F(3,24) = 12.1, p=5.20×10−5. Planned contrasts in HCT116 WT cells: siNC and siVAPA: t(24) = 2.02, uncorrected p=0.054 with a priori Bonferroni adjusted significance threshold of 0.0083, Bonferroni corrected p=0.325; siNC and siCNOT6L: t(24) = 0.506, uncorrected p=0.618, Bonferroni corrected p>0.99; siNC and siPTEN: t(24) = 3.03, uncorrected p=0.0057, Bonferroni corrected p=0.034. Planned contrasts in HCT116 DICEREx5 cells: siPTEN and siVAPA: t(24) = 2.43, uncorrected p=0.023, Bonferroni corrected p=0.138; siPTEN and siCNOT6L: t(24) = 4.57, uncorrected p=1.25×10−4, Bonferroni corrected p=7.48×10−4; siNC and siPTEN: t(24) = 4.31, uncorrected p=2.42×10−4, Bonferroni corrected p=0.0015. Additional details for this experiment can be found at https://osf.io/5c7sb/.
Figure 4—figure supplement 1.  Knockdown efficiency and individual repeats of cell growth assay in cells transfected with siRNA against PTEN ceRNAs.
This is the same experiment as Figure 5. (A) Independent biological repeats of cell growth assay. Dot plot with means reported as crossbars and error bars represent SD. (B) Independent biological repeats of RT-qPCR analysis. Expression of each transcript after transfection of its respective siRNA relative to negative control transfection (siNC) is presented. Transcripts listed on y-axis. Means reported and error bars represent SD from three technical replicates. One-sample t-tests of transcript expression data after transfection of respective siRNA to a constant of 1 (relative value of siNC). DU145 cells: VAPA: t(4) = 58.6, uncorrected p=5.09×10−7, Bonferroni corrected p=1.53×10−6; CNOT6L: t(4) = 36.3, uncorrected p=3.42×10−6, Bonferroni corrected p=1.03×10−5; PTEN: t(4) = 9.05, uncorrected p=8.27×10−4, Bonferroni corrected p=0.0025. Wild-type HCT116 cells: VAPA: t(3) = 18.6, uncorrected p=3.40×10−4, Bonferroni corrected p=0.0010; CNOT6L: t(3) = 108, uncorrected p=1.73×10−6, Bonferroni corrected p=5.19×10−6; PTEN: t(3) = 28.1, uncorrected p=9.90×10−5, Bonferroni corrected p=2.97×10−4. DicerEx5 HCT116 cells: VAPA: t(3) = 14.8, uncorrected p=6.73×10−4, Bonferroni corrected p=0.0020; CNOT6L: t(3) = 27.9, uncorrected p=1.02×10−4, Bonferroni corrected p=3.05×10−4; PTEN: t(3) = 178, uncorrected p=3.89×10−7, Bonferroni corrected p=1.17×10−6. Additional details for this experiment can be found at https://osf.io/5c7sb/. For wild-type HCT116 cells, we found that compared to cells transfected with control siRNA, depletion of VAPA, CNOT6L, or PTEN resulted in different levels of increased cell proliferation (Figure 4, Figure 4—figure supplement 1A). For DicerEx5 HCT116 cells, depletion of PTEN resulted in increased cell proliferation compared to control siRNA with a similar magnitude as wild-type cells, depletion of VAPA resulted in an increased proliferation compared to control siRNA, but not at the same magnitude as occurred in wild-type cells, while depletion of CNOT6L resulted in a slight decrease in cell proliferation compared to control siRNA. To test if depletion of the putative PTEN ceRNAs increased proliferation in wild-type HCT116 cells and were attenuated in the DicerEx5 HCT116 cells, we performed an analysis of variance (ANOVA) on the AUC for each biological repeat. The ANOVA result was statistically significant for the siRNA main effect (F(3,24) = 12.1, p=5.20×10−5). Thus, the null hypothesis that there is no difference in cell proliferation when the putative PTEN ceRNAs or PTEN was depleted, whether or not it was conducted in wild-type or DicerEx5 HCT116 cells, can be rejected. The main effect for cell type (F(1,24) = 1.81, p=0.191) was not statistically significant, indicating the null hypothesis that there is no difference in cell proliferation between wild-type or DicerEx5 HCT116 cells can not be rejected, and the interaction effect was not statistically significant (F(3,24) = 0.734, p=0.542). These results suggest that while there were differences in cell proliferation when the putative PTEN ceRNAs or PTEN were depleted, it was similar between cell lines suggesting a lack of an attenuated ceRNA effect. We also conducted six comparisons using the Bonferroni correction to adjust for multiple comparisons, making the a priori adjusted significance threshold 0.0083. According to this criterion, depletion of PTEN in wild-type or DicerEx5 HCT116 cells resulted in statistically significant increases in cell proliferation compared to control siRNA. Depletion of VAPA or CNOT6L did not result in a statistically significant increase in cell proliferation compared to control siRNA in wild-type HCT116 cells. Additionally, depletion of CNOT6L, but not VAPA, resulted in a statistically significant decrease in cell proliferation compared to PTEN-depleted DicerEx5 HCT116 cells. The original study reported reduced expression of VAPA or CNOT6L in wild-type HCT116 cells resulted in a statistically significant increase in cell proliferation compared to control siRNA similar to what was observed with PTEN siRNA, which was statistically significantly attenuated in the DicerEx5 HCT116 cells (Tay et al., 2011). Further, the original study (DicerEx5: 1–15%; wild-type: 5–10%) and this replication attempt (DicerEx5: 6–19%; wild-type: 5–10%) observed similar RSDs. To summarize, for this experiment we found results that varied in statistical significance and varied in direction relative to the original study for the putative PTEN ceRNAs, but were in the same direction as the original study for cells transfected with siPTEN.

Meta-analyses of original and replication effects
We performed a meta-analysis using a random-effects model, where possible, to combine each of the effects described above as pre-specified in the confirmatory analysis plan (Phelps et al., 2016). To provide a standardized measure of the effect, a common effect size was calculated for each effect from the original and replication studies. Cohen’s d is the standardized difference between two means using the pooled sample standard deviation, while the effect size Glass’ delta is the standardized difference between two means using the standard deviation of only the control group. Glass’ delta was used when the variance between the control and treatment conditions were not equal in the original or replication study experiments. The estimate of the effect size of one study, as well as the associated uncertainty (i.e. confidence interval), compared to the effect size of the other study provides one approach to compare the original and replication results (Errington et al., 2014; Valentine et al., 2011). Importantly, the width of the confidence interval (CI) for each study is a reflection of not only the confidence level (e.g. 95%), but also variability of the sample (e.g. SD) and sample size.
There were five comparisons of the PTEN-3’UTR luciferase reporter activity when putative PTEN ceRNAs were depleted, which were reported in Figure 1 of this study and Figure 3C of Tay et al., 2011. Only one of the effects, control siRNA compared to ZNF460 siRNA, was consistent in direction and when considering if the effect size point estimate of each study was within the confidence interval of the other study, suggesting the null hypothesis that there is no difference in reporter activity can not be rejected (Figure 5A). The other effects were inconsistent in whether the direction of the effect was the same between the two studies, if the effect size of one study was within the confidence interval of the other study, or both. Additionally, the meta-analyses were not statistically significant, with all but one of the effects having large confidence intervals around the meta-analysis effect size along with statistically significant Cochran’s Q tests (siNC and siSERINC1, p=0.013; siNC and siVAPA, p=0.0019; siNC and siPTEN, p=0.0054) suggesting heterogeneity between the original and replication studies.
Figure 5.  Meta-analyses of each effect.
Effect size and 95% confidence interval are presented for Tay et al., 2011, this replication study (RP:CB), and a random effects meta-analysis of those two effects. For each effect, Cohen’s d or Glass’ delta, which are standardized differences between the two indicated measurements, is reported. Sample sizes used in Tay et al., 2011 and RP:CB are reported under the study name. (A) These effects are related to the change in luciferase activity between the conditions reported in Figure 1 of this study and Figure 3C of Tay et al., 2011. Meta-analysis p values: siNC and siSER (p=0.374); siNC and siZNF (p=0.233); siNC and siVAPA (p=0.316); siNC and siCNO (p=0.253); siNC and siPTEN (p=0.079). (B) These effects are related to the change in luciferase activity between the conditions reported in Figure 2 of this study and Figure 3D of Tay et al., 2011. Meta-analysis p values: SER 3’U and EV (p=0.881); VAPA 3’U1 and EV (p=0.624); VAPA 3’U2 and EV (p=0.754); CNO 3’U1 and EV (p=0.790); CNO 3’U2 and EV (p=0.716); PTEN 3’U and EV (p=0.766). (C) These effects are related to the differences in PTEN protein expression between the conditions reported in Figure 3 of this study and Figure 3H of Tay et al., 2011. Meta-analysis p values: WT HCT116: siNC and siSER (p=0.091); siNC and siVAPA (p=2.99×10−7); siNC and siCNO (p=0.049); siNC and siPTEN (p=0.0041): DicerEx5 HCT116: siNC and siSER (p=2.78×10−4); siNC and siVAPA (p=0.215); siNC and siCNO (p=3.70×10−4); siNC and siPTEN (p=0.229). (D) These effects are related to the differences in cell growth between the conditions reported in Figure 4 of this study and Figure 5B of Tay et al., 2011. Meta-analysis p values: DU145: siVAPA and siNC (p=0.255); siCNO and siNC (p=0.554); siPTEN and siNC (p=0.082): WT HCT116: siVAPA and siNC (p=0.045); siCNO and siNC (p=0.287); siPTEN and siNC (p=0.082): DicerEx5 HCT116: siVAPA and siPTEN (p=0.075); siCNO and siPTEN (p=0.001); siPTEN and siNC (p=0.048). Additional details for these meta-analyses can be found at https://osf.io/xgrqp/. PTEN-3’UTR luciferase reporter activity was also tested when the 3’UTR of putative PTEN ceRNAs were ectopically overexpressed, reported in Figure 2 of this study and Figure 3D of Tay et al., 2011. The direction of all six comparisons were in the opposite direction of the original study with none of the effect size point estimates within the confidence intervals of the other study (Figure 5B). The meta-analyses were not statistically significant, with all effects having large confidence intervals around the meta-analysis effect size along with statistically significant Cochran’s Q tests (SERINC1 3’UTR and empty vector, p=0.0011; VAPA 3’UTR1 and empty vector, p=2.55×10−4; VAPA 3’UTR2 and empty vector, p=1.28×10−6; CNOT6L 3’UTR1 and empty vector, p=1.07×10−5; CNOT6L 3’UTR2 and empty vector, p=2.54×10−5; PTEN 3’UTR and empty vector, p=1.86×10−6) suggesting heterogeneity between the original and replication studies.
PTEN protein expression was examined in two cell lines, wild-type and DicerEx5 HCT116 cells, following depletion of putative PTEN ceRNAs with four comparisons made in each cell line, which were reported in Figure 3 of this study and Figure 3G–H of Tay et al., 2011. In wild-type cells, all effects were consistent when considering the direction of the effect and varied in whether the studies were within the confidence interval of the other study (Figure 5C). The meta-analysis of one of the effects, control siRNA compared to SERINC1 siRNA, was not statistically significant suggesting the null hypothesis that there is no difference in PTEN protein expression can not be rejected; however, the large confidence intervals along with a statistically significant Cochran’s Q tests (p=0.024) suggests heterogeneity between the original and replication studies. The meta-analyses of the other effects were statistically significant suggesting the null hypothesis can be rejected and that these ceRNAs regulate PTEN protein expression in HCT116 cells. In DicerEx5 cells all effects were consistent when considering the direction of the effect and three had effect size point estimates of the original study that was within the confidence interval of the replication and vice versa. The meta-analyses of control siRNA compared to SERINC1 or CNOT6L were statistically significant, suggesting the null hypothesis that there is no difference in PTEN protein expression when the microRNA machinery is disrupted can be rejected. The other two meta-analyses were not statistically significant suggesting the null hypothesis can not be rejected; however, for the control siRNA to PTEN siRNA comparison, the large confidence intervals along with a statistically significant Cochran’s Q tests (p=2.11×10−9) suggests heterogeneity between the original and replication studies.
Cell proliferation was also tested when putative PTEN ceRNAs were depleted, with three comparisons made in each cell line as reported in Figure 4 of this study and Figure 5B of Tay et al., 2011. In wild-type and DicerEx5 HCT116 cells all effects were consistent when considering the direction of the effect and varied in whether the studies were within the confidence interval of the other study (Figure 5D). Additionally, the meta-analyses varied in terms of statistical significance with some meta-analyses having wide confidence intervals and statistically significant Cochran’s Q tests (WT HCT116: siVAPA and siNC, p=0.048; siCNOT6L and siNC, p=2.79×10−4; siPTEN and siNC, p=0.0079; DicerEx5 HCT116: siPTEN and siNC, p=0.029). In DU145 cells, the effects were inconsistent in whether the direction of the effect was the same between the two studies or if the effect size of one study was within the confidence interval of the other study. For all effects, the meta-analysis was not statistically significant, with wide confidence intervals, and statistically significant Cochran’s Q tests (DU145: siVAPA and siNC, p=0.046; siCNOT6L and siNC, p=0.015, siPTEN and siNC, p=0.037), suggesting study heterogeneity.
This direct replication provides an opportunity to understand the present evidence of these effects. Any known differences, including reagents and protocol differences, were identified prior to conducting the experimental work and described in the Registered Report (Phelps et al., 2016). However, this is limited to what was obtainable from the original paper and through communication with the original authors, which means there might be particular features of the original experimental protocol that could be critical, but unidentified. So while some aspects, such as cell lines, antibodies, and the specific siRNA sequences and plasmids were maintained, others were unknown or not easily controlled for. These include variables such as cell line genetic drift (Ben-David et al., 2018; Hughes et al., 2007; Kleensang et al., 2016) and impacts of atmospheric oxygen on cell viability and growth (Boregowda et al., 2012). Whether these or other factors influence the outcomes of this study is open to hypothesizing and further investigation, which is facilitated by direct replications and transparent reporting.