4. Criteria and Methodology for Evaluation of Performance Measures
As mentioned before, the main purpose of kinome array experiments is to detect differentially phosphorylated peptides. Variance stabilization is a preprocessing step to increase accuracy of various downstream analyses to detect such peptides. Therefore, we suggest and use a set of quantitative performance measures to evaluate the effect of variance stabilization methods on the performance of peptide classification.

4.1. Criteria for Evaluation of Performance Measures
In this paper, we use sensitivity, specificity, precision, and accuracy as performance measures for peptide classification [26]. We interpret the word “positive” as “differentially phosphorylated” and the term “negative” as “non-differentially phosphorylated”. In addition, we use the following notations: ∥∥ operator denotes the size of a set.
TP (True Positives): the set of all differentially phosphorylated peptides predicted as differentially phosphorylated.
FN (False Negatives): the set of all differentially phosphorylated peptides predicted as non-differentially phosphorylated.
TN (True Negatives): the set of all non-differentially phosphorylated peptides predicted as non-differentially phosphorylated.
FP (False Positives): the set of all non-differentially phosphorylated peptides predicted as differentially phosphorylated.
The specificity criterion shows the proportion of all true negatives classified correctly, and is defined as follows: (2) Specificity=∥TN∥∥TN∥+∥FP∥
The sensitivity score, which is also referred to as recall, shows the proportion of all positives classified correctly, and is defined as follows: (3) Sensitivity=∥TP∥∥TP∥+∥FN∥
The precision criterion shows the proportion of all true positive samples against all the positive results, and is defined as follows: (4) Precision=∥TP∥∥TP∥+∥FP∥
Accuracy is the proportion of all samples classified correctly, and is defined as follows: (5) Accuracy=∥TP∥+∥TN∥∥TP∥+∥TN∥+∥FP∥+∥FN∥

4.2. Evaluation of Performance Measures
In order to compare the effect of variance stabilization methods on specificity, sensitivity, precision, and accuracy, we use the following procedure:Procedure:  Performance evaluation
Input:  {A1,⋯,An} , a set of n actual kinome arrays
nd, the maximum number of differentially phosphorylated peptides on each pair of arrays
T, the threshold value for significant fold-change
θ, percentage of noisy peptides
α, a significance level
n′, number of synthesized arrays (n′≤n)
Output:  Calculated value of specificity, sensitivity, accuracy, and precision for each pair of inter-array technical replicates, and for each normalization method
Step 1:  For each q, where 1≤q≤n′, do Step 2 through Step 8:
Step 2:  Using Algorithm 1, create an inter-array technical replicate Yq, where Yq is an inter-array technical replicate for Aq, considering {A1,⋯,An}, T, θ and α.
Step 3:  Phosphorylate a random subset of peptides on Yq using Algorithm 2, considering {A1,⋯,An}, Aq, T, and α; exclude peptides that cannot be differentially phosphorylated by Algorithm 2 from the random subset and record the differentially phosphorylated peptides regardless of their fold-change direction in a set Pq; name the output as Yq′.
Step 4:  For each normalization method do steps 5 to 8.
Step 5:  Normalize the pair (Aq,Yq′), and denote the normalized array pair (Aq*,Yq*).
Step 6:  For the pair (Aq*,Yq*) detect the phosphorylated peptides and save them in a set Fq.
Step 7:  Calculate true positive (TPq), false positive (FPq), true negative (TNq), and false negative (FNq) sets as follows: TPq=Pq∩FqFPq=(Nq−Pq)∩FqTNq=(Nq−Pq)∩(Nq−Fq)FNq=(Pq)∩(Nq−Fq) where Nq is the set of all peptides on array Yq.
Step 8:  Using TPq, FPq, TNq, and FNq, calculate specificity, sensitivity, accuracy, and precision.