4) Develop threshold-free procedures. To improve stability of results, one strategy is to develop threshold-free procedures with few, if any, a priori selected parameters. For example, in the commonly used over-representation analysis, a significance threshold is first selected and used to classify whether or not genes are significantly associated with a particular disease, followed by comparing the proportion of disease associated genes in the gene set with the proportion in the rest of the genome by Fisher’s exact test. The identification of an optimal threshold is often a difficult task. Holmans et al. [31] suggested investigators to apply a range of cutoff values and then select the cutoff value that gives the most significant increase in over-represented gene sets. A more comprehensive approach, albeit computationally intensive, is to choose a threshold value that could make a reasonable compromise among power, type I error rate, and stability of gene set analysis results using a cross-validation scheme.
