Figure 5 shows the results of an uncorrected chi-squared test and Fisher’s exact test, both two-sided and one-sided, for 10,000 comparisons of two samples of size 1,000 with the same binary outcome variable. The two sided probabilities for Fisher’s exact test and the chi-squared test will be approximately the same for a large sample. Despite the size of the sample and the large number of possible tables, the distribution shows pronounced peaks and troughs. It does not conform closely to a Uniform distribution, as does Figure 1, for example. Another run gave the peaks and troughs in the same places. This applied to both the chi-squared test and to Fisher’s exact test.
