Johan Verbeeck

To analyze multivariate outcomes in clinical trials, several authors have suggested generalizations of the univariate Mann–Whitney test. As the Mann–Whitney statistic compares the subjects’ outcome pairwise, the multivariate generalizations are known as generalized pairwise comparisons (GPC) statistics. For GPC statistics such as the net treatment benefit, the win ratio, and the win odds, asymptotic based or re-sampling tests have been suggested in the literature. However, asymptotic methods require a sufficiently high sample size to be accurate, and re-sampling methods come with a high computational burden. We use graph theory notation to obtain closed-form formulas for the expectation and the variance of the permutation and bootstrap sampling distribution of the GPC statistics, which can be utilized to develop fast and accurate inferential tests for each of the GPC statistics. A simple example and a simulation study demonstrate the accuracy of the exact permutation and bootstrap methods, even in very small samples. As the time complexity is O(N2), where N is the total number of patients, the exact methods are fast. In situations where asymptotic methods have been used to obtain these variance matrices, the new methods will be more accurate and equally fast. In situations where bootstrap has been used, the new methods will be both more accurate and much faster.

Ordinal data in a repeated measures design of a crossover study for rare diseases usually do not allow for the use of standard parametric methods, and hence, nonparametric methods should be considered instead. However, only limited simulation studies in settings with small sample sizes exist. Therefore, starting from an Epidermolysis Bullosa simplex trial with the above‐mentioned design, a rank‐based approach using the R package nparLD and different generalized pairwise comparisons (GPC) methods were compared impartially in a simulation study. The results revealed that there was not one single best method for this particular design, because a trade‐off exists between achieving high power, accounting for period effects, and for missing data. Specifically, nparLD as well as the unmatched GPC approaches do not address crossover aspects, and the univariate GPC variants partly ignore the longitudinal information. The matched GPC approaches, on the other hand, take the crossover effect into account in the sense of incorporating the within‐subject association. Overall, the prioritized unmatched GPC method achieved the highest power in the simulation scenarios, although this may be due to the specified prioritization. The rank‐based approach yielded good power even at a sample size of , whereas the matched GPC method could not control the type I error.