Kai T. Horstmann

Objective Theories about within‐person (WP) variation are often tested using between‐person (BP) research, despite the well‐established fact that results may not generalize across levels of analysis. One possible explanation is vague theories that do not specify which level of analysis is of interest. We illustrate such a case using the construct of self‐compassion. The factor structure at the BP level has been highly debated, although the theory is actually concerned with relationships at the WP level. Method Multilevel confirmatory factor analysis was applied to experience‐sampling data of self‐compassion ( N = 213, with n = 4052 measurement occasions). Results At both levels of analysis, evidence for a two‐factor model was found. However, the factors were moderately related at the WP level ( r = 0.37, p < 0.001) but largely independent at the BP level ( r = 0.04, p = 0.696). Exploratory analyses revealed considerable heterogeneity in the WP relationship among individuals. Conclusion We discuss how our results provide new impulses to move the debate around self‐compassion forward. Lastly, we outline how the WP level—which is of major interest for self‐compassion and other constructs in psychology—can guide the conceptualization and assessment to promote advancements of the theory and resulting applications.