Seweryn Krupnik

Qualitative comparative analysis is gradually becoming more established in the evaluation field. The purpose of this article is to highlight the potential for evaluation research of engaging in consecutive rounds of this analysis. This is possible when approaching qualitative comparative analysis as a systematic strategy for configurational theorizing. To substantiate this potential, we present two evaluation studies on Research and Development subsidies for companies in Poland. Compared with the results of the first study, the findings of the subsequent consecutive qualitative comparative analysis studies were much more nuanced and helped in developing a full-fledged configurational program theory. In addition to elaborating on the strengths of a consecutive qualitative comparative analysis approach and the relevance of configurational program theories for evaluators, this article shares the main lessons learned in overcoming challenges common to such designs. Thus, concrete guidance is offered to researchers and evaluators who are willing to take configurational theorizing seriously.

Qualitative comparative analysis is increasingly popular as a methodological option in the evaluator’s toolkit. However, evaluators who are willing to apply it face inconsistent suggestions regarding the choice of the ‘solution term’. These inconsistent suggestions reflect a current broad debate among proponents of two approaches to qualitative comparative analysis. The first approach focuses on substantial interpretability and the second on redundancy-free results. We offer three questions to guide the choice of a solution term in the context of impact multi-method evaluation research. They are related to the intended use of the findings, goals of the analysis and regularity theory of causality. Finally, we showcase guidelines through three potential applications of qualitative comparative analysis. The guiding questions would almost always lead to choosing the substantial interpretability approach. However, the redundancy-free approach should not be disregarded. Whatever the choice, researchers should be aware of the assumptions each approach is based on and the risks involved.