Effect size assessment in quasi-experimental studies
https://doi.org/10.37661/1816-0301-2023-20-1-27-39
Abstract
Objectives. To estimate effect sizes in quasi-experimental studies.
Methods. Methods of the theory of estimation, methods of mathematical statistics.
Results. Estimation of the effect size on an ordinal scale, estimation of the effect size on a binary in the case of opposite direction effects in groups, in quasi-experimental studies for the analytical method "differences in differences".
Conclusion. The paper considers approaches to assessing absolute and standardized effect sizes in experimental and quasi-experimental studies. A brief review of the estimators of absolute and standardized effect sizes for quantitative and binary study variables is provided. The applied approach is proposed to assess the effect sizes of a binary variable in the case of opposite direction effects in groups within a quasi-experimental studies for the "differences in differences" analytical method. An example of assessment of absolute and standardized effect sizes of quantitative and binary variables in quasi-experimental studies in clinical epidemiology is considered.
About the Author
O. V. Krasko
The United Institute of Informatics Problems of the National Academy of Sciences of Belarus
Belarus
Belarus
Olga V. Krasko, Ph. D. (Eng.), Assoc. Prof., Leading Researcher
st. Surganova, 6, Minsk, 220012
References
1. Lederer D. J. et al. Control of confounding and reporting of results in causal inference studies. Guidance for authors from editors of respiratory, sleep, and critical care journals //Annals of the American Thoracic Society. – 2019. – Т. 16. – №. 1. – С. 22-28.
2. Fletcher G. S. Clinical epidemiology: the essentials. –Williams & Wilkins, 1996.
3. Campbell D. T., Riecken H. W. Quasi-experimental design //International encyclopedia of the social sciences. – 1968. – Т. 5. – №. 3. – С. 259-263.
4. Pearl, J. (2009) Causal inference in statistics: An overview. Statistics surveys 3, 96-146.
5. Shadish W. R., Luellen J. K. Quasi-experimental design //Handbook of complementary methods in education research. – Routledge, 2012. – С. 539-550.
6. Kelley K., Preacher K. J. On effect size //Psychological methods. – 2012. – Т. 17. – №. 2. – С. 137.
7. Hodges, J.L., and Lehmann, E.L. (1963), Estimates of location based on rank tests. The Annals of Mathematical Statistics, 34, 598–611.
8. Hollander M., Wolfe D. A., Chicken E. Nonparametric statistical methods. – John Wiley & Sons, 2013.
9. Cohen J. Statistical power analysis for the behavioral sciences (Revised ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. – 1988.
10. Glass G. V. Primary, secondary, and meta-analysis of research //Educational researcher. – 1976. – Т. 5. – №. 10. – С. 3-8.
11. Hedges L. V. Distribution theory for Glass's estimator of effect size and related estimators //journal of Educational Statistics. – 1981. – Т. 6. – №. 2. – С. 107-128.
12. Cliff N. Ordinal Methods for Behavioral Data Analysis. 1–197 //Norman Cliff. – 1996.
13. Cureton E. E. Rank-biserial correlation //Psychometrika. – 1956. – Т. 21. – №. 3. – С. 287-290.
14. Cureton E. E. Rank-biserial correlation when ties are present //Educational and Psychological Measurement. – 1968. – Т. 28. – №. 1. – С. 77-79.
15. Glass G. V. Note on rank biserial correlation //Educational and Psychological Measurement. – 1966. – Т. 26. – №. 3. – С. 623-631.
16. Willson V. L. Critical values of the rank-biserial correlation coefficient //Educational and Psychological Measurement. – 1976. – Т. 36. – №. 2. – С. 297-300.
17. Sawilowsky S. S. New effect size rules of thumb //Journal of modern applied statistical methods. – 2009. – Т. 8. – №. 2. – С. 26.
18. Lovakov A., Agadullina E. R. Empirically derived guidelines for effect size interpretation in social psychology //European Journal of Social Psychology. – 2021. – Т. 51. – №. 3. – С. 485-504.
19. Gignac G. E., Szodorai E. T. Effect size guidelines for individual differences researchers //Personality and individual differences. – 2016. – Т. 102. – С. 74-78.
20. Funder D. C., Ozer D. J. Evaluating effect size in psychological research: Sense and nonsense //Advances in Methods and Practices in Psychological Science. – 2019. – Т. 2. – №. 2. – С. 156-168.
21. Evans J. D. Straightforward statistics for the behavioral sciences. – Thomson Brooks/Cole Publishing Co, 1996.
22. Hess M. R., Kromrey J. D. Robust confidence intervals for effect sizes: A comparative study of Cohen’sd and Cliff’s delta under non-normality and heterogeneous variances //Annual meeting of the American Educational Research Association. – 2004. – Т. 1.
23. Romano J. et al. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys //Annual meeting of the Florida Association of Institutional Research. – 2006. – Т. 177. – С. 34.
24. Marfo P., Okyere G. A. The accuracy of effect-size estimates under normals and contaminated normals in meta-analysis //Heliyon. – 2019. – Т. 5. – №. 6. – С. e01838.
25. McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages //Psychometrika. – 1947. – Т. 12. – №. 2. – С. 153-157.
26. Anscombe F. J. The transformation of Poisson, binomial and negative-binomial data //Biometrika. – 1948. – Т. 35. – №. 3/4. – С. 246-254.
27. Abadie A. Semiparametric difference-in-differences estimators //The Review of Economic Studies. – 2005. – Т. 72. – №. 1. – С. 1-19.
28. Sant’Anna P. H. C., Zhao J. Doubly robust difference-in-differences estimators //Journal of Econometrics. – 2020. – Т. 219. – №. 1. – С. 101-122.
29. Rosenbaum P. R., Rubin D. B. The central role of the propensity score in observational studies for causal effects //Biometrika. – 1983. – Т. 70. – №. 1. – С. 41-55.
Review
For citations:
Krasko O.V. Effect size assessment in quasi-experimental studies. Informatics. 2023;20(1):27-39. (In Russ.) https://doi.org/10.37661/1816-0301-2023-20-1-27-39
Views: 467
Email this article 