informИнформатикаInformatics1816-03012617-6963UIIP NASB10.37661/1816-0301-2023-20-1-27-39inform-1226Research ArticleБИОИНФОРМАТИКАBIOINFORMATICSОценка размера эффекта воздействия в квазиэкспериментальных исследованияхEffect size assessment in quasi-experimental studieshttps://orcid.org/0000-0002-4150-282XКраськоО. В.KraskoO. V.

Красько Ольга Владимировна, кандидат технических  наук, доцент, ведущий научный сотрудник

ул. Сурганова, 6, Минск, 220012

Olga V. Krasko, Ph. D. (Eng.), Assoc. Prof., Leading Researcher

st. Surganova, 6, Minsk, 220012

krasko@NEWMAN.bas-net.byОбъединенный институт проблем информатики Национальной академии наук БеларусиThe United Institute of Informatics Problems of the National Academy of Sciences of Belarus2023290320232012739Copyright © Красько О.В., 20232023Красько О.В.Krasko O.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://inf.grid.by/jour/article/view/1226Цели

Цели. Поставлена задача оценить размер эффекта в квазиэкспериментальных исследованиях.

Методы

Методы. Использованы методы теории оценивания и методы математической статистики.

Результаты

Результаты. Оценен размер эффекта на порядковой и бинарной шкалах в случае разнонаправленных эффектов в группах в квазиэкспериментальных исследованиях для аналитического метода «различие в различиях». 

Заключение

Заключение. В работе представлены подходы к оценке абсолютных и стандартизированных размеров эффектов в экспериментальных и квазиэкспериментальных исследованиях. Дан краткий обзор оценок абсолютных и стандартизированных размеров эффектов для количественных и бинарных переменных исследования. Рассмотрен практический подход к оценке размеров эффектов порядковой и бинарной переменных в случае разнонаправленных эффектов в группах в квазиэкспериментальных исследованиях для аналитического метода «различие в различиях». Приведен пример расчетов абсолютных и стандартизированных размеров эффектов количественной и бинарной переменных в квазиэкспериметальных исследованиях в клинической эпидемиологии. 

Objectives

Objectives. To estimate effect sizes in quasi-experimental studies.

Methods

Methods. Methods of the theory of estimation, methods of mathematical statistics.

Results

Results. Estimation of the effect size on an ordinal scale, estimation of the effect size on a binary in the case of opposite direction effects in groups, in quasi-experimental studies for the analytical method "differences in  differences".

Conclusion

Conclusion. The paper considers approaches to assessing absolute and standardized effect sizes in experimental and quasi-experimental studies. A brief review of the estimators of absolute and standardized effect sizes for quantitative and binary study variables is provided. The applied approach is proposed to assess the effect sizes of a binary variable in the case of opposite direction effects in groups within a quasi-experimental studies for the "differences in differences" analytical method. An example of assessment of absolute and standardized effect sizes of quantitative and binary variables in quasi-experimental studies in clinical epidemiology is considered. 

экспериментальные исследованияквазиэкспериментальные исследованияабсолютный размер эффектастандартизированный размер эффектааналитический метод «различие в различиях»experimental study designquasi-experimental study designabsolute effect sizestandardized effect size"difference-in-differences" analytical methodReferencesLederer D. J., Bell S. C., Branson R. D., Chalmers J. D., Marshall R., …, Vincent J.-L. Control of confounding and reporting of results in causal inference studies. Guidance for authors from editors of respiratory, sleep, and critical care journals. Annals of the American Thoracic Society, 2019, vol. 16, no. 1, рр. 22–28.Lederer D. J. et al. Control of confounding and reporting of results in causal inference studies. Guidance for authors from editors of respiratory, sleep, and critical care journals //Annals of the American Thoracic Society. – 2019. – Т. 16. – №. 1. – С. 22-28.Fletcher R. H., Fletcher S. W., Wagner E. H. Clinical Epidemiology: The Essentials, 3rd ed. Baltimore, Williams & Wilkins, 1996, 276 р.Fletcher G. S. Clinical epidemiology: the essentials. –Williams & Wilkins, 1996.Campbell D. T., Riecken H. W. Quasi-experimental design. International Encyclopedia of the Social Sciences, 1968, vol. 5, no. 3, рр. 259–263.Campbell D. T., Riecken H. W. Quasi-experimental design //International encyclopedia of the social sciences. – 1968. – Т. 5. – №. 3. – С. 259-263.Pearl J. Causal inference in statistics: An overview. Statistics Surveys, 2009, vol. 3, рр. 96–146.Pearl, J. (2009) Causal inference in statistics: An overview. Statistics surveys 3, 96-146.Shadish W. R., Luellen J. K. Quasi-experimental design. Handbook of Complementary Methods in Education Research. Mahwah, Routledge, 2012, рр. 539–550.Shadish W. R., Luellen J. K. Quasi-experimental design //Handbook of complementary methods in education research. – Routledge, 2012. – С. 539-550.Kelley K., Preacher K. J. On effect size. Psychological Methods, 2012, vol. 17, no. 2, рр. 137–152.Kelley K., Preacher K. J. On effect size //Psychological methods. – 2012. – Т. 17. – №. 2. – С. 137.Hodges J. L., Lehmann E. L. Estimates of location based on rank tests. The Annals of Mathematical Statistics, 1963, vol. 34, рр. 598–611.Hodges, J.L., and Lehmann, E.L. (1963), Estimates of location based on rank tests. The Annals of Mathematical Statistics, 34, 598–611.Hollander M., Wolfe D. A., Chicken E. Nonparametric Statistical Methods, 3rd ed. John Wiley & Sons, 2013, 848 р.Hollander M., Wolfe D. A., Chicken E. Nonparametric statistical methods. – John Wiley & Sons, 2013.Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, Lawrence Earlbaum Associates, 1988, 590 р.Cohen J. Statistical power analysis for the behavioral sciences (Revised ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. – 1988.Glass G. V. Primary, secondary, and meta-analysis of research. Educational Researcher, 1976, vol. 5, no. 10, рр. 3–8.Glass G. V. Primary, secondary, and meta-analysis of research //Educational researcher. – 1976. – Т. 5. – №. 10. – С. 3-8.Hedges L. V. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 1981, vol. 6, no. 2, рр. 107–128.Hedges L. V. Distribution theory for Glass's estimator of effect size and related estimators //journal of Educational Statistics. – 1981. – Т. 6. – №. 2. – С. 107-128.Cliff N. Ordinal Methods for Behavioral Data Analysis, 1st ed. Psychology Press, 1996, 212 р.Cliff N. Ordinal Methods for Behavioral Data Analysis. 1–197 //Norman Cliff. – 1996.Cureton E. E. Rank-biserial correlation. Psychometrika, 1956, vol. 21, no. 3, рр. 287–290.Cureton E. E. Rank-biserial correlation //Psychometrika. – 1956. – Т. 21. – №. 3. – С. 287-290.Cureton E. E. Rank-biserial correlation when ties are present. Educational and Psychological Measurement, 1968, vol. 28, no. 1, рр. 77–79.Cureton E. E. Rank-biserial correlation when ties are present //Educational and Psychological Measurement. – 1968. – Т. 28. – №. 1. – С. 77-79.Glass G. V. Note on rank biserial correlation. Educational and Psychological Measurement, 1966, vol. 26, no. 3, рр. 623–631.Glass G. V. Note on rank biserial correlation //Educational and Psychological Measurement. – 1966. – Т. 26. – №. 3. – С. 623-631.Willson V. L. Critical values of the rank-biserial correlation coefficient. Educational and Psychological Measurement, 1976, vol. 36, no. 2, рр. 297–300.Willson V. L. Critical values of the rank-biserial correlation coefficient //Educational and Psychological Measurement. – 1976. – Т. 36. – №. 2. – С. 297-300.Sawilowsky S. S. New effect size rules of thumb. Journal of Modern Applied Statistical Methods, 2009, vol. 8, no. 2, рр. 597–599.Sawilowsky S. S. New effect size rules of thumb //Journal of modern applied statistical methods. – 2009. – Т. 8. – №. 2. – С. 26.Lovakov A., Agadullina E. R. Empirically derived guidelines for effect size interpretation in social psychology. European Journal of Social Psychology, 2021, vol. 51, no. 3, рр. 485–504.Lovakov A., Agadullina E. R. Empirically derived guidelines for effect size interpretation in social psychology //European Journal of Social Psychology. – 2021. – Т. 51. – №. 3. – С. 485-504.Gignac G. E., Szodorai E. T. Effect size guidelines for individual differences researchers. Personality and Individual Differences, 2016, vol. 102, рр. 74–78.Gignac G. E., Szodorai E. T. Effect size guidelines for individual differences researchers //Personality and individual differences. – 2016. – Т. 102. – С. 74-78.Funder D. C., Ozer D. J. Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2019, vol. 2, no. 2, рр. 156–168.Funder D. C., Ozer D. J. Evaluating effect size in psychological research: Sense and nonsense //Advances in Methods and Practices in Psychological Science. – 2019. – Т. 2. – №. 2. – С. 156-168.Evans J. D. Straightforward Statistics for the Behavioral Sciences. Thomson Brooks/Cole Publishing Co, 1996, 624 р.Evans J. D. Straightforward statistics for the behavioral sciences. – Thomson Brooks/Cole Publishing Co, 1996.Hess M. R., Kromrey J. D. Robust confidence intervals for effect sizes: A comparative study of Cohen’sd and Cliff’s delta under non-normality and heterogeneous variances. Annual meeting of the American Educational Research Association, 2004, vol. 1.Hess M. R., Kromrey J. D. Robust confidence intervals for effect sizes: A comparative study of Cohen’sd and Cliff’s delta under non-normality and heterogeneous variances //Annual meeting of the American Educational Research Association. – 2004. – Т. 1.Romano J., Kromrey J., Coraggio J., Skowronek J. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys. Annual meeting of the Florida Association of Institutional Research, 2006, vol. 177, р. 34.Romano J. et al. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen’s d for evaluating group differences on the NSSE and other surveys //Annual meeting of the Florida Association of Institutional Research. – 2006. – Т. 177. – С. 34.Marfo P., Okyere G. A. The accuracy of effect-size estimates under normals and contaminated normals in meta-analysis. Heliyon, 2019, vol. 5, no. 6, р. e01838.Marfo P., Okyere G. A. The accuracy of effect-size estimates under normals and contaminated normals in meta-analysis //Heliyon. – 2019. – Т. 5. – №. 6. – С. e01838.Couper M. P., Tourangeau R., Conrad F. G., Singer E. Evaluating the effectiveness of visual analog scales: A web experiment. Social Science Computer Review, 2006, vol. 24, no. 2, рр. 227–245.McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages //Psychometrika. – 1947. – Т. 12. – №. 2. – С. 153-157.McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 1947, vol. 12, no. 2, рр. 153–157.Anscombe F. J. The transformation of Poisson, binomial and negative-binomial data //Biometrika. – 1948. – Т. 35. – №. 3/4. – С. 246-254.Anscombe F. J. The transformation of Poisson, binomial and negative-binomial data. Biometrika, 1948, vol. 35, no. 3/4, рр. 246–254.Abadie A. Semiparametric difference-in-differences estimators //The Review of Economic Studies. – 2005. – Т. 72. – №. 1. – С. 1-19.Abadie A. Semiparametric difference-in-differences estimators. The Review of Economic Studies, 2005, vol. 72, no. 1, рр. 1–19.Sant’Anna P. H. C., Zhao J. Doubly robust difference-in-differences estimators //Journal of Econometrics. – 2020. – Т. 219. – №. 1. – С. 101-122.Sant’Anna P. H. C., Zhao J. Doubly robust difference-in-differences estimators. Journal of Econometrics, 2020, vol. 219, no. 1, рр. 101–122.Rosenbaum P. R., Rubin D. B. The central role of the propensity score in observational studies for causal effects //Biometrika. – 1983. – Т. 70. – №. 1. – С. 41-55.Rosenbaum P. R., Rubin D. B. The central role of the propensity score in observational studies for causal effects. Biometrika, 1983, vol. 70, no. 1, рр. 41–55.Rosenbaum P. R., Rubin D. B. The central role of the propensity score in observational studies for causal effects. Biometrika, 1983, vol. 70, no. 1, рр. 41–55.

The authors declare that there are no conflicts of interest present.