informИнформатикаInformatics1816-03012617-6963UIIP NASB10.37661/1816-0301-2024-21-4-58-71inform-1311Research ArticleОБРАБОТКА СИГНАЛОВ, ИЗОБРАЖЕНИЙ, РЕЧИ, ТЕКСТА И РАСПОЗНАВАНИЕ ОБРАЗОВSIGNAL, IMAGE, SPEECH, TEXT PROCESSING AND PATTERN RECOGNITIONАўтаматызацыя аналізу галасавых сігналаў птушакAutomation of bird voice signal analysisГецэвічЮ. С.HetsevichY. S.

Гецэвіч Юрась Станіслававіч, кандыдат тэхнічных навук, дацэнт, загадчык лабараторыі распазнавання і сінтэзу маўлення

вул. Сурганава, 6, Мінск, 220012

Yuras S. Hetsevich, Ph. D. (Eng.), Assoc. Prof., Head of the Speech Synthesis and Recognition Laboratory

st. Surganova, 6, Minsk, 220012

yuras.hetsevich@gmail.comhttps://orcid.org/0009-0009-1189-0807ЗяноўкаЯ. С.ZianoukaYa. S.

Зяноўка Яўгенія Сяргееўна, малодшы навуковы супрацоўнік

вул. Сурганава, 6, Мінск, 220012

Yauheniya S. Zianouka, Junior Researcher

st. Surganova, 6, Minsk, 220012

 

 

evgeniakacan@gmail.comБакуновічА. А.BakunovichA. A.

Бакуновіч Андрэй Аляксеевіч, малодшы навуковы супрацоўнік

вул. Сурганава, 6, Мінск, 220012

Andrei A. Bakunovich, Junior Researcher

st. Surganova, 6, Minsk, 220012

bakunovich.andrei.work@gmail.comЖалаваД. А.ZhalavaD. A.

Жалава Дар’я Аляксандраўна, інжынер-праграміст

вул. Сурганава, 6, Мінск, 220012

Zhalava Darja, Software Engineer

st. Surganova, 6, Minsk, 220012

daryazhalova@gmail.comШагаваТ. Г.ShagavaT. G.

Шагава Таццяна Грыгор’еўна, інжынер-праграміст

вул. Сурганава, 6, Мінск, 220012

Tatsiana G. Shagаva, Software Engineer

st. Surganova, 6, Minsk, 220012

tanya.shagova@gmail.comАб’яднаны інстытут праблем інфарматыкі Нацыянальнай акадэміі навук БеларусіThe United Institute of Informatics Problems of the National Academy of Sciences of Belarus2024301220242145871Copyright © Гецэвіч Ю.С., Зяноўка Я.С., Бакуновіч А.А., Жалава Д.А., Шагава Т.Г., 20242024Гецэвіч Ю.С., Зяноўка Я.С., Бакуновіч А.А., Жалава Д.А., Шагава Т.Г.Hetsevich Y.S., Zianouka Y.S., Bakunovich A.A., Zhalava D.A., Shagava T.G.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://inf.grid.by/jour/article/view/1311Мэты

Мэты. Мэтай працы з’яўляецца стварэнне эксперыментальнага праграмнага забеспячэння аўтаматы заванага распазнавання галасавых сігналаў, якое валодае магчымасцямі доўгачасовага кругласутачнага і кругласезоннага маніторынгу відавой разнастайнасці жывѐл у абраных месцапражываннях і экасістэмах.

Метады

Метады. У працы выкарыстоўваюцца метады глыбокага машыннага навучання скруткавых нейронных сетак, навучаных на падставе мел-спектраграм вакалізацый птушак, якія пабудаваны з дапамогай хуткага пераўтварэння Фур’е.

Вынікі

Вынікі. Апісаны працэс, метады і падыходы да трэніроўкі мадэлі глыбокага машыннага навучання для перспектыўнай сістэмы пасіўнага акустычнага маніторынгу папуляцый птушак на тэрыторыі Беларусі, а таксама ў ходзе тэсціравання прататыпа праграмнага забеспячэння выяўлены складанасці і дасягнуты вынікі.

Заключэнне

Заключэнне. Прадстаўлены працоўны прататып праграмнага забеспячэння аўтаматызаванага распазна вання галасавых сігналаў птушак. Ён ажыццяўляе аналіз акустычных запісаў галасоў птушак з выдачай імавернаснай ацэнкі відавой прыналежнасці для прысутных на запісах вакалізацый жывѐльнага паходжання. Праграмнае забеспячэнне накіравана на павышаную эфектыўнасць маніторынгу птушак, што забяспечвае выкананне прыродаахоўчых і даследчых мерапрыемстваў на аснове дакладных і ак туальных даных відавога распаўсюджвання.

Objectives

Objectives. The purpose of the work is to create an experimental software for automated recognition of voice signals, which has the capabilities of long-term round-the-clock and round-the-season monitoring of animal species diversity in selected habitats and ecosystems.

Methods

Methods. The work uses methods of deep machine learning of convolutional neural networks trained on mel-spectrograms of bird vocalizations, which are built using fast Fourier transform.

Results

Results. The process, methods and approaches to training a deep machine learning model for a system of passive acoustic monitoring of bird populations in Belarus are described, as well as the difficulties identified during testing of the software prototype and the results that were achieved.

Conclusion

Conclusion. A working prototype of the software for automatic recognition of animal (bird) voice signals is presented. It performs the analysis of acoustic recordings of bird voices with the issue of probabilistic assessment of species belonging to animal vocalizations present in the recordings. The software is aimed at increasing the efficiency of bird monitoring, which ensures the implementation of conservation and research activities based on accurate and up-to-date data on species distribution.

аўтаматызаванае распазнаванне галасавых сігналаўскруткавыя нейронныя сеткіглыбокае машыннае навучаннемел-спектраграмаанатаванне галасавых сігналаўautomatic recognition of voice signalsconvolutional neural networksdeep machine learningmel-spectrogramannotation of voice signalsThe research is carried out within the research work under contract No. 221/35 dated September 6, 2021 on the title "Development of technology for automated recognition of animal voice signals for autonomous continuous monitoring of rare, threatened and indicator species and the state of biodiversity in forest ecosystems (in terms of technical implementation)" jointly with the Scientific and Practical Center of the National Academy of Sciences of Belarus for Bioresources.ReferencesPriyadarshani, N. Automated birdsong recognition in complex acoustic environments: a review / N. Priyadarshani, S. Marsland, I. Castro // Journal of Avian Biology. – 2018. – URL: https://nsojournals.onlinelibrary.wiley.com/doi/full/10.1111/jav.01447 (date of access: 19.03.2021).Priyadarshani N., Marsland S., Castro I. Automated birdsong recognition in complex acoustic environments: a review. Journal of Avian Biology, 2018, vol. 49, no. 5. Available at: https://nsojournals.onlinelibrary.wiley.com/doi/full/10.1111/jav.01447 (accessed 19.03.2021).Sharma, S. A methodological literature review of acoustic wildlife monitoring using artificial intelligence tools and techniques / S. Sharma, K. Sato, B. P. Gautam // Sustainability. – 2023. – Vol. 15, no. 9. – Р. 7128 DOI: 10.3390/su15097128.Sharma S., Sato K., Gautam B. P. A methodological literature review of acoustic wildlife monitoring using artificial intelligence tools and techniques. Sustainability, 2023, vol. 15, no. 9, р. 7128 . DOI: 10.3390/su15097128.Briggs, F. Audio classification of bird species: a statistical manifold approach / F. Briggs, R. Raich, X. Z. Fern // 2009 Ninth IEEE Intern. Conf. on Data Mining, Miami Beach, FL, USA, 6–9 Dec. 2009. – Miami Beach, 2009. – P. 51–60.Briggs F., Raich R., Fern X. Z. Audio classification of bird species: a statistical manifold approach. 2009 Ninth IEEE International Conference on Data Mining, Miami Beach, FL, USA, 6–9 December 2009. Miami Beach, 2009, pp. 51–60.BirdNET: A deep learning solution for avian diversity monitoring / S. Kahl, C. M. Wood, M. Eibl, H. Klinck // Ecological Informatics. – March 2021. – Vol. 61. – Р. 101236. – DOI: 10.1016/j.ecoinf.2021.101236.Kahl S., Wood C. M., Eibl M., Klinck H. BirdNET: A deep learning solution for avian diversity monitoring. Ecological Informatics, vol. 61, March 2021, р. 101236. DOI: 10.1016/j.ecoinf.2021.101236.Insights and approaches using deep learning to classify wildlife / Z. Miao, K. M. Gaynor, J. Wang [et al.] // Scientific Reports. – 2019. – Vol. 9, no. 1. – URL: https://www.nature.com/articles/s41598-019-44565-w (date of access: 13.02.2021).Miao Z., Gaynor K. M., Wang J., Liu Z., Muellerklein O., Norouzzadeh M. S. Insights and approaches using deep learning to classify wildlife. Scientific Reports, 2019, vol. 9, no. 1. Available at: https://www.nature. com/articles/s41598-019-44565-w (accessed 13.02.2021). DOI: 10.1038/s41598-019-44565-w.Тэхналогіі аўтаматычнай апрацоўкі і аналізу маўлення з прымяненнем штучнага інтэлекту / Ю. С. Гецэвіч, В. В. Дыдо, Д. А. Бяляўскі [і інш.] // II Форум IT-Академграда «Искусственный интеллект в Беларуси» : доклады, Минск, 12–13 окт. 2023 г. – Минск : ОИПИ НАН Беларуси, 2023. – С. 71–78.Hetsevich Yu. S., Dydo O. V., Bialiauski D. A., Zjanowka Ja. S., Ljucich M. S., …, Nazaraw U. U. Technologies of automatic speech processing and analysis using artificial intelligence. Doklady II Foruma IT-Akademgrada «Iskusstvennyj intellekt v Belarusi» [Reports of the II IT Academy Forum "Artificial Intelligence in Belarus", Minsk, 12–13 October 2023]. Minsk, The United Institute of Informatics Problems of the National Academy of Sciences of Belarus, 2023, pp. 71–78 (In Bel).Klein, D. J. Deep learning for large scale biodiversity monitoring / D. J. Klein, M. Mckown, B. Tershy // Bloomberg Data for Good Exchange Conf., N. Y., NY, USA, 28 Sept. 2015. – N. Y., 2015. – 7 р. – DOI: 10.13140/RG.2.1.1051.7201.Klein D. J., Mckown M. W., Tershy B. R. Deep learning for large scale biodiversity monitoring. Bloomberg Data for Good Exchange Conference, New York, NY, USA, 28 September 2015. New York, 2015, 7 р. DOI: 10.13140/RG.2.1.1051.7201.Artificial intelligence (BirdNET) supplements manual methods to maximize bird species richness from acoustic data sets generated from regional monitoring / L. Ware, C. L. Mahon, L. McLeod, J. F. Jetté // The Canadian Journal of Zoology. – 2023. – Vol. 101, no. 12. – P. 1031–1051.Ware L., Mahon C. L., McLeod L., Jetté J. F. Artificial intelligence (BirdNET) supplements manual methods to maximize bird species richness from acoustic data sets generated from regional monitoring. The Canadian Journal of Zoology, 2023, vol. 101, no. 12, pp. 1031–1051.Stowell, D. An open dataset for research on audio field recording archives: freefield1010 / D. Stowell, M. D. Plumbley. – 2013. – URL: https://arxiv.org/abs/1309.5275 (date of access: 06.06.2024).Stowell D., Plumbley M. D. An open dataset for research on audio field recording archives: freefield1010, 2013. Available at: https://arxiv.org/abs/1309.5275 (accessed 06.06.2024). DOI: 10.48550/arXiv.1309.5275.The machine learning-powered BirdNET App reduces barriers to global bird research by enabling citizen science participation / C. M. Wood, S. Kahl, A. Rahaman, H. Klinck // PLoS Biology. – 2022. – Vol. 20, no. 6. – 10 р. – DOI: 10.48550/arXiv.1309.5275.Wood C. M., Kahl S., Rahaman A., Klinck H. The machine learning-powered BirdNET App reduces barriers to global bird research by enabling citizen science participation. PLoS Biology, 2022, Vol. 20, no. 6, 10 р. DOI: 10.48550/arXiv.1309.5275.The Belarusian list of bird species approved by the Belarusian ornitho-faunistic commission for 2021 and 2022 / N. V. Karlionova, A. V. Borodin, I. E. Samusenko, M. Y. Nikiforov // Zoological Readings. – Grodno : GrGU, 2023. – P. 113.Karlionova N. V., Borodin A. V., Samusenko I. E., Nikiforov M. Y. The Belarusian list of bird species approved by the Belarusian Ornitho-Faunistic Commission for 2021 and 2022. Zoological Readings, Grodno, Grodnenskij gosudarstvennyj universitet imeni Janki Kupaly, 2023, р. 113.Мадэль баз даных для тэхналогіі аўтаматызаванага распазнавання галасавых сігналаў жывѐл / С. А. Гайдураў, Д. І. Латышэвіч, А. А. Бакуновіч [і інш.] // Развитие информатизации и государственной системы научно-технической информации (РИНТИ-2022) : докл. ХXI Междунар. науч.-техн. конф., Минск, 17 нояб. 2022 г. – Минск : ОИПИ НАН Беларуси, 2022. – С. 236–240.Gaidurov S. A., Latyshevich D. I, Bakunovich A. A., Kaigorodova L. I., Khokhlov V. A., …, Hetsevich Yu. S. A database model for automated recognition of animal voice signals. Razvitie informatizacii i gosudarstvennoj sistemy nauchno-tehnicheskoj informacii RINTI-2022 : doklady ХХI Mezhdunarodnoj nauchno-tehnicheskoj konferencii, Minsk, 17 nojabrja 2022 g. [Development of Informatization and the State System of Scientific and Technical Information (RINTI-2022) : Reports of the XXI International Scientific and Technical Conference, Minsk, 17 November 2022], The United Institute of Informatics Problems of the National Academy of Sciences of Belarus, 2022, pp. 236–240 (In Bel.).Stowell, D. Computational bioacoustics with deep learning: a review and roadmap / D. Stowel // PeerJ. – 2022. – Vol. 10 – P. 46.Stowell D. Computational bioacoustics with deep learning: a review and roadmap. PeerJ, 2022, vol. 10, р. 46.FSD50K: an open dataset of human-labeled sound events / E. Fonseca, X. Favory, J. Pons [et al.] // ACM Transactions on Audio, Speech, and Language Processing. – 2022. – Vol. 30 – P. 829–852.Fonseca E., Favory X., Pons J., Font F., Serra X. FSD50K: an open dataset of human-labeled sound events. ACM Transactions on Audio, Speech, and Language Processing, 2022, vol. 30, pp. 829–852.Hearing to the unseen: audiomoth and BirdNET as a cheap and easy method for monitoring cryptic bird species / G. Bota, R. Manzano-Rubio, L. Catalán [et al.] // Sensors. – 2023. – Vol. 23, no. 16. – 11 р. – DOI: 10.3390/s23167176.Bota G, Manzano-Rubio R., Catalán L., Gómez-Catasús J., Pérez-Granados C. Hearing to the unseen: audiomoth and BirdNET as a cheap and easy method for monitoring cryptic bird species. Sensors, vol. 23, no. 16, 11 p. DOI: 10.3390/s23167176.Tan, M. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks / M. Tan, Q. V. Le // Proc. of the 36th Intern. Conf. on Machine Learning, ICML 2019, Long Beach, 9–15 June 2019. – Long Beach, 2019. – Р. 6105–6114.Tan M., Le Q. V. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, 9–15 June 2019. Long Beach, 2019, рр. 6105–6114.Multi-class imbalanced data classification: a systematic mapping study / Y. Wang, M. M. Rosli, N. Musa, F. Li // Engineering, Technology & Applied Science Research. – 2024. – Vol. 14. – P. 14183–14190. – DOI: 10.48084/etasr.7206.Wang Y., Rosli M. M., Musa N., Li F. Multi-class imbalanced data classification: a systematic mapping study. Engineering, Technology & Applied Science Research, 2024, vol. 14, рр. 14183–14190. DOI: 10.48084/etasr.7206Mao, J.-X. Learning label-specific multiple local metrics for multi-label classification / J.-X. Mao, J.-Y. Hang, M.-L. Zhang // Thirty-Third Intern. Joint Conf. on Artificial Intelligence {IJCAI-24}, Jeju, Korea, 3–9 Aug. 2024. – Jeju, 2024. – P. 4742–4750. – DOI: 10.24963/ijcai.2024/524.Mao J.-X., Hang J.-Y., Zhang M.-L. Learning label-specific multiple local metrics for multi-label classification. Thirty-Third International Joint Conference on Artificial Intelligence {IJCAI-24}, Jeju, Korea, 3–9 Aug. 2024. Jeju, 2024, pp. 4742–4750. DOI: 10.24963/ijcai.2024/524.Jia, B.-B. Towards exploiting linear regression for multi-class/multi-label classification: an empirical analysis / B.-B. Jia, J.-Y. Liu, M.-L. Zhang // International Journal of Machine Learning and Cybernetics. – March 2024. – Vol. 15. – P. 3671–3700. – DOI: 10.1007/s13042-024-02114-6.Jia B.-B., Liu J.-Y., Zhang M.-L. Towards exploiting linear regression for multi-class/multi-label classification: an empirical analysis. International Journal of Machine Learning and Cybernetics, March 2024, vol. 15, рр. 3671–3700. DOI: 10.1007/s13042-024-02114-6.Developing birds sound recognition system using an ontological approach / Ya. Zianouka, D. Bialiauski, L. Kajharodava [et al.] // Open Semantic Technologies for Intelligent Systems. – Minsk, Belarusian State University of Informatics and Radioelectronics, 2023. – Iss. 7. – P. 165–170.ZianoukaYa., Bialiauski D., Kajharodava L., Chachlou V., Hetsevich Yu., …, Zhaksylyk K. Developing birds sound recognition system using an ontological approach. Open Semantic Technologies for Intelligent Systems, Minsk, Belarusian State University of Informatics and Radioelectronics, 2023, iss. 7, pp. 165–170.

The authors declare that there are no conflicts of interest present.