Object detection and tracking in video sequences: formalization, metrics and results
https://doi.org/10.37661/1816-0301-2021-18-1-43-60
Abstract
One of the promising areas of development and implementation of artificial intelligence is the automatic detection and tracking of moving objects in video sequence. The paper presents a formalization of the detection and tracking of one and many objects in video. The following metrics are considered: the quality of detection of tracked objects, the accuracy of determining the location of the object in a frame, the trajectory of movement, the accuracy of tracking multiple objects. Based on the considered generalization, an algorithm for tracking people has been developed that uses the tracking through detection method and convolutional neural networks to detect people and form features. Neural network features are included in a composite descriptor that also contains geometric and color features to describe each detected person in the frame. The results of experiments based on the considered criteria are presented, and it is experimentally confirmed that the improvement of the detector operation makes it possible to increase the accuracy of tracking objects. Examples of frames of processed video sequences with visualization of human movement trajectories are presented.
About the Authors
R. P. Bohush
Polotsk State University
Belarus
Belarus
Rykhard P. Bohush, Cand. Sci. (Eng.), Associate Professor, Head of the Department of Computer Systems and Networks
st. Blokhin, 29, Novopolotsk, 211440
S. V. Ablameyko
Belarusian State University; The United Institute of Informatics Problems of the National Academy of Sciences of Belarus
Belarus
Belarus
Sergey V. Ablameyko, Academician of the National Academy of Sciences of Belarus, Dr. Sci. (Eng.), Professor, Сhief Researcher; Professor of the Faculty of Mechanics and Mathematics
av. Nezaliezhnasti, 4, 220030, Minsk
st. Surganova, 6, Minsk, 220012
References
1. Maggio E., Cavallaro А. Video Tracking: Theory and Practice. Wiley, 2011, 295 р.
2. Datta A. K., Datta M., Banerjee P. K. Face Detection and Recognition: Theory and Practice. CRC Press, 2016, 350 p.
3. Kuplyakov D. A., Shalnov E. V., Konushin A. S. Algoritm soprovozhdenija ljudej v video na osnove metoda Monte-Karlo dlja Markovskih cepej [Markov chain Monte Carlo based video tracking algorithm]. Programmirovanie [Programming and Computer Software], 2017, no. 4, pр. 13–21 (in Russian). https://doi.org/10.1134/s0361768817040053
4. Kirpichnikov A. P., Lyasheva S. А., Shleymovich М. P. Obnaruzhenie i soprovozhdenie ljudej v intellektual'nyh detektorah vneshtatnyh situacij [Detection and tracking of people in intelligent detectors of emergency situations]. Vestnik Kazanskogo tehnologicheskogo universiteta [Kazan Technological University Bulletin], 2014, vol. 17, pр. 351–356 (in Russian).
5. Ravish Aradhya H. V. Object detection and tracking using deep learning and artificial intelligence for video surveillance applications. International Journal of Advanced Computer Science and Applications, 2019, vol. 10, no. 12, pр. 517–530. https://doi.org/10.14569/IJACSA.2019.0101269
6. Mane S., Mangale S. Moving object detection and tracking using convolutional neural networks. Second International Conference on Intelligent Computing and Control Systems (ICICCS). Madurai, India, 2018, pр. 1809–1813. https://doi.org/10.1109/ICCONS.2018.8662921
7. Bogush R., Maltsev S., Kastryuk A., Brovko N., Gluhau D. Motion detection and tracking algorithms in video streams. VNU Journal of Science, Mathematics – Physics, 2009, vol. 25, no. 3, pр. 143–151.
8. Bogush R. P., Zakharova I. Yu., Ablameyko S. V. Algoritm soprovozhdenija ljudej na videoposledovatel'nosti s ispol'zovaniem identifikacii po licam dlja nabljudenija vnutri pomeshhenij [Algorithm for person tracking on video sequences using face identification for indoor surveillance]. Vestnik informacionnyh i komp'juternyh tehnologij [Herald of Computer and Information Technologies], 2020, no. 7, pр. 3–14 (in Russian). https://doi.org/10.14489/vkit.2020.07
9. Ye S., Zhican B., Chen C., Bohush R., Ablameyko S. An effective algorithm to detect both smoke and flame using color and wavelet analysis. Pattern Recognition and Image Analysis, 2017, vol. 27, no. 1, pр. 131–138. https://doi.org/10.1134/S1054661817010138
10. Zalesky B. A., Kravchonok A. I. Otslezhivanie dinamicheskih objektov i ih raspoznavanie s pomoshh'ju grafovyh algoritmov [Tracking dynamical objects and their recognition by graph algorithms]. Informatika [Informatics], 2006, no. 2(10), p. 7–26 (in Russian).
11. Zalesky B. A. Algoritm otslezhivanija objektov dvizhushhejsja videokameroj [Object tracking algorithm by moving video camera]. Doklady Nacional'noj akademii nauk Belarusi [Doklady of the National Academy of Sciences of Belarus], 2020, vol. 64, no. 2, pр. 144–149 (in Russian). https://doi.org/10.29235/1561-8323-2020-64-2-144-149
12. Agafonov V. Yu., Rozaliev V. L., Zaboleeva-Zotova A. V. Ispol'zovanie fil'tra Kalmana v zadachah trekinga objektov [Using the Kalman filter in object tracking tasks. Intelligent systems]. Intellektual'nye sistemy. Teorija i prilozhenija [Theory and Applications], 2016, vol. 20, no. 4, рр. 13–17 (in Russian).
13. Gustafsson F., Gunnarsson F., Bergman N. Particle filters for positioning, navigation and tracking. IEEE Transactions on Signal Processing, 2002, vol. 50(2), pр. 425–437. https://doi.org/10.1109/78.978396
14. Ristani E., Solera F., Zou R., Cucchiara R., Tomasi C. Performance measures and a data set for multi-target, multi-camera tracking. Computer Vision ECCV 2016 Workshops. Lecture Notes in Computer Science, 2016, vol. 9914, рp. 17–35.
15. Dendorfer P., Os ep ., ilan ., Schindler ., remers ., , Leal-Taixé L. MOTChallenge: A benchmark for single-camera multiple target tracking. International Journal of Computer Vision, 2020. Available at: https://link.springer.com/article/10.1007/s11263-020-01393-0 (accessed 12.06.2020). https://doi.org/10.1007/s11263-020-01393-0
16. Shchelkunov A. E., Kovalev V. V., Morev K. I., Sidko I. V. Metriki ocenki algoritmov avtomaticheskogo soprovozhdenija [The metrics for tracking algorithms evaluation]. Izvestija Juzhnogo federal'nogo universiteta. Tehnicheskie nauki [News of the Southern Federal University. Engineering Sciences], 2020, no. 1(211), pр. 233–245 (in Russian).
17. Bochkovskiy A., Wang Ch.-Y., Liao H.-Y. M. YOLOv4: Optimal Speed and Accuracy of Object Detection. Available at: https://arxiv.org/abs/2004.10934 (accessed 12.06.2020).
18. Bohush R. P., Zakharava I. Y. Algoritm soprovozhdenija ljudej na videoposledovatel'nostjah s ispol'zovaniem svertochnyh nejronnyh setej dlja videonabljudenija vnutri pomeshhenij [Person tracking algorithm based on convolutional neural network for indoor video surveillance]. Komp'juternaja optika [Computer Optics], 2020, vol. 44, no. 1, рр. 109–116 (in Russian). https://doi.org/10.18287/2412-6179-CO-565
19. Kuhn H. W. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 1955, no. 2, рр. 83–97.
20. Wojke N., Bewley A., Paulus D., Wojke N. Simple online and realtime tracking with a deep association metric. Proceedings of the IEEE International Conference on Image Processing (ICIP). Beijing, China, 2017, pр. 3645–3649. https://doi.org/10.1109/ICIP.2017.8296962
Review
For citations:
Bohush R.P., Ablameyko S.V. Object detection and tracking in video sequences: formalization, metrics and results. Informatics. 2021;18(1):43-60. (In Russ.) https://doi.org/10.37661/1816-0301-2021-18-1-43-60
Views: 1107
Email this article 