Clustering-based interactive image segmentation

Objectives. The task of color image segmentation without the use of preliminary training is considered. It arises, for example, when it is necessary to perform image segmentation with semantic and color properties unknown in advance immediately after their acquisition, or when the set of images intended for segmentation is too small, as well as when performing preliminary "exploratory" analysis of images. In such cases, powerful neural network and other segmentation tools that require deep learning can not be used.
Methods. An algorithm for interactive image segmentation is proposed, based on the analysis of the colors of areas selected interactively. First, in interactive mode, the image areas belonging to the objects are selected very approximately, and then regions belonging to the background are chosen. In the next step, the set of colors of the selected object areas and the set of colors of the selected background areas are clustered separately by one of the clustering algorithms, for example, k-means, fuzzy c-means, or the multi-level clustering algorithm proposed by the author. After this, non-informative elements are removed from the set of cluster centers describing the objects and the set of clusters presenting the background. The modified sets of object and background cluster centers are used for image segmentation.
Results . The constructed algorithm allows selection of the required objects in color images if the colors of the objects and the background are different. Interactive selection of object areas and background areas does not require accuracy or much effort and usually takes several tens of seconds. For selection, rectangular areas that lie entirely inside the object images, and rectangular areas that belong completely to the background can be used. Below an example of interactive regions selection and color image segmentation is shown.
Conclusion. The experiments performed showed the effectiveness of the proposed approach to segmenting color images. It can be used in cases where the semantic and color properties of images are not known in advance, and in cases where the use of more powerful deep learning methods, including neural networks, is too expensive or impossible.

1. Gonzales R. C., Woods R. E. Digital Image Processing. Upper Saddle River, New Jersey, Prentice Hall, 2002, 814 p.

2. Shapiro L. S., Stockman G. C. Computer Vision. Upper Saddle River, New Jersey, Prentice Hall, 2001, 608 p.

3. Selyankin V. V. Komp'juternoe zrenie. Analiz i obrabotka izobrazhenij. Computer Vision. Image Analysis and Processing. Saint Petersburg, Lan', 2019, 152 p. (In Russ.).

4. Snyder W. E., Qi H. Fundamentals of Computer Vision. Cambridge, Cambridge University Press, 2017, 386 р.

5. Bezdek J. C. Pattern Recognition with Fuzzy Objective Function Algorithms. New York, Springer New York, 1981, 272 p.

6. Zalesky B. A. Object tracking algorithm by moving video camera. Doklady Nacionalʹnoj akademii nauk Belarusi [Doklady of the National Academy of Sciences of Belarus], 2020, vol. 64, no. 2, pp. 144–149 (In Russ.).

7. Zalesky B. A. Multilevel algorithm for color clustering of images. Doklady Nacionalʹnoj akademii nauk Belarusi [Doklady of the National Academy of Sciences of Belarus], 2021, vol. 65, no. 3, pp. 209–274 (In Russ.).