Segmentation of Overlapping Tree Images in the Digital Photographs of Forest Areas
DOI:
https://doi.org/10.37482/0536-1036-2024-1-126-140Keywords:
deep learning, artificial intelligence, convolutional neural network, segmentation of overlapping objects, decision support system for logging machine operatorsAbstract
The use of decision support systems based on computer vision and artificial intelligence significantly improves the working conditions for the operators of technological machines in the timber sector, whose work implies high intensity and psycho-emotional overload. By means of computer vision and artificial intelligence the operator can quickly and easily obtain the data on the state of the cutting area and adopt the optimal solution for holding the working operation. This facilitates his work and reduces the time spent searching and analyzing the data on the cutting area. Meanwhile, one of the key elements of such a system is a subsystem for automatic segmentation of objects in the photograph. We have explored the possibility of segmenting overlapping objects in the photographs of forest areas using a convolutional neural network based on the Mask R-CNN architecture. Unlike in most works on similar topics, the objects of this study are color photographs taken by an RGB camera rather than a lidar. This creates the prospect for reducing the cost of hardware and software systems used to support decision-making by the operators of logging machines. The images of the stems and crowns of coniferous and deciduous trees overlapping each other are the segmented objects under consideration. Using the GIMP graphic editor, we have manually marked the color photographs depicting a total of 134 trees of 4 different species: spruce, aspen, birch and pine. Utilizing the developed database, we have carried out an experiment to further train the Mask R-CNN convolutional neural network for segmentation of overlapping parts of the trees in the digital photographs of forest areas. The neural network has been pre-trained using the Microsoft COCO dataset containing more than 200,000 images of 80 different classes of objects such as people, cars, animals and various items. While training the neural network, the images supplied to its input were subjected to a series of various linear and nonlinear geometric transformations, which made it possible to increase the volume of training data by 11 times. As a result, the accuracy of segmentation of the images of the stems and crowns of coniferous and deciduous trees overlapping each other has reached 79 %, which allows the use of neural networks based on a similar architecture in decision support systems for logging machine operators.
Downloads
References
Иванов К.О. Классификация фрагментов электроэнцефалограммы по степени патологической значимости с помощью последовательных нейронных сетей // Вестн. ПГТУ. Сер.: Радиотехн. и инфокоммуникац. системы. 2022. № 4(56). С. 24–37. Ivanov K.O. Classification of Electroencephalogram Segments Based on Pathological Significance Using Sequential Neural Networks. Vestnik Povolzhskogo gosudarstvennogo tekhnologicheskogo universiteta. Seriya: Radiotekhnicheskie i infokommunikatsionnye sistemy = Vestnik of Volga State University of Technology. Series: Radio Engineering and Infocommunication Systems, 2022, no. 4(56), pp. 24–37. (In Russ.). https://doi.org/10.25686/2306-2819.2022.4.24
Николенко С., Кадурин А., Архангельская Е. Глубокое обучение. Погружение в мир нейронных сетей. СПб.: Питер, 2022. 480 c. Nikolenko S., Kadurin A., Arkhangelskaya E. Deep Learning. Dive into the World of Neural Networks. St. Petersburg, Piter Publ., 2022. 480 p. (In Russ).
Портнов А.М., Иванова Н.В., Шашков М.П. Опыт использования нейронной сети DeepForest для детектирования деревьев в широколиственном лесу // Математическая биология и биоинформатика: докл. Междунар. конф. / под ред. В.Д. Лахно. Т. 9. Пущино: ИМПБ РАН, 2022. Ст. № e45. Portnov A.M., Ivanova N.V., Shashkov M.P. Experience of Using the DeepForest Neural Network to Detect Trees in a Broad-Leaved Forest. Proceedings of the International Conference “Mathematical Biology and Bioinformatics”. Ed. V.D. Lakhno. Pushchino, IMPB RAS, 2022, vol. 9, art. no. e45. (In Russ). https://doi.org/10.17537/icmbb22.12
Тимофеев В.С., Сивак М.А. Робастная нейронная сеть с простой архитектурой // Сиб. журн. индустр. математики. 2021. T. 24, № 4. С. 126–138. Timofeev V.S., Sivak M.A. Robust Neural Network with Simple Architecture. Sibirskiy zhurnal industrialnoy matematiki = Journal of Applied and Industrial Mathematics, 2021, vol. 24, no. 4, pp. 126–138. (In Russ). https://doi.org/10.33048/SIBJIM.2021.24.409
Callan R. The Essence of Neural Network. Upper Saddle River: Prentice Hall Europe, 1998. 232 p.
Chollet F. Deep Learning with Python: Second Edition. New York, Manning Publications Co., 2021. 504 p.
COCO: Common Objects in Context. Available at: https://cocodataset.org/#home (accessed 17.01.23).
Daubechies I., DeVore R., Foucart S., Hanin B., Petrova G. Nonlinear Approximation and (Deep) ReLU Networks. Constructive Approximation, 2022, vol. 55, pp. 127–172. https://doi.org/10.1007/s00365-021-09548-z
Diez Y., Kentsch S., Fukuda M., Caceres M.L.L., Moritake K., Cabezas M. Deep Learning in Forestry Using UAV-Acquired RGB Data: A Practical Review. Remote Sensing, 2021, vol. 13, no. 14, art. no. 2837. https://doi.org/10.3390/rs13142837
Girshick R. Fast R-CNN. 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1440–1448.
Goodfellow I., Benigo Y., Courville A. Deep Learning. Cambridge, MIT Press Publ., 2017. 785 p.
Grondin V., Fortin J.-M., Pomerleau F., Giguère P. Tree Detection and Diameter Estimation Based on Deep Learning. Forestry: An International Journal of Forest Research, 2023, vol. 96, iss. 2, pp. 264–276. https://doi.org/10.1093/forestry/cpac043
Haykin S. Neural Networks: A Comprehensive Foundation (3rd Edition). Hoboken: Prentice-Hall Inc., 2007. 938 p.
He K., Gkioxari G., Dollar P., Girshick R. Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–2988.
Imgaug: a Library for Image Augmentation in Machine Learning Experiments. Available at: https://imgaug.readthedocs.io/en/latest/ (accessed 17.07.23).
Ivanov K.O., Kazarinov A.A., Rozhentsov A.A., Dubrovin V.N., Baev A.A., Evdokimov A.O. An Algorithm for Segmentation of Kidney Tissues on CT Images Based on a U-Net Convolutional Neural Network. Biomedical Engineering, 2023, vol. 56, pp. 424–428. https://doi.org/10.1007/s10527-023-10249-z
Mask R-CNN for Object Detection and Segmentation. Available at: https://github.com/matterport/Mask_RCNN (accessed 17.07.23).
Moussaid A., El Fkihi S., Zennayi Y. Tree Crowns Segmentation and Classification in Overlapping Orchards Based on Satellite Images and Unsupervised Learning Algorithms. Journal of Imaging, 2021, vol. 7, no. 241. https://doi.org/10.3390/jimaging7110241
Ochoa C.G., Caruso P., Ray G., Deboodt T., Jarvis T.W., Guldan S.J. Ecohydrologic Connections in Semiarid Watershed Systems of Central Oregon USA. Water, 2018, vol. 2, no. 10, art. no. 181. https://doi.org/10.3390/w10020181
Pointer I. Programming PyTorch for Deep Learning: Creating and Deploying Deep Learning Applications. O’Reilly Media Inc., 2019. 217 p.
Qassim H., Verma A., Feinzimer D. Compressed Residual-VGG16 CNN Model for Big Data Places Image Recognition. 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), 2018, pp. 169–175.
Qianwei L., Weifeng M., Jianpeng Z., Yicheng L., Dongfan X., Jinliang W. Point-Cloud Segmentation of Individual Trees in Complex Natural Forest Scenes Based on a Trunk Growth Method. Journal of Forestry Research, 2021, vol. 32, pp. 2403–2414. https://doi.org/10.1007/s11676-021-01303-1
Rao D., McMahan B. Natural Language Processing with PyTorch: Build Intelligent Language Applications Using Deep Learning. O’Reilly Media Inc., 2019. 254 p.
Ren S., Kaiming H., Ross G., Jian S. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems. Great Britain, Oxford, December, 2015, vol. 1, pp. 91–99.
Rojas R. The Backpropagation Algorithm. Neural Networks, 1996, pp. 149–182. https://doi.org/10.1007/978-3-642-61068-4_7
Tarasova A., Strielkowski W., Larionova N. Digital Relations and Innovative Ecosystem for Sustainable Engineering Education: A Case of Russia. Increasing Supply Chain Performance in Digital Society. IGI Global, 2022, pp. 304–318.
Wang D., Liang X., Mofack II G., Martin-Ducup O. Individual Tree Extraction from Terrestrial Laser Scanning Data via Graph Pathing. Forest Ecosystems, 2021, vol. 8, no. 67. https://doi.org/10.1186/s40663-021-00340-w
Weinstein B.G., Graves S.J., Marconi S., Singh A., Zare A., Stewart D. A Benchmark Dataset for Canopy Crown Detection and Delineation in Co-Registered Airborne RGB, LiDAR and Hyperspectral Imagery from the National Ecological Observation Network. PLoS Computational Biology, 2021, vol. 7, no. 17. https://doi.org/10.1371/journal.pcbi.1009180
Windrim L., Bryson M. Detection, Segmentation, and Model Fitting of Individual Tree Stems from Airborne Laser Scanning of Forests Using Deep Learning. Remote Sensing, 2020, vol. 12, no. 9, art. no. 1469. https://doi.org/10.3390/rs12091469
Xu X., Iuricich F., Floriani L.D. A Topology-Based Approach to Individual Tree Segmentation from Airborne LiDAR Data. GeoInformatica, 2023, vol. 27, pp. 759–788. https://doi.org/10.1007/s10707-023-00487-4
Yang Z., Dong R., Xu H., Gu J. Instance Segmentation Method Based on Improved Mask R-CNN for the Stacked Electronic Components. Electronics, 2020, vol. 9, no. 6, art. no. 886. https://doi.org/10.3390/electronics9060886
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 И.В. Петухов, К.О. Иванов, Д.М. Ворожцов, A.А. Роженцов, Н.И. Роженцова, Л.А. Стешина (Автор)
This work is licensed under a Creative Commons Attribution 4.0 International License.
