Journals →  Tsvetnye Metally →  2023 →  #4 →  Back

ArticleName Augmented reality as a means of metallurgical equipment servicing
DOI 10.17580/tsm.2023.04.02
ArticleAuthor Koteleva N. I., Valnev V. V., Korolev N. A.

Saint Petersburg Mining University, Saint Petersburg, Russia:

N. I. Koteleva, Associate Professor at the Department of Process and Plant Automation, Candidate of Technical Sciences, Associate Professor, e-mail:
V. V. Valnev, Postgraduate Student of the Department of Process and Plant Automation
N. A. Korolev, Lead Researcher at the Research and Training Centre for Digital Technology, Candidate of Technical Sciences


In the era of digital technology, more and more of it finds application in various industries. This paper proposes to use technology of augmented reality for maintenance of stirred tank reactors. The proposed approach can be applied to any type of equipment, as it can be easily integrated with the existing automation systems and does not require much investment at the initial stage, implying gradual optimization and functionality build-up. This paper describes the basic set of functional requirements of an augmented reality-based maintenance system, methods of assessing the system performance, as well as scaling-up and streamlining prospects. The paper also describes how such systems can be integrated with existing control systems of a production company. The effectiveness of the developed augmented reality-based system was verified by determining the average execution time of each service stage and processing the outcomes using the Mann-Whitney U test. The use of the augmented reality system resulted in the reduction of the average service time by 2.3 times, while the maintenance efficiency increased by 5%.

keywords Аugmented reality, maintenance, metallurgy, digitalization, IoT, control systems, automation, industry 4.0.

1. Ralph B. J., Woschank M., Miklautsch Ph., Konig A. et al. MUL 4.0: Systematic digitalization of a value chain from raw material to recycling. Procedia Manufacturing. 2021. Vol. 55. pp. 335–342.
2. Phua A., Davies C. H. J., Delaney G. W. A digital twin hierarchy for metal additive manufacturing. Computers in Industry. 2022. Vol. 140, Iss. 3. DOI: 10.1016/j.compind.2022.103667
3. Zhukovskii Y. L., Sizyakova E. V. The introduction of the system of energy saving and energy efficiency at the enterprises of metallurgy. Journal of Mining Institute. 2013. Vol. 202. pp. 153–155.
4. Litvinenko V. S. Digital economy as a factor in the technological development of the mineral sector. Natural Resources Research. 2020. Vol. 29, No. 3. pp. 1521–1541.
5. Kunshin A. et al. Development of monitoring and forecasting technology energy efficiency of well drilling using mechanical specific energy. Energies. 2022. Vol. 15, Iss. 19. DOI: 10.3390/en15197408
6. Dvoynikov M., Kunshin A. A., Blinov P., Morozov V. Development of mathematical model for controlling drilling parameters with screw downhole motor. International Journal of Engineering, Transactions A: Basics. 2020. Vol. 33, No. 7. pp. 1423–1430.
7. Sun Y., Wang J., Wang X. Fault diagnosis of mechanical equipment in high energy consumption industries in China: A review. Mechanical Systems and Signal Processing. 2023. Vol. 186. 109833.
8. Aguilera J. J., Meesenburg W., Ommen T. et al. A review of common faults in large-scale heat pumps. Renewable and Sustainable Energy Reviews. 2022. Vol. 168, Iss. 12. 112826.
9. Gangsar P., Tiwari R. Signal based condition monitoring techniques for fault detection and diagnosis of induction motors: A state-of-the-art review. Mechanical Systems and Signal Processing. 2020. Vol. 144, Iss. 4. 106908.
10. Khokhlov S., Abiev Z., Makkoev V. The choice of optical flame detectors for automatic explosion containment systems based on the results of explosion radiation analysis of methane-and dust-air mixtures. Applied Sciences (Switzerland). 2022. Vol. 12, No. 3. 1515.
11. Kottre A., Scholer T., Legat C. Applying engineering knowledge in alarm flood reduction to reduce machine downtime. IFAC-Papers OnLine. 2022. Vol. 55, No. 2. pp. 54–59.

12. Krishnan Umachandran, Roosefert Mohan T., Preetha Roselyn J., Uthra R. A. et al. Intelligent machine learning based total productive maintenance approach for achieving zero downtime in industrial machinery. Computers & Industrial Engineering. 2021. Vol. 157. DOI: 10.1016/j.cie.2021.107267
13. Plotnikova I., Sheveleva E., Narimanov R. Application of the system for electrical equipment diagnostics and its analysis. Studies in Systems, Decision and Control. 2023. Vol. 433. pp. 111–119.
14. Lipis E. A., Schislyaeva E. Qualification and mobility of transport complex personnel in the digitalization of shipbuilding. Transportation Research Procedia. 2022. Vol. 63. pp. 2138–2150.
15. Ramere M. D., Laseinde O. T. Optimization of condition-based maintenance strategy prediction for aging automotive industrial equipment using FMEA. Procedia Computer Science. 2021. Vol. 180. pp. 229–238.
16. Geng S., Wang X. Predictive maintenance scheduling for multiple power equipment based on data-driven fault prediction. Computers & Industrial Engineering. 2022. Vol. 164. 107898.
17. Saäski J., Salonen T., Liinasuo M., Pakkanen J. et al. Augmented reality efficiency in manufacturing industry: a case study. DS 50: Proceedings of Nord-Design 2008 Conference, Tallinn, Estonia. 21–23.08.2008. 2008. pp. 99–109.
18. Zheng L., Liu X., An Z., Li S. et al. A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection. Virtual Reality & Intelligent Hardware. 2020. Vol. 2, No. 1. pp. 12–27.
19. Costa C. M., Viega G., Sousa A. J., Rocha L. F. Modeling of video projectors in OpenGL for implementing a spatial augmented reality teaching system for assembly operations. 19th IEEE International Conference on Autonomous Robot Systems and Competitions. 2019. DOI: 10.1109/ICARSC.2019.8733617
20. Pogodaev A., Muzyleva I., Yazykova L., Kondratev S. The use of augmented reality technologies in electrical engineering. 2nd International Conference on Control Systems, Mathematical Modelling, Automation and Energy Efficiency. 2020. pp. 646–650.
21. Romo J. E., Tipantasi G. R., Andsluz V. H., Sanchez J. S. Virtual training on pumping stations for drinking water supply systems. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2019. Vol. 11614 LNCS. pp. 10–429.
22. Ko C., Lee H., Lim Y., Lee W. B. Development of augmented virtual reality-based operator training system for accident prevention in a refinery. Korean Journal of Chemical Engineering. 2021. Vol. 38, Iss. 8. pp. 1566–1577.
23. Beloglazov I. I., Petrov P. A., Bazhin V. Yu. The concept of digital twins for tech operator training simulator design for mining and processing industry. Eurasian Mining. 2020. No. 2. pp. 50–54. DOI: 10.17580/em.2020.02.12
24. Khokhlov V., Lukin V., Khokhlov S. Modelling full-colour images of Earth: simulation of radiation brightness field of Earth’s atmosphere and underlying surface. Annals of GIS. 2022. Vol. 29, Iss. 2. DOI: 10.1080/19475683.2022.2064911
25. Turman E., Wayne S. CFD modeling of LDPE autoclave reactor to reduce ethylene decomposition: Part 1 validating computational methods. Chemical Engineering Science. 2022. Vol. 257, Iss. 9. 117720.
26. Kondrasheva N. K., Kireeva E. V., Zyryanova O. V. Development of new compositions for dust control in the mining and mineral transportation industry. Journal of Mining Institute. 2021. Vol. 248, No. 2. pp. 272–280.

27. Zhdaneev О. V., Zaytsev А. V., Prodan Т. Possibilities for creating Russian high-tech bottomhole assembly. Journal of Mining Institute. 2021. Vol. 252, No. 6. pp. 872–884.
28. Bushuev A. B., Boykov V. I., Bystrov S. V., Grigoriev V. V. et al. Synthesis of optimal information and energy schemes of measuring and converting devices. Mekhatronika, Avtomatizatsiya, Upravlenie. 2021. Vol. 22, No. 10. pp. 518–526.
29. Rajan V., Sobhana N. V., Jayakrishnan R. Machine fault diagnostics and condition monitoring using augmented reality and IoT. 2018 Second International Conference on Intelligent Computing and Control Systems. 2019. pp. 910–914.
30. Larsson T., Hestetun K., Hovland E., Skogestad S. Self-optimizing control of a large-scale plant: The Tennessee Eastman process. Industrial & Engineering Chemistry Research. 2001. Vol. 40, No. 22. pp. 4889–4901.
31. Sotaniemi V.-H., Taskila S., Ojamo H., Tanskanen J. Controlled feeding of lignocellulosic substrate enhances the performance of fed-batch enzymatic hydrolysis in a stirred tank reactor. Biomass & Bioenergy. 2016. Vol. 91. pp. 271–277.
32. Skamyin A., Belsky A., Dobush V., Gurevich I. Computation of nonlinear load harmonic currents in the presence of external distortions. Computation. 2022. Vol. 10, Iss. 3. 41.
33. Kurilin S., Dli M., Sokolov A. Linear induction motors for non-ferrous metallurgy. Non-ferrous Metals. 2021. No. 1. pp. 67–73. DOI: 10.17580/nfm.2021.01.09
34. Turman E. M., Wayne S. Leveraging fuzzy Logic PID controllers for accelerating chemical reactor CFD. Chemical Engineering Science. 2022. Vol. 262. p. 118029.
35. Romashev A. O., Nikolaeva N. V., Gatiatullin B. L. Adaptive approach formation using machine vision technology to determine the parameters of enrichment products deposition. Journal of Mining Institute. 2022. Vol. 256. pp. 677–685.
36. Paduloh P., Fatahillah H., Ramadhan M. A., Muhendra R. et al. Designing of temperature control for agitator machine using Internet of Thing. IOP Conference Series: Earth and Environmental Science. 2022. Vol. 1063, Iss. 1. 012053.
37. Shilpa V., Vidya A., Pattar S. MQTT based secure transport layer communication for mutual authentication in IoT network. Global Transitions Proceedings. 2022. Vol. 3, Iss. 1. pp. 60–66.
38. Koo J., Kim Y.-G. Resource identifier interoperability among heterogeneous IoT platforms. International Computer Engineering Conference. 2022. Vol. 34, Iss. 7. pp. 4191–4208.
39. Mohammadian M., Parsaei H., Mokarami H., Kazemi R. Cognitive demands and mental workload: A filed study of the mining control room operators. Heliyon. 2022. Vol. 8, Iss. 2. e08860.
40. Hart S. G., Staveland L. E. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Advances in Psychology. 1988. Vol. 52. pp. 139–183.
41. Ruxton G. D. The unequal variance t-test is an underused alternative to Student’s t-test and the Mann-Whitney U test. Behavioral Ecology. 2006. Vol. 17, Iss. 4. pp. 688–690.
42. Al-Badi A., Khan A., Eid-Alotaibi. Perceptions of learners and instructors towards artificial intelligence in personalized learning. Procedia Computer Science. 2022. Vol. 201. pp. 445–451.

Full content Augmented reality as a means of metallurgical equipment servicing