A Systematic Literature Review of Autonomous Line-Following Robots using Reinforcement Learning

Matheus Amorim Pereira; Hianna Araujo dos Reis Soares; Sara Dereste dos Santos; Ricardo Pires

doi:10.22456/2175-2745.150105

Authors

Matheus Amorim Pereira Instituto Federal de Educação, Ciência e Tecnologia de São Paulo (IFSP) https://orcid.org/0009-0002-1819-1549
Hianna Araujo dos Reis Soares Instituto Federal de Educação, Ciência e Tecnologia de São Paulo (IFSP) https://orcid.org/0009-0006-2723-4378
Sara Dereste dos Santos Instituto Federal de Educação, Ciência e Tecnologia de São Paulo (IFSP)
Ricardo Pires Instituto Federal de Educação, Ciência e Tecnologia de São Paulo (IFSP) https://orcid.org/0000-0003-4677-8435

DOI:

https://doi.org/10.22456/2175-2745.150105

Keywords:

Reinforcement Learning, Line-Following Robot, Machine Learning, Artificial Intelligence, Education, Systematic Literature Review

Abstract

Mobile robots have gained importance due to its capability to perform autonomous tasks. Line-following robots are mobile robots which have become a growing topic of research, as they serve as a platform for understanding complex concepts, such as control, which makes them suitable for the application of Artificial Intelligence (AI). There are competitions around the world focused on this kind of robots. Accurate mathematical modeling of line-following robots and the environment in which they will be used can be difficult. Reinforcement learning is an AI technique that deals precisely with creating model-free algorithms, which are suitable for this application. This review article aimed to look for articles on line-following robots using reinforcement learning in order to analyze the state of the art of this branch of artificial intelligence in robotics.

Downloads

Download data is not yet available.

References

[1] ABDULSAHEB, J. A.; KADHIM, D. J. Classical and heuristic approaches for mobile robot path planning: A survey. Robotics, MDPI, v. 12, n. 4, p. 93, 2023. ⟨https://doi.org/10.3390/robotics12040093⟩.

[2] ANTONYSHYN, L. et al. Multiple mobile robot task and motion planning: A survey. ACM Computing Surveys, ACM New York, NY, v. 55, n. 10, p. 1–35, 2023. ⟨https://doi.org/10.1145/3564696⟩.

[3] KARUR, K. et al. A survey of path planning algorithms for mobile robots. Vehicles, MDPI, v. 3, n. 3, p. 448–468, 2021. ⟨https://doi.org/10.3390/vehicles3030027⟩.

[4] KABIR, M. A. et al. Design and implementation of automatic line follower robot for assistance of covid-19 patients. In: Sustainable Communication Networks and Application: Proceedings of ICSCN 2021. [S.l.]: Springer, 2022. p. 243–255. ⟨https://doi.org/10.1007/978-981-16-6605-6_17⟩.

[5] SAQIB, N.; YOUSUF, M. M. Design and implementation of shortest path line follower autonomous rover using decision making algorithms. In: IEEE. 2021 Asian Conference on Innovation in Technology (ASIANCON). [S.l.], 2021. p. 1–6. ⟨https://doi.org/10.1109/ASIANCON51346.2021.9544672⟩.

[6] OSWAL, S.; SARAVANAKUMAR, D. Line following robots on factory floors: Significance and simulation study using coppeliasim. In: IOP PUBLISHING. IOP Conference Series: Materials Science and Engineering. [S.l.], 2021. v. 1012, n. 1, p. 012008.

[7] IBM. What is machine learning? Available in: ⟨https://www.ibm.com/think/topics/machine-learning⟩. Accessed in March 12, 2025.

[8] NEXUS. Speed and Precision: Follow-Line Robot Challenge at NSRC 2024. 2024. Available in: ⟨https://news.utm.my/2024/12/speed-and-precision-follow-line-robot-challenge-at-nsrc-2024/⟩. Accessed in September 5, 2025.

[9] TECHNOXIAN. Fastest Line Follower. 2024. Available in: ⟨https://www.technoxian.com/fastest-line-follower⟩. Accessed in May 14, 2025.

[10] Harbour Edutech Pvt Ltd. Fastest Line Follower Championship. 2024. Available in: ⟨https://techradiance.in/line-follower-robot-competition/⟩. Accessed in May 14, 2025.

[11] UKMARS. Line Follower. 2020. Available in: ⟨https://ukmars.org/contests/line-follower/⟩. Accessed in May 14, 2025.

[12] Robotex International. PCBWay LEGO Line Following. 2025. Available in: ⟨https://robotex.international/lego-line-following/⟩. Accessed in May 14, 2025.

[13] Yelgea Event. Line Follower. 2025. Available in: ⟨http://www.robotchallenge.org.cn/competition-LineFollower.html⟩. Accessed in May 14, 2025.

[14] NASA. Robotics Alliance Project. 2025. Available in: ⟨https://robotics.nasa.gov/⟩. Accessed in May 14, 2025.

[15] OGATA, K. et al. Modern control engineering. [S.l.]: Prentice Hall, 2009.

[16] ERTEL, W. Introduction to Artificial Intelligence, 2nd edition. Switzerland: Springer International Publishing, 2017.

[17] CANESE, L. et al. Multi-agent reinforcement learning: A review of challenges and applications. Applied Sciences, v. 11, n. 11, 2021. ISSN 2076-3417. ⟨https://doi.org/10.3390/app11114948⟩.

[18] SHAKYA, A. K.; PILLAI, G.; CHAKRABARTY, S. Reinforcement learning algorithms: A brief survey. Expert Systems with Applications, Elsevier, v. 231, p. 120495, 2023. ⟨https://doi.org/10.1016/j.eswa.2023.120495⟩.

[19] FRANÇOIS-LAVET, V. et al. An introduction to deep reinforcement learning. Foundations and Trends® in Machine Learning, Now Publishers, Inc., v. 11, n. 3-4, p. 219–354, 2018.

[20] COTA, J. L. et al. Roadmap for development of skills in artificial intelligence by means of a reinforcement learning model using a deepracer autonomous vehicle. In: 2022 IEEE Global Engineering Education Conference (EDUCON). [S.l.: s.n.], 2022. p. 1355–1364. ⟨https://doi.org/10.1109/EDUCON52537.2022.9766659⟩.

[21] TAN, F.; YAN, P.; GUAN, X. Deep reinforcement learning: From q-learning to deep q-learning. In: SPRINGER. Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14–18, 2017, Proceedings, Part IV 24. [S.l.], 2017. p. 475–483. ⟨https://doi.org/10.1007/978-3-319-70093-9_50⟩.

[22] HASSEL, T.; HOFMANN, O. Reinforcement learning of robot behavior based on a digital twin. In: ICPRAM. [S.l.: s.n.], 2020. p. 381–386. ⟨https://doi.org/10.5220/0008880903810386⟩.

[23] KRAWCZYK, A. et al. Continual reinforcement learning without replay buffers. In: 2024 IEEE 12th International Conference on Intelligent Systems (IS). [S.l.: s.n.], 2024. p. 1–9. ⟨https://doi.org/10.1109/IS61756.2024.10705256⟩.

[24] SAADATMAND, S. et al. Autonomous control of a line follower robot using a q-learning controller. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC). [S.l.: s.n.], 2020. p. 0556–0561. ⟨https://doi.org/10.1109/CCWC47524.2020.9031160⟩.

[25] LEE, C.-T.; SUNG, W.-T. Controller design of tracking wmr system based on deep reinforcement learning. Electronics, MDPI, v. 11, n. 6, p. 928, 2022. ⟨https://doi.org/10.3390/electronics11060928⟩.

[26] HUBER, L.; SLOTINE, J.-J.; BILLARD, A. Avoidance of concave obstacles through rotation of nonlinear dynamics. IEEE Transactions on Robotics, IEEE, v. 40, p. 1983–2002, 2023. ⟨https://doi.org/10.1109/TRO.2023.3344034⟩.

[27] Open Robotics. ROS - Robot Operating System. 2025. Available in: ⟨https://www.ros.org/⟩. Accessed in Aug. 16, 2025.

[28] JAFARI-TABRIZI, A.; GRUBER, D. Reinforcement-learning-based control of an industrial robotic arm for following a randomly-generated 2d-trajectory. In: . [S.l.: s.n.], 2021. p. 1–6. ⟨https://doi.org/10.1109/COINS51742.2021.9524158⟩.

[29] MEDEIROS, T. F. de; MÁXIMO, M. R. O. de A.; YONEYAMA, T. Deep reinforcement learning applied to ieee very small size soccer strategy. In: 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE). [S.l.: s.n.], 2020. p. 1–6. ⟨https://doi.org/10.1109/LARS/SBR/WRE51543.2020.9306954⟩.

[30] MEDEIROS, T. F. de; YONEYAMA, T.; MÁXIMO, M. R. Deep reinforcement learning applied to ieee’s very small size soccer in cooperative attack strategy. In: IEEE. 2023 Latin American Robotics Symposium (LARS), 2023 Brazilian Symposium on Robotics (SBR), and 2023 Workshop on Robotics in Education (WRE). [S.l.], 2023. p. 349–354. ⟨https://doi.org/10.1109/LARS/SBR/WRE59448.2023.10332923⟩.

[31] KONG, S.-C.; YANG, Y. Using the robot-assisted attention-engagement-error-feedback-reflection (aeer) pedagogical design to develop machine learning concepts and facilitate reflection on learning-to-learn skills: Evaluation of an empirical study in hong kong primary schools. In: CSEDU (2). [S.l.: s.n.], 2024. p. 155–162. ⟨https://doi.org/10.5220/0012505700003693⟩.

[32] LOPEZ-RODRIGUEZ, F. M.; CUESTA, F. An android and arduino based low-cost educational robot with applied intelligent control and machine learning. Applied Sciences, v. 11, n. 1, 2021. ISSN 2076-3417. ⟨https://doi.org/10.3390/app11010048⟩.

[33] JOVENTINO, C. F. et al. A sim-to-real practical approach to teach robotics into k-12: A case study of simulators, educational and diy robotics in competition-based learning. Journal of Intelligent & Robotic Systems, Springer, v. 107, n. 1, p. 14, 2023. ⟨https://doi.org/10.1007/s10846-022-01790-2⟩.

[34] BUXTON, E.; JAVADI, E.; HAGAMAN, M. Foundations of autonomous vehicles: A curriculum model for developing competencies in artificial intelligence and the internet of things for grades 7–10. In: Proceedings of the AAAI Conference on Artificial Intelligence. [S.l.: s.n.], 2024. v. 38, n. 21, p. 23276–23284. ⟨https://doi.org/10.1609/aaai.v38i21.30375⟩.

[35] ZHENG, Q. et al. Continuous reinforcement learning based ramp jump control for single-track two-wheeled robots. Transactions of the Institute of Measurement and Control, SAGE Publications Sage UK: London, England, v. 44, n. 4, p. 892–904, 2022. ⟨https://doi.org/10.1177/01423312211037847⟩.

[36] ZHENG, Q. et al. Reinforcement learning-based control of single-track two-wheeled robots in narrow terrain. Actuators, v. 12, n. 3, 2023. ISSN 2076-0825. ⟨https://doi.org/10.3390/act12030109⟩.

[37] WANG, Z.; LING, Y.; MA, M. A deep learning optimized lqr method for enhanced formation control with embedded systems. Engineering Research Express, IOP Publishing, v. 6, n. 2, p. 025203, 2024. ⟨https://doi.org/10.1088/2631-8695/ad3c12⟩.

[38] BEZERRA, C. D. de S.; CARDOSO, I. H. L.; VIEIRA, F. H. T. Deep reinforcement learning with convolutional networks applied to autonomous navigation of real robots using virtual scenario training. In: IEEE. 2023 Latin American Robotics Symposium (LARS), 2023 Brazilian Symposium on Robotics (SBR), and 2023 Workshop on Robotics in Education (WRE). [S.l.], 2023. p. 302–307. ⟨https://doi.org/10.1109/LARS/SBR/WRE59448.2023.10332922⟩.

[39] YILDIZ, H. et al. Sliding mode control of a line following robot. Journal of the Brazilian Society of Mechanical Sciences and Engineering, Springer, v. 42, n. 11, p. 561, 2020. ⟨https://doi.org/10.1007/s40430-020-02645-3⟩.

[40] SINGH, R.; BERA, T. K.; CHATTI, N. A real-time obstacle avoidance and path tracking strategy for a mobile robot using machine-learning and vision-based approach. Simulation, SAGE Publications Sage UK: London, England, v. 98, n. 9, p. 789–805, 2022. ⟨https://doi.org/10.1177/00375497221091592⟩.