Reinforcement Learning-Based Control for Smooth and Accurate Circular Trajectory Tracking in Mobile Robots
Online First: 22/10/2026
Corressponding author's email:
ngunguyenviet77@gmail.comDOI:
https://doi.org/10.54644/jte.2026.2455Keywords:
Mobile robot, Trajectory tracking, Reinforcement learning, Motion smoothness, Soft Actor-CriticAbstract
This paper investigates reinforcement learning-based control for smooth and accurate trajectory tracking of a nonholonomic mobile robot. Three reinforcement learning algorithms, Deep Q-Network (DQN), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC), are implemented and evaluated, with conventional PID and Sliding Mode Control (SMC) controllers included as comparative baselines. The robot is modeled using unicycle kinematics, and the tracking task is formulated through body-frame longitudinal, lateral, and heading errors. A reward function is designed to penalize tracking errors, radial deviation, and excessive control effort while encouraging forward progression. Simulations are conducted primarily on a circular reference trajectory, with a figure-eight trajectory included for additional comparison. The results show that all controllers can follow the reference trajectories, although their tracking accuracy and motion smoothness differ. DQN exhibits noticeable control switching, while PPO produces smoother actions but retains tracking deviations. Among the evaluated controllers, SAC achieves lower tracking errors and smaller control variations. These findings suggest that SAC is a promising approach for accurate and smooth trajectory tracking in mobile robots.
Downloads: 0
References
R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza, Introduction to Autonomous Mobile Robots, 2nd ed. Cambridge, MA, USA: The MIT Press, 2011.
J. A. Abdulsaheb and D. J. Kadhim, "Classical and Heuristic Approaches for Mobile Robot Path Planning: A Survey," Robotics, vol. 12, no. 4, Art. no. 93, 2023, doi: 10.3390/robotics12040093.
G. Yang and Y. Guo, "Deep Reinforcement Learning Based Mobile Robot Navigation in Crowd Environments," in Proc. 21st Int. Conf. Ubiquitous Robots (UR), 2024, pp. 248–254, doi: 10.1109/UR61395.2024.10597510.
Y. Zhang, C. Li, G. Zhang, R. Zhou, and Z. Liang, "Research on the Local Path Planning for Mobile Robots Based on PRO-Dueling Deep Q-Network (DQN) Algorithm," Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, pp. 381–389, 2023, doi: 10.14569/IJACSA.2023.0140842.
T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor," in Proc. 35th Int. Conf. Mach. Learn. (ICML), ser. Proceedings of Machine Learning Research (PMLR), vol. 80, 2018, pp. 1861–1870.
T. M. T. Le, D. H. Pham, and V. N. Nguyen, "PPO Reinforcement Learning for Smooth Control of Two-Wheeled Mobile Robot," J. Meas., Control Autom., vol. 30, no. 1, pp. 57–65, 2026.
Y. Zhu, X. Wang, and H. Li, "Deep Reinforcement Learning of Mobile Robot Navigation: A Review," Sensors, vol. 25, no. 11, Art. no. 3394, 2025, doi: 10.3390/s25113394.
X. Yu, Y. Fan, S. Xu, and L. Ou, "A Self-adaptive SAC-PID Control Approach Based on Reinforcement Learning for Mobile Robots," arXiv:2103.10686, 2021.
T. Nakamura, M. Kobayashi, and N. Motoi, "Path Planning for Mobile Robot Considering Turnabouts on Narrow Road by Deep Q-Network," IEEE Access, vol. 11, pp. 19111–19121, 2023, doi: 10.1109/ACCESS.2023.3247730.
H. Xu, T. Terakawa, and M. Komori, "Deep-Reinforcement-Learning-Based Trajectory Tracking Control for Slidable-Wheel Omnidirectional Mobile Robot," J. Adv. Mech. Des., Syst., Manuf., vol. 19, no. 4, Art. no. JAMDSM0031, 2025, doi: 10.1299/jamdsm.2025jamdsm0031.
J. Bao and R. Yonetani, "Path Planning using Instruction-Guided Probabilistic Roadmaps," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2025, pp. 1–7.
H. Liu, Y. Shen, S. Yu, Z. Gao, and T. Wu, "Deep Reinforcement Learning for Mobile Robot Path Planning," arXiv:2404.06974, 2024.
X. Hua, T. Zhang, X. Cheng, and X. Ning, "Path Tracking Control for Four-Wheel Independent Steering and Driving Vehicles Based on Improved Deep Reinforcement Learning," Technologies, vol. 12, no. 11, Art. no. 218, 2024, doi: 10.3390/technologies12110218.
Y. Hu and S. X. Yang, "A Novel Knowledge-Based Genetic Algorithm for Robot Path Planning in Complex Environments," arXiv:2209.01482, 2022.
L. Zhao, J. Zhao, Z. Liu, Y. Wang, and H. Liu, "Solving the Real-Time Motion Planning Problem for Non-Holonomic Robots With Collision Avoidance in Dynamic Scenes," IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 9429–9436, Oct. 2022, doi: 10.1109/LRA.2022.3194313.
Downloads
Published
How to Cite
License
Copyright (c) 2026 Journal of Technical Education Science
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright © JTE.
