Reinforcement Learning-Based Control for Smooth and Accurate Circular Trajectory Tracking in Mobile Robots
Online First: 22/10/2026
Email tác giả liên hệ:
ngunguyenviet77@gmail.comDOI:
https://doi.org/10.54644/jte.2026.2455Từ khóa:
Mobile robot, Trajectory tracking, Reinforcement learning, Motion smoothness, Soft Actor-CriticTóm tắt
This paper investigates reinforcement learning-based control for smooth and accurate trajectory tracking of a nonholonomic mobile robot. Three reinforcement learning algorithms, Deep Q-Network (DQN), Proximal Policy Optimization (PPO), and Soft Actor-Critic (SAC), are implemented and evaluated, with conventional PID and Sliding Mode Control (SMC) controllers included as comparative baselines. The robot is modeled using unicycle kinematics, and the tracking task is formulated through body-frame longitudinal, lateral, and heading errors. A reward function is designed to penalize tracking errors, radial deviation, and excessive control effort while encouraging forward progression. Simulations are conducted primarily on a circular reference trajectory, with a figure-eight trajectory included for additional comparison. The results show that all controllers can follow the reference trajectories, although their tracking accuracy and motion smoothness differ. DQN exhibits noticeable control switching, while PPO produces smoother actions but retains tracking deviations. Among the evaluated controllers, SAC achieves lower tracking errors and smaller control variations. These findings suggest that SAC is a promising approach for accurate and smooth trajectory tracking in mobile robots.
Tải xuống: 0
Tài liệu tham khảo
R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza, Introduction to Autonomous Mobile Robots, 2nd ed. Cambridge, MA, USA: The MIT Press, 2011.
J. A. Abdulsaheb and D. J. Kadhim, "Classical and Heuristic Approaches for Mobile Robot Path Planning: A Survey," Robotics, vol. 12, no. 4, Art. no. 93, 2023, doi: 10.3390/robotics12040093.
G. Yang and Y. Guo, "Deep Reinforcement Learning Based Mobile Robot Navigation in Crowd Environments," in Proc. 21st Int. Conf. Ubiquitous Robots (UR), 2024, pp. 248–254, doi: 10.1109/UR61395.2024.10597510.
Y. Zhang, C. Li, G. Zhang, R. Zhou, and Z. Liang, "Research on the Local Path Planning for Mobile Robots Based on PRO-Dueling Deep Q-Network (DQN) Algorithm," Int. J. Adv. Comput. Sci. Appl., vol. 14, no. 8, pp. 381–389, 2023, doi: 10.14569/IJACSA.2023.0140842.
T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, "Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor," in Proc. 35th Int. Conf. Mach. Learn. (ICML), ser. Proceedings of Machine Learning Research (PMLR), vol. 80, 2018, pp. 1861–1870.
T. M. T. Le, D. H. Pham, and V. N. Nguyen, "PPO Reinforcement Learning for Smooth Control of Two-Wheeled Mobile Robot," J. Meas., Control Autom., vol. 30, no. 1, pp. 57–65, 2026.
Y. Zhu, X. Wang, and H. Li, "Deep Reinforcement Learning of Mobile Robot Navigation: A Review," Sensors, vol. 25, no. 11, Art. no. 3394, 2025, doi: 10.3390/s25113394.
X. Yu, Y. Fan, S. Xu, and L. Ou, "A Self-adaptive SAC-PID Control Approach Based on Reinforcement Learning for Mobile Robots," arXiv:2103.10686, 2021.
T. Nakamura, M. Kobayashi, and N. Motoi, "Path Planning for Mobile Robot Considering Turnabouts on Narrow Road by Deep Q-Network," IEEE Access, vol. 11, pp. 19111–19121, 2023, doi: 10.1109/ACCESS.2023.3247730.
H. Xu, T. Terakawa, and M. Komori, "Deep-Reinforcement-Learning-Based Trajectory Tracking Control for Slidable-Wheel Omnidirectional Mobile Robot," J. Adv. Mech. Des., Syst., Manuf., vol. 19, no. 4, Art. no. JAMDSM0031, 2025, doi: 10.1299/jamdsm.2025jamdsm0031.
J. Bao and R. Yonetani, "Path Planning using Instruction-Guided Probabilistic Roadmaps," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2025, pp. 1–7.
H. Liu, Y. Shen, S. Yu, Z. Gao, and T. Wu, "Deep Reinforcement Learning for Mobile Robot Path Planning," arXiv:2404.06974, 2024.
X. Hua, T. Zhang, X. Cheng, and X. Ning, "Path Tracking Control for Four-Wheel Independent Steering and Driving Vehicles Based on Improved Deep Reinforcement Learning," Technologies, vol. 12, no. 11, Art. no. 218, 2024, doi: 10.3390/technologies12110218.
Y. Hu and S. X. Yang, "A Novel Knowledge-Based Genetic Algorithm for Robot Path Planning in Complex Environments," arXiv:2209.01482, 2022.
L. Zhao, J. Zhao, Z. Liu, Y. Wang, and H. Liu, "Solving the Real-Time Motion Planning Problem for Non-Holonomic Robots With Collision Avoidance in Dynamic Scenes," IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 9429–9436, Oct. 2022, doi: 10.1109/LRA.2022.3194313.
Tải xuống
Đã Xuất bản
Cách trích dẫn
Giấy phép
Bản quyền (c) 2026 Tạp chí Khoa học Giáo dục Kỹ Thuật
Tác phẩm này được cấp phép theo Giấy phép quốc tế Creative Commons Attribution-NonCommercial-NoDeri Phái sinh 4.0 .
Bản quyền thuộc về JTE.
