Virtual Evaluation of Deep Learning Techniques for Vision-Based Trajectory Tracking 2022-01-0369

Artificial intelligence (AI) enhanced control system deployments are emerging as a viable substitute to more traditional control system. In particular, deep learning techniques offer an alternate approach to tune the ever increasing sets of control system parameters to extract performance. However, the systematic verification and validation (to establish the reliability and robustness) of deep learning based controllers in actual deployments remains a challenge. This is exacerbated by the need to evaluate and optimize control systems embedded within an operational environment (with its own sets of additional unknown or uncertain parameters). Existing literature comparisons of deep learning against traditional controllers, where they may exist, do not offer structured approaches to comparative performance evaluation and improvement. It is also crucial to develop a standardized controlled test environment within which various controllers are evaluated against a common metric. Hence, in this paper, we evaluate deep learning based controllers by a structured selection of pantheon of evaluation metrics and employ the insights to propose further modifications to the deep learning algorithms. We first evaluate a high resolution proportional-integral-derivative (PID) controller to serve as a common benchmark and develop the suite of evaluation metrics for a mobile robot to create a point-to-point trajectory within in the simulation environment. Against this backdrop, we then evaluate alternate variants of imitation learning and deep reinforcement learning controllers which use camera image stream as an input to develop or tune trajectory tracking output parameters. Subsequent deployments of control strategies on actual hardware (in a Sim2Real transition) is anticipated, where the developed evaluation metrics can be used as a validation tool when deploying the control strategy on actual hardware.