[MS/PhD] Defense Announcement: Accelerating Reinforcement Learning using Heterogeneous Platforms: Co-Designing Hardware, Algorithm, and System Solutions

Title: Accelerating Reinforcement Learning using Heterogeneous Platforms: Co-Designing Hardware, Algorithm, and System Solutions

PhD Candidate: Yuan Meng

Committee Members: Prof Viktor K. Prasanna (Chair), Prof. Bhaskar Krishnamachari, Prof. Yue Zhao

Date: Thursday, June 20th, 2024

Time: 2pm

Location:  EEB 132

Zoom Link: https://usc.zoom.us/j/8629150353

Abstract:

Reinforcement Learning (RL) is an area of AI that constitutes a wide range of algorithms, enabling autonomous agents to learn optimal decisions through online environment interactions, data collection, and training. Recently, certain categories of RL algorithms have witnessed widespread adoption due to their generalizability and reliability, including model-free RL based on policy/value optimizations and model-based RL using Monte Carlo Tree Search. General-purpose processors fail to optimally achieve efficient execution speed for RL due to the intrinsic heterogeneous characteristics among various RL primitives and algorithms. Optimized acceleration systems that exploit heterogeneity across different architectures to support the variations of compute kernels and memory characteristics in RL are crucial to fast and efficient application development.

In this dissertation, we develop acceleration frameworks for two key categories of RL algorithms, i.e., model-free Deep RL, and model-based RL using Monte Carlo Tree Search (MCTS). We implement these frameworks by addressing two objectives: 1. We develop algorithm-hardware co-optimized accelerators for the fundamental primitives in the key categories of RL algorithms. This includes inference and training of DNN policy models, as well as dynamic tree-based operations in MCTS. 2. We create portable system solutions that identify the optimal primitive scheduling, mapping, and design configurations onto heterogeneous devices based on the task dependency, compute, and memory characteristics of the target RL algorithms.

Experiments on various platforms consisting of interconnected CPUs, FPGAs, and GPUs showcase superior performance enhancements across diverse models, algorithms, hardware platforms, and benchmark environments compared to state-of-the-art RL libraries.

Bio: Yuan Meng is a fifth-year PhD candidate in Computer Engineering, advised by Professor Viktor K. Prasanna. She obtained her BS degree in electrical and computer engineering at Rensselaer Polytechnic Institute. Her research interests include parallel computing, deep learning acceleration, heterogeneous computing, and reinforcement learning.

Published on June 10th, 2024Last updated on June 10th, 2024