I was lurking on the interweb one day and ended up browsing through the AWS DeepRacer Student League page, so I decided to try it out. The AWS DeepRacer Student League is basically an AI and ML scholarship program from AWS that rewards qualified students with a Udacity Nanodegree scholarship and mentorship. Pretty neat! The Udacity AI Programming Nanodegree program offers 4 courses: Introduction to Python; Anaconda, Jupyter Notebook, NumPy, Pandas and Matplotlib; Linear Algebra Essentials; and Calculus Essentials and Neural Networks.
To be eligible, you have to be over 16 years old and currently enrolled in a high school or higher education institution. After signing up (AWS Account Signup Link), you have to complete 2 learning modules: Introduction to Machine Learning and Reinforcement Learning. Each module takes 10 hours to complete and covers topics like what machine learning is, the machine learning process, case studies, the basics of reinforcement learning, how the AWS DeepRacer model works and tips on tweaking it to perform well.
Once I finished the modules, I could finally create my own model, yay! Create a model link
It's pretty much a step-by-step process: choose the direction of the track (clockwise or counterclockwise), choose the algorithm (Proximal Policy Optimization (PPO) or Soft Actor Critic (SAC)), customize the reward function and specify the training duration. Seems pretty easy…
Or so I thought 😄 Take a look at my overly complicated model's performance below. Rank 1 finished with an overall time of 00.39.887. Mine was 00.57.066, which was actually my best time out of the 8 submissions I made. So yeah, maybe I should have simplified my reward function instead of piling on hyperparameters. I'll try that next time...probably.
If you're curious about how the reward function gets tweaked, check out AWS's docs on it. It's basically a Python function that takes in params like the car's x and y coordinates, speed, steering angle and more. Input parameters of the AWS DeepRacer reward function
I had fun with this whole thing. Tweaking a small part of the model doesn't sound like much, but as someone new to machine learning, it gave me a glimpse into how ML engineers actually approach a problem, one reward function and one hyperparameter at a time. I also just enjoyed watching the agent (the car) learn the track lap after lap. If you're eligible, I'd encourage you to give it a try yourself.