AWS DeepRacer Student League
I was lurking on the interweb one day and ended up browsing through the AWS DeepRacer Student League page, so I decided to try it out. AWS Deep Racer Student is basically AI and ML Scholarship Program from AWS that aims to reward qualified students with Udacity Nanodegree scholarship and mentorship. Pretty neat! The Udacity AI Programming Nanodegree program offers 4 courses: Introduction to Python; Anaconda, Jupyter Notebook, NumPy, Pandas and Matplotlib, Linear Algebra Essentials, Calculus Essentials and Neural Network.
To be eligible, you have to be over 16 years of age and are currently enrolled in a high school or higher education institution. After signing up (AWS Account Signup Link), students have to complete 2 learning modules about Introduction to Machine Learning and Reinforcement Learning. Each module requires 10 hours to complete and consist of topics such as what is machine learning, the machine learning process, case studies, basics of reinforcement learning, how AWS DeepRacer model works, and tips on how to tweak it to perform well.
Once done with the module, we can now create our own model, yay! Create a model link
This is pretty much a step-by-step process, all you need to do is to choose the direction of the track (either clockwise or counterclockwise), choose the algorithm (either Proximal Policy Optimization (PPO) or Soft Actor Critic (SAC)), customize the reward function and specify the training duration. Seems pretty easy…
Or so I thought 😄 Take a look at my overly complicated model performance, which is definitely too far away from Rank 1 which has an overall time of 00.39.887 — mine was 00.57.066 which is basically the best model out of the 8 submissions. So yeah, maybe I should have simplified my reward function more instead of adding too many hyperparameters. I will try that next time...probably.
If you're curious about how the reward function is tweaked, you can check this link. It's basically a Python function that accepts params such as the car's x and y coordinates, speed, steering angle, etc. Input parameters of the AWS DeepRacer reward function
Overall, I had fun with this activity. It may just be tweaking a small part of the model. Still, as someone new to machine learning, it gives a glimpse into how machine learning engineers approach a problem. Also, I enjoy watching the agent (i.e. the car) learn the track. I encourage you to try it yourself too.