29 Oct 2021
Over 90 students put on their virtual helmets as they raced against one another at the Amazon Web Services (AWS) DeepRacer League Workshop!
Jointly organised by DSTA, AWS, and SG Code Campus, the virtual event was held on the sidelines of the Singapore Defence Technology Summit 2021 to put the students in pole position as they got hands-on experience with Reinforcement Learning (RL) – an advanced machine learning technique – to build and train their very own race cars known as AWS DeepRacers.
Addressing the participants, Head Capability Development (Data Centre & Cloud) Adrian Toh, who was part of the workshop’s organising committee, said: “RL is growing in significance as it allows autonomous vehicles such as drones and self-driving cars to adapt swiftly based on real-world complexities and dynamics. We hope that through this workshop, you will be able to learn about RL in an interesting and fun way using a cloud-based 3D racing simulator, fully autonomous race cars driven by RL, and a global racing league!”
From 8 to 10 October 2021, participants attended online classes in cloud computing, machine learning, RL, and Python. They also got their hands dirty by training their agents – in this case a virtual race car – to race through a track in the quickest time possible using a cloud-based 3D racing simulator.
As RL enables the agent to learn through trial and error, participants had to configure a model – the agent’s ‘brain’ essentially – by developing algorithms that reward and punish the agent. This process would help the agent determine the appropriate steering angles and speeds to navigate around the race track successfully.
Participants could also finetune their agents using seven hyper parameters. For example, participants could determine the rate at which their agents learn. While a higher learning rate would enable the agent to learn faster, its learning might not be sufficient for the challenges ahead.
After some training, participants tested the waters with their agents, with each having a mere three minutes to navigate around a virtual race track. Their overall timing was determined by averaging their three fastest lap timings.
The workshop culminated in a physical race on the final day, which pitted the 12 fastest agents against one another. The models of these agents were uploaded to AWS DeepRacers, which are autonomous 1/18th scale race cars. With cameras to view the track and a reinforcement model to control throttle and steering, the AWS DeepRacer shows how a model trained in a simulated environment could perform in the real world.
After a neck and neck race, Nanyang Technological University student Seah Jun Sheng took first place with a timing of 18.54s. Reflecting on his experience, he said: “I was able to learn about RL in a fun and interesting way. The trainers were also very friendly, and their insights into the world of machine learning and artificial intelligence helped to enrich the lessons. This experience definitely got me interested to find out more about the applications of RL and machine learning. I would encourage others to participate in future workshops!”
Adrian added: “It’s heartening to see the next generation investing their time to learn more about RL. Such workshops will undoubtedly strengthen their theoretical learning and help them realise the significance of RL in the real world.”