About Me
I am a Founding Member of Technical Staff at Oak Lab, where we are building algorithms that will enable all kinds of artificial systems to learn continually in real time—from their own experience—with orders of magnitude less compute and energy than contemporary methods.
Recently, I finished a postdoc at the National Research Council Canada, where I did reinforcement-learning research at the intersection of robotics and space applications. Check out this video of our robot dog (Rocky!) traversing lunar-like terrain zero-shot after learning to walk purely in simulation in less than 20 minutes using a single GPU!
Previously, I obtained my Ph.D. at the University of Alberta with Rich Sutton, where I developed simple and practical algorithms from first principles for long-lived artificial decision-making systems. In particular, I developed the algorithms within the reinforcement-learning framework for continuing (non-episodic) problems—in which the agent-environment interaction goes on ad infinitum—with the goal of maximizing the average reward obtained per step. Empirically, my algorithms are easy to implement and use.
I love space! I envision a future where artificial systems will have human-like intelligence and adaptability, making space exploration significantly easier and safer for our species.
Here my resume (last updated: May 2026).
Some Updates
- (Nov 2026) Our robot-dog work was featured in Nature’s weekly AI&Robotics newsletter!
- (Oct 2026) NRC published a wonderful article about our RL-for-space-robotics work for a very general audience!
- (Oct 2026) Joined Oak Lab as a Founding Member of Technical Staff! Really excited for what we’ll build!
- (Sep 2026) Presented our robot-dog work at the Space Robotics Workshop at IROS 2026!
- (May 2026) Two papers that I co-authored are accepted in the Reinforcement Learning Journal: one on robotic manipulation and one on sim2real transfer.
- (Jan 2026) I am hosting two interns over the Winter term. This is my first time supervising students!
- (Oct 2025) My first research proposal was approved! I am the lead PI for a six-month $150k project on safe RL-based robotics.
- (Feb 2025) I had the pleasure of being interviewed for a TalkRL podcast episode!
- (Sep 2024) Started a postdoc on RL research for space applications at NRC!
Ph.D. Dissertation
This dissertation develops simple and practical learning algorithms from first principles for long-lived agents. Formally, the algorithms are developed within the reinforcement learning framework for continuing (non-episodic) problems, in which the agent-environment interaction goes on ad infinitum, with the goal of maximizing the average reward obtained per step.
There are three main contributions:
- Foundational one-step tabular learning algorithms for average-reward prediction and control.
- Multi-step prediction algorithms for average-reward prediction, some of which are proved to converge with linear function approximation.
- Reward centering to improve discounted-reward algorithms.
All of the above contributions are grounded in theory. My experiments show that the performance of the proposed algorithms is robust to the choice of their parameters—making them easy to use.
Defended in March 2024. [Dissertation PDF, Defense Slides, Defense Seminar video]
Selected Research
-
Some more interesting work on the way… :)
-
Towards Reinforcement Learning for Space Robotics: Fast Training and Reliable Real-World Transfer [PDF]
Abhishek Naik, Michael Wu, Michael O’Sullivan, Colin Bellinger, Yunli Wang
In Space Robotics Workshop, IROS 2026.
(A pop-science article about this work was featured in Nature’s AI & Robotics newsletter on November 3rd!) -
Rethinking the Suitability of RL Algorithms Under Practical Transfer Constraints [PDF]
Hany Hamed, Abhishek Naik, Colin Bellinger, A. Rupam Mahmood
In Reinforcement Learning Conference (RLC), 2026. -
Dynamic Object Masks as Goal Representations for Visual Goal-Conditioned Reinforcement Learning [PDF]
Fahim Shahriar, Cheryl Wang, Seyed Alireza Azimi, Gautham Vasan, Hany Hamed, Abhishek Naik, A. Rupam Mahmood, Colin Bellinger
In Reinforcement Learning Conference (RLC), 2026. -
Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation [PDF]
Seyed Alireza Azimi, Homayoon Farrahi, Abhishek Naik, Colin Bellinger, A. Rupam Mahmood
Under review. -
Reward Centering [PDF]
Abhishek Naik, Yi Wan, Manan Tomar, Richard S. Sutton
In Reinforcement Learning Conference (RLC), 2024.
-
Multi-Step Average-Reward Prediction via Differential TD(lambda) [PDF]
Abhishek Naik, Richard S. Sutton
In The Conference on Reinforcement Learning and Decision Making (RLDM), 2022. -
Average-Reward Learning and Planning with Options [PDF]
Yi Wan, Abhishek Naik, Richard S. Sutton
In Advances in Neural Information Processing Systems (NeurIPS), 2021. -
Towards Reinforcement Learning in the Continuing Setting [PDF]
Abhishek Naik, Zaheer Abbas, Adam White, Richard S. Sutton
In Never-Ending Reinforcement Learning (NERL) Workshop, ICLR 2021. -
Learning and Planning in Average-Reward Markov Decision Processes [PDF]
Yi Wan*, Abhishek Naik*, Richard S. Sutton
In International Conference on Machine Learning (ICML), 2021. -
Discounted Reinforcement Learning is Not an Optimization Problem [PDF]
Abhishek Naik, Roshan Shariff, Niko Yasui, Richard S. Sutton
In Optimization Foundations of Reinforcement Learning Workshop, NeurIPS 2019. -
MADRaS: Multi Agent DRiving Simulator [PDF]
Anirban Santara, Sohan Rudra, Sree Aditya Buridi, Meha Kaushik, Abhishek Naik, Bharat Kaul, Balaraman Ravindran
In Journal of Artificial Intelligence Research (JAIR), 2021.
Selected Talks
-
Reinforcement Learning for Continuing Tasks
Carleton University’s NTN Seminar, Oct 2026
[Slides] -
Robot Learning at Scale: Learning To Walk From Scratch Within 11 Minutes of Training
NRC Knowledge-Exchange Seminar, May 2026
[Slides coming soon] -
Intro to Reinforcement Learning: What, When, How?
NRC Knowledge-Exchange Seminar, Dec 2024
[Slides] -
Demystifying the Discount Factor
Guest Lecture at a graduate RL course at University of Alberta, Nov 2024
[Slides] -
Reinforcement Learning in Continuing Problems Using Average Reward
-
An Experimentalist’s Venture into RL Theory: Two Successes and a Failure
Amii AI Seminar, Feb 2024
[Video, Slides] -
Unifying Perspectives on Intelligence: What RL adds to the Common Model of the Agent
Science of Intelligence Institute’s Summer School, Aug 2023
[Slides] -
Essentials of Reinforcement Learning
3rd Nepal Winter School in AI, Dec 2021
[Slides]
-
Personalized Brain State Targeting via Reinforcement Learning
The 3rd Neuromatch Conference, Oct 2020
[Video (more Q/A here), Slides] -
Learning and Planning in Average-Reward MDPs
Tea Time Talks, RLAI lab and Amii, Aug 2020
[Video, Slides] -
On Intelligence: A Glimpse of the Diversity in Natural Intelligence
-
Figuring Out How the Mind Works: At the Exciting Intersection of RL, Psychology, and Neuroscience
Cognitive Psychology Seminar, Dept. of Psychology, University of Alberta, March 2020
[Video, Slides] -
Discounting — Does It Make Sense?
Tea Time Talks, RLAI lab and Amii, Aug 2019
[Video, Slides]
Work Experience
Oak Lab
Founding Member and Member of Technical Staff
Oct 2026 – ongoing; Toronto, Canada
- At Oak Lab, we are building algorithms that will enable all kinds of artificial systems to learn continually in real time—from their own experience—with orders of magnitude less compute and energy than contemporary methods. Stay tuned for exciting updates!
National Research Council of Canada
Postdoc Fellow
Sep 2024 – Sep 2026; Ottawa, Canada
- At NRC, I did RL research with a particular focus on applications in the space industry. I was involved in several projects, including a $150k project that I led on RL-based quadruped robotics which was featured in Nature’s weekly newsletter! I also mentored/co-supervised students through internships at NRC and through collaborations with universities.
AlbertaSat
Member: Software, Automation, and Testing Team
April 2023 – Aug 2024; Edmonton, Canada
- AlbertaSat is University of Alberta’s student group that designs, builds, and operates nano satellites. My role was to simulate various operational and safety scenarios of our upcoming satellite to ensure it can robustly achieve all the mission objectives. I also did some explorative RL/AI work for them on the side!
Google Research, Brain Team
Research Scientist Intern
June 2022 – Sep 2022; Toronto, Canada
With Bo Chang and Alexandros Karatzoglou.
- Investigated methods for action-space generalization in RL for large-scale recommender systems like YouTube.
Huawei Research
Research Scientist Intern
May 2019 – Sep 2019; Edmonton, Canada
With Hengshuai Yao.
- Worked on establishing an appropriate problem formulation for control in continuing tasks with function approximation.
- Surveyed the literature on the average reward problem formulation for MDPs, and its connection with reinforcement learning.
- Some of the work started here was presented at the NeurIPS 2019 Workshop on Optimization Foundations of Reinforcement Learning (OPTRL 2019).
Intel Labs
Research Intern
May 2017 – Jul 2017; Bengaluru, India
With Bharat Kaul
- Started work on a multi-agent version of the TORCS driving simulator (MADRaS) compatible with OpenAI Gym.
- Proposed and implemented a novel risk-averse imitation learning framework, achieving upto 89% improvement over the state-of-the-art in terms of tail-end risk at several physics-based control tasks.
- This project was presented at AAMAS 2018.
Purdue University
Research Intern
May 2016 – Jul 2016; Indiana, USA
With Bruno Ribeiro
- Engineered temporal features to design a binary probabilistic classifier to categorise the expected lifespan of new users based on their initial activity.
- Created and curated one of the richest social-media datasets and released it for public use via a technical paper.
Amazon Development Centre
Technical Internship; May 2015 – Jul 2015; Chennai, India
With Sravan Bodapati and Venkatraman Kalyanapasupathy
- Built a classifier to determine the start-reading-location of books.
- Now in production, this feature helps Kindle users start reading a book quicker after downloading it, without having to flip through pages like acknowledgements or copyright notices. If you use a Kindle, you have used this feature :)
Master’s Thesis
This thesis was a part of my integrated Bachelor’s + Master’s program in the Dept. of Computer Science and Engineering at the Indian Institute of Technology Madras in Chennai, India, supervised by Professor Balaraman Ravindran. Defended in May 2018.
My goal was to make self-driving cars a reality in my country, India. Towards this end, I modeled it as a multi-agent learning problem in a safety-critical application and:
- proposed a risk-averse imitation learning algorithm that had lower tail-end risk w.r.t. the then state-of-the-art,
- trialled a curriculum-based learning approach for multi-agent RoboSoccer, and
- extended the TORCS simulator to release the first open-source driving simulator that supports multi-agent training — MADRaS (has 100+ stars on Github).
Teaching Experience
-
Reinforcement Learning II (CMPUT609) (x4)
Jan - Apr 2024, 2023, 2021, 2020; Dept. of Computing Science, University of Alberta
As a teaching assistant for Professor Rich Sutton’s class of ~30 graduate students, I presented some lectures, helped create the assignments and lectures, and spent a surprisingly large amount of time grading. -
Reinforcement Learning I (CMPUT397)
Sep 2020 - Dec 2020; Dept. of Computing Science, University of Alberta
Helped Professor Martha White teach a class of ~150 undergraduate students. -
Reinforcement Learning (CS6700)
Jan 2018 - May 2018; Dept. of CSE, IIT Madras
As the Head Teaching Assistant of this course offered by Professor Balaraman Ravindran, I created and evaluated tutorials, programming assignments, and exams for a class of about 90 undergraduates and graduates. -
Principles of Machine Learning (CS4011)
Aug 2017 - Nov 2017; Dept. of CSE, IIT Madras
As one of Teaching Assistants of this course offered by Professor Balaraman Ravindran and Professor Mitesh Khapra, I created and evaluated tutorials, programming assignments, and quizzes for a class of about 90 undergraduates.
-
Reinforcement Learning Specialization on Coursera [Link]
Jan 2019 - Oct 2019; University of Alberta
As one of the ‘Subject Matter Expert’s, I developed programming assignments, multiple-choice quizzes, and slides for the four courses that form the RL Specialization, released in late 2019. As of Dec 2025, there have been more than 100k enrolments!
Community Service
-
Reviewer
I review reinforcement-learning papers semi-regularly for conferences, workshops, and journals, including for RLC, EWRL, ICLR, AAAI, ICML, NeurIPS, Artificial Intelligence.
-
Co-organizer, ICML 2021 Social on Continuing (Non-episodic) Problems in RL
July 2021
Had insightful discussions with a bunch of people about the state of research in continuing problems and where we should go from here. -
Co-organizer, NeurIPS 2020 Tutorial on Policy Optimization in RL
Dec 2020
Alan Chan, Shivam Garg, Dhawal Gupta and I created a set of notebooks to highlight some aspects of policy-gradient methods, such as the effects of a baseline. Thanks to Sham Kakade, Martha White, and Nicolas Le Roux for giving us the opportunity! -
Organizer, Tea Time Talks 2020, Amii and RLAI lab
June 2020 – Aug 2020
Organized and moderated the talks of 40+ speakers over the course of 12 weeks (in a virtual format for the first time). Full playlist here. -
Executive Member, Computer Science Graduate Students’ Association, University of Alberta
Apr 2019 – Apr 2020
Along with representing the interests of the graduate students to the department, I helped organize activities which support their well-being – physically and emotionally, academically and personally – to make University of Alberta a home away from home, especially for international students. -
Volunteer, Centre for Autism Services Alberta
Jan 2019 - Mar 2020
As a part of the Centre’s Community and Therapeutic program, I helped organize recreational activities for individuals in the age range of 5-20 affected with the Autism Spectrum Disorder. The aim was to create a fun and supportive atmosphere for the individuals to interact with each other and have a good time.
Some Interests and Hobbies
Books
If I had to pick one thing of the few things I could do all my life, it would be reading (sports comes first). With three fat bookshelves overflowing with books back home, and many more in my handy Kindle, there are actually times when I am happy to see long queues, presenting another opportunity to dive into my latest book. Some of my favorite authors are Adrian Tchaikovsky, Ted Chiang, Andy Weir, Michael Crichton.
I also read non-fiction, mostly about space or minds.
I have had the pleasure of leading the Making Minds reading group for 3+ years at the University of Alberta.
Connect with me on Goodreads!
Space
Space has fascinated me since I was a kid. Over the past few years, my go-to sci-fi subgenre has been first contact and inter-galactic travel. But my interest in space has had a massive resurgence thanks to Kerbal Space Program and Everyday Astronaut. Instead of core AI, I might want to start have started a career in AIxSpace!
Formula 1
There is hardly anything as spectacular as this confluence of science and engineering which gives the world these lean, mean, and beautiful machines, with some of fittest athletes on the planet battling fearlessly at speeds excessive of 300 kmph over 20+ challenging tracks all over the world. Current favorite track: Spa Francorchamps Circuit Gilles-Villeneuve, Team: Forza Ferrari forever! (sigh)
How things work
I am marvelled by the incredible displays of science and engineering all around us, so I spend inordinate amounts of time looking up how things work, like Sterling Engines, Fluorescence, Lagrange Points, water-based mills, series-parallel power-split hybrid powertrains, etc. Sometimes I learn enough (and have enough time) to try and explain them to the world through my blog.
Contact Me
Recent Posts
Understanding the one of the only ways to simulate space on Earth
April 19, 2026
My journey towards understanding these fascinating points in the universe.
November 29, 2025
How to use KSP's hyper-realistic physics engine as a simulator for RL
November 26, 2023
Linear Algebra yields a hilariously fast method to compute Fibonacci numbers!
May 12, 2023