About Me
I am a Ph.D. student at the University of Alberta, working with Professor Rich Sutton in the Reinforcement Learning and Artificial Intelligence (RLAI) lab. I want to discover the general principles that underlie goal-seeking behaviour. I study how the mind works and how intelligent systems can learn to perform a myriad of tasks over their lifetime. I believe research progress in these areas will benefit humanity significantly and tangibly, for good.
You can find my resume here.
Recent News
- (Dec 2020) Helped organize the Policy Optimization in RL tutorial at NeurIPS 2020. We made some cool interactive notebooks; links on the website!
- (Oct 2020) Presented our work on ‘Personalized Brain State Targeting via Reinforcement Learning’ at the 3rd Neuromatch conference (more Q/A at the 9:58:41 mark)
- (Sep 2020) Reviewed for AAAI 2021
- (Sep 2020) Started TA-ing for Martha White’s CMPUT397 RL-1 course
- (Aug 2020) Finished organizing the Amii Tea Time Talk series. Check out all the videos here!
Publications and Pre-prints
Currently, I’m interested in learning and planning methods for continuing (non-episodic) problems in RL.
-
Learning and Planning in Average-Reward Markov Decision Processes [PDF]
Yi Wan*, Abhishek Naik*, Richard S. Sutton
Under review. -
Discounted Reinforcement Learning is Not an Optimization Problem [PDF]
Abhishek Naik, Roshan Shariff, Niko Yasui, Richard S. Sutton
In the Optimization Foundations of Reinforcement Learning Workshop, NeurIPS, 2019. -
MADRaS: Multi Agent DRiving Simulator [PDF]
Anirban Santara, Sohan Rudra, Sree Aditya Buridi, Meha Kaushik, Abhishek Naik, Bharat Kaul, Balaraman Ravindran
Under review. -
RAIL: Risk-Averse Imitation Learning [PDF]
Anirban Santara*, Abhishek Naik*, Balaraman Ravindran, Dipankar Das, Dheevatsa Mudigere, Sasikanth Avancha, Bharat Kaul
In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), 2018. -
Identifying User Survival Types via Clustering of Censored Social Network Data [PDF]
S Chandra Mouli, Abhishek Naik, Bruno Ribeiro, Jennifer Neville
CoRR abs/1703.03401, 2017.
Talks
-
Personalized Brain State Targeting via Reinforcement Learning
The 3rd Neuromatch Conference, Oct 2020
[Video (more Q/A here), Slides] -
Learning and Planning in Average-Reward MDPs
Tea Time Talks, RLAI lab and Amii, Aug 2020
[Video, Slides] -
On Intelligence: A Glimpse of the Diversity in Natural Intelligence
-
Figuring Out How the Mind Works: At the Exciting Intersection of RL, Psychology, and Neuroscience
Cognitive Psychology Seminar, Dept. of Psychology, University of Alberta, March 2020
[Video, Slides] -
Discounting — Does It Make Sense?
Tea Time Talks, RLAI lab and Amii, Aug 2019
[Video, Slides]
Master’s Thesis
This thesis was a part of my integrated Bachelor’s + Master’s program in the Dept. of Computer Science and Engineering at the Indian Institute of Technology Madras in Chennai, India, supervised by Professor Balaraman Ravindran. Presented in May 2018.
My goal was to contribute in making self-driving cars a reality in my country, India. I modeled this as a multi-agent learning problem in a safety-critical application and:
- proposed a risk-averse imitation learning algorithm that had lower tail-end risk w.r.t. the then state-of-the-art,
- trialled a curriculum-based learning approach for multi-agent RoboSoccer, and
- extended the TORCS simulator to release the first open-source driving simulator that supports multi-agent training.
Work Experience
Huawei Research
Research Internship; May 2019 – Sep 2019; Edmonton, Canada
With Hengshuai Yao.
- Worked on establishing an appropriate problem formulation for control in continuing tasks with function approximation.
- Surveyed the literature on the average reward problem formulation for MDPs, and its connection with reinforcement learning.
- Some of the work done here will be presented at the NeurIPS 2019 Workshop on Optimization Foundations of Reinforcement Learning (OPTRL 2019).
Intel Labs
Research Internship; May 2017 – Jul 2017; Bengaluru, India
With Bharat Kaul
- Developed a multi-agent version of the TORCS driving simulator (MADRaS) compatible with OpenAI Gym. The repo has 100+ stars!
- Proposed and implemented a novel risk-averse imitation learning framework, achieving upto 89% improvement over the state-of-the-art in terms of tail-end risk at several physics-based control tasks.
- This project was presented at AAMAS 2018.
Purdue University
Research Internship; May 2016 – Jul 2016; Indiana, USA
With Bruno Ribeiro
- Engineered temporal features to design a binary probabilistic classifier to categorise the expected lifespan of new users based on their initial activity.
- Created and curated one of the richest social-media datasets and released it for public use via a technical paper.
Amazon Development Centre
Technical Internship; May 2015 – Jul 2015; Chennai, India
With Sravan Bodapati and Venkatraman Kalyanapasupathy
- Worked on the design and implementation of a machine learning classifier to determine the relevance of a text-block with a bunch of other text-blocks, in a book.
- The project, now in production(!), helps Kindle users to directly start reading a book after downloading it, without having to flip through a lot of irrelevant pages.
Teaching Experience
-
Reinforcement Learning I (CMPUT397)
Sep 2020 - Dec 2020; Dept. of Computing Science, University of Alberta
Helping teach Professor Martha White a class of ~150 undergraduate students. -
Reinforcement Learning II (CMPUT609)
Jan 2020 - Apr 2020; Dept. of Computing Science, University of Alberta
As one of the Teaching Assistants of this course offered by Professor Rich Sutton, I helped create the course content and guided research projects for a class of about 40 graduate students. -
Reinforcement Learning (CS6700)
Jan 2018 - May 2018; Dept. of CSE, IIT Madras
As the Head Teaching Assistant of this course offered by Professor Balaraman Ravindran, I created and evaluated tutorials, programming assignments, and exams for a class of about 90 undergraduates and graduates. -
Principles of Machine Learning (CS4011)
Aug 2017 - Nov 2017; Dept. of CSE, IIT Madras
As one of Teaching Assistants of this course offered by Professor Balaraman Ravindran and Professor Mitesh Khapra, I created and evaluated tutorials, programming assignments, and quizzes for a class of about 90 undergraduates.
-
Reinforcement Learning Specialization on Coursera [Link]
Jan 2019 - Oct 2019; University of Alberta
As one of the ‘Subject Matter Expert’s, I helped develop programming assignments, multiple-choice quizzes, and slides for the four courses that form the RL Specialization, released in late 2019. There have been more than 10k enrollments till now!
Community Service
-
Organizer, Tea Time Talks 2020, Amii and RLAI lab
June 2020 – Aug 2020
Organized and moderated the talks of 40+ speakers over the course of 12 weeks (in a virtual format for the first time). Full playlist here. -
Executive Member, Computer Science Graduate Students’ Association, University of Alberta
Apr 2019 – Apr 2020
Along with representing the interests of the graduate students to the department, I helped organize activities which support their well-being – physically and emotionally, academically and personally – to make University of Alberta a home away from home, especially for international students. -
Volunteer, Centre for Autism Services Alberta
Jan 2019 - present
As a part of the Centre’s Community and Therapeutic program, I help organize recreational activities for individuals in the age range of 5-20 affected with the Autism Spectrum Disorder. We try to create a fun and supportive atmosphere for the individuals to interact with each other and have a good time.
Interests and Hobbies
Formula 1
There’s hardly anything as spectacular as this confluence of science and engineering which gives the world these lean, mean, and beautiful machines, with some of fittest athletes on the planet battling fearlessly at speeds excessive of 300 kmph over 20+ challenging tracks all over the world. Forza Ferrari!
Ice-hockey
One of the fastest sport in the world, with an exhausting 60 minutes of action (yes, even while watching). The wizardry these athletes pull off while on skates is a delight to watch (shoutout to Connor McDavid! #LetsGoOilers). I am currently learning ice-skating in order to start playing ice-hockey by early 2020! learning to play ice-hockey!
Football
I love watching and playing the beautiful game. Come rain or shine, my heart beats for Real Madrid. I also root for Liverpool, Juventus, and Portugal. My idol is, you guessed it, Cristiano Ronaldo — one of the greatest footballers of all time.
Books
If I had to pick one thing of the few things I could do all my life, it would be reading (sports comes first). With three fat bookshelves overflowing with books back home, and many more in my handy Kindle, there are actually times when I am happy to see long queues, presenting another opportunity to dive into my latest book.
Music
During the pandemic, I have started learning to play the piano. I hope to play my favourite piano piece Nuvole Bianche by the end of 2020!
Photography and Traveling
I love visiting and documenting quaint, spectacular places; meeting local people; digging into the native cuisine. You can find some of my photos here. I also love trekking and hiking into the wilderness. After skydiving, scuba diving, and parasailing, I’m looking forward to hang gliding and bungee jumping!
Contact Me
Recent Posts
Interesting research-relevant lessons from Peter Thiel's notes on startups and building the future
December 28, 2020
Time travel implies no free will, and that P equals NP?
April 30, 2020
A parrot, with a brain the size of a walnut, showed cognitive and linguistic skills matching those of chimps and 5-year old human toddlers.
January 15, 2020