About Me

I just finished my Ph.D. at the University of Alberta with Richard Sutton, in which I developed simple and practical algorithms from first principles for long-lived artificial decision-making systems.

In particular, I developed algorithms within the reinforcement-learning framework for continuing (non-episodic) problems—in which the agent-environment interaction goes on ad infinitum—with the goal of maximizing the average reward obtained per step. Empirically, the algorithms are easy to implement and use.

I love space! And I want to use my AI expertise in space sciences and technology. I envision a future where artificial systems will have human-like intelligence and adaptability, making space exploration significantly easier and safer for our species. To this end, I am currently working at the National Research Council of Canada (NRC) as a postdoc fellow, where I do RL research for improving space science and technology 🚀 🛰️

You can find my resume here (last updated: May 2024).



Some updates



Ph.D. Dissertation

This dissertation develops simple and practical learning algorithms from first principles for long-lived agents. Formally, the algorithms are developed within the reinforcement learning framework for continuing (non-episodic) problems, in which the agent-environment interaction goes on ad infinitum, with the goal of maximizing the average reward obtained per step.

There are three main contributions:

  1. Foundational one-step tabular learning algorithms for average-reward prediction and control.
  2. Multi-step prediction algorithms for average-reward prediction, some of which are proved to converge with linear function approximation.
  3. Reward centering to improve discounted-reward algorithms.

All of the above contributions are grounded in theory. My experiments show that the performance of the proposed algorithms is robust to the choice of their parameters—making them easy to use.

Defended in March 2024. [Dissertation PDF, Defense Slides, Defense Seminar video]



Publications and Pre-prints

During my Ph.D., I focused on algorithms that can learn continually throughout an agent’s lifetime. In particular, I designed algorithms for non-episodic problems such that an agent can learn to achieve its goals from a single stream of experience (without resets or timeouts).



Talks



Work Experience

National Research Council of Canada

Postdoc Fellow; Sep 2024 – ongoing; Ottawa, Canada

AlbertaSat

Software, Automation, and Testing Team Member; April 2023 – ongoing; Edmonton, Canada

Google Research, Brain Team

Research Scientist Intern; June 2022 – Sep 2022; Toronto, Canada
With Bo Chang and Alexandros Karatzoglou.

Huawei Research

Research Internship; May 2019 – Sep 2019; Edmonton, Canada
With Hengshuai Yao.

Intel Labs

Research Internship; May 2017 – Jul 2017; Bengaluru, India
With Bharat Kaul

Purdue University

Research Internship; May 2016 – Jul 2016; Indiana, USA
With Bruno Ribeiro

Amazon Development Centre

Technical Internship; May 2015 – Jul 2015; Chennai, India
With Sravan Bodapati and Venkatraman Kalyanapasupathy



Master’s Thesis

This thesis was a part of my integrated Bachelor’s + Master’s program in the Dept. of Computer Science and Engineering at the Indian Institute of Technology Madras in Chennai, India, supervised by Professor Balaraman Ravindran. Defended in May 2018.

My goal was to make self-driving cars a reality in my country, India. Towards this end, I modeled it as a multi-agent learning problem in a safety-critical application and:

  1. proposed a risk-averse imitation learning algorithm that had lower tail-end risk w.r.t. the then state-of-the-art,
  2. trialled a curriculum-based learning approach for multi-agent RoboSoccer, and
  3. extended the TORCS simulator to release the first open-source driving simulator that supports multi-agent training — MADRaS (has 100+ stars on Github).

[Thesis PDF, Defense Slides]



Teaching Experience




Community Service


Interests and Hobbies

Ice-hockey

One of the fastest sport in the world, with an exhausting 60 minutes of action (yes, even while watching). The wizardry these athletes pull off while on skates is a delight to watch (shoutout to Connor McDavid! #LetsGoOilers). I am currently learning ice-skating in order to start playing ice-hockey by early 2020! learning to play ice-hockey! playing hockey in a league!

Formula 1

There’s hardly anything as spectacular as this confluence of science and engineering which gives the world these lean, mean, and beautiful machines, with some of fittest athletes on the planet battling fearlessly at speeds excessive of 300 kmph over 20+ challenging tracks all over the world. Current favorite track: Spa Francorchamps, Team: Forza Ferrari forever!

Books

If I had to pick one thing of the few things I could do all my life, it would be reading (sports comes first). With three fat bookshelves overflowing with books back home, and many more in my handy Kindle, there are actually times when I am happy to see long queues, presenting another opportunity to dive into my latest book. Some of my favorite authors are Adrian Tchaikovsky, Ted Chiang, Andy Weir, Michael Crichton. I also read non-fiction, mostly about intelligence. I have had the pleasure of leading the Making Minds reading group for 3+ years at the University of Alberta. Check out my Goodreads page!

Space

I’ve found space fascinating since I was a kid. Over the past few years, my go-to sci-fi subgenre is first contact and inter-galactical travel. But my interest in space has had a massive resurgence thanks to Kerbal Space Program and Everyday Astronaut. Instead of core AI, I might want to start have started a career in Space x AI!

Photography and Traveling

I love visiting and documenting quaint, spectacular places, and digging into the local cuisine. Till I figure out where to showcase some of my favorite pictures, here is my old Flickr account. I also enjoy trekking and hiking into the wilderness. After skydiving, bungee jumping, scuba diving, parasailing, I’m looking forward to hang gliding and cliff jumping!


Contact Me

Email ID


Recent Posts

  • Setting up RL experiments with KSP
    How to use KSP's hyper-realistic physics engine as a simulator for RL
    November 26, 2023

  • Computing Fibonacci numbers using Linear Algebra
    Linear Algebra yields a hilariously fast method to compute Fibonacci numbers!
    May 12, 2023

  • My book reviews
    A post about some books that I read recently
    August 28, 2021

  • On Immortality
    Examples in nature that can live forever, and if we humans really aspire that
    July 17, 2021

  • See All Posts