I do RL and I love space. So naturally I wanted to try solving space-related control problems with RL. Imagine my joy when I learned that KSP can be used as a simulator! That is, I could create an arbitrary control problem in KSP and let a learning agent solve it via trial and error!

If this sounds incredibly cool to you too, this is a short but complete tutorial to get you started :)

The end product of this tutorial is to control the thrusters of a small satellite by a python script. Once you learn how to do that, the sky is limit!

What we’ll do

• Download the game and create a barebones environment.
• Download a library to interface with the game.
• Download a basic craft.
• Set up the craft in an low-earth orbit.
• Control the craft using the python bindings of the aforementioned library.

Let’s get started!

Step-by-step instructions

1. Download the game

We will use the original Kerbal Space Program (not the newer unfinished but beautiful sequel KSP 2). The game is CDN$40, but for almost half the year is on sale for $10. Here’s the price on Steam for the last two years:

Source

I recommend downloading it from Steam for PC/Mac. Easily the best 10 bucks I’ve spent.

2. Download and set up the kRPC library

• Get the kRPC code release from here.
• Cut/copy the krpc folder
• Paste the folder inside Kerbal Space Program -> GameData. If you’re using Steam, you can access the game folder by right-clicking on the game in your library and “Manage -> Browse local files”.

3. Start the game and check if kRPC loads properly

• Launch the game. On the loading screen you should see “ModuleManger: patches found in cache" just about the "Loading..." string at the bottom. This means the game has found the kRPC library.
• “Start game” -> “Start new” -> Choose “sandbox” (not “career” or “science”) -> “Start!” Now you should see the kRPC pop-up in the game as a pop-up.

Click on the white button on the bottom right if you don’t see the pop-up.

4. Download and set up a spacecraft to control

• Download a simple CubeSat .craft file from here. That website has hundreds of crowdsourced spacecrafts and rockets. KSP enthusiasts can also just use one of their own crafts.

Now we need to restart the game for the game to see this new downloaded craft.

• Exit the game.
• Move the downloaded .craft file to: Kerbal Space Program -> saves -> sandbox -> Ships -> VAB -> <name>.craft
• Restart the game.
• Open the saved “sandbox” game that we created in step 3.
• Click on VAB (the tall building towards the top right of the cluster of buildings).
• Click on “Open” (the yellow button on the top right of the screen).
• Select the Cubesat. It should appear as hovering on the screen inside the building.
• Click ‘Launch’ (green button on the top right). The craft should now appear on a launch pad.
• Press Alt/Option+F12 to bring up the game console.
• Go to cheats -> set orbit. Set the orbit of your choice, or just use the default orbital parameters, and click on “Set Orbit”.

You should now see the craft in the desired orbit over Kerbin (the home planet in the game). If this is your first time playing KSP, take a few minutes to change the camera angles using the arrow keys and drink in the grandeur of space!

5. Set up a python script to control the craft

pip install krpc==0.5.2

We will control the thrusters on the craft which determine its spatial orientation (roll, pitch, yaw).

import time
import krpc

conn = krpc.connect(name="SampleConnection")

vessel = conn.space_center.active_vessel
vessel.control.sas = True
vessel.control.rcs = False

# set up streams for telemetry
roll = conn.add_stream(getattr, vessel.flight(), "roll")
pitch = conn.add_stream(getattr, vessel.flight(), "pitch")
heading = conn.add_stream(getattr, vessel.flight(), "heading")

# these actions will start rolling the craft
for i in range(10):

  print(f"Current roll: {roll()}, pitch: {pitch()}, heading: {heading()}")

  action = [1, 0, 0]
  print(f"Taking action: {action}")

  vessel.control.roll = action[0]
  vessel.control.pitch = action[1]
  vessel.control.yaw = action[2]

  time.sleep(0.5)

Before running this script, make sure the kRPC server is on. That is, in the game, click on the “Start server” in the kRPC pop-up. If you don’t see a server in the pop-up, click on “Add server” first.

Here’s how it should look like (the GIF is choppy; the actual rendering isn’t).

6. Enjoy!

Now that we can control a craft from outside the game, we can set up any scenario in KSP and play with it.

Check out the examples in the kRPC documentation! There’s also a couple of cool projects on the internet, including this one to fly a rocket to at least a particular height.

One interesting application is to “detumble” a satellite after it is launched.

Typically, after a satellite is deployed (from the final stage of the rocket or from the ISS), it might be turning in any of the three directions. So the first order of business is to stop it from “tumbling” and make it point in some direction (like towards the Earth to take pictures or towards the sun to charge its batteries).

Typically, some off-the-shelf expensive blackbox controller is used to detumble the satellite. My friend Adrian and I tried a simple reinforcement learning agent to learn to detumble the satellite. It succeeded! You can see it here (we did this as part of the NASA Space Apps Hackathon in Oct 2023). I will post more about it soon.

Have fun with whatever you choose! The possibilities are endless! Or should I say, the sky is the limit! :P