Looks promising, but what's the signup wall for?
Also, I guess I can't try this until I get an FPV controller/game controller that I can connect to my computer?
Yes, one of the axis on FPV controllers usually is not springing back to neutral position, for throttle. This may seem like a small difference it is important.
Very cool, I'm working on a similar project but using MCP as the flight input layer. Would love to dm and discuss more about how you built it or collaborate.
I was looking into the MCP route too, and found some libraries abstracting mavlink for this use case (there’s at least one white paper documenting failure modes of LLMs trying to issue mavlink commands without an abstraction), but realized that autopilot like PX4 exists. My use case was more about autonomous flight, and it seemed better to just set waypoints and put some guards on other inputs. When paired with QGroundControl plans, all I needed to do for most flight paths was generate or update a .plan file using an LLM and other methodologies. I wasn’t super happy with the QGroundControl -> Gazebo rendering (no tie into real world terrain out of the box), but it did sort of work out of the box without too much effort!
Interesting. I've been building around that MCP abstraction and have had some early success flying in Cosys-Airsim (and Gazebo before that): https://github.com/jakedcmp/droneserver . I am starting to realize I need to break apart the MCP interface from all the other pieces of the stack for cleaner architecture but thats pending work. My flow goes like this: LLM -> MCP tools -> droneserver -> MAVSDK/MAVLink -> PX4/Ardupilot -> Cosys-Airsim Software in the Loop testing. What is novel for me is not having to learn how to fly a drone and bringing that capability into already existing technologies like PX4 autopilot. I have been attempting to code my own mission planning so I will check out QGroundControl as that might already be a solved problem and not worth building from scratch. I have also built the foundation of video streaming back from the drone so I can run video/image perception. Once I get perception working I am hoping I can build intent level autonomy where images are analyzed according to high level mission plan and potentially re-task the drone based on that. For example, the user issues simple command to fly around the property and scan for broken parts of the fence. During flight if an image of a broken fence is perceived, then the drone stops its flight and goes closer to capture additional imaging/video and document a gps location. Still hacking things together towards a real demo so the code probably wont port over well but idk. To anyone in this thread who wants to discuss further or collaborate let me know, it seems we are all working in a similar domain but from slightly different angles. Exciting area to build, I know there is big demand for solutions in this space over the next decade.
Cool project man! It's well designed and well made also can you explain how did you arrange all the required libraries and stuff before beginning the work?
I am a bit scared of drones. They have had a huge impact.
We could see this in Ukraine versus Russia; and more recently
USA versus Iran. I understand that technology won't change now,
but it is a "disruptive technology" indeed. It's like this is
the real skynet from Terminator. Soon AI controlled. I don't
like this really. (AI is also disruptive of course.)
It's sad to see drones being used in wars! But maybe the way to combat that is to power everyone with it, then nobody dare to step out to do anything sketchy with it like nuclear weapons.
Looks promising, but what's the signup wall for? Also, I guess I can't try this until I get an FPV controller/game controller that I can connect to my computer?
> Looks promising, but what's the signup wall for?
Major turn off for me as well. Instant No.
Is an FPV controller any different than a regular video game controller? I interested in how hard it is to fly a drone even in a sim environment.
The left stick is generally used for throttle and is not centered with springs but rather adjustable and holds its position in the Y-axis direction.
You should be able to use the controller calibrator to calibrate a regular video game controller and use it.
Only use the upper half of the left stick per the spring issue folks mentioned above.
Yes, one of the axis on FPV controllers usually is not springing back to neutral position, for throttle. This may seem like a small difference it is important.
You can also use your keyboard or the virtual joystick if you are on a touch screen!
Looks cool! Feel free to submit this to https://hnarcade.com :)
Thank you mate!
Very cool, I'm working on a similar project but using MCP as the flight input layer. Would love to dm and discuss more about how you built it or collaborate.
I was looking into the MCP route too, and found some libraries abstracting mavlink for this use case (there’s at least one white paper documenting failure modes of LLMs trying to issue mavlink commands without an abstraction), but realized that autopilot like PX4 exists. My use case was more about autonomous flight, and it seemed better to just set waypoints and put some guards on other inputs. When paired with QGroundControl plans, all I needed to do for most flight paths was generate or update a .plan file using an LLM and other methodologies. I wasn’t super happy with the QGroundControl -> Gazebo rendering (no tie into real world terrain out of the box), but it did sort of work out of the box without too much effort!
Interesting. I've been building around that MCP abstraction and have had some early success flying in Cosys-Airsim (and Gazebo before that): https://github.com/jakedcmp/droneserver . I am starting to realize I need to break apart the MCP interface from all the other pieces of the stack for cleaner architecture but thats pending work. My flow goes like this: LLM -> MCP tools -> droneserver -> MAVSDK/MAVLink -> PX4/Ardupilot -> Cosys-Airsim Software in the Loop testing. What is novel for me is not having to learn how to fly a drone and bringing that capability into already existing technologies like PX4 autopilot. I have been attempting to code my own mission planning so I will check out QGroundControl as that might already be a solved problem and not worth building from scratch. I have also built the foundation of video streaming back from the drone so I can run video/image perception. Once I get perception working I am hoping I can build intent level autonomy where images are analyzed according to high level mission plan and potentially re-task the drone based on that. For example, the user issues simple command to fly around the property and scan for broken parts of the fence. During flight if an image of a broken fence is perceived, then the drone stops its flight and goes closer to capture additional imaging/video and document a gps location. Still hacking things together towards a real demo so the code probably wont port over well but idk. To anyone in this thread who wants to discuss further or collaborate let me know, it seems we are all working in a similar domain but from slightly different angles. Exciting area to build, I know there is big demand for solutions in this space over the next decade.
Cool project man! It's well designed and well made also can you explain how did you arrange all the required libraries and stuff before beginning the work?
Thank you mate! It's a iterative process over quite a few years as a side project. Things mostly come by one by one.
Shitty dark patterns
I am a bit scared of drones. They have had a huge impact.
We could see this in Ukraine versus Russia; and more recently USA versus Iran. I understand that technology won't change now, but it is a "disruptive technology" indeed. It's like this is the real skynet from Terminator. Soon AI controlled. I don't like this really. (AI is also disruptive of course.)
It's sad to see drones being used in wars! But maybe the way to combat that is to power everyone with it, then nobody dare to step out to do anything sketchy with it like nuclear weapons.