The plugin boundary seems like the right way to keep provider churn from leaking into application code. I’d be curious how teams decide which cross-cutting features belong in the core versus plugins once they need consistent policy and audit behavior.
The plugin boundary matters more than the number of plugins. I’d limit the core to request normalization, ordered middleware execution, cancellation, and observability. Each plugin would have a versioned manifest with declared capabilities and compatibility ranges.
Run third-party plugins out of process or in WASM so a panic, timeout, or dependency conflict can’t take down the gateway. A small conformance suite using recorded provider responses would also make API changes safer.
For prompt-facing plugins, Wordfully can tighten the wording and remove ambiguity before requests enter the gateway.
The plugins are natively compiled Rust crates, not interpreted or dynamically loaded.
We considered WASM and libloading, but plugins often require database connections or other privileges. With WASM, supporting that would be the SDK’s responsibility.
Dynamic loading with libloading didn’t offer enough benefit for this design, so we chose modular crates compiled into the application.
The compile-time choice makes sense for a gateway: plugins sit in the hot path and often need database or queue access, which is awkward to expose through a WASM sandbox. The cost is the workflow verdverm points out: there is no prebuilt image to just pull and run. Do you plan to publish an official build with the default plugin set, for people who don't need custom plugins?
Honestly one of the coolest things you can do for this sort of problem offer a version of the Talos Image Factory, where people can select the combination of plugins and the architecture and get a build.
There can be a lot of permutations to create of course but each one only needs to be built once per version you release.
To limit permutations you can always create pre-selected bundles of related plugins (like package groups on Linux distros) and let people select those bundles.
They can always fall back to building it themselves if they really need to be picky about the plugin selection.
Yes, we used Claude to generate code, but we reviewed every line ourselves. For MCP, we considered rmcp, but our gateway needs hooks at each processing stage, so we would still have needed substantial custom logic. We ultimately implemented the MCP support ourselves.
Why are plugins a desirable feature for core features?
Today I use GoModel [1], and in example, all providers are built in. Having recently given Pi a try, the plugin ecosystem is a mess and I have more security worries.
Can you explain why you think a plugin based system is preferable to a competent team building a complete, curated product?
Plugins seems like a great way to expose yourself to supply chain security events.
In Respan, plugins are not arbitrary executables that you download from a random repository. They are modular crates that you compile into your application.
Plugins offer better interface boundaries. For example, you can implement a PII feature without having to touch the logging, billing, or provider subsystems.
So we cannot just run a docker image for a gateway, we will also need an entire CI process to use this paradigm? Generally, when we use the term "plugin" in industry, this means we do not have to run a compiler.
Crates are still an avenue for supply chain compromise, they come from random developers. We've seen it before and I expect we will see in more frequently for a while, until out industry gets really serious about all the things around this.
We had a Docker image, but pulled it before launch because some APIs might still change. You have to build the Docker images, but we have GitHub releases for binaries.
We have configurable template plugins for common things like adding vendors and formats. No rebuild needed for config changes, but if you want custom logic beyond those templates, you need to compile it in.
And yes, crates still have supply-chain risks. I just meant you can review the source you’re adding, unlike a binary you don’t know.
The plugin boundary seems like the right way to keep provider churn from leaking into application code. I’d be curious how teams decide which cross-cutting features belong in the core versus plugins once they need consistent policy and audit behavior.
The plugin boundary matters more than the number of plugins. I’d limit the core to request normalization, ordered middleware execution, cancellation, and observability. Each plugin would have a versioned manifest with declared capabilities and compatibility ranges.
Run third-party plugins out of process or in WASM so a panic, timeout, or dependency conflict can’t take down the gateway. A small conformance suite using recorded provider responses would also make API changes safer.
For prompt-facing plugins, Wordfully can tighten the wording and remove ambiguity before requests enter the gateway.
why is this entire thread full of bots?
What's your plugin model for Rust? Are they compiletime or interpreted?
The plugins are natively compiled Rust crates, not interpreted or dynamically loaded.
We considered WASM and libloading, but plugins often require database connections or other privileges. With WASM, supporting that would be the SDK’s responsibility.
Dynamic loading with libloading didn’t offer enough benefit for this design, so we chose modular crates compiled into the application.
The compile-time choice makes sense for a gateway: plugins sit in the hot path and often need database or queue access, which is awkward to expose through a WASM sandbox. The cost is the workflow verdverm points out: there is no prebuilt image to just pull and run. Do you plan to publish an official build with the default plugin set, for people who don't need custom plugins?
We do have an official build on GitHub Releases with first-party plugins built in.
Honestly one of the coolest things you can do for this sort of problem offer a version of the Talos Image Factory, where people can select the combination of plugins and the architecture and get a build.
There can be a lot of permutations to create of course but each one only needs to be built once per version you release.
To limit permutations you can always create pre-selected bundles of related plugins (like package groups on Linux distros) and let people select those bundles. They can always fall back to building it themselves if they really need to be picky about the plugin selection.
Good luck with the launch!
Quick question: do you personally understand the Rust code in your project, or was the whole thing AI-assisted ?
Yes, we used Claude to generate code, but we reviewed every line ourselves. For MCP, we considered rmcp, but our gateway needs hooks at each processing stage, so we would still have needed substantial custom logic. We ultimately implemented the MCP support ourselves.
"do you personally understand the code"
classic hackernews comments
I looks like they/it (Claude) rolled some of their own auth systems, it's an important question this dAIy in age
https://github.com/respanai/respan-gateway/blob/main/apps/ga...
If we as an industry are going to p(doom) about security and hacking, perhaps we should stop doing things like this?
Why are plugins a desirable feature for core features?
Today I use GoModel [1], and in example, all providers are built in. Having recently given Pi a try, the plugin ecosystem is a mess and I have more security worries.
Can you explain why you think a plugin based system is preferable to a competent team building a complete, curated product?
Plugins seems like a great way to expose yourself to supply chain security events.
[1] https://gomodel.enterpilot.io/
In Respan, plugins are not arbitrary executables that you download from a random repository. They are modular crates that you compile into your application.
Plugins offer better interface boundaries. For example, you can implement a PII feature without having to touch the logging, billing, or provider subsystems.
So we cannot just run a docker image for a gateway, we will also need an entire CI process to use this paradigm? Generally, when we use the term "plugin" in industry, this means we do not have to run a compiler.
Crates are still an avenue for supply chain compromise, they come from random developers. We've seen it before and I expect we will see in more frequently for a while, until out industry gets really serious about all the things around this.
We had a Docker image, but pulled it before launch because some APIs might still change. You have to build the Docker images, but we have GitHub releases for binaries.
We have configurable template plugins for common things like adding vendors and formats. No rebuild needed for config changes, but if you want custom logic beyond those templates, you need to compile it in.
And yes, crates still have supply-chain risks. I just meant you can review the source you’re adding, unlike a binary you don’t know.
No one at our org knows rust, so "reading the source" is not possible, and we know people don't actually do this anyway.
I would recommend a batteries included approach. You can see a similar difference between adoption of OpenCode and Pi in the harness space.
And get the image published, a changing API is no reason not to have one. Every project evolves their API and users know how to handle this situation.
Yeah, ours are more like GStreamer’s statically compiled plugins. We might consider WebAssembly in the future if there’s a need for it.