Shepherd Model Gateway
Rust/Tokio-based model gateway that isolates streaming and the agentic loop from the data-plane API tier. Led the migration of OCI Generative AI’s OpenAI-compatible agent execution onto it — 15% lower p90 latency and 90% less memory.
- Rust
- Tokio
- OpenAI API
- LLM serving
