Routing every AI feature through a single self-hosted open LLM is an architectural decision, not merely a cost saving.
Why it matters
- Privacy. No user data leaves the machine.
- Predictable cost. No per-token bill that scales with success.
- Offline capability. The workspace stays useful without a connection.
The inference gateway
A circuit breaker in front of the AI layer detects failures and degrades to an explicit "AI unavailable" state — never silently falling through to an external provider.
if breaker.is_open():
return AIUnavailable(reason="inference gateway tripped")
The point is not the fallback. The point is that failure is visible.