llm observability starts with a gateway in front of every model call, because that’s the only place you can see the whole picture without instrumenting each script. I run LiteLLM as its own container in my homelab, and the request log behind it has changed how I think about my automation’s model usage.
What a gateway actually gives you
Before the gateway, my automation called models directly. Each script had its own API key and its own error handling. When something failed, I had to grep through whatever logging that particular script happened to do. When I wanted to switch from one model to another, I had to edit a config file in each project.
Putting LiteLLM in front of everything changes that. Now every call goes through one endpoint. The gateway logs every request with the model name, the token counts, and the cost. That’s the observability part: one log file that shows me exactly what my automation is spending on model calls, per call, per model, per day.
It also gives me one place to swap models. If I decide to move from GPT-4 to Claude or to a local model, I change it in the gateway config. The scripts don’t care. They just hit the same URL with the same API key. That’s saved me from editing a dozen different automation scripts.
Retries are another thing the gateway handles. I can configure it to retry a failed call a few times before giving up. That’s useful for transient errors, like a model API returning a 503. My scripts don’t need to implement their own retry logic anymore.
What the log finally told me
The request log is the part I didn’t expect to be so useful. I knew I was making a lot of model calls, but I didn’t know which ones were actually costing money. The log showed me that a few of my automation scripts were making far more calls than I thought. One script that summarizes RSS feeds was calling the model for every single feed item, even the ones that were just a title and a link. That was a waste.
Another thing the log showed me: token counts. Some of my prompts were way longer than they needed to be. I was sending entire web pages to the model when I only needed a paragraph. The log made that obvious, because I could see the input token count per call.
Cost attribution is the big one. Before the gateway, I had no idea how much I was spending on model calls. The API bills were just a total. Now I can see, per script, per day, what the cost is. That’s the kind of observability that changes behavior. I cut my model spending by more than half just by seeing where the money was going.
What a gateway does not give you
A gateway is not a full observability platform. It doesn’t give you tracing across services, or a fancy dashboard with charts. It doesn’t alert you when something goes wrong. It’s just a proxy with a log. That’s enough for a homelab, but if you’re running a production system, you might want more.
Hosted observability platforms like LangSmith or Helicone offer a lot more: tracing, evaluation, prompt management. But I’m not using any of those here. I looked at their documentation, and they’re overkill for what I need. A gateway and its log are enough.
Another thing a gateway doesn’t do: it doesn’t make your prompts better. It just shows you what you’re sending. You still have to figure out how to improve them. The log is a mirror, not a coach.
Setting up the gateway
I run LiteLLM in a container on my Proxmox host. The setup is straightforward. I have a config file that lists the models I want to expose, with their API keys and parameters. The gateway reads that file and starts listening on a port. My automation scripts point to that port instead of the model APIs directly.
The log is just a file that LiteLLM writes to. I can tail it to see requests in real time, or grep it for specific patterns. It’s plain text, so I can use standard Unix tools on it. No special query language needed.
One thing I learned: the log format is not documented as well as I’d like. I had to experiment a bit to figure out what each field means. But once I did, it was easy to parse with a simple script if I wanted to.
Why this matters for a homelab
In a homelab, you’re not running a business on these model calls. But you’re still spending money, and you’re still trying to keep things running. A gateway gives you the visibility to see what’s actually happening with your automation’s model usage. It’s the difference between guessing and knowing.
I’ve written before about self-hosted automation software and the costs that only appear in month three. Model costs are one of those hidden costs. The gateway makes them visible from day one.
And when something goes wrong, the log is the first place I look. Just like checking logs and watchdog first when a Proxmox hypervisor restarts itself, the gateway log is where I start when a model call fails. It usually tells me what happened.
If you’d rather pay for this
If you don’t want to run your own gateway, there are hosted options. LangSmith is one, from the people behind LangChain. It’s billed per seat per month, with a free tier that’s limited. Helicone is another, billed per request or per month depending on the plan. These give you a lot more than a gateway: tracing, evaluation, prompt management, and a nice UI.
For a homelab, I don’t think they’re worth the money. The gateway I run is free and open source, and it gives me the observability I need: per-call cost attribution, one place to swap models, and retries. If you’re running a production system with multiple developers, the hosted platforms might be worth it. But for one person automating things at home, a self-hosted gateway is plenty.
