Twenty-one seconds to get everything on one screen
I wanted my scattered social stats on one screen. The four Instagram accounts reuse insights-collection code I had already written, and Threads gets a live status view. Access goes straight to a tailnet IP with no proxy in front. It is one Python file served on port 8892.
The catch was the cold load: collecting everything from scratch took about 21 seconds. It walks several accounts and calls external APIs, so there was not much my own code could do to make that shorter.
So the question changed from how to make collection faster to something else. When a request hits your dashboard, does it start collecting data at that moment, or does it only show what was already collected?
The trap is not the 21 seconds, it is who waits for them
The simplest version receives a request, collects, renders, and responds. Whoever opens the browser waits the full 21 seconds.
Adding a lock to prevent concurrent collection, which is the usual move, makes it worse. If the whole collection sits inside the lock, a second request waits at the lock until the first collection finishes. If requests that only read the cache also take that same lock, reads block during every refresh too. In the end, every request for one screen is tied to the slowest external call.
Since the collection time belongs to the external APIs, there was no cheap fix here. The only option left was to move that time off the request path.
What would you do: spend time making collection faster first, or change the structure so nobody waits for it?
The structure I switched to
The commit records four rules.
- Collection runs outside the lock. Nothing is locked during the 21-second job. The lock is held only for the short moment when finished results go into the cache.
- Requests read only the cache and never block. The request handler does not call the collector. It returns whatever is in the cache right now.
- An empty cache returns a loading page immediately. Even right after startup, before anything has been collected, a request does not hang for 21 seconds.
- A warmer thread refreshes in the background. Refreshing is the job of a separate thread, not of requests.
The cold load still takes about 21 seconds, but that time no longer adds to any response. That is what the commit message means by making it have no impact. Nothing got faster. Nobody waits anymore.
Self-check
- Does your request handler call a collector that hits external APIs directly?
- Is there a lock wrapped around the whole slow job, and do requests that only read the cache wait on that lock too?
- When the cache is empty, does a request wait for collection to finish, or does it respond right away?
The honest part
I did not measure or record response times after the change. The 21-second figure is a rough number for one cold load, and I do not know how much it varies with account count or API conditions. There are clear costs. The first visitor right after startup sees a loading page instead of data, and during a refresh the screen shows the previously collected data. For a dashboard only I look at, I decided that was fine. On a screen where freshness matters, you might not reach the same conclusion.