The reflexive choice in modern Rust services is #[tokio::main], and the reflexive architecture is “every I/O call is async, every handler is a future, every blocking call is wrapped in spawn_blocking and apologized for”; on a system monitoring daemon that polls /proc and /sys and pushes a number to clients over a loopback socket, every one of those reflex choices is overhead you do not need, and the result is a hot path that does the same work in roughly a ninth of the time with a tenth of the resident footprint, because the runtime was never the right tool for the job in the first place.
What the workload actually is
LinSight is a metric daemon with a sampler loop and a request loop, and the sampler loop is the part that does the work: at a fixed cadence, usually one to five seconds, it reads /proc/stat, /proc/meminfo, /proc/diskstats, /sys/class/thermal/thermal_zone*/temp, the network counters from /proc/net/dev, and a handful of in-process sources, then computes deltas and pushes the result into a ring buffer that the request loop serves back to clients; the work is strictly periodic, with no event arrivals between ticks, no upstream notifications to react to, and no I/O that is slower than a microsecond except the rare disk-stat read that touches a slow block device and is the only place a synchronous wait ever happens.
The naive Rust implementation of this, using Tokio and the sysinfo crate and a Notify channel between sampler and server, runs about 12 megabytes of resident memory on Linux and pushes the same metric in roughly 700 microseconds per request at p99, and the synchronous version sits at 1.4 megabytes and answers the same request in roughly 80 microseconds at p99, which is a 9x speedup on a workload where latency budget matters more than throughput budget because the daemon is local to the box and one slow sample shows up as visible jitter on every graph it feeds; the size difference is mostly Tokio’s own working set, the timer wheel and task queue and per-task allocations, all of which exist to multiplex I/O that the daemon never asked to multiplex, and the reduction in resident memory from dropping the executor is real while the reduction in on-disk binary size is much smaller and not what I measured.
The shape of the synchronous code
The sampler is a std::thread running a plain loop, holding a crossbeam::queue::ArrayQueue of precomputed snapshots, and reading at a fixed period through a std::time::Instant::now() plus a std::thread::sleep of the remaining budget; there is no scheduler tick, no reactor wakeup, no work-stealing across threads, and the cost of a tick is exactly the cost of the read, because that is the only thing happening on the thread for the entire duration of a tick. Sources register themselves through a MetricSource trait with one method that returns a Vec<Sample>, and the runtime iterates the registered sources in priority order, computes the deltas against the previous tick’s snapshot, writes the new snapshot into the ring, and goes back to sleep; one thread, one loop, no allocation per tick because everything is preallocated up front in Vec::with_capacity(64) and reused.
use crossbeam::queue::{ArrayQueue, PopError};
use std::sync::Arc;
use std::time::{Duration, Instant};
pub trait MetricSource {
fn name(&self) -> &'static str;
fn collect(&mut self, out: &mut Vec<Sample>);
}
pub struct Sampler {
sources: Vec<Box<dyn MetricSource>>,
ring: Arc<ArrayQueue<Arc<Snapshot>>>,
period: Duration,
}
impl Sampler {
fn run(mut self) {
let mut out = Vec::with_capacity(64);
let mut next_tick = Instant::now();
loop {
next_tick += self.period;
for src in &mut self.sources {
src.collect(&mut out);
}
let snap = Arc::new(Snapshot::from(&out));
// Evict oldest entry to make room; ArrayQueue drops the value
// we attempt to push when full, so we must pop first.
while self.ring.push(Arc::clone(&snap)).is_err() {
let _ = self.ring.pop();
}
out.clear();
let now = Instant::now();
if next_tick > now {
std::thread::sleep(next_tick - now);
} else {
next_tick = now;
}
}
}
}
The client side is a Unix listener on a path under /run/linsight/, accepting one connection at a time, reading a postcard-framed request, looking up the requested metric by integer id in a static table built at startup, and pushing the corresponding Sample out the same socket. There is no request handler future, no async cancellation to defend against, and no executor shutting down with in-flight requests that need to drain; when the client hangs up, the server notices on the next read, drops the connection, and moves on. The ArrayQueue is multi-producer multi-consumer under the hood, but in practice only the sampler thread pushes and only the request loop pops, so the access pattern is a structural single-producer single-consumer one even when the underlying implementation is more general.
Why this is sane and not lazy
The conventional wisdom is that “blocking I/O on a server is wrong,” and the conventional wisdom is right for the workload it was built for: thousands of concurrent connections, mostly idle, where the cost of a thread per connection is wasteful and the cost of polling through a reactor is fine, because a reactor exists to solve the mostly-idle problem. LinSight is not mostly idle. Its sampler thread is fully busy every tick, and its listener accepts one local connection at a time and serves the request in tens of microseconds, which is faster than the time it would take for the kernel to wake a reactor on the same machine, let alone schedule the corresponding task; the synchronization cost of “ring buffer push from sampler, ring buffer pop from server” involves a couple of atomic operations per side (fetch-add and compare-and-swap on head and tail), and that is the entire coordination cost for the system, instead of a Tokio reactor that schedules MPSC channels, task wakes, and timer drift across threads that share an L3 cache on the same die.
The single place where blocking would be wrong is the disk-stat read, which can take a few milliseconds on a stalled block device; the synchronous sampler does block on it, and the fix is to read through a dedicated worker thread with std::thread::spawn when the metric is on a rotation-bound device, with the result passed back through a second ArrayQueue or signaled through a std::sync::Condvar, and the same pattern applies for any source that might block rather than reaching for a Tokio primitive that would require the executor to exist.
What you give up
The honest list of tradeoffs is short and worth stating: you give up the ability to multiplex thousands of clients without a thread per connection, but local metric consumers number in the dozens, not the thousands, and the Unix socket limit is fine; you give up cancellation tokens, structured concurrency, and the rest of the ergonomic surface Tokio offers for I/O orchestration, and in exchange you give up roughly 90% of the resident working set, 9x of the request latency, and the entire category of “the executor is in a weird state and the daemon isn’t responding to SIGTERM” bugs that show up in production Tokio services; you do not give up testability, since the sampler is just a function of inputs over time and the request loop is just a function of the ring and the request bytes, and both can be driven synchronously in unit tests with no runtime mocking required.
The daemon is roughly 1,400 lines of Rust on top of crossbeam, postcard, and the standard library, and the entire thing compiles in about ten seconds on a small cloud box and starts in under five milliseconds, which is the time it takes to allocate the ring buffer and bind the socket, because there is no executor to warm up and no async traits to wire through the type system; if the workload were different, I would reach for Tokio, and on a network-facing API server I do reach for Tokio, but on a metric daemon whose clients are loopback and whose periodic sampler is the real work, synchronous Rust is the boring choice, and the boring choice is the one that delivers the latency budget without the operational surprises.