PERFORMANCE: Cache file handles in posix_storage to eliminate per-block fopen/fclose

[gubatron]

Component(s): src/posix_storage.cpp, src/storage_utils.cpp, include/libtorrent/aux_/posix_storage.hpp

Severity: critical

Category: I/O Labels: perf, syscall, linux|macos

Personal motivation: improve disk I/O performance in macos with jlibtorrent 2.0, we had to use the posix_storage there because of unmanageable issues with custom signal handlers with JNI/Java Runtime Environment.

Problem

posix_storage::read/write (src/posix_storage.cpp:165-285) invoke readwrite() (src/storage_utils.cpp:60-158), which opens and closes FILE* via open_file() for every buffer chunk.

Under high-throughput swarms this means thousands of fopen/seek/fread/fclose syscalls per piece, saturating the kernel and trashing the page cache.

Evidence/Heuristics: repeated loop-triggered fopen/fread in readwrite(), flamegraphs show reopen overhead, reports from production indicating double-digit % CPU in fopen on Linux.

Proposed Solution

Introduce a per-storage small LRU of file_pointer objects (or reuse aux::file_pool) keyed by file_index_t so reads/writes reuse descriptors across blocks; batch sequential read in async_hash via preadv/readv; ensure handles respect TORRENT_USE_ASSERTS.

Fewer syscalls ⇒ lower latency and better page cache hit rate.

Risks/Tradeoffs: must guard against too many open FDs (respect settings_pack::file_pool_size); ensure thread safety.

Implementation Notes

Touch points: In the RC_2_0 branch. posix_storage::read/write (src/posix_storage.cpp:165-285), aux::readwrite (src/storage_utils.cpp:60-156), posix_storage::open_file (src/posix_storage.cpp:440-523).

Store cached handles in posix_storage members (maybe std::vector<file_pointer>), flush on remove_torrent/abort.

API impact: none

Compatibility: Works for Linux/macOS POSIX backends; no change for Windows.

Bench/Test Plan

Microbench (Google Benchmark):

Inputs: 16 MiB piece, block size 16 KiB, vary sequential block counts.
Metric(s): ns/op, syscalls/s
Harness: google-benchmark with warmup/iterations.

#include <benchmark/benchmark.h>
// #include <libtorrent/posix_storage.hpp>

static void BM_PosixRead(benchmark::State& state) {
  for (auto _ : state) {
    // invoke posix_storage::read on sequential blocks
    benchmark::DoNotOptimize(state.iterations());
  }
} 
BENCHMARK(BM_PosixRead);
BENCHMARK_MAIN();

Macro/probe: run examples/client_test doing sustained download, capture perf record/syscalls before/after.

Success criteria: ≥30% fewer syscalls per piece and lower wall time.

Telemetry & Repro

Add TORRENT_PERF_TRACE counters around read/write batched spans via steady_clock.

CLI:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DTORRENT_PERF_TRACE=ON
cmake --build build -j
./build/examples/client_test --buffers 1024 --download-rate 0

References

src/posix_storage.cpp (lines 165-285, 440-523)
src/storage_utils.cpp (lines 60-156)