pread-disk-io #7013

arvidn · 2022-08-14T19:01:06Z

add a multi-threaded, pread()-based, disk I/O backend (pread_disk_io)

glassez · 2022-08-15T11:12:04Z

add a multi-threaded, pread()-based, disk I/O backend (pread_disk_io)

Is it something similar to libtorrent 1.2 disk I/O?

arvidn · 2022-08-15T11:32:26Z

yes, it is similar. This doesn't cache blocks in memory though. It just has a store-buffer while waiting for blocks to be written to disk.

xavier2k6 · 2022-08-15T12:48:03Z

Failing to build with qBittorrent master:
https://github.com/xavier2k6/qBittorrent/actions/runs/2860408368

glassez · 2022-08-15T13:29:47Z

Failing to build with qBittorrent master: https://github.com/xavier2k6/qBittorrent/actions/runs/2860408368

We never previously care about supporting libtorrent master. I'll try to deal with it ASAP since I really interested with this new disk I/O type.

xavier2k6 · 2022-08-15T13:32:34Z

@glassez feel free to make any changes/pushes to my branch if needed for testing.

arvidn · 2022-08-15T13:58:51Z

I thought pausing a session was unusual. Maybe I should add that back.

@xavier2k6 is that a build with deprecated functions disabled?

glassez · 2022-08-15T14:04:07Z

I thought pausing a session was unusual.

Do you mean create session in paused state? Of course it is useful! It was even added at our request, if memory serves.

xavier2k6 · 2022-08-15T14:22:43Z

is that a build with deprecated functions disabled?

-Ddeprecated-functions=OFF

https://github.com/xavier2k6/qBittorrent/blob/6fdf4f1e2d741b7854999cefa75bf2fbbee7dae2/.github/workflows/ci_windows.yaml#L101

arvidn · 2022-08-15T14:24:01Z

Do you mean create session in paused state? Of course it is useful! It was even added at our request, if memory serves.

I think this feature got lost in a merge. I'll fix that in master

userdocs · 2022-08-15T22:28:45Z

I made some nox linux multiarch builds to test using the pr branch.

https://github.com/userdocs/qbt_static_test/releases/tag/release-4.4.3.1_pread-disk-io
https://github.com/userdocs/qbt_static_test/releases/tag/master_pread-disk-io

glassez · 2022-08-16T08:01:15Z

We never previously care about supporting libtorrent master. I'll try to deal with it ASAP since I really interested with this new disk I/O type.

It seems that it will not be possible to keep the qBittorrent code compatible with both master and RC_2_0 branch at the same time due to the fact that master declares the same version as RC_2_0. @arvidn, could you fix that?

glassez · 2022-08-16T17:43:30Z

@glassez feel free to make any changes/pushes to my branch if needed for testing.

I cannot push directly to this branch so I created PR.

I tested it a little with checking a large v2 torrent (30+ GB), but I didn't finish it, because I had a slow computer and little time. BUT nevertheless, I noticed significant hangups of UI during checking, so I believe that we still have some other problems besides the checking speed itself. I'll deal with related issues later. As for the checking speed itself, you should compare it on the same torrent, using both libtorrent-1.2 and libtorrent-2.0 with different disk I/O types. The RAM limit should be set to the highest possible value so as not to be a bottleneck in these tests.

james58899 · 2022-08-17T03:16:54Z

I noticed this PR always used 16KB block size for reads, which is very inefficient, especially on HDD.

I think the block size too small is the root cause for the difference in performance between 1.2 and 2.0. In 1.2 with disk cache enable average size is >256KB, 2.0 mmap average size ~128KB and pread only 16KB. That caused the number of IOs to increase by more than ten times and kill disk performance.

brvphoenix · 2022-08-17T03:27:36Z

@glassez feel free to make any changes/pushes to my branch if needed for testing.

I cannot push directly to this branch so I created PR.

I tested it a little with checking a large v2 torrent (30+ GB), but I didn't finish it, because I had a slow computer and little time. BUT nevertheless, I noticed significant hangups of UI during checking, so I believe that we still have some other problems besides the checking speed itself. I'll deal with related issues later. As for the checking speed itself, you should compare it on the same torrent, using both libtorrent-1.2 and libtorrent-2.0 with different disk I/O types. The RAM limit should be set to the highest possible value so as not to be a bottleneck in these tests.

glassez/qBittorrent@97e4855#diff-0efd426259b58f47b1f848560cf9c51a92268084060cebf91c7ef22f092da1f7R1410

Doesn't this line should be sessionParams.disk_io_constructor = customPReadDiskIOConstructor;?

glassez · 2022-08-17T03:59:42Z

glassez/qBittorrent@97e4855#diff-0efd426259b58f47b1f848560cf9c51a92268084060cebf91c7ef22f092da1f7R1410

Doesn't this line should be sessionParams.disk_io_constructor = customPReadDiskIOConstructor;?

Damn it! You're right. I forgot to change this in a hurry after copy-pasting.
Fixed.

glassez · 2022-08-17T08:49:12Z

I believe that increasing the checking speed is not the only expected advantage of this disk I/O type. So we should test other cases where MMap based I/O messes up, for example, using external and especially network drives to download torrents, problems related to I/O on macOS.

arvidn · 2022-08-17T08:53:35Z

in parallel to this I'm working on modifying the mmap_disk_io in 2.0 to use pwrite() for writes and mmap for reads. I think that could give use the best of both worlds (as long as the OS is sophisticated enough I suppose).

glassez · 2022-08-17T11:15:39Z

I'm working on modifying the mmap_disk_io in 2.0 to use pwrite() for writes and mmap for reads.

IIRC, another problem of current MMap disk I/O implementation is that you map the entire file, which can still be a stumbling block, even if memory mapping will be used only for reading files.

xavier2k6 · 2022-08-17T11:30:00Z

@glassez I merged your PR in my branch, build is available below:
https://github.com/xavier2k6/qBittorrent/suites/7850730272/artifacts/332517996

Is there any Large V2 torrent publicly available that we can all use for testing purposes for consistency?

BTW, does OpenSSL have anything to do with the hashing? (official qBittorrent use 1.1.1 where as qBittorrent CI (Windows) builds use 3.0.5)

glassez · 2022-08-17T11:47:59Z

Is there any Large V2 torrent publicly available that we can all use for testing purposes for consistency?

I used one specially created to test another problem with large torrents. But they must exist in reality, otherwise where would that problem come from?

Another thing is since we are interested in comparing libtorrent 1.2 vs 2.0 in the checking speed problem, we will have to use a pure v1 torrent in testing.

glassez · 2022-08-18T11:44:01Z

It seems that it will not be possible to keep the qBittorrent code compatible with both master and RC_2_0 branch at the same time due to the fact that master declares the same version as RC_2_0. @arvidn, could you fix that?

#7018 will do (if there are no pitfalls).

ghost · 2022-08-25T05:21:13Z

I think the hangups during checking are due to high page faults due to working set limit. It should be disabled if one isn’t using mmap I/O.

tristanleboss · 2022-08-25T06:35:31Z

@xavier2k6 There is a torrent https://the-eye.eu/public/Random/torrents/oldversion.com_Sept2019.torrent with the dump of all stuffs from oldversion.com. The torrent file itself is 5.5mb and there is 30k files inside along with as many folders (each version is in its own folder) for a gross total of 472.18gb. Those stuffs are just demo, freeware or shareware... no cracked softwares. The file sizes range from 1mb to hundreds of megabytes. It works flawlessly with qBt 4.4.4 RC_1_2.

tristanleboss · 2022-08-25T16:01:12Z

@summerqb I have it in my qBt 4.4.4 RC_1_2 and it works just fine. But RC_2_0 was freezing and never starting. That's why it's probably a really good test torrent!

ghost · 2022-08-25T16:02:54Z

Tested all 3 branches and RC_1_2 still beats the others.
RC_1_2 is 33% faster than this one and 50% faster than RC_2_0.

glassez · 2025-06-28T09:32:24Z

@arvidn
How far is this job from completion?

Aleksman4o · 2025-07-10T23:43:03Z

I've successfully build it on arch linux. Is it enough safe for testing?

arvidn · 2025-07-10T23:50:03Z

There are two problems I've been trying to get to the bottom of.

It seems some write jobs may post more than one completion. Causing the file progress tracking to increment the download counter more than once for the same piece.
It also seems write jobs can sometimes get stuck in the cache, not being flushed and cleared as they're supposed to.

Aleksman4o · 2025-11-29T13:53:38Z

Is there some progress after 4 months? )

Aleksman4o · 2025-11-30T16:22:35Z

I'm trying to help and i know nothing about C++, don't scold me for using AI, please ))

src/disk_cache.cpp: disk_cache::insert() never checks cached_piece_entry::flushing. While flush_piece_impl() (around lines 330-440) drops m_mutex to write, a new async_write can overwrite cached_block_entry::write_job on the same piece/block. When the flush thread comes back it walks flushed_blocks and posts completions for whatever pointer is now stored, even if that job was never written. The “new” job may then be flushed later as well, so its handler fires twice (and the job written during the flush may be lost). This race also means completions do not correspond to the bytes that actually hit disk.

src/pread_disk_io.cpp (lines 601-673) (async_write) only parks write jobs in the cache; they are never queued to any thread. Cache flushing is driven by try_flush_cache() in thread_fun() (≈1445ff), which only runs when a generic-thread job is dequeued or when m_flush_target is set (buffer pool above the high watermark). If the workload is just writes + hashing and the buffer pool stays below the watermark, no generic job ever wakes and m_flush_target stays nullopt; hash threads don’t trigger flushing unless flush_piece is set. The result is dirty blocks sitting in m_cache indefinitely—no thread ever calls flush_to_disk() for them.

Fix:
Block overwriting in-flight write jobs during flush. Right now flush_piece_impl() drops m_mutex, and insert() can install a new write_job in the same block. When the flush completes, it reads the new pointer and posts completion for the wrong op. Fix: if cached_piece_entry::flushing is set, don’t write into blocks[...]; queue the write in a per-piece pending_writes list instead. After flush_piece_impl() finishes (with the mutex held again), drain pending and insert them normally, updating ready_to_flush/num_jobs etc. That keeps the write_job* being flushed intact and prevents double completions. Changes: add a small pending_writes container to cached_piece_entry; in disk_cache::insert() if flushing is true, push (block_idx, write_job) to pending and return; in flush_piece_impl(), after processing flushed_blocks, insert pending entries using the existing logic.

Ensure dirty blocks get flushed even when there are no other generic jobs and the buffer pool is below the high watermark. After a successful m_cache.insert() in async_write, if m_flush_target is unset, set it to “flush at least something” (e.g. m_cache.size() - 1) and call m_generic_threads.interrupt(). That wakes a thread, which sees m_flush_target in thread_fun and calls try_flush_cache() even without queued jobs. Ready-to-flush pieces still flush fully (the first flush_to_disk branch ignores the target). This prevents dirty blocks from sitting in m_cache forever.

Combined, this removes duplicate completions from the flush/insert race and forces flushing to run without external triggers.

Corrected files (don't know if they are working, but may be you can diff with your files and understand what's going on. there are a few lines of difference)
pread_disk_io.cpp
disk_cache.cpp
disk_cache.hpp.txt *.txt is just for uploading

And thank you for your work :)

Rootax · 2026-01-25T16:46:51Z

So it seems you're pushing it to master ? :) Good news :) I hope perfs will be back to 1.2 level or even better.

Copilot

Pull request overview

This PR introduces a new experimental multi-threaded disk I/O backend called pread_disk_io that uses pread()/pwrite() system calls with an integrated write cache. This is a substantial addition to libtorrent's disk I/O subsystem.

Changes:

Adds a complete new disk I/O backend (pread_disk_io) with caching, multi-threading, and vectorized I/O support
Implements platform-specific optimizations using preadv/pwritev where available
Updates benchmarking tools to test all three disk I/O backends (mmap, posix, pread)

Reviewed changes

Copilot reviewed 51 out of 51 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
src/pread_disk_io.cpp	Core implementation of the multi-threaded pread-based disk I/O backend with job scheduling and cache management
src/pread_storage.cpp	Storage layer implementation handling file operations, part files, and file priority management
src/disk_cache.cpp	Write cache implementation with piece-level caching, hashing coordination, and flushing logic
src/file.cpp	Low-level pwritev_all implementation with platform-specific optimizations
include/libtorrent/pread_disk_io.hpp	Public API for the pread disk I/O constructor
include/libtorrent/aux_/pread_storage.hpp	Storage class interface for pread-based operations
include/libtorrent/aux_/disk_cache.hpp	Disk cache data structures and multi-index container for efficient piece lookup
include/libtorrent/aux_/visit_block_iovecs.hpp	Helper for iterating over block I/O vectors with gap handling
include/libtorrent/config.hpp	Platform detection for preadv/pwritev support across Linux, BSD, macOS, and Windows
src/settings_pack.cpp	Increases default max_queued_disk_bytes from 1MB to 50MB
src/torrent.cpp	Adds debug logging for piece operations and fixes piece completion logic
test/test_disk_cache.cpp	Comprehensive tests for visit_block_iovecs functionality
test/test_transfer.cpp	Integration tests for pread_disk_io with various storage modes
test/test_storage.cpp	Storage-level tests for pread backend including rename, remove, and move operations
tools/run_benchmark.py	Updates benchmark suite to test all three disk I/O backends
CMakeLists.txt, Jamfile, Makefile	Build system updates to include new source and header files

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-25T22:03:38Z

src/pread_storage.cpp

+#endif
+	}
+
+	// TODO: move this o aux_/file.hpp


The comment has a typo: "o aux_/file.hpp" should be "to aux_/file.hpp"

Copilot · 2026-01-25T22:03:38Z

include/libtorrent/aux_/store_buffer.hpp

+			// the mutex. It would require a reference counter on the store
+			// buffer entries and that we potentially erases it after this call.
+			// it would also require the store buffer being able to take over
+			// ownership of the buffer when the owner erases it. Perhase erase()


The comment contains a spelling error: "Perhase" should be "Perhaps"

Suggested change

// ownership of the buffer when the owner erases it. Perhase erase()

// ownership of the buffer when the owner erases it. Perhaps erase()

Copilot · 2026-01-25T22:03:38Z

test/web_seed_suite.cpp

 		{
 			settings_pack pack = settings();
-			pack.set_int(settings_pack::max_queued_disk_bytes, 256 * 1024);
 			pack.set_str(settings_pack::listen_interfaces, test_listen_interface());


The removal of the max_queued_disk_bytes setting appears to be related to the settings_pack.cpp change where the default was increased. However, removing this setting entirely could affect existing code that relied on this explicit configuration. If this is intentional, there should be documentation explaining why it's no longer needed.

Copilot · 2026-01-25T22:03:39Z

src/copy_file.cpp

-#define _XOPEN_SOURCE 600
-#endif
-
 #include "libtorrent/aux_/file_descriptor.hpp"


The removal of _XOPEN_SOURCE definitions could affect portability. This macro is often required on some Unix systems to enable certain POSIX features. While this change might work on modern systems, it could break builds on older or less common platforms. Consider testing on various platforms or documenting why this is safe to remove.

Copilot · 2026-01-25T22:03:39Z

src/disk_cache.cpp

+	if (piece_iter->hashing)
+		return;
+
+	// this piece is done hasing


The comment has a spelling error: "hasing" should be "hashing"

Suggested change

// this piece is done hasing

// this piece is done hashing

Copilot · 2026-01-25T22:03:40Z

test/test_add_torrent.cpp

 	}

-	lt::session_params p;
+	lt::session_params p = settings();


The test_add_torrent function now uses settings() for initialization instead of a default-constructed session_params. While this may be necessary for the pread_disk_io backend, this changes the test behavior and could mask issues with default initialization. Consider whether this change is necessary or if it should be conditionally applied only when testing pread_disk_io.

Copilot · 2026-01-25T22:03:41Z

src/pread_storage.cpp

+
+namespace {
+
+	// TODO: move this o aux_/file.hpp


The comment has a typo: "o aux_/file.hpp" should be "to aux_/file.hpp"

Copilot · 2026-01-25T22:03:41Z

src/settings_pack.cpp

 		SET(allowed_fast_set_size, 5, nullptr),
 		SET(suggest_mode, settings_pack::no_piece_suggestions, nullptr),
-		SET(max_queued_disk_bytes, 1024 * 1024, nullptr),
+		SET(max_queued_disk_bytes, 50 * 1024 * 1024, nullptr),


The default value for max_queued_disk_bytes has been increased from 1MB to 50MB. This is a significant 50x increase that could have substantial memory implications. While this may be appropriate for the new pread_disk_io backend with caching, the change affects all disk I/O backends. Consider whether this should be configurable per backend or if there should be documentation explaining the rationale for this large increase.

Copilot · 2026-01-25T22:03:41Z

include/libtorrent/aux_/disk_cache.hpp

+
+	// when this is true, there is a thread currently hashing blocks and
+	// updating the hash context in "ph". Other threads may not touch "ph",
+	// "hasing_cursor", and may only read "hasing".


The comment has a typo: "hasing" should be "hashing" (appears twice in the comment)

Suggested change

// "hasing_cursor", and may only read "hasing".

// "hashing_cursor", and may only read "hashing".

Copilot · 2026-01-25T22:03:41Z

include/libtorrent/config.hpp

 #define TORRENT_HAVE_MMAP 1
 #endif

+#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,30) && !defined __ANDROID__


The definition of TORRENT_USE_PWRITEV for Android is set to 0, which means the pwritev() system call won't be used on Android even though it was added in Android API level 24 (Android 7.0, 2016). Consider updating this to check for the Android API level and enable pwritev where supported.

Suggested change

#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,30) && !defined __ANDROID__

#if defined __ANDROID__

# if defined __ANDROID_API__ && __ANDROID_API__ >= 24

# define TORRENT_USE_PWRITEV 1

# else

# define TORRENT_USE_PWRITEV 0

# endif

#elif LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,30)

@copilot where is the Android kernel API documented?

copilot doesn't respond I take it?

https://docs.github.com/en/copilot/how-tos/use-copilot-agents/request-a-code-review/use-code-review#providing-feedback-on-copilots-reviews
Looks like you have to give it a thumbs down and tell it to stop hallucinating about android, and there is supposed to be a circle arrow button to make it retry

https://android.googlesource.com/platform/bionic/%2B/00fedf5%5E%21/

arvidn mentioned this pull request Aug 14, 2022

File checking speed for 4.4.0 is much slower than 4.3.9 qbittorrent/qBittorrent#16043

Closed

arvidn force-pushed the pread-disk-io branch from 05638e5 to 6240f30 Compare August 15, 2022 07:25

james58899 mentioned this pull request Aug 24, 2022

Go back to using libtorrent 1.2.x until qBittorrent 5.x qbittorrent/qBittorrent#17545

Closed

This comment was marked as off-topic.

Sign in to view

arvidn force-pushed the pread-disk-io branch from a3080b8 to 3e82f19 Compare June 27, 2024 08:21

arvidn force-pushed the pread-disk-io branch from 51dc3af to 796c953 Compare July 21, 2024 18:32

HanabishiRecca mentioned this pull request Sep 5, 2024

Add "Simple pread/pwrite" disk IO type qbittorrent/qBittorrent#21300

Merged

arvidn force-pushed the pread-disk-io branch from 796c953 to 01753c5 Compare October 6, 2024 09:42

arvidn force-pushed the pread-disk-io branch 3 times, most recently from 47cdb93 to 264c66a Compare October 26, 2024 11:30

arvidn force-pushed the pread-disk-io branch 2 times, most recently from 2a6b2e7 to 4e9d240 Compare November 2, 2024 11:08

userdocs mentioned this pull request Jan 25, 2025

boost-1.87.0.beta1 libtorrent RC_1_2 #7796

Open

arvidn force-pushed the pread-disk-io branch from 4e9d240 to c24093d Compare January 30, 2025 01:31

HanabishiRecca mentioned this pull request Mar 14, 2025

Libtorrent 2.x memory-mapped files and RAM usage #6667

Open

arvidn force-pushed the pread-disk-io branch from c24093d to a8c4638 Compare June 29, 2025 19:30

NLZ mentioned this pull request Sep 27, 2025

qbittorent to use libtorrent 1.2 home-operations/containers#848

Closed

qBittUser mentioned this pull request Nov 8, 2025

libtorrent 2.1.0 - when? #8053

Open

arvidn force-pushed the pread-disk-io branch 2 times, most recently from b01c81a to fa4674d Compare January 25, 2026 16:38

arvidn force-pushed the pread-disk-io branch from fa4674d to 4bffd27 Compare January 25, 2026 17:10

add new multi-threaded disk I/O subsystem using preadv and pwritev

a4a9820

arvidn force-pushed the pread-disk-io branch from 4bffd27 to a4a9820 Compare January 25, 2026 21:49

arvidn requested a review from Copilot January 25, 2026 21:52

arvidn marked this pull request as ready for review January 25, 2026 21:52

Copilot started reviewing on behalf of arvidn January 25, 2026 21:56 View session

Copilot AI reviewed Jan 25, 2026

View reviewed changes

	// ownership of the buffer when the owner erases it. Perhase erase()
	// ownership of the buffer when the owner erases it. Perhaps erase()

	// "hasing_cursor", and may only read "hasing".
	// "hashing_cursor", and may only read "hashing".

-#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,30) && !defined __ANDROID__
+#if defined __ANDROID__
+# if defined __ANDROID_API__ && __ANDROID_API__ >= 24
+#  define TORRENT_USE_PWRITEV 1
+# else
+#  define TORRENT_USE_PWRITEV 0
+# endif
+#elif LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,30)

Uh oh!

pread-disk-io #7013

Are you sure you want to change the base?

pread-disk-io #7013

Conversation

arvidn commented Aug 14, 2022

Uh oh!

glassez commented Aug 15, 2022

Uh oh!

arvidn commented Aug 15, 2022

Uh oh!

xavier2k6 commented Aug 15, 2022

Uh oh!

glassez commented Aug 15, 2022

Uh oh!

xavier2k6 commented Aug 15, 2022

Uh oh!

arvidn commented Aug 15, 2022

Uh oh!

glassez commented Aug 15, 2022

Uh oh!

xavier2k6 commented Aug 15, 2022

Uh oh!

arvidn commented Aug 15, 2022

Uh oh!

userdocs commented Aug 15, 2022

Uh oh!

glassez commented Aug 16, 2022

Uh oh!

glassez commented Aug 16, 2022

Uh oh!

james58899 commented Aug 17, 2022

Uh oh!

brvphoenix commented Aug 17, 2022

Uh oh!

glassez commented Aug 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

glassez commented Aug 17, 2022

Uh oh!

arvidn commented Aug 17, 2022

Uh oh!

glassez commented Aug 17, 2022

Uh oh!

xavier2k6 commented Aug 17, 2022

Uh oh!

glassez commented Aug 17, 2022

Uh oh!

glassez commented Aug 18, 2022

Uh oh!

ghost commented Aug 25, 2022

Uh oh!

tristanleboss commented Aug 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as off-topic.

tristanleboss commented Aug 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Aug 25, 2022

Uh oh!

This comment was marked as off-topic.

glassez commented Jun 28, 2025

Uh oh!

Aleksman4o commented Jul 10, 2025

Uh oh!

arvidn commented Jul 10, 2025

Uh oh!

Aleksman4o commented Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Aleksman4o commented Nov 30, 2025

Uh oh!

Rootax commented Jan 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

glassez commented Aug 17, 2022 •

edited

Loading

tristanleboss commented Aug 25, 2022 •

edited

Loading

tristanleboss commented Aug 25, 2022 •

edited

Loading

Aleksman4o commented Nov 29, 2025 •

edited

Loading

arvidn Jan 26, 2026 •

edited

Loading

NLZ Jan 26, 2026 •

edited

Loading