[rollout] refactor: bucketed transfer utils

[pengwu22]

What does this PR do?

• Abstract the current vllm weight update helper out for clear interfaces, and more importantly unittests

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

• Extra unittests covering shm and ipc

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)

[gemini-code-assist]

Code Review

This pull request refactors the weight transfer logic into a new, dedicated module bucketed_weight_transfer.py, improving code organization and testability. However, a critical security vulnerability has been identified in the new bucketed weight transfer mechanism due to insecure deserialization and arbitrary code execution. The use of ZMQ's recv_pyobj (which uses pickle) over a predictable IPC socket path in /tmp/ allows any user on the same host to achieve code execution. This must be addressed by using secure serialization and avoiding the transmission of executable callables. Additionally, two high-severity robustness issues were found: one concerning the reuse of a helper function for shared memory creation to improve error handling, and another regarding the fragility of relying on a hardcoded index for CUDA IPC.

[wuxibin89]

Hold this PR until #5029 merged.