Skip to content

Conversation

@Mercykid-bash
Copy link
Contributor

@Mercykid-bash Mercykid-bash commented Feb 2, 2026

This fix addresses the missing pre-compilation of EPLB algorithm in subprocesses under graph mode. With the correction, the kernel pre-compilation procedure runs properly in child processes, eliminating related runtime performance and stability risks.

Signed-off-by: Mercykid-bash <[email protected]>
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @Mercykid-bash, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a targeted warm-up mechanism within the worker process. It ensures that a specific initialization routine is performed for a particular policy type before the worker enters its main processing loop, aiming to optimize performance or resource readiness under those specific conditions.

Highlights

  • Conditional Warm-up Execution: A new conditional block has been added to the worker_process method. This block executes a warm-up routine specifically when self.policy_type is set to 3.
  • FlashLB Policy Integration: The warm-up function, warm_up(), is dynamically imported from vllm_ascend.eplb.core.policy.policy_flashlb, indicating that this warm-up is tailored for the FlashLB policy.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a warm-up mechanism for the FlashLB policy by adding a warm_up() call in the worker_process. This is a good improvement to ensure Numba JIT-compiled functions are ready before use in the worker process. I have one suggestion regarding code redundancy. Additionally, the pull request title and description could be improved to provide more context, for which I have provided suggestions below as per the repository's style guide.

Suggested PR Title:

[EPLB][Misc] Add warm-up for FlashLB policy in worker process

Suggested PR Summary:

### What this PR does / why we need it?

This PR adds a warm-up call for the FlashLB policy (`policy_type == 3`). The warm-up is performed at the beginning of the EPLB worker process.

This is necessary to pre-compile the Numba JIT-accelerated functions used by the FlashLB policy within the worker process, avoiding performance hits on the first execution.

This change makes the existing `warm_up()` call in `policy_factory.py` redundant. That call runs in the main process and should be removed to avoid unnecessary startup overhead.

### Does this PR introduce _any_ user-facing change?

No, this is a performance optimization for startup and does not affect user-facing functionality.

### How was this patch tested?

CI should pass. Manual verification should show reduced latency on the first balancing operation when using the FlashLB policy.

@Mercykid-bash Mercykid-bash changed the title warm up Bugfix: Pre-compile EPLB algorithm successfully in subprocess under graph mode Feb 2, 2026
@Mercykid-bash Mercykid-bash marked this pull request as ready for review February 2, 2026 03:01
@wangxiyuan wangxiyuan added the ready read for review label Feb 2, 2026
@wangxiyuan wangxiyuan merged commit 6b20f41 into vllm-project:releases/v0.13.0 Feb 2, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready read for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants