Fix inconsistent BABE fork-choice by storing block weights in shared state

[Nathy-bajo]

Attempts to resolve #6013

Description

Introduced SharedBlockWeightStorage - a thread-safe, in-memory cache for block weights that provides immediate visibility across all import operations.

[Nathy-bajo]

I'm not done with the implementation and I am yet to test it.

[Nathy-bajo]

Please review @bkchr

[bkchr]

A good thing to start would be to try to write some unit test for the issue.

[Nathy-bajo]

A good thing to start would be to try to write some unit test for the issue.

Yeah, I think I'm getting to understand the issue better now.

[rockbmb]

Could you add some test(s) that verify #6013 is now fixed?

I ran revert_prunes_epoch_changes_and_removes_weights against this PR's latest commit (e025b63) and it passes; however, running it against latest master also sees it pass, which means it wasn't catching #6013 regardless.

Ideally the test(s), when run against latest master, fails in a situation described in #6013, and passes against this PR's tip.

[Nathy-bajo]

@rockbmb Done

[rockbmb]

@Nathy-bajo thanks; this change breaks some things, though; e.g. this refactor

polkadot-sdk/substrate/bin/node/cli/src/command.rs

Lines 225 to 228 in 0130e98

	let aux_revert = Box::new(|client: Arc<FullClient>, backend, blocks| {
	sc_consensus_babe::revert(client.clone(), backend, blocks)?;
	sc_consensus_grandpa::revert(client, blocks)?;
	Ok(())

[bkchr]

Where are you testing this?

[bkchr]

@Nathy-bajo my comment here is still not done? Because you just say "test xy", but you don't test it? You just finalize, but have no idea if it gets pruned.

[Nathy-bajo]

@bkchr All done. Please review

[bkchr]

It isn't helping when you just ignore my comments. You still don't have a test for the original problematic behavior.

[bkchr]

@Nathy-bajo my comment here is still not done? Because you just say "test xy", but you don't test it? You just finalize, but have no idea if it gets pruned.

[bkchr]

This will for sure make this test flaky.

[Nathy-bajo]

@bkchr I have removed the flaky tests and added tests for the original problematic behavior.