Bump rng_salt to 0.45.0
#8908
Bump rng_salt to 0.45.0
#8908
Conversation
|
Hello. You may have forgotten to update the changelog!
|
|
This time, there are only 7 different stochastic failures |
### Before submitting
Please complete the following checklist when submitting a PR:
- [ ] All new features must include a unit test.
If you've fixed a bug or added code that should be tested, add a test to
the
test directory!
- [ ] All new functions and code must be clearly commented and
documented.
If you do make documentation changes, make sure that the docs build and
render correctly by running `make docs`.
- [ ] Ensure that the test suite passes, by running `make test`.
- [ ] Add a new entry to the `doc/releases/changelog-dev.md` file,
summarizing the
change, and including a link back to the PR.
- [ ] The PennyLane source code conforms to
[PEP8 standards](https://www.python.org/dev/peps/pep-0008/).
We check all of our code against [Pylint](https://www.pylint.org/).
To lint modified files, simply `pip install pylint`, and then
run `pylint pennylane/path/to/file.py`.
When all the above are checked, delete everything above the dashed
line and fill in the pull request template.
------------------------------------------------------------------------------------------------------------
**Context:**
**Description of the Change:**
**Benefits:**
**Possible Drawbacks:**
**Related GitHub Issues:**
…_sqeuence` with torch (#8937) First observed in the rng salt update of 0.45 A local run reveals that there's around 5% chance to fail this stochastic test (50 out of 1000): ``` Tests failed for seeds: [np.int64(6), np.int64(22), np.int64(33), np.int64(46), np.int64(96), np.int64(130), np.int64(137), np.int64(180), np.int64(230), np.int64(243), np.int64(256), np.int64(263), np.int64(264), np.int64(266), np.int64(281), np.int64(328), np.int64(392), np.int64(408), np.int64(429), np.int64(432), np.int64(456), np.int64(470), np.int64(475), np.int64(510), np.int64(522), np.int64(540), np.int64(550), np.int64(554), np.int64(578), np.int64(614), np.int64(623), np.int64(649), np.int64(658), np.int64(712), np.int64(724), np.int64(757), np.int64(768), np.int64(790), np.int64(795), np.int64(801), np.int64(812), np.int64(840), np.int64(864), np.int64(872), np.int64(881), np.int64(908), np.int64(925), np.int64(942), np.int64(972), np.int64(974)] Number of failed tests: 50 ``` which is at the marginal rate for us to further investigate it deeper. Hence I suggest we just salt it and leave note [sc-107856]
| # Use 4-sigma tolerance to prevent flaky tests | ||
| # We use atol (absolute) because noise does not scale with the expectation value | ||
| assert np.allclose(res, expected, atol=3 * std_error) | ||
| assert np.allclose(res, expected, atol=4 * std_error) |
There was a problem hiding this comment.
My local testify turns out the std is basically OK, and indeed below 1% chance to fail. Seems like this test is extremely unlucky (could suffer from very usual actual distribution or something). Therefore bump to 4-sigma for a ~0.01% chance to fail
astralcai
left a comment
astralcai
left a comment
There was a problem hiding this comment.
Have we also removed local_salt from the tests that don't need it to pass anymore?
That's a good point. Let me scan it. |
A salt on qdrift test has been removed. Another salt from midmeasure will be removed by the other PR maintaining that file specifically. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8908 +/- ##
=======================================
Coverage 99.44% 99.44%
=======================================
Files 601 601
Lines 64762 64762
=======================================
Hits 64401 64401
Misses 361 361 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
NOTE: remember to revert the fast fail option after fixing everythingDoneShortly after release we should start investigating the new
rng_saltalong with the stochastic failures it would reveal and fix things up if necessary[sc-107811]