Skip to content

Commit 0d8896b

Browse files
committed
stabilize observability
1 parent 7288aa8 commit 0d8896b

35 files changed

+657
-682
lines changed

hypothesis-python/RELEASE.rst

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
RELEASE_TYPE: minor
2+
3+
This release adds |settings.observability|, which can be used to enable :ref:`observability <observability>`.
4+
5+
When observability is enabled, Hypothesis writes data about each test case to the ``.hypothesis/observed`` directory in an analysis-ready `jsonlines <https://jsonlines.org/>`_ format. This data is intended to help users who want to dive deep into understanding their tests. It's also intended for people building tools or research on top of Hypothesis.
6+
7+
Observability can be controlled in two ways:
8+
9+
* via the new |settings.observability| argument,
10+
* or via the ``HYPOTHESIS_OBSERVABILITY`` environment variable.
11+
12+
See :ref:`Configuring observability <observability-configuration>` for details.
13+
14+
If you use VSCode, we recommend the `Tyche <https://github.com/tyche-pbt/tyche-extension>`__ extension, a PBT-specific visualization tool designed for Hypothesis's observability interface.

hypothesis-python/docs/changelog.rst

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -529,7 +529,7 @@ Fixes a bug with solver-based :ref:`alternative backends <alternative-backends>`
529529
6.137.0 - 2025-08-05
530530
--------------------
531531

532-
Add the |add_observability_callback|, |remove_observability_callback|, |with_observability_callback|, and |observability_enabled| methods to the :ref:`observability <observability>` interface. The previous |TESTCASE_CALLBACKS| is deprecated.
532+
Add the ``add_observability_callback``, ``remove_observability_callback``, ``with_observability_callback``, and ``observability_enabled`` methods to the :ref:`observability <observability>` interface. The previous ``TESTCASE_CALLBACKS`` is deprecated.
533533

534534
This release also adds better threading support to observability callbacks. An observability callback will now only be called for observations generated by the same thread.
535535

@@ -864,11 +864,11 @@ Further improve the performance of the constants-collection feature introduced i
864864
6.135.3 - 2025-06-08
865865
--------------------
866866

867-
This release adds the experimental and unstable |OBSERVABILITY_CHOICES| option for :ref:`observability <observability>`. If set, the choice sequence is included in ``metadata.choice_nodes``, and choice sequence spans are included in ``metadata.choice_spans``.
867+
This release adds the experimental and unstable ``OBSERVABILITY_CHOICES`` option for :ref:`observability <observability>`. If set, the choice sequence is included in ``metadata.choice_nodes``, and choice sequence spans are included in ``metadata.choice_spans``.
868868

869869
These are relatively low-level implementation detail of Hypothesis, and are exposed in observability for users building tools or research on top of Hypothesis. See |PrimitiveProvider| for more details about the choice sequence and choice spans.
870870

871-
We are actively working towards a better interface for this. Feel free to use |OBSERVABILITY_CHOICES| to experiment, but don't rely on it yet!
871+
We are actively working towards a better interface for this. Feel free to use ``OBSERVABILITY_CHOICES`` to experiment, but don't rely on it yet!
872872

873873
.. _v6.135.2:
874874

@@ -985,7 +985,7 @@ This patch resolves a Pandas FutureWarning (:issue:`4400`) caused by indexing wi
985985
6.131.29 - 2025-05-27
986986
---------------------
987987

988-
The observations passed to |TESTCASE_CALLBACKS| are now dataclasses, rather than dictionaries. The content written to ``.hypothesis/observed`` under ``HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY`` remains the same.
988+
The observations passed to ``TESTCASE_CALLBACKS`` are now dataclasses, rather than dictionaries. The content written to ``.hypothesis/observed`` under ``HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY`` remains the same.
989989

990990
.. _v6.131.28:
991991

hypothesis-python/docs/prolog.rst

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
.. |settings.suppress_health_check| replace:: :obj:`settings.suppress_health_check <hypothesis.settings.suppress_health_check>`
2828
.. |settings.stateful_step_count| replace:: :obj:`settings.stateful_step_count <hypothesis.settings.stateful_step_count>`
2929
.. |settings.backend| replace:: :obj:`settings.backend <hypothesis.settings.backend>`
30+
.. |settings.observability| replace:: :obj:`settings.observability <hypothesis.settings.observability>`
3031

3132
.. |~settings.max_examples| replace:: :obj:`~hypothesis.settings.max_examples`
3233
.. |~settings.database| replace:: :obj:`~hypothesis.settings.database`
@@ -39,6 +40,7 @@
3940
.. |~settings.suppress_health_check| replace:: :obj:`~hypothesis.settings.suppress_health_check`
4041
.. |~settings.stateful_step_count| replace:: :obj:`~hypothesis.settings.stateful_step_count`
4142
.. |~settings.backend| replace:: :obj:`~hypothesis.settings.backend`
43+
.. |~settings.observability| replace:: :obj:`~hypothesis.settings.observability`
4244

4345
.. |settings.register_profile| replace:: :func:`~hypothesis.settings.register_profile`
4446
.. |settings.get_profile| replace:: :func:`~hypothesis.settings.get_profile`
@@ -66,6 +68,8 @@
6668
.. |Verbosity.normal| replace:: :obj:`Verbosity.normal <hypothesis.Verbosity.normal>`
6769
.. |Verbosity.quiet| replace:: :obj:`Verbosity.quiet <hypothesis.Verbosity.quiet>`
6870

71+
.. |ObservabilityConfig| replace:: :obj:`ObservabilityConfig <hypothesis.ObservabilityConfig>`
72+
6973
.. |HypothesisException| replace:: :obj:`HypothesisException <hypothesis.errors.HypothesisException>`
7074
.. |HypothesisDeprecationWarning| replace:: :obj:`HypothesisDeprecationWarning <hypothesis.errors.HypothesisDeprecationWarning>`
7175
.. |Flaky| replace:: :obj:`Flaky <hypothesis.errors.Flaky>`
@@ -143,12 +147,6 @@
143147
.. |AVAILABLE_PROVIDERS| replace:: :data:`~hypothesis.internal.conjecture.providers.AVAILABLE_PROVIDERS`
144148
.. |run_conformance_test| replace:: :func:`~hypothesis.internal.conjecture.provider_conformance.run_conformance_test`
145149

146-
.. |add_observability_callback| replace:: :data:`~hypothesis.internal.observability.add_observability_callback`
147-
.. |remove_observability_callback| replace:: :data:`~hypothesis.internal.observability.remove_observability_callback`
148-
.. |with_observability_callback| replace:: :data:`~hypothesis.internal.observability.with_observability_callback`
149-
.. |observability_enabled| replace:: :data:`~hypothesis.internal.observability.observability_enabled`
150-
.. |TESTCASE_CALLBACKS| replace:: :data:`~hypothesis.internal.observability.TESTCASE_CALLBACKS`
151-
.. |OBSERVABILITY_CHOICES| replace:: :data:`~hypothesis.internal.observability.OBSERVABILITY_CHOICES`
152150
.. |BUFFER_SIZE| replace:: :data:`~hypothesis.internal.conjecture.engine.BUFFER_SIZE`
153151
.. |MAX_SHRINKS| replace:: :data:`~hypothesis.internal.conjecture.engine.MAX_SHRINKS`
154152
.. |MAX_SHRINKING_SECONDS| replace:: :data:`~hypothesis.internal.conjecture.engine.MAX_SHRINKING_SECONDS`

hypothesis-python/docs/reference/api.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,8 @@ Settings
174174
.. autoclass:: hypothesis.Verbosity
175175
:members:
176176

177+
.. autoclass:: hypothesis.ObservabilityConfig
178+
177179
.. autoclass:: hypothesis.HealthCheck
178180
:undoc-members:
179181
:inherited-members:

hypothesis-python/docs/reference/integrations.rst

Lines changed: 10 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -75,17 +75,10 @@ If you're interested in similar questions, `drop me an email`_!
7575
Observability
7676
-------------
7777

78-
.. note::
78+
.. tip::
7979

8080
The `Tyche <https://github.com/tyche-pbt/tyche-extension>`__ VSCode extension provides an in-editor UI for observability results generated by Hypothesis. If you want to *view* observability results, rather than programmatically consume or display them, we recommend using Tyche.
8181

82-
.. warning::
83-
84-
This feature is experimental, and could have breaking changes or even be removed
85-
without notice. Try it out, let us know what you think, but don't rely on it
86-
just yet!
87-
88-
8982
Motivation
9083
~~~~~~~~~~
9184

@@ -108,24 +101,23 @@ debuggers such as `rr <https://rr-project.org/>`__ or `pytrace <https://pytrace.
108101
because there's no good way to compare multiple traces from these tools and their
109102
Python support is relatively immature.
110103

104+
.. _observability-configuration:
111105

112-
Configuration
113-
~~~~~~~~~~~~~
106+
Configuring observability
107+
~~~~~~~~~~~~~~~~~~~~~~~~~
114108

115-
If you set the ``HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY`` environment variable,
116-
Hypothesis will log various observations to jsonlines files in the
109+
The standard way to configure observability is with |settings.observability|.
110+
111+
Alternatively, observability can be configured by setting the ``HYPOTHESIS_OBSERVABILITY`` environment variable. If ``HYPOTHESIS_OBSERVABILITY`` is set to one of ``True``, ``true``, or ``1``, |settings.observability| defaults to ``True``. Note that unlike |settings.observability|, ``HYPOTHESIS_OBSERVABILITY`` only configures whether observability is enabled or disabled, not additional options like |ObservabilityConfig|.
112+
113+
When observability is enabled, and ``callbacks`` in |ObservabilityConfig| has not been configured to remove the default callback, Hypothesis will log various observations to jsonlines files in the
117114
``.hypothesis/observed/`` directory. You can load and explore these with e.g.
118115
:func:`pd.read_json(".hypothesis/observed/*_testcases.jsonl", lines=True) <pandas.read_json>`,
119116
or by using the :pypi:`sqlite-utils` and :pypi:`datasette` libraries::
120117

121118
sqlite-utils insert testcases.db testcases .hypothesis/observed/*_testcases.jsonl --nl --flatten
122119
datasette serve testcases.db
123120

124-
If you are experiencing a significant slow-down, you can try setting
125-
``HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER`` instead; this will disable coverage information
126-
collection. This should not be necessary on Python 3.12 or later, where coverage collection is very fast.
127-
128-
129121
Collecting more information
130122
^^^^^^^^^^^^^^^^^^^^^^^^^^^
131123

@@ -181,7 +173,7 @@ While the observability format is agnostic to the property-based testing library
181173
Choices metadata
182174
++++++++++++++++
183175

184-
These additional metadata elements are included in ``metadata`` (as e.g. ``metadata["choice_nodes"]`` or ``metadata["choice_spans"]``), if and only if |OBSERVABILITY_CHOICES| is set.
176+
These additional metadata elements are included in ``metadata`` (as e.g. ``metadata["choice_nodes"]`` or ``metadata["choice_spans"]``), if and only if observability is configured to include choices (see |ObservabilityConfig|).
185177

186178
.. jsonschema:: ./schema_metadata_choices.json
187179
:hide_key: /additionalProperties, /type

hypothesis-python/docs/reference/internals.rst

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -27,18 +27,6 @@ Alternative backends
2727
.. autoclass:: hypothesis.errors.BackendCannotProceed
2828
.. autoclass:: hypothesis.internal.intervalsets.IntervalSet
2929

30-
Observability
31-
-------------
32-
33-
.. autofunction:: hypothesis.internal.observability.add_observability_callback
34-
.. autofunction:: hypothesis.internal.observability.remove_observability_callback
35-
.. autofunction:: hypothesis.internal.observability.with_observability_callback
36-
.. autofunction:: hypothesis.internal.observability.observability_enabled
37-
38-
.. autodata:: hypothesis.internal.observability.TESTCASE_CALLBACKS
39-
.. autodata:: hypothesis.internal.observability.OBSERVABILITY_COLLECT_COVERAGE
40-
.. autodata:: hypothesis.internal.observability.OBSERVABILITY_CHOICES
41-
4230
Engine constants
4331
----------------
4432

hypothesis-python/docs/reference/schema_metadata_choices.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"properties": {
44
"choice_nodes": {
55
"type": ["array", "null"],
6-
"description": ".. warning::\n\n EXPERIMENTAL AND UNSTABLE. This attribute may change format or disappear without warning.\n\nThe sequence of choices made during this test case. This includes the choice value, as well as its constraints and whether it was forced or not.\n\nOnly present if |OBSERVABILITY_CHOICES| is ``True``.\n\n.. note::\n\n The choice sequence is a relatively low-level implementation detail of Hypothesis, and is exposed in observability for users building tools or research on top of Hypothesis. See |PrimitiveProvider| for more details about the choice sequence.",
6+
"description": ".. warning::\n\n EXPERIMENTAL AND UNSTABLE. This attribute may change format without warning.\n\nThe sequence of choices made during this test case. This includes the choice value, as well as its constraints and whether it was forced or not.\n\n.. note::\n\n Only present if observability is configured to include choices (see |ObservabilityConfig|).\n\n.. note::\n\n The choice sequence is a relatively low-level implementation detail of Hypothesis, and is exposed in observability for users building tools or research on top of Hypothesis. See |PrimitiveProvider| for more details about the choice sequence.",
77
"items": {
88
"type": "object",
99
"properties": {
@@ -31,7 +31,7 @@
3131
"choice_spans": {
3232
"type": "array",
3333
"items": {"type": "array"},
34-
"description": ".. warning::\n\n EXPERIMENTAL AND UNSTABLE. This attribute may change format or disappear without warning.\n\nThe semantically-meaningful spans of the choice sequence of this test case.\n\nEach span has the format ``[label, start, end, discarded]``, where:\n\n* ``label`` is an opaque integer-value string shared by all spans drawn from a particular strategy.\n* ``start`` and ``end`` are indices into the choice sequence for this span, such that ``choices[start:end]`` are the corresponding choices.\n* ``discarded`` is a boolean indicating whether this span was discarded (see |PrimitiveProvider.span_end|).\n\nOnly present if |OBSERVABILITY_CHOICES| is ``True``.\n\n.. note::\n\n Spans are a relatively low-level implementation detail of Hypothesis, and are exposed in observability for users building tools or research on top of Hypothesis. See |PrimitiveProvider| (and particularly |PrimitiveProvider.span_start| and |PrimitiveProvider.span_end|) for more details about spans."
34+
"description": ".. warning::\n\n EXPERIMENTAL AND UNSTABLE. This attribute may change format without warning.\n\nThe semantically-meaningful spans of the choice sequence of this test case.\n\nEach span has the format ``[label, start, end, discarded]``, where:\n\n* ``label`` is an opaque integer-value string shared by all spans drawn from a particular strategy.\n* ``start`` and ``end`` are indices into the choice sequence for this span, such that ``choices[start:end]`` are the corresponding choices.\n* ``discarded`` is a boolean indicating whether this span was discarded (see |PrimitiveProvider.span_end|).\n\n.. note::\n\n Only present if observability is configured to include choices (see |ObservabilityConfig|).\n\n.. note::\n\n Spans are a relatively low-level implementation detail of Hypothesis, and are exposed in observability for users building tools or research on top of Hypothesis. See |PrimitiveProvider| (and particularly |PrimitiveProvider.span_start| and |PrimitiveProvider.span_end|) for more details about spans."
3535
}
3636
},
3737
"required": ["traceback", "reproduction_decorator", "predicates", "backend", "sys.argv", "os.getpid()", "imported_at", "data_status", "interesting_origin", "choice_nodes", "choice_spans"],

hypothesis-python/docs/reference/schema_observations.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@
3636
},
3737
"coverage": {
3838
"type": ["object", "null"],
39-
"description": "Mapping of filename to list of covered line numbers, if coverage information is available, or None if not. Hypothesis deliberately omits stdlib and site-packages code.",
39+
"description": ".. warning::\n\n EXPERIMENTAL AND UNSTABLE. This attribute might change format without warning, as we figure out the right format to support both line and branch coverage.\n\n Mapping of filename to list of covered line numbers, if coverage information is available, or None if not. Hypothesis deliberately omits stdlib and site-packages code.\n\n.. note::\n\n Only present if observability is configured to include coverage (see |ObservabilityConfig|).",
4040
"additionalProperties": {
4141
"type": "array",
4242
"items": {"type": "integer", "minimum": 1},

hypothesis-python/pyproject.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ pandas = ["pandas>=1.1"]
110110
pytest = ["pytest>=4.6"]
111111
dpcontracts = ["dpcontracts>=0.4"]
112112
redis = ["redis>=3.0.0"]
113-
crosshair = ["hypothesis-crosshair>=0.0.26", "crosshair-tool>=0.0.99"]
113+
crosshair = ["hypothesis-crosshair>=0.0.27", "crosshair-tool>=0.0.100"]
114114
# zoneinfo is an odd one: every dependency is platform-conditional.
115115
zoneinfo = ["tzdata>=2025.2; sys_platform == 'win32' or sys_platform == 'emscripten'"]
116116
# We only support Django versions with upstream support - see
@@ -120,7 +120,7 @@ zoneinfo = ["tzdata>=2025.2; sys_platform == 'win32' or sys_platform == 'emscrip
120120
django = ["django>=4.2"]
121121
watchdog = ["watchdog>=4.0.0"]
122122
# Avoid changing this by hand. This is automatically updated by update_changelog_and_version
123-
all = ["black>=20.8b0", "click>=7.0", "crosshair-tool>=0.0.99", "django>=4.2", "dpcontracts>=0.4", "hypothesis-crosshair>=0.0.26", "lark>=0.10.1", "libcst>=0.3.16", "numpy>=1.21.6", "pandas>=1.1", "pytest>=4.6", "python-dateutil>=1.4", "pytz>=2014.1", "redis>=3.0.0", "rich>=9.0.0", "tzdata>=2025.2; sys_platform == 'win32' or sys_platform == 'emscripten'", "watchdog>=4.0.0"]
123+
all = ["black>=20.8b0", "click>=7.0", "crosshair-tool>=0.0.100", "django>=4.2", "dpcontracts>=0.4", "hypothesis-crosshair>=0.0.27", "lark>=0.10.1", "libcst>=0.3.16", "numpy>=1.21.6", "pandas>=1.1", "pytest>=4.6", "python-dateutil>=1.4", "pytz>=2014.1", "redis>=3.0.0", "rich>=9.0.0", "tzdata>=2025.2; sys_platform == 'win32' or sys_platform == 'emscripten'", "watchdog>=4.0.0"]
124124

125125
[tool.setuptools.dynamic]
126126
version = {attr = "hypothesis.version.__version__"}

hypothesis-python/src/hypothesis/__init__.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,12 @@
1717

1818
import _hypothesis_globals
1919

20-
from hypothesis._settings import HealthCheck, Phase, Verbosity, settings
20+
from hypothesis._settings import (
21+
HealthCheck,
22+
Phase,
23+
Verbosity,
24+
settings,
25+
)
2126
from hypothesis.control import (
2227
assume,
2328
currently_in_test_context,
@@ -30,11 +35,13 @@
3035
from hypothesis.entry_points import run
3136
from hypothesis.internal.detection import is_hypothesis_test
3237
from hypothesis.internal.entropy import register_random
38+
from hypothesis.internal.observability import ObservabilityConfig
3339
from hypothesis.utils.conventions import infer
3440
from hypothesis.version import __version__, __version_info__
3541

3642
__all__ = [
3743
"HealthCheck",
44+
"ObservabilityConfig",
3845
"Phase",
3946
"Verbosity",
4047
"__version__",

0 commit comments

Comments
 (0)