v1/ast: Add lazy hash evaluation to Array type #8155

alex60217101990 · 2025-12-23T08:47:41Z

Array Lazy Hash Computation Optimization

Why the changes in this PR are needed?

The current Array implementation maintains a separate hashs []int slice that stores precomputed hashes for each element. This leads to:

Excessive memory consumption: 8 bytes × N elements for every array
Unnecessary computations: Hashes are computed even if never used
Extra allocations: During array creation, copying, and slicing operations

Many arrays are created for temporary use and never participate in hash-based operations (e.g., map lookups or set operations).

What are the changes in this PR?

Refactor the Array structure to use lazy hash computation:

Key Changes:

Removed hashs []int field from the Array struct
- Memory savings: 8*N bytes per array (where N is the number of elements)
Added hashValid bool flag to track hash computation state
- Flag indicates whether the hash has been computed
Implemented lazy evaluation in the Hash() method
- Hash is computed only on first access
- Subsequent calls return the cached value
Incremental hash updates in Array.Append()
- If hash is already computed, it's updated incrementally: hash += newElement.Hash()
- If hash was not computed, computation is deferred
Updated all related methods:
- NewArray() - no longer computes hashes at creation time
- Copy() - copies the hashValid flag
- Sorted() - preserves computed hash (sorting doesn't change hash)
- Slice() - creates a slice with invalid hash
- rehash() - simplified to just invalidate the cache
- set() - invalidates hash instead of recomputing

Benchmark Results

Key Improvements:

Array Creation (ArrayCreation)

10 elements: -68% time, -67% memory
100 elements: -82% time, -95% memory
1000 elements: -85% time, -99% memory
10000 elements: -93% time, -99% memory

Append Operations (ArrayAppend)

10 elements: -65% time, -52% memory, -40% allocs
100 elements: -71% time, -59% memory, -40% allocs
1000 elements: -77% time, -62% memory, -40% allocs

Array Copy (ArrayCopy)

10 elements: -13% time, -21% memory
100 elements: -9% time, -21% memory
1000 elements: -4% time, -20% memory
10000 elements: -10% time, -20% memory

Slice Operations (ArraySlice)

100 elements: -69% time, -25% memory
1000 elements: -96% time, -25% memory
10000 elements: -99.5% time, -25% memory

Set Operations (ArraySet)

10 elements: -91% time
100 elements: -97% time
1000 elements: -99.7% time

Operations Without Hash Access (ArrayNoHashAccess)

This benchmark demonstrates the real benefit of lazy evaluation - when hash is not needed:

10 elements: -68% time, -67% memory, -50% allocs
100 elements: -82% time, -95% memory, -50% allocs
1000 elements: -86% time, -99% memory, -50% allocs
10000 elements: -94% time, -99% memory, -50% allocs

Overall Geometric Mean:

Execution time: -59%
Memory usage: -59%
Number of allocations: -19%

Full benchstat Results

Detailed results are available in:

netlify · 2025-12-23T08:50:25Z

✅ Deploy Preview for openpolicyagent ready!

Name	Link
🔨 Latest commit	`4141a17`
🔍 Latest deploy log	https://app.netlify.com/projects/openpolicyagent/deploys/694a5730fb498e0008a6d775
😎 Deploy Preview	https://deploy-preview-8155--openpolicyagent.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

netlify · 2025-12-23T08:52:58Z

✅ Deploy Preview for openpolicyagent ready!

Name	Link
🔨 Latest commit	`2f319a0`
🔍 Latest deploy log	https://app.netlify.com/projects/openpolicyagent/deploys/696cfef8ca019f00080b0eae
😎 Deploy Preview	https://deploy-preview-8155--openpolicyagent.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

srenatus · 2026-01-05T14:22:51Z

That race detected in tests is probably genuine. Can you have a look? Also, let's better discuss improvements like this before diving into it, I could have made you aware of the proneness to data races in this approach 💭

alex60217101990 · 2026-01-10T15:56:32Z

@srenatus
You were right, there was indeed a data race. I've fixed it using atomic operations. Attached are the benchmark results before and after the fix, along with benchstat diff showing the performance improvements from these changes.
benchmarks_array_lazy_hash_baseline.txt
benchmarks_array_lazy_hash_optimized.txt
benchstat_array_lazy_hash.txt

johanfylling

Hi! 👋
Some questions regarding concurrency.

johanfylling · 2026-01-26T15:59:44Z

v1/ast/term.go

-	ground bool
+	elems     []*Term
+	ground    bool
+	hash      atomic.Int64  // cached hash value (atomic for race-free access)


Do we have an understanding of how this affects performance for concurrent eval; e.g. when OPA is run as a server and multiple queries might contend over this value?
Maybe there should be a separate path of instantiation for when the array is generated in topdown during evaluation and when the array is coming from storage and is never expected to change. In the latter case, pre-hashing is likely always preferred.

johanfylling · 2026-01-26T16:08:34Z

v1/ast/term.go

+	elems     []*Term
+	ground    bool
+	hash      atomic.Int64  // cached hash value (atomic for race-free access)
+	hashValid atomic.Uint32 // 0 = invalid, 1 = valid


Even though hash and hashValid are atomic, they could still drift in a concurrent context, right? If the array was read from storage without pre-calculated hash, is it possible for one query to get an updated hashValid value, but a stale hash value, making a comparison invalid?

johanfylling · 2026-01-26T16:11:38Z

v1/ast/term.go

+	// Try to atomically set valid flag using CAS
+	if arr.hashValid.CompareAndSwap(0, 1) {
+		// We won the race, store the hash
+		arr.hash.Store(int64(h))


This doesn't look like it's happening atomically within the swap. Could we not be returning a stale hash value in another routine up top in this func (arr.hashValid.Load() == 1)?

Refactor Array structure to use lazy hash evaluation instead of maintaining a separate hashs slice, reducing memory overhead by 8*N bytes per array. Key changes: - Remove hashs []int slice from Array struct - Add hashValid bool flag to track hash computation state - Implement lazy hash computation in Hash() method - Optimize Append() with incremental hash updates when hash is already computed - Update Copy(), Sorted(), Slice() to work with new hash model - Simplify rehash() to just invalidate the hash cache This optimization reduces memory allocations while maintaining performance through intelligent caching and incremental updates. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

Add comprehensive benchmarks to measure the impact of lazy hash computation optimization for Array operations. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

Eliminate data race in lazy hash computation by using atomic operations for the hashValid flag. This ensures thread-safe access when multiple goroutines call Hash() concurrently on the same Array. Changes: - Use atomic.LoadUint32/StoreUint32 for hashValid flag - Implement CompareAndSwapUint32 for race-free hash computation - Preserve lazy evaluation and O(1) cached access performance Performance impact: - Maintains 3ns hash access time for cached values - 54% faster overall array operations (geomean) - 64% less memory usage across benchmarks - Zero lock contention (lock-free fast path) Passes all tests with -race flag enabled. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

Remove obsolete hashs field initialization in NewArrayWithCapacity that was missed during the merge with upstream main branch. The hashs field was removed as part of the lazy hash computation optimization for Array type. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

Replace raw atomic function calls with atomic types to eliminate data races in concurrent hash access and computation. Changes: - Use atomic.Int64 for hash field instead of int - Use atomic.Uint32 for hashValid field instead of uint32 - Replace atomic.Load/Store/CompareAndSwap functions with methods - Fix data races in Hash(), Copy(), Sorted(), Append(), and rehash() All ast tests pass with race detector. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

Fix linter errors related to copying lock values in Array struct. Changes: - Rewrite Array.Append() to avoid copying atomic fields by value - Change SubResultMap.Update() to accept *ast.Array pointer - Update all call sites to pass pointer instead of dereferenced value - Fix gofmt formatting for atomic field alignment Resolves govet copylocks warnings while maintaining thread-safety. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

Replace separate hash/hashValid atomics with single atomic.Int64 using math.MinInt64 sentinel. Eliminates drift between hash and validity flag, prevents stale reads, ensures atomic operations. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

netlify · 2026-01-26T18:22:14Z

✅ Deploy Preview for openpolicyagent ready!

Name	Link
🔨 Latest commit	`73faa63`
🔍 Latest deploy log	https://app.netlify.com/projects/openpolicyagent/deploys/6977b0384eb92b0008296494
😎 Deploy Preview	https://deploy-preview-8155--openpolicyagent.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

alex60217101990 · 2026-01-26T18:30:23Z

@johanfylling Thank you for the thorough review! You're absolutely right about the race condition - very constructive feedback.

I've refactored the implementation to fix the issues you pointed out. Replaced the separate hash/hashValid atomics with a single atomic.Int64 using math.MinInt64 as a sentinel value for "not computed" state. This eliminates the drift problem and ensures atomic operations throughout.

Also added concurrent tests covering multiple goroutines accessing the same array simultaneously, which all pass cleanly with -race.

When you have a chance, could you take a look? I believe this addresses all the concerns you raised.

alex60217101990 force-pushed the ast/lazy-eval-array-hash branch from 4141a17 to cb4a46b Compare December 23, 2025 08:49

alex60217101990 force-pushed the ast/lazy-eval-array-hash branch 2 times, most recently from d489ff0 to 9dc3e60 Compare January 10, 2026 14:11

alex60217101990 force-pushed the ast/lazy-eval-array-hash branch 2 times, most recently from 73b2485 to 9f31e14 Compare January 12, 2026 14:08

johanfylling reviewed Jan 26, 2026

View reviewed changes

alex60217101990 added 7 commits January 26, 2026 19:40

v1/ast: Add benchmarks for Array lazy hash computation

e6be9b6

Add comprehensive benchmarks to measure the impact of lazy hash computation optimization for Array operations. Signed-off-by: alex60217101990 <alex6021710@gmail.com>

alex60217101990 force-pushed the ast/lazy-eval-array-hash branch from 2f319a0 to 73faa63 Compare January 26, 2026 18:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v1/ast: Add lazy hash evaluation to Array type #8155

v1/ast: Add lazy hash evaluation to Array type #8155

alex60217101990 commented Dec 23, 2025

Uh oh!

netlify bot commented Dec 23, 2025

Uh oh!

netlify bot commented Dec 23, 2025 •

edited

Loading

Uh oh!

srenatus commented Jan 5, 2026

Uh oh!

alex60217101990 commented Jan 10, 2026

Uh oh!

johanfylling left a comment

Uh oh!

johanfylling Jan 26, 2026

Uh oh!

johanfylling Jan 26, 2026

Uh oh!

johanfylling Jan 26, 2026

Uh oh!

netlify bot commented Jan 26, 2026

Uh oh!

alex60217101990 commented Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

v1/ast: Add lazy hash evaluation to Array type #8155

Are you sure you want to change the base?

v1/ast: Add lazy hash evaluation to Array type #8155

Conversation

alex60217101990 commented Dec 23, 2025

Array Lazy Hash Computation Optimization

Why the changes in this PR are needed?

What are the changes in this PR?

Key Changes:

Benchmark Results

Key Improvements:

Array Creation (ArrayCreation)

Append Operations (ArrayAppend)

Array Copy (ArrayCopy)

Slice Operations (ArraySlice)

Set Operations (ArraySet)

Operations Without Hash Access (ArrayNoHashAccess)

Overall Geometric Mean:

Full benchstat Results

Uh oh!

netlify bot commented Dec 23, 2025

✅ Deploy Preview for openpolicyagent ready!

Uh oh!

netlify bot commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for openpolicyagent ready!

Uh oh!

srenatus commented Jan 5, 2026

Uh oh!

alex60217101990 commented Jan 10, 2026

Uh oh!

johanfylling left a comment

Choose a reason for hiding this comment

Uh oh!

johanfylling Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

johanfylling Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

johanfylling Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

netlify bot commented Jan 26, 2026

✅ Deploy Preview for openpolicyagent ready!

Uh oh!

alex60217101990 commented Jan 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

netlify bot commented Dec 23, 2025 •

edited

Loading