Releases: GreptimeTeam/greptimedb
Release v1.0.0-beta.3-nightly-20251222
What's Changed
- chore: add wait_initialized method for frontend client by @fengys1996 in #7414
- feat: introduce granularity for memory manager by @fengjiachun in #7416
- refactor(cli): unify storage configuration for export command by @McKnight22 in #7280
- chore: use official etcd-client by @MichaelScofield in #7432
- chore: remove
canonicalizeby @shuiyisong in #7430 - chore: expose
disable_ec2_metadataoption by @shuiyisong in #7439 - chore(mito): nit remove extra hashset in gc workers by @lyang24 in #7399
- ci: ensure commits from main branch for whitelisted git dependencies by @sunng87 in #7434
- feat: make distributed time constants and client timeouts configurable by @WenyXu in #7433
- test: reduce execution time of test
test_suspend_frontendby @MichaelScofield in #7444 - fix: flat format use correct encoding in indexer for tags by @evenyag in #7440
- chore: fix bincode version by @discord9 in #7445
- chore: release push check against Cargo.toml by @discord9 in #7426
Full Changelog: v1.0.0-beta.3...v1.0.0-beta.3-nightly-20251222
Release v1.0.0-beta.3
v1.0.0-beta.3
Release date: December 17, 2025
It's recommended to upgrade to v1.0.0-beta.4
Breaking changes
- feat!: download file to fill the cache on write cache miss by @evenyag in #7294
- Added:
region_engine.mito.manifest_cache_size(default: 256MB) - Removed:
storage.cache_path, storage.enable_read_cache,storage.cache_capacity
- Added:
🚀 Features
- feat: implement a cache for manifest files by @evenyag in #7326
- feat: add more verbose metrics to scanners by @evenyag in #7336
- feat: decode pk eagerly by @waynexia in #7350
- feat: run histogram quantile in safe mode for incomplete data by @waynexia in #7297
- feat: update dashboard to v0.11.9 by @ZonaHex in #7364
- feat: allow publishing new nightly release when some platforms are absent by @waynexia in #7354
- feat: allow one to many VRL pipeline by @v0y4g3r in #7342
- feat: collect merge and dedup metrics by @evenyag in #7375
- feat: suspend frontend and datanode by @MichaelScofield in #7370
- feat: table/column/flow COMMENT by @waynexia in #7060
- feat: grafana postgresql data source query builder support by @sunng87 in #7379
- feat: mark index outdated by @discord9 in #7383
- feat: mem manager on compaction by @fengjiachun in #7305
- feat: optimize and fix part sort on overlapping time windows by @waynexia in #7387
- feat: per file scan metrics by @evenyag in #7396
- feat: move memory_manager to common crate by @fengjiachun in #7408
- feat: introduce
copy_region_fromfor mito engine by @WenyXu in #7389 - feat: support function aliases and add MySQL-compatible aliases by @killme2008 in #7410
- feat(vector_index): adds the foundational types and SQL parsing support for vector index by @killme2008 in #7366
🐛 Bug Fixes
- fix: reset cached channel on errors with VIP by @fengjiachun in #7335
- fix: configure HTTP/2 keep-alive for heartbeat client to detect network failures faster by @WenyXu in #7344
- fix: regression with shortcutted statement on postgres extended query by @sunng87 in #7340
- fix: use saturating in gc tracker by @discord9 in #7369
- fix: part sort behavior by @waynexia in #7374
- fix: improve network failure detection by @WenyXu in #7382
- fix(procedure): update procedure state correctly during execution and on failure by @WenyXu in #7376
- fix: gc listing op first by @discord9 in #7385
- fix: parse "KEEP FIRING FOR" by @fengys1996 in #7386
- fix: promql histogram with aggregation by @waynexia in #7393
- fix: promql offset direction by @waynexia in #7392
- fix: TLS option validate and merge by @shuiyisong in #7401
- fix: cpu cores got wrongly calculated to 0 by @MichaelScofield in #7405
- fix: use verified recycling method for PostgreSQL connection pool by @WenyXu in #7407
- fix(servers): flight stuck on waiting for first message by @v0y4g3r in #7413
- fix: using anonymous s3 access when ak and sk is not provided by @shuiyisong in #7425
🚜 Refactor
- refactor(servers): allow custom flight service by @v0y4g3r in #7333
- refactor(servers): bulk insert service by @v0y4g3r in #7329
- refactor: remove datafusion data frame wrapper by @waynexia in #7347
- refactor: use versioned index for index file by @discord9 in #7309
- refactor: extract file watcher to common-config by @shuiyisong in #7357
- refactor: remove duplication coverage and code from window sort tests by @waynexia in #7384
- refactor: optimize heartbeat channel and etcd client keepalive settings by @WenyXu in #7390
⚡ Performance
🧪 Testing
- test: gc worker scheduler mock test by @discord9 in #7292
- test: test_tracker_cleanup skip non linux by @discord9 in #7398
⚙️ Miscellaneous Tasks
- chore: members and committers update by @fengjiachun in #7341
- chore: rm unnecessary warning by @discord9 in #7352
- chore: rename to avoid git reserved name by @discord9 in #7359
- chore(mito): expose some symbols by @v0y4g3r in #7373
- chore: saturating duration since by @discord9 in #7380
- chore(mito): nit avoid clone the batch object on inverted index building by @lyang24 in #7388
- chore: sort histogram sqlness result by @waynexia in #7406
- chore: add
is_initializedmethod for frontend client by @fengys1996 in #7409 - chore: expose symbols by @v0y4g3r in #7417
- chore: change etcd endpoints to array in the test scripts by @daviderli614 in #7419
- chore: bump version to beta.3 by @discord9 in #7423
- chore: feature gate vector_index by @discord9 in #7428
All Contributors
We would like to thank the following contributors from the GreptimeDB community:
@MichaelScofield, @WenyXu, @ZonaHex, @daviderli614, [@discord9](ht...
Release v1.0.0-beta.2-nightly-20251215
What's Changed
- feat: allow publishing new nightly release when some platforms are absent by @waynexia in #7354
- fix: use saturating in gc tracker by @discord9 in #7369
- refactor: use versioned index for index file by @discord9 in #7309
- refactor: extract file watcher to common-config by @shuiyisong in #7357
- chore(mito): expose some symbols by @v0y4g3r in #7373
- feat: allow one to many VRL pipeline by @v0y4g3r in #7342
- chore: saturating duration since by @discord9 in #7380
- fix: part sort behavior by @waynexia in #7374
- feat: collect merge and dedup metrics by @evenyag in #7375
- fix: improve network failure detection by @WenyXu in #7382
- refactor: remove duplication coverage and code from window sort tests by @waynexia in #7384
- feat: suspend frontend and datanode by @MichaelScofield in #7370
- feat: table/column/flow COMMENT by @waynexia in #7060
- fix(procedure): update procedure state correctly during execution and on failure by @WenyXu in #7376
- feat: grafana postgresql data source query builder support by @sunng87 in #7379
- fix: gc listing op first by @discord9 in #7385
- fix: parse "KEEP FIRING FOR" by @fengys1996 in #7386
- feat: mark index outdated by @discord9 in #7383
- refactor: optimize heartbeat channel and etcd client keepalive settings by @WenyXu in #7390
- chore(mito): nit avoid clone the batch object on inverted index building by @lyang24 in #7388
- test: test_tracker_cleanup skip non linux by @discord9 in #7398
- feat: mem manager on compaction by @fengjiachun in #7305
- fix: promql histogram with aggregation by @waynexia in #7393
- fix: promql offset direction by @waynexia in #7392
- feat: optimize and fix part sort on overlapping time windows by @waynexia in #7387
Full Changelog: v1.0.0-beta.2-8af1ab0ff-20251209-1765256263...v1.0.0-beta.2-nightly-20251215
Release v1.0.0-beta.2-b60178160-20251212-1765553490
What's Changed
- feat: allow publishing new nightly release when some platforms are absent by @waynexia in #7354
- fix: use saturating in gc tracker by @discord9 in #7369
- refactor: use versioned index for index file by @discord9 in #7309
- refactor: extract file watcher to common-config by @shuiyisong in #7357
- chore(mito): expose some symbols by @v0y4g3r in #7373
- feat: allow one to many VRL pipeline by @v0y4g3r in #7342
- chore: saturating duration since by @discord9 in #7380
- fix: part sort behavior by @waynexia in #7374
- feat: collect merge and dedup metrics by @evenyag in #7375
- fix: improve network failure detection by @WenyXu in #7382
- refactor: remove duplication coverage and code from window sort tests by @waynexia in #7384
- feat: suspend frontend and datanode by @MichaelScofield in #7370
- feat: table/column/flow COMMENT by @waynexia in #7060
- fix(procedure): update procedure state correctly during execution and on failure by @WenyXu in #7376
- feat: grafana postgresql data source query builder support by @sunng87 in #7379
- fix: gc listing op first by @discord9 in #7385
- fix: parse "KEEP FIRING FOR" by @fengys1996 in #7386
- feat: mark index outdated by @discord9 in #7383
- refactor: optimize heartbeat channel and etcd client keepalive settings by @WenyXu in #7390
- chore(mito): nit avoid clone the batch object on inverted index building by @lyang24 in #7388
- test: test_tracker_cleanup skip non linux by @discord9 in #7398
- feat: mem manager on compaction by @fengjiachun in #7305
- fix: promql histogram with aggregation by @waynexia in #7393
- fix: promql offset direction by @waynexia in #7392
- feat: optimize and fix part sort on overlapping time windows by @waynexia in #7387
Full Changelog: v1.0.0-beta.2-8af1ab0ff-20251209-1765256263...v1.0.0-beta.2-b60178160-20251212-1765553490
Release v1.0.0-beta.2-8af1ab0ff-20251209-1765256263
What's Changed
Full Changelog: v1.0.0-beta.2-8af1ab0ff-20251208-1765190404...v1.0.0-beta.2-8af1ab0ff-20251209-1765256263
Release v1.0.0-beta.2-8af1ab0ff-20251208-1765190404
What's Changed
- refactor(servers): allow custom flight service by @v0y4g3r in #7333
- fix: reset cached channel on errors with VIP by @fengjiachun in #7335
- chore: members and committers update by @fengjiachun in #7341
- refactor(servers): bulk insert service by @v0y4g3r in #7329
- fix: configure HTTP/2 keep-alive for heartbeat client to detect network failures faster by @WenyXu in #7344
- refactor: remove datafusion data frame wrapper by @waynexia in #7347
- perf: treat DISTINCT as comm/part-comm by @waynexia in #7348
- test: gc worker scheduler mock test by @discord9 in #7292
- feat: implement a cache for manifest files by @evenyag in #7326
- feat: add more verbose metrics to scanners by @evenyag in #7336
- fix: regression with shortcutted statement on postgres extended query by @sunng87 in #7340
- chore: rm unnecessary warning by @discord9 in #7352
- feat: decode pk eagerly by @waynexia in #7350
- feat: run histogram quantile in safe mode for incomplete data by @waynexia in #7297
- chore: rename to avoid git reserved name by @discord9 in #7359
Full Changelog: v1.0.0-beta.2...v1.0.0-beta.2-8af1ab0ff-20251208-1765190404
Release v1.0.0-beta.2
v1.0.0-beta.2
Release date: December 03, 2025
Breaking changes
- fix!: align numeric type aliases with PostgreSQL and MySQL by @killme2008 in #7270
- feat!: improve mysql/pg compatibility by @killme2008 in #7315
- perf(metric-engine)!: Replace mur3 with fxhash for faster TSID generation by @v0y4g3r in #7316
👍 Highlights
🚀 Key New Features
- Region Migration:
- Introduced batch region migration
- JSON Handling:
- New UDF:
json_get_object.
- New UDF:
- Operations:
- Supported dynamic enabling or disabling of tracing.
- Enabled parallel table operations in
COPY DATABASE. - Supported the ability to alter database compaction options.
⚡ Performance Improvements
- Enhanced efficiency by parallelizing file source region and building partition sources in parallel.
- Optimized query execution by avoiding unnecessary merge sort.
- Implemented tracking for the query memory pool.
🐛 Notable Bug Fixes
- Fixed a critical write stall issue that prevented recovery due to flush logic problems.
- Resolved a deadlock in the metric engine when altering a group of tables.
- Fixed multiple compatibility issues, including corrections for PostgreSQL extended query parameter parsing, timezone settings, and MySQL binary date type handling.
🚀 Features
- feat: load latest index file first by @evenyag in #7221
- feat: introduce batch region migration by @WenyXu in #7176
- feat: build partition sources in parallel by @waynexia in #7243
- feat: dynamic enable or disable trace by @waynexia in #6609
- feat: support parallel table operations in
COPY DATABASEby @WenyXu in #7213 - feat: gc worker only local regions&test by @discord9 in #7203
- feat: implement compressed CSV/JSON export functionality by @McKnight22 in #7162
- feat: split batches before merge by @evenyag in #7225
- feat: gc scheduler ctx&procedure by @discord9 in #7252
- feat: gc get ref from manifest by @discord9 in #7260
- feat: support alter database compaction options by @WaterWhisperer in #7251
- feat: don't validate external table's region schema by @waynexia in #7268
- feat: simplify merge scan stream by @waynexia in #7269
- feat: udf
json_get_objectby @MichaelScofield in #7241 - feat: support Dictionary type by @waynexia in #7277
- feat: simplify file_stream::create_stream by @waynexia in #7275
- feat: track query memory pool by @fengjiachun in #7219
- feat: basic gc scheduler by @discord9 in #7263
- feat: reloadable tls client config by @shuiyisong in #7230
- feat:
decode_primary_keymethod for debugging by @waynexia in #7284 - feat: update dashboard to v0.11.8 by @ZonaHex in #7281
- feat: introduce
remap_manifestsforRegionEngineby @WenyXu in #7265 - feat: implement manual type for async index build by @SNC123 in #7104
- feat: add building option to build images base on distroless image by @WaterWhisperer in #7240
- feat: batch region migration for failover by @WenyXu in #7245
- feat: add batch gc procedure by @discord9 in #7296
- feat: add region repartition group procedure infrastructure by @WenyXu in #7299
- feat: add some configurable points by @fengys1996 in #7227
- feat: simple read write new json type values by @MichaelScofield in #7175
- feat: introduce
EnterStagingRequestforRegionEngineby @WenyXu in #7261 - feat: implement metadata update for repartition group procedure by @WenyXu in #7311
- feat: implement
Displaytrait forFlushRegionsby @WenyXu in #7320 - feat: update pg-catalog for describe table by @sunng87 in #7321
🐛 Bug Fixes
- fix: obtain system time after fetching lease values by @WenyXu in #7223
- fix: clone the page before putting into the index cache by @evenyag in #7229
- fix: correct signature of current_schemas function by @sunng87 in #7233
- fix: allow compacting L1 files under append mode by @evenyag in #7239
- fix: dynamic reload tracing layer loses trace id by @waynexia in #7257
- fix: log not print by @killme2008 in #7272
- fix: return sqlalchemy compatible version string in version() by @sunng87 in #7271
- fix: postgres extended query paramater parsing and type check by @sunng87 in #7276
- fix: unlimit
trace_idquery in jaeger API by @shuiyisong in #7283 - fix: postgres show statement describe and timestamp text parsing by @sunng87 in #7286
- fix: postgres timezone setting by default by @killme2008 in #7289
- fix: pre-commit all files failed by @yihong0618 in #7290
- fix: mysql binary date type and multi-lang ci tests by @killme2008 in #7291
- fix: partition tree metric should the delta by @evenyag in #7307
- fix: implement bulk write for time partitions and bulk memtable by @evenyag in #7293
- fix: metric engine deadlock when altering a group of tables by @waynexia in #7308
- fix: request limiter test case fix by @fengjiachun in #7323
- fix: fix write stall that never recovers due to flush logic issues by @WenyXu in #7322
🚜 Refactor
- refactor: remove
export_metricsand related configuration by @WaterWhisperer in #7236 - refactor: create
JsonValuefor json value by [@...
Release v1.0.0-beta.2-nightly-20251201
What's Changed
- refactor: remove
export_metricsand related configuration by @WaterWhisperer in #7236 - ci: dev-build with large page size by @MichaelScofield in #7228
- refactor: create
JsonValuefor json value by @MichaelScofield in #7214 - chore: bump version to beta.2 by @discord9 in #7238
- feat: build partition sources in parallel by @waynexia in #7243
- feat: dynamic enable or disable trace by @waynexia in #6609
- feat: support parallel table operations in
COPY DATABASEby @WenyXu in #7213 - fix: allow compacting L1 files under append mode by @evenyag in #7239
- build: update opensrv-mysql to 0.10 by @killme2008 in #7246
- feat: gc worker only local regions&test by @discord9 in #7203
- refactor: make
show tablesfast under large tables by @MichaelScofield in #7231 - feat: implement compressed CSV/JSON export functionality by @McKnight22 in #7162
- feat: split batches before merge by @evenyag in #7225
- ci: update review code owners by @evenyag in #7250
- fix: dynamic reload tracing layer loses trace id by @waynexia in #7257
- feat: gc scheduler ctx&procedure by @discord9 in #7252
- feat: gc get ref from manifest by @discord9 in #7260
- feat: support alter database compaction options by @WaterWhisperer in #7251
- chore: add tests for election reset and region lease failure handling by @WenyXu in #7266
- fix: log not print by @killme2008 in #7272
- feat: don't validate external table's region schema by @waynexia in #7268
- fix: return sqlalchemy compatible version string in version() by @sunng87 in #7271
- feat: simplify merge scan stream by @waynexia in #7269
- feat: udf
json_get_objectby @MichaelScofield in #7241 - fix!: align numeric type aliases with PostgreSQL and MySQL by @killme2008 in #7270
- chore: add default value to sparse_primary_key_encoding config item by @fengys1996 in #7273
- refactor: remove
Vectors fromRecordBatchcompletely by @MichaelScofield in #7184 - perf: avoid unnecessary merge sort by @waynexia in #7274
- feat: support Dictionary type by @waynexia in #7277
- feat: simplify file_stream::create_stream by @waynexia in #7275
- docs(config): clarify
store_addrsformat by @fengys1996 in #7279 - fix: postgres extended query paramater parsing and type check by @sunng87 in #7276
- refactor: make json value use json type by @MichaelScofield in #7248
- fix: unlimit
trace_idquery in jaeger API by @shuiyisong in #7283 - feat: track query memory pool by @fengjiachun in #7219
- feat: basic gc scheduler by @discord9 in #7263
- refactor: pub
HttpOutputWriterfor external use by @MichaelScofield in #7287 - chore: add
INFORMATION_SCHEMA_ALERTS_TABLE_IDconst value by @fengys1996 in #7288 - perf: parallelize file source region by @waynexia in #7285
- feat: reloadable tls client config by @shuiyisong in #7230
- feat:
decode_primary_keymethod for debugging by @waynexia in #7284 - feat: update dashboard to v0.11.8 by @ZonaHex in #7281
- fix: postgres show statement describe and timestamp text parsing by @sunng87 in #7286
- fix: postgres timezone setting by default by @killme2008 in #7289
- fix: pre-commit all files failed by @yihong0618 in #7290
- fix: mysql binary date type and multi-lang ci tests by @killme2008 in #7291
- feat: introduce
remap_manifestsforRegionEngineby @WenyXu in #7265 - feat: implement manual type for async index build by @SNC123 in #7104
- chore: return meaningful message when content type mismatch in otel by @killme2008 in #7301
- ci: add multi lang tests workflow into release and nightly workflows by @killme2008 in #7300
- feat: add building option to build images base on distroless image by @WaterWhisperer in #7240
- chore: optimize search traces from Grafana by @shuiyisong in #7298
- feat: batch region migration for failover by @WenyXu in #7245
- refactor: load metadata using offical impl by @discord9 in #7302
- chore: return 404 if trace not found by @shuiyisong in #7304
- fix: partition tree metric should the delta by @evenyag in #7307
- feat: add batch gc procedure by @discord9 in #7296
- feat: add region repartition group procedure infrastructure by @WenyXu in #7299
- fix: implement bulk write for time partitions and bulk memtable by @evenyag in #7293
- feat: add some configurable points by @fengys1996 in #7227
- fix: metric engine deadlock when altering a group of tables by @waynexia in #7308
- feat: simple read write new json type values by @MichaelScofield in #7175
- feat: introduce
EnterStagingRequestforRegionEngineby @WenyXu in #7261
New Contributors
- @McKnight22 made their first contribution in #7162
Full Changelog: v1.0.0-beta.1-nightly-20251117...v1.0.0-beta.2-nightly-20251201
Release v1.0.0-beta.1-nightly-20251117
What's Changed
- chore: allow unlimited return if timerange is applied by @shuiyisong in #7222
- docs: update project status and tweak readme by @killme2008 in #7216
- chore: add debug log on receiving logs by @shuiyisong in #7211
- feat: load latest index file first by @evenyag in #7221
- fix: obtain system time after fetching lease values by @WenyXu in #7223
- feat: introduce batch region migration by @WenyXu in #7176
- chore: add
tls-watchoption in cmd by @shuiyisong in #7226 - ci: update helm-charts and homebrew-greptime pull request reviewer by @daviderli614 in #7232
- fix: clone the page before putting into the index cache by @evenyag in #7229
- chore: fix SQLness test for
COPYcommand from CSV file by @Standing-Man in #7235 - fix: correct signature of current_schemas function by @sunng87 in #7233
Full Changelog: v1.0.0-beta.1...v1.0.0-beta.1-nightly-20251117
Release v1.0.0-beta.1
Release date: November 11, 2025
🚨 Breaking changes
- refactor(pipeline)!: change dispatch table name format by @paomian in #6901
- feat!: improve
greptime_identitypipeline behavior by @waynexia in #6932 - refactor!: add
enable_read_cacheconfig to support disable read cache explicitly by @zyy17 in #6834 - refactor!: remove pb_value to json conversion, keep json output consistent by @sunng87 in #7063
- refactor!: unify the API of getting total cpu and memory by @zyy17 in #7049
- refactor!: add a
opentelemetry_traces_operationstable to aggregate(service_name, span_name, span_kind)to improve query performance by @zyy17 in #7144 - feat(metric)!: enable sparse primary key encoding by default by @WenyXu in #7195
👍 Highlights
Dashboard v0.11.7 Released
- Metrics UI optimization: separated table and chart tabs, independent instant/range queries, support for time picker and multi-value display.
- Timezone optimization: support for timezone validation and local storage persistence.
- Flow management: UI for flow task CRUD operations.
Bulk Memtable
For scenarios with high cardinality primary keys, this release introduces experimental Bulk Memtable and a new data organization format (flat format). Both must be used together. Bulk Memtable offers lower memory usage when dealing with high cardinality primary keys. When primary key cardinality exceeds two million, memory usage can be reduced by more than 75%. Currently, Bulk Memtable performs better with larger write batch sizes, and we recommend setting batch sizes to 1024 rows or more when using Bulk Memtable. Additionally, the new data organization format provides better query performance in high cardinality scenarios compared to the original format.
Users can enable the new data format and Bulk Memtable by specifying sst_format as flat when creating tables.
CREATE TABLE flat_format_table(
request_id STRING,
content STRING,
greptime_timestamp TIMESTAMP TIME INDEX,
PRIMARY KEY (request_id))
WITH ('sst_format' = 'flat');Additionally, for tables using the old format, you can switch to the flat format and Bulk Memtable using an ALTER statement.
ALTER TABLE old_format_table SET 'sst_format' = 'flat';Tables using the flat format cannot be converted back to the old format. We will gradually switch the default format to the new format in future releases.
Independent Index File Caching
This release implements independent local caching for index files on object storage, allowing index files to be cached in local disk cache as much as possible, reducing the probability of accessing object storage during index queries. By default, the database allocates 20% of disk cache space to index files. Users can adjust this ratio by setting the index_cache_percent parameter.
In previous versions, when users increased the local disk cache size, only newly generated data files could enter the local write cache, providing limited improvement for querying historical data. In this version, the database loads index files from object storage to local storage in the background after startup, reducing the time required for historical data queries.
Read-Write Permission Control Mode
This release introduces permission mode support for the static user provider, implementing read-write access control with support for read-only, write-only, and read-write permissions. This enhancement enables administrators to create users with specific access levels, improving security and data governance.
The static user provider now accepts the format username:permission_mode=password, where the optional permission_mode can be:
rw(read-write): Full access to read and write operationsro(read-only): Restricted to read operations onlywo(write-only): Restricted to write operations only
Examples:
# Read-write user (default, backward compatible)
greptime_user=greptime_pwd
# Or explicitly specified
greptime_user:rw=greptime_pwd
# Read-only user
greptime_user:ro=greptime_pwd
# Write-only user
greptime_user:wo=greptime_pwd
This feature is fully backward compatible with the old format - users without an explicitly specified permission mode default to read-write access (rw).
TQL Supports Value Aliasing
TQL now supports AS aliases for clearer column names and easier SQL integration.
TQL EVAL (0, 30, '10s') http_requests_total AS requests;New objbench Subcommand (Datanode)
This release adds the greptime datanode objbench subcommand for conducting read/write performance benchmarks on specified SST files in object storage. This tool can be used to analyze storage layer performance, troubleshoot slow queries or I/O latency issues, and supports generating flame graphs for deeper performance diagnostics.
Main Features
- Perform read/write performance tests on individual SST files
- Support detailed output (-v/--verbose)
- Support generating SVG flame graphs (--pprof-file)
- Can load datanode configuration files (--config)
# Basic test
greptime datanode objbench --config datanode.toml --source <path>.parquet
# Generate flame graph
greptime datanode objbench --config datanode.toml --source <path>.parquet --pprof-file flamegraph.svg🚀 Features
- feat: Update parquet writer and indexer to support the flat format by @evenyag in #6866
- feat: unify FlushRegions instructions by @aaraujo in #6819
- feat: implement basic write/read methods for bulk memtable by @evenyag in #6888
- feat: Supports flat format in SeqScan and UnorderedScan by @evenyag in #6905
- feat: put sqlness into a separated dir by @waynexia in #6911
- feat: humanize analyze numbers by @waynexia in #6889
- feat: file ref mgr by @discord9 in #6844
- feat: add
written_bytes_since_opencolumn toregion_statisticstable by @WenyXu in #6904 - feat: support function alias by @MichaelScofield in #6917
- feat: add CPU, memory and node status info to
cluster_infoby @WenyXu in #6897 - feat: add udtf (table function) registration by @sunng87 in #6922
- feat(pipeline): generate create table sql from pipeline config by @shuiyisong in #6930
- feat(mito): backfill partition expr on region open by @zhongzc in #6862
- feat: add InformationExtension.inspect_datanode for datanode inspection by @zhongzc in #6921
- feat: Implements compaction for bulk memtable by @evenyag in #6923
- feat: add origin_region_id and node_id to sst entry by @zhongzc in #6937
- feat: store partition expr per file in region manifest by @waynexia in #6849
- feat: exiting staging mode on success case by @waynexia in #6913
- feat: expose workload filter to selector options by @WenyXu in #6951
- feat: support flat format for SeriesScan by @evenyag in #6938
- feat: support flush and compact flat format files by @evenyag in #6949
- feat: add visible to sst entry for staging mode by @zhongzc in #6964
- feat: add an flag to enable the experimental flat format by @evenyag in #6976
- feat: add TLS support for mysql backend by @WenyXu in #6979
- feat: datanode side local gc worker by @discord9 in #6940
- feat: extract standalone...