Skip to content

Commit f7c1db7

Browse files
committed
format
1 parent 76b3957 commit f7c1db7

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+3080
-2917
lines changed

.journal/2025-01-27.md

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,18 @@
33
## Breaking Change: IoInterface Redesign for Transferable Objects Support
44

55
### Core Decision/Topic
6+
67
Redesigned the `IoInterface` contract to support transferable objects and structured clone messaging, moving from centralized buffer handling to adapter-level data format conversion.
78

89
### Options Considered
910

1011
1. **Keep centralized conversion** (main branch approach)
12+
1113
- Pros: No breaking changes, simple for adapters
1214
- Cons: Can't support transferables, type contracts are unclear, hidden complexity
1315

1416
2. **Move conversion to adapters** (chosen approach)
17+
1518
- Pros: Clean separation of concerns, enables transferables, explicit type contracts
1619
- Cons: Breaking change, requires adapter updates
1720

@@ -24,6 +27,7 @@ Redesigned the `IoInterface` contract to support transferable objects and struct
2427
**Chose option 2**: Move data format conversion responsibility to adapters.
2528

2629
**Rationale:**
30+
2731
- Enables transferable objects feature (primary goal)
2832
- Cleaner architecture with explicit responsibilities
2933
- Better type safety and clearer contracts
@@ -33,27 +37,30 @@ Redesigned the `IoInterface` contract to support transferable objects and struct
3337
### Key Changes Made
3438

3539
#### Interface Changes (BREAKING)
40+
3641
```typescript
3742
// OLD
3843
interface IoInterface {
39-
read(): Promise<Uint8Array | string | null>
40-
write(data: string): Promise<void>
44+
read(): Promise<Uint8Array | string | null>
45+
write(data: string): Promise<void>
4146
}
4247

43-
// NEW
48+
// NEW
4449
interface IoInterface {
45-
read(): Promise<string | IoMessage | null>
46-
write(message: string | IoMessage): Promise<void>
47-
capabilities?: IoCapabilities
50+
read(): Promise<string | IoMessage | null>
51+
write(message: string | IoMessage): Promise<void>
52+
capabilities?: IoCapabilities
4853
}
4954
```
5055

5156
#### Channel Changes
57+
5258
- Removed centralized `TextDecoder` buffer conversion
5359
- Now expects clean string/IoMessage from adapters
5460
- Added support for structured clone and transferable objects
5561

5662
#### Adapter Updates Required
63+
5764
- **WebSocket**: Added Buffer→string conversion in `onmessage` handlers
5865
- **All adapters**: Must now handle their own data format conversion
5966
- **New adapters**: Can declare capabilities for transferables
@@ -63,19 +70,21 @@ interface IoInterface {
6370
**Severity: MAJOR** (requires major version bump)
6471

6572
**Affected:**
73+
6674
- All custom `IoInterface` implementations
6775
- Any code depending on `Uint8Array` return type from `read()`
6876
- Adapters that relied on centralized buffer conversion
6977

7078
**Migration Required:**
79+
7180
```typescript
7281
// OLD adapter
7382
async read(): Promise<Uint8Array | string | null> {
7483
const data = await this.source.read()
7584
return data // Could be Buffer/Uint8Array
7685
}
7786

78-
// NEW adapter
87+
// NEW adapter
7988
async read(): Promise<string | IoMessage | null> {
8089
const data = await this.source.read()
8190
// Must convert to string here
@@ -90,10 +99,11 @@ async read(): Promise<string | IoMessage | null> {
9099
**Root Cause:** The old design had a type lie - WebSocket adapter was typed as returning `string` but actually returned `Buffer` on server side.
91100

92101
**Solution:** Added explicit Buffer→string conversion in WebSocket adapters:
102+
93103
```typescript
94104
// Works in both Node.js and browsers
95105
if (typeof message === "object" && message !== null && "toString" in message) {
96-
message = message.toString("utf-8")
106+
message = message.toString("utf-8")
97107
}
98108
```
99109

@@ -121,4 +131,4 @@ if (typeof message === "object" && message !== null && "toString" in message) {
121131

122132
---
123133

124-
*This entry documents the architectural decision to redesign IoInterface for transferable objects support, including the breaking change impact and migration requirements.*
134+
_This entry documents the architectural decision to redesign IoInterface for transferable objects support, including the breaking change impact and migration requirements._

.journal/2025-11-02.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,17 @@
33
## Core Decision/Topic: Elysia WebSocket Adapter Development
44

55
### Overview
6+
67
Today marked significant progress on adding **Elysia WebSocket adapter support** to the kkrpc library, expanding our cross-runtime compatibility to include the modern TypeScript web framework Elysia.
78

89
### Options Considered
10+
911
1. **Elysia WebSocket Integration**: Adding native support for Elysia's built-in WebSocket functionality powered by uWebSocket
1012
2. **Standard WebSocket Compatibility**: Ensuring the adapter works with standard WebSocket API for broad compatibility
1113
3. **Bidirectional Communication**: Maintaining kkrpc's core principle of bi-directional RPC communication
1214

1315
### Final Decision & Rationale
16+
1417
**Decided to implement a comprehensive Elysia WebSocket adapter** with the following architecture:
1518

1619
- **ElysiaWebSocketServerIO**: Server-side adapter that integrates with Elysia's WebSocket lifecycle
@@ -22,18 +25,22 @@ Today marked significant progress on adding **Elysia WebSocket adapter support**
2225
### Key Changes Made
2326

2427
#### New Files Created:
28+
2529
- `packages/kkrpc/src/adapters/elysia-websocket.ts` - Main adapter implementation (470 lines)
2630
- `packages/kkrpc/__tests__/elysia-websocket.test.ts` - Comprehensive test suite (363 lines)
2731
- `packages/kkrpc/elysia-websocket.ts` - Public export file for the adapter
2832

2933
#### Core Features Implemented:
34+
3035
1. **Server-Side Integration**:
36+
3137
- Automatic message routing through Elysia's WebSocket handlers
3238
- Connection information extraction (remote address, query params, headers)
3339
- Graceful error handling and cleanup
3440
- Support for both `ws.raw` and standard WebSocket interfaces
3541

3642
2. **Client-Side Implementation**:
43+
3744
- Standard WebSocket API compatibility across runtimes (Bun/Deno/Node.js)
3845
- Automatic connection waiting with Promise-based readiness
3946
- Cross-platform WebSocket instantiation
@@ -46,35 +53,41 @@ Today marked significant progress on adding **Elysia WebSocket adapter support**
4653
- Bidirectional communication verification
4754

4855
#### Technical Implementation Details:
56+
4957
- **Message Processing**: Handles string, ArrayBuffer, and object messages with automatic conversion
5058
- **Reference Management**: Proper cleanup of WebSocket references to prevent memory leaks
5159
- **Capability Declaration**: Explicitly declares no support for structuredClone or transfer for now
5260
- **Destroy Signals**: Implements clean shutdown signaling mechanism
5361

5462
### Package Updates:
63+
5564
- Updated `packages/kkrpc/mod.ts` to include Elysia WebSocket exports
5665
- Modified `packages/kkrpc/package.json` dependencies for Elysia support
5766
- Updated build configuration in `tsdown.config.ts`
5867

5968
### Development Environment Setup:
69+
6070
- Added `.python-version` and Python-related files for potential cross-language tooling
6171
- Enhanced `.claude/` configuration for improved AI assistance
6272
- Updated `.gitignore` to exclude new development artifacts
6373

6474
### Future Considerations
75+
6576
1. **Performance Optimization**: Consider adding support for transferable objects and structured clone for zero-copy messaging
6677
2. **Documentation**: Add comprehensive examples to the main documentation
6778
3. **TypeScript Enhancements**: Improve type inference for Elysia-specific features
6879
4. **Error Handling**: Enhance error serialization across Elysia WebSocket boundaries
6980

7081
### Technical Debt & Remaining Work
82+
7183
- Need to update main README.md with Elysia examples
7284
- Consider adding Elysia-specific demos to the examples directory
7385
- Potential integration with Elysia's plugin system for even tighter integration
7486

7587
### Impact Assessment
88+
7689
This implementation significantly expands kkrpc's ecosystem compatibility, making it a compelling choice for developers using modern TypeScript frameworks. The adapter maintains the library's core principles while providing idiomatic Elysia integration patterns.
7790

7891
---
7992

80-
*Entry created: 2025-11-02T14:30:00Z*
93+
_Entry created: 2025-11-02T14:30:00Z_

.journal/2025-11-04-redis-streams-improvements.md

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
**问题**: 原实现的 `messageQueue` 没有大小限制,如果消息到达速度超过消费速度,可能导致内存溢出。
1212

1313
**解决方案**:
14+
1415
- 添加 `maxQueueSize` 配置选项(默认 1000 条消息)
1516
- 队列满时自动丢弃最老的消息,并记录警告日志
1617
- 提示用户考虑增加 `maxQueueSize` 或加快消息处理速度
@@ -27,6 +28,7 @@ interface RedisStreamsOptions {
2728
```
2829

2930
**实现细节**:
31+
3032
```typescript
3133
private handleMessage(message: string): void {
3234
// ...
@@ -47,29 +49,31 @@ private handleMessage(message: string): void {
4749
**问题**: 原实现没有对配置选项进行验证,可能导致运行时错误。
4850

4951
**解决方案**:
52+
5053
- 添加 `validateOptions` 方法,在构造函数中验证所有配置
5154
- 验证类型、范围、整数要求等
5255
- 无效配置立即抛出异常,提供清晰的错误信息
5356

5457
**验证规则**:
58+
5559
- `blockTimeout`: 必须是非负整数
5660
- `maxLen`: 必须是正整数
5761
- `maxQueueSize`: 必须是正整数
5862
- `url`, `stream`, `consumerGroup`, `consumerName`: 必须是字符串
5963

6064
```typescript
6165
private validateOptions(options: RedisStreamsOptions): void {
62-
if (options.blockTimeout !== undefined &&
66+
if (options.blockTimeout !== undefined &&
6367
(options.blockTimeout < 0 || !Number.isInteger(options.blockTimeout))) {
6468
throw new Error("blockTimeout must be a non-negative integer")
6569
}
6670

67-
if (options.maxLen !== undefined &&
71+
if (options.maxLen !== undefined &&
6872
(options.maxLen <= 0 || !Number.isInteger(options.maxLen))) {
6973
throw new Error("maxLen must be a positive integer")
7074
}
7175

72-
if (options.maxQueueSize !== undefined &&
76+
if (options.maxQueueSize !== undefined &&
7377
(options.maxQueueSize <= 0 || !Number.isInteger(options.maxQueueSize))) {
7478
throw new Error("maxQueueSize must be a positive integer")
7579
}
@@ -83,6 +87,7 @@ private validateOptions(options: RedisStreamsOptions): void {
8387
**问题**: 原实现只支持 XREAD(pub/sub 模式),在高吞吐量场景下可能不够高效。
8488

8589
**解决方案**:
90+
8691
- 添加 `useConsumerGroup` 配置选项
8792
- 支持两种消息消费模式:
8893
- **Pub/Sub 模式**(默认,`useConsumerGroup: false`): 使用 XREAD,所有 consumer 都能收到所有消息
@@ -101,6 +106,7 @@ interface RedisStreamsOptions {
101106
```
102107

103108
**实现细节**:
109+
104110
- 在 pub/sub 模式下,使用 XREAD,不创建 consumer group
105111
- 在 consumer group 模式下,使用 XREADGROUP,自动创建 consumer group,并在处理完消息后 ACK
106112
- 根据模式选择不同的连接初始化和消息读取逻辑
@@ -139,10 +145,12 @@ else {
139145
添加了完整的测试覆盖:
140146

141147
1. **配置验证测试**:
148+
142149
- 测试所有无效配置会抛出正确的错误
143150
- 测试有效配置能正常创建 adapter
144151

145152
2. **内存管理测试**:
153+
146154
- 测试队列大小限制功能
147155
- 验证队列满时正确丢弃老消息
148156

@@ -155,15 +163,15 @@ else {
155163
```typescript
156164
/**
157165
* Redis Streams implementation of IoInterface
158-
*
166+
*
159167
* 支持两种消息消费模式:
160168
* 1. Pub/Sub 模式 (默认): 使用 XREAD,所有 consumer 都能收到所有消息
161169
* 2. Consumer Group 模式: 使用 XREADGROUP,每条消息只被一个 consumer 处理 (负载均衡)
162-
*
170+
*
163171
* 内存管理:
164172
* - 支持最大队列大小限制 (maxQueueSize),防止消息积压导致内存问题
165173
* - 队列满时自动丢弃最老的消息并记录警告
166-
*
174+
*
167175
* 配置验证:
168176
* - 构造时验证所有配置选项的类型和范围
169177
* - 无效配置会立即抛出异常
@@ -173,6 +181,7 @@ else {
173181
## 使用示例
174182

175183
### 基本用法(Pub/Sub 模式)
184+
176185
```typescript
177186
const io = new RedisStreamsIO({
178187
url: "redis://localhost:6379",
@@ -182,6 +191,7 @@ const io = new RedisStreamsIO({
182191
```
183192

184193
### Consumer Group 模式(负载均衡)
194+
185195
```typescript
186196
const io = new RedisStreamsIO({
187197
url: "redis://localhost:6379",
@@ -196,13 +206,15 @@ const io = new RedisStreamsIO({
196206
## 测试结果
197207

198208
所有新测试通过:
209+
199210
- ✅ 配置验证测试(8 个测试用例)
200211
- ✅ 内存管理测试(队列大小限制)
201212
- ✅ Consumer Group 模式测试(2 个场景)
202213

203214
## 总结
204215

205216
这些改进使 Redis Streams adapter 更加健壮和灵活:
217+
206218
1. **更安全**: 配置验证防止运行时错误,队列大小限制防止内存溢出
207219
2. **更灵活**: 支持两种消息消费模式,适应不同的使用场景
208220
3. **更可靠**: 完整的测试覆盖确保功能正常工作
@@ -211,8 +223,8 @@ const io = new RedisStreamsIO({
211223
## 未来考虑
212224

213225
Code review 还提到的其他建议(暂未实现):
226+
214227
- **Enhanced Error Scenarios**: 添加更多网络故障、Redis 不可用等错误场景的测试
215228
- **Connection Pooling**: 实现连接池以提高资源管理效率
216229
- **Monitoring**: 添加 metrics 和监控钩子
217230
- **Dead Letter Queues**: 支持失败消息处理的 DLQ
218-

0 commit comments

Comments
 (0)