Skip to content

perf: reduce unnecessary serialization in KLL sketch - #132

Merged
proost merged 2 commits into
apache:mainfrom
proost:perf-reduce-serialization
Mar 6, 2026
Merged

proost merged 2 commits into
apache:mainfrom
proost:perf-reduce-serialization

Conversation

@proost

@proost proost commented Mar 5, 2026

Copy link
Copy Markdown
Member

We can reduce serialization twice to once.

benchmark code:

func BenchmarkGetSerializedSizeBytes(b *testing.B) {
	comparator := common.ItemSketchStringComparator(false)
	serde := common.ItemSketchStringSerDe{}
	sketch, _ := NewKllItemsSketch[string](_DEFAULT_K, _DEFAULT_M, comparator, serde)
	for i := 0; i < 10000; i++ {
		sketch.Update(fmt.Sprintf("%05d", i))
	}
	b.ReportAllocs()
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		sketch.GetSerializedSizeBytes()
	}
}

AS-IS:

goos: darwin
goarch: arm64
pkg: github.com/apache/datasketches-go/kll
cpu: Apple M4 Pro
BenchmarkGetSerializedSizeBytes
BenchmarkGetSerializedSizeBytes-12    	       1	1372407375 ns/op	   32224 B/op	     542 allocs/op
PASS

TO-BE:

goos: darwin
goarch: arm64
pkg: github.com/apache/datasketches-go/kll
cpu: Apple M4 Pro
BenchmarkGetSerializedSizeBytes
BenchmarkGetSerializedSizeBytes-12    	 1000000	      1003 ns/op	       0 B/op	       0 allocs/op
PASS

@proost proost self-assigned this Mar 5, 2026
@proost
proost merged commit d5f6e04 into apache:main Mar 6, 2026
@proost
proost deleted the perf-reduce-serialization branch March 6, 2026 03:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant