You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Postgres simple-query path materializes the entire result set in memory before sending (OOM on a large SELECT) #6679
The Postgres simple-query ('Q') path materializes the entire result set into a List<Result> in memory before writing the first DataRow, so a large SELECT over the simple protocol can OOM the server. The Bolt executor streams; the Postgres extended protocol is bounded by the portal max-rows; only the simple-query path is unbounded.
Where (HEAD 86a704526)
postgresw/src/main/java/com/arcadedb/postgres/PostgresNetworkExecutor.java:670 → final List<Result> cachedResultSet = browseAndCacheResultSet(resultSet, 0); (limit 0 = unlimited) then writeRowDescription(...) / writeDataRows(cachedResultSet, ...). All row objects are held simultaneously before the first byte is sent.
Repro
A plain SELECT over the simple query protocol (e.g. psql -c "SELECT FROM HugeType") against a large table → the whole result is buffered into an ArrayList server-side before streaming, risking OutOfMemoryError. The Bolt path (handlePull) streams row-by-row and does not exhibit this (verified).
Suggested fix
For the 'Q' path, iterate the ResultSet and write DataRows incrementally instead of pre-caching with limit 0; or cap with a configurable fetch bound.
Scope / confidence
High that everything is materialized (read at HEAD). Honest caveat: this is a long-standing, documented design (the browse-cache predates the #4197 audit and is deliberate for portal semantics), so it may be considered known/intended — reporting because it is a real "whole result materialized → OOM on a big simple-query SELECT" gap and the sibling transports avoid it.
A very large SELECT over the Postgres wire protocol (simple query mode) could use unbounded memory. There's now a configurable limit (arcadedb.postgres.simpleQueryMaxRows, default 1,000,000 rows) - a result larger than that is refused with a clear error instead of risking a server crash. For very large result sets, use the extended query protocol with a bounded fetch size instead.
❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.
Re-validated against main at c50338bd8: the 'Q' path is properly bounded now. The extended-protocol sibling (:415, :530) still buffers unbounded, and the new config doc-text recommends that path as the remedy - filed as a follow-up.
Activity
review: strengthen #6665 count assertions, stop scanning early on #6679…
review(#6679): prove the socket stays usable after a refused oversize…
review(#6679): assert no DataRow leaks before a refused oversized result
Merge pull request #6693 from ArcadeData/fix/6665-6680-6679-6676-6671…
lvca commentedon Aug 24, 2026
Fixed in #6693 (merged to main).
A very large
SELECTover the Postgres wire protocol (simple query mode) could use unbounded memory. There's now a configurable limit (arcadedb.postgres.simpleQueryMaxRows, default 1,000,000 rows) - a result larger than that is refused with a clear error instead of risking a server crash. For very large result sets, use the extended query protocol with a bounded fetch size instead.❤️ Thanks for helping make ArcadeDB better! ArcadeDB is free and open source, sustained by its community. If it's useful to you or your company, please consider becoming a sponsor to keep development going.
fix(ArcadeData#6680,ArcadeData#6679,ArcadeData#6676,ArcadeData#6671):…
ruispereira commentedon Sep 2, 2026
Re-validated against
mainatc50338bd8: the 'Q' path is properly bounded now. The extended-protocol sibling (:415,:530) still buffers unbounded, and the new config doc-text recommends that path as the remedy - filed as a follow-up.