create table on duckdb 0.33-dev105 web shell and count() ok, but cannot run sum()

[l1t1]

create table on duckdb 0.33-dev105 web shell and count() ok, but cannot run sum(), it reported tried to read past the end
reproduction

DuckDB Web Shell
Database: v0.3.3-dev105
Package:  @duckdb/[email protected]

Connected to a transient in-memory database.
Enter .help for usage hints.

duckdb> .timer on
Timer enabled

duckdb> create table t as select range i from range(1000000000);
┌────────────┐
│ Count      │
╞════════════╡
│ 1000000000 │
└────────────┘
Elapsed: 15.654 s

duckdb> describe t;
┌───────┬────────┬──────┬─────┬─────────┬───────┐
│ Field ┆ Type   ┆ Null ┆ Key ┆ Default ┆ Extra │
╞═══════╪════════╪══════╪═════╪═════════╪═══════╡
│ i     ┆ BIGINT ┆ YES  ┆     ┆         ┆       │
└───────┴────────┴──────┴─────┴─────────┴───────┘
Elapsed: 3 ms

duckdb> select sum(i) from t;
tried to read past the end

duckdb> select count(*) from t;
┌──────────────┐
│ count_star() │
╞══════════════╡
│   1000000000 │
└──────────────┘
Elapsed: 02.598 s

duckdb> select sum(i) from t where i<100000;
tried to read past the end

and duckdb32 on windows is ok.

D:\duckdb>duckdb32
v0.3.2 5aebf7dac
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D .timer on
D create table t as select range i from range(1000000000);
Run Time: real 16.436 user 11.263272 sys 5.148033
D select sum(i) from t;
┌────────────────────┐
│       sum(i)       │
├────────────────────┤
│ 499999999500000000 │
└────────────────────┘
Run Time: real 1.008 user 22.354943 sys 0.015600
D select avg(i) from t;
┌─────────────┐
│   avg(i)    │
├─────────────┤
│ 499999999.5 │
└─────────────┘
Run Time: real 1.081 user 23.634152 sys 0.015600
D select count(i) from t;
┌────────────┐
│  count(i)  │
├────────────┤
│ 1000000000 │
└────────────┘
Run Time: real 0.969 user 20.966534 sys 0.000000

[ankoh]

Hi @l1t1, thanks for reporting this!

This is a limitation that we can't work around at the moment.
1'000'000'000 bigints will take 8GB of space uncompressed which exceeds the wasm heap size limits of 4GB.
We can certainly do a bit better with the reported error here.

We may be able to support this in the future through persistent writes to disk but that's not possible today.

[ankoh]

6962efe will change the error to:

Out of Memory Error: could not allocate block of 262144 bytes
Database is launched in in-memory mode and no temporary directory is specified.
Unused blocks cannot be offloaded to disk.

[ankoh]

Yes, that's also expected.
DuckDB-Wasm doesn't support tables larger than the wasm heap limits today.
The tmp filesystem in faked in the browser and won't work here.

We're waiting for this to be resolved:
whatwg/fs#7

We can work towards spooling to disk with DuckDB-Wasm but (for now) only if browsers start supporting a fully synchronous filesystem api.