Seems to fail in ip-location-api/src/db.mjs:500
After processing the same netblock at line 499,956 of GeoLite2-Country-Blocks-IPv4.csv, so likely a change in how buffers alloc/write could of been changed at 1.2.0?

Notes

• Fails at the same netblock on 1.2.0 and 1.2.7

Around

if(ipv4){
        buffer1 = Buffer.allocUnsafe(4)
        buffer1.writeUInt32LE(start)
        buffer2 = Buffer.allocUnsafe(4)
        buffer2.writeUInt32LE(end)
} else {
        buffer1 = Buffer.allocUnsafe(8)
        buffer1.writeBigUInt64LE(start)
        buffer2 = Buffer.allocUnsafe(8)
        buffer2.writeBigUInt64LE(end)
}
buffer3 = Buffer.allocUnsafe(2)
buffer3.write(cc)
if(preBuffer1){
        if(!ws1.write(preBuffer1)) rs.pause()
        if (setting.smallMemory) {
                ws = createSmallMemoryFile(ws, ipv4, lineCount++, preBuffer2, preBuffer3)
        } else {
                if(!ws2.write(preBuffer2)) rs.pause()
                if(!ws3.write(preBuffer3)) rs.pause()
        }
}

Try replacing Buffer.allocUnsafe() with Buffer.alloc().

This will zero-fill memory to determine if it is a memory handling issue:

buffer= buffer.alloc(4)
buffer1.writeUint32LE(start)

@Fendaril just tried your suggestion, replaced each of the .allocUnsafe with .alloc and same thing. Interestingly when processing each of them it gets to the point that it pauses, then it waits 5-6s before exiting, so it could be hanging, either internally in bun, or something else in ip-location-api.
Its exit code is 0

Try this to see exactly where logs stop.

console.log(`[${lineCount}] Processing: ${start} - ${end}`); 

if (ipv4) {
    buffer1 = Buffer.alloc(4);
    buffer1.writeUInt32LE(start);
    buffer2 = Buffer.alloc(4);
    buffer2.writeUInt32LE(end);
} else {
    buffer1 = Buffer.alloc(8);
    buffer1.writeBigUInt64LE(start);
    buffer2 = Buffer.alloc(8);
    buffer2.writeBigUInt64LE(end);
}

console.log(`[${lineCount}] Buffers allocated, writing to stream...`);

// Add logging before writes
console.log(`[${lineCount}] Writing preBuffer1`);
if (!ws1.write(preBuffer1)) {
    console.log(`[${lineCount}] Pausing stream...`);
    rs.pause();
    ws1.once('drain', () => {
        console.log(`[${lineCount}] Resuming stream`);
        rs.resume();
    });
}

// Similar logs for other writes
console.log(`[${lineCount}] Writing preBuffer2`);
if (!ws2.write(preBuffer2)) rs.pause();

console.log(`[${lineCount}] Writing preBuffer3`);
if (!ws3.write(preBuffer3)) rs.pause();

console.log(`[${lineCount}] Done processing`);

If logs stop before writing buffers, then something is breaking before buffer allocation.

If logs stop after .pause() but before .resume(), then Bun 1.2+ isn’t handling backpressure correctly.

It fails writing the buffer on 1.2.7 and logs 516, on 1.1.45 it never logs that line

if(!ws1.write(preBuffer1)) {
    console.log(`[${lineCount}] 516`);
    rs.pause()
}

Since the log only appears in Bun 1.2.7 and not in 1.1.45 it could signal that that Bun 1.2+ changed handling of backpressure, which causes the internal buffer stream ws1 to reach it's internal limit faster.

The reason why it is pausing for 5-6 seconds and exiting silently could be because ws1.write() is returning false because the stream is overwhelmed. This means that rs.pause() is called but never resumed.

Here are some things that you can try:

Manually resume on drain:

currently, the script pauses but never resumes when the stream is prepared:

Try listening for the drain event so you can resume the stream

if (!ws1.write(prebuffer1)) { console.log([${lineCount}] 516- backpressure detected, pausing); rs.pause(); ws1.once('drain', () => { console.log([${lineCount}] resuming readable stream); rs.resume(); }); }
This may work because drain is called when the buffer is ready, making it safe to call resume.

You could also try increasing the readable buffer size:

const ws1= fs.createWriteStream('output.dat', {highWaterMark: 16 * 1024}); //64KB buffer

Since the default highWaterMark is 16KB for writable streams increasing it will reduce backpressure occurrences.

@JasonTJames It already has handling for drain when it gets triggered or pause. Also they are setting highWaterMark to 1mb already so 64kb would make the issue happen more often? Ill try increasing that higher to see if it changes anything. Does it work as expected for you?

@JasonTJames nice call on highWaterMark upping it above the set value of 1mb makes the problem get pushed back, setting to 4mb makes it go away, I'm going to patch it to 128mb to work around this issue until its sorted in bun.