Chapter 30: Node.js Buffer Module
1. What is a Buffer in Node.js? (the honest explanation)
A Buffer is Node.js’s way of handling raw binary data.
In JavaScript, normal strings are Unicode (UTF-16 internally). They are perfect for text, but very inefficient for binary data (images, videos, PDFs, network packets, file contents, crypto operations…).
Buffer is a fixed-size chunk of raw memory (outside the V8 heap) that lets you work with bytes directly — exactly like arrays of 0–255 numbers.
Key mental model (2025–2026):
- String = text (humans read it)
- Buffer = bytes (computers & protocols read it)
Almost every time you deal with:
- Files (especially non-text)
- Network (HTTP, TCP, WebSocket)
- Crypto
- Images / video processing
- Binary protocols (Protobuf, MessagePack, BSON…)
- Encoding / decoding
…you are going to meet Buffer.
2. Modern import style (2025–2026)
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 |
// Recommended way import { Buffer } from 'node:buffer'; // Old style (still works everywhere) const { Buffer } = require('buffer'); |
Always use node:buffer prefix — clearer and prevents name collisions.
3. Creating Buffers – the different ways you’ll see
| Method | When to use it | Example |
|---|---|---|
| Buffer.from(…) | Most common – from string, array, etc. | Buffer.from(‘hello’) |
| Buffer.alloc(size) | Create empty buffer of exact size | Buffer.alloc(1024) |
| Buffer.allocUnsafe(size) | Faster, but contains old memory garbage | Only when you will overwrite everything |
| Buffer.from(array) | From array of bytes (0–255) | Buffer.from([0x48, 0x65, 0x6c, 0x6c, 0x6f]) |
| Buffer.concat(list) | Combine multiple buffers | Merging chunks from stream |
Real examples
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
// 1. From string (default UTF-8) const buf1 = Buffer.from('Hello Hyderabad!'); console.log(buf1); // <Buffer 48 65 6c 6c 6f 20 48 79 64 65 72 61 62 61 64 21> console.log(buf1.toString()); // Hello Hyderabad! // 2. From string with encoding const buf2 = Buffer.from('नमस्ते', 'utf8'); console.log(buf2.toString('utf8')); // नमस्ते // 3. Empty buffer – safe (zeroed) const buf3 = Buffer.alloc(16); console.log(buf3); // <Buffer 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00> // 4. Unsafe (faster, but contains random old memory) const buf4 = Buffer.allocUnsafe(16); console.log(buf4); // contains random bytes – dangerous if you forget to overwrite! // 5. From array of bytes const buf5 = Buffer.from([0x48, 0x65, 0x6c, 0x6c, 0x6f]); // Hello console.log(buf5.toString()); // Hello |
4. Most important properties & methods
| Property / Method | What it does | Example |
|---|---|---|
| buf.length | Size in bytes | buf.length === 5 |
| buf.toString(encoding?) | Convert to string | buf.toString(‘utf8’) |
| buf.toJSON() | Get { type: ‘Buffer’, data: [..] } | Useful for JSON.stringify |
| buf[index] | Read / write single byte (0–255) | buf[0] = 65 → ‘A’ |
| buf.slice(start, end) | Create view (not copy!) | Zero-copy subregion |
| buf.copy(target) | Copy bytes to another buffer | Manual copy |
| Buffer.concat(list) | Merge multiple buffers | Very common with streams |
| buf.equals(other) | Deep equality check | Compare binary content |
| buf.indexOf(value) | Find position of byte / string / buffer | Search inside binary data |
Real example – byte manipulation
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
const buf = Buffer.from('Hello'); // Change first byte to 'J' buf[0] = 74; // ASCII 'J' console.log(buf.toString()); // → Jello // Replace substring buf.fill('x', 1, 3); // fill positions 1–2 with 'x' console.log(buf.toString()); // → Jxxlo |
5. Most common real-world use cases (2025–2026)
5.1 Working with file streams
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
import fs from 'node:fs'; const readStream = fs.createReadStream('image.jpg'); readStream.on('data', (chunk) => { // chunk is a Buffer! console.log(`Received {chunk.length} bytes`); // → do something with raw bytes (hash, resize, encrypt…) }); |
5.2 Network / HTTP (very common)
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
import https from 'node:https'; https.get('https://example.com/image.png', (res) => { const chunks = []; res.on('data', (chunk) => { chunks.push(chunk); // chunk is Buffer }); res.on('end', () => { const fullImage = Buffer.concat(chunks); console.log('Image size:', fullImage.length, 'bytes'); }); }); |
5.3 Crypto operations
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
import { createHash } from 'node:crypto'; const fileBuffer = await fs.readFile('document.pdf'); const hash = createHash('sha256') .update(fileBuffer) .digest('hex'); console.log('SHA-256:', hash); |
5.4 Encoding / decoding (base64, hex, utf8…)
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 |
const text = 'नमस्ते Hyderabad'; const buf = Buffer.from(text, 'utf8'); console.log(buf.toString('base64')); // base64 encoded console.log(buf.toString('hex')); // hex encoded console.log(Buffer.from(buf.toString('base64'), 'base64').toString('utf8')); // back to original |
6. Important modern helpers & patterns
Pattern 1 – Safe Buffer creation (avoid allocUnsafe unless necessary)
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 |
// Safe – always zero-filled const buf = Buffer.alloc(1024); // Unsafe – faster, but must overwrite everything const bufUnsafe = Buffer.allocUnsafe(1024); bufUnsafe.fill(0); // ← must do this manually if you want zeros |
Pattern 2 – Buffer.concat() with streams (very common)
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
const chunks = []; req.on('data', chunk => chunks.push(chunk)); req.on('end', () => { const fullBody = Buffer.concat(chunks); console.log('Received body size:', fullBody.length); }); |
Pattern 3 – Zero-copy slicing (very efficient)
JavaScript
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
const bigBuffer = Buffer.alloc(1000000); // ... fill it ... const header = bigBuffer.subarray(0, 64); // view, not copy! const payload = bigBuffer.subarray(64); console.log(header.length); // 64 |
subarray() creates a view — zero memory copy — very fast!
7. Common mistakes & traps (very important)
| Mistake | Consequence | Correct way / Fix |
|---|---|---|
| Treating Buffer like normal array | Wrong behavior (Buffer is not resizable) | Use Buffer.concat() to grow |
| Logging Buffer directly in production | Huge logs, sensitive data leak | Always .toString(‘utf8’) or .toJSON() |
| Forgetting encoding when creating | Corrupted text (default is utf8) | Explicitly pass encoding: Buffer.from(str, ‘utf8’) |
| Using slice() and modifying | Original buffer also changes | Use .subarray() (view) or .copy() if needed |
| Assuming Buffer is immutable | Bugs when modifying shared buffers | Buffers are mutable — treat carefully |
Summary – Quick cheat sheet 2025–2026
| You want to… | Recommended way | Typical real-world use case |
|---|---|---|
| Create buffer from string / array | Buffer.from(str) or Buffer.from([…]) | Text → bytes, byte array from protocol |
| Create empty buffer | Buffer.alloc(size) | Preparing space for writing |
| Combine multiple chunks | Buffer.concat([buf1, buf2, …]) | Collecting stream chunks (HTTP body, file parts) |
| Get view without copying | buf.subarray(start, end) | Parsing headers / binary protocols |
| Convert to string / base64 / hex | buf.toString(‘utf8’), .toString(‘base64’) | Logging, API responses, storage |
| Work with raw binary data | Read / write single bytes buf[i] | Crypto, image processing, custom protocols |
Would you like to go much deeper into any specific area?
- Binary protocol parsing (real example with length-prefixed messages)
- Working with Buffers in streams (chunk by chunk processing)
- Buffer vs TypedArray / Uint8Array – when to use which
- High-performance patterns (zero-copy, pooling, Unsafe allocations)
- Complete example – simple binary protocol server/client using Buffer
Just tell me which direction feels most useful right now — I’ll continue with detailed, production-ready code examples. 😊
