Scribd
Upload
3 views

NodeJS Streaming Cluster

The document discusses streaming and clustering in Node.js, highlighting their importance for handling large data flows and improving application scalability. It details the types of streams (readable, writable, duplex, and transform), key methods and events, and provides examples for each. Additionally, it explains the Cluster module, its architecture, and advantages, emphasizing its role in utilizing multi-core CPUs for better performance.

Uploaded by

Alwani Anis
Copyright
© © All Rights Reserved
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
3 views

NodeJS Streaming Cluster

The document discusses streaming and clustering in Node.js, highlighting their importance for handling large data flows and improving application scalability. It details the types of streams (readable, writable, duplex, and transform), key methods and events, and provides examples for each. Additionally, it explains the Cluster module, its architecture, and advantages, emphasizing its role in utilizing multi-core CPUs for better performance.

Uploaded by

Alwani Anis
Copyright
© © All Rights Reserved
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 4

### Streaming in Node.

js

Streams in Node.js provide a powerful way to handle continuous data flow,


especially when working with large files or real-time data.
They are used for reading or writing sequential data in chunks, making them memory
efficient.

#### Types of Streams


1. **Readable Streams**
- Data flows from a source (e.g., file, network) to the application.
- Example: `fs.createReadStream`

2. **Writable Streams**
- Data flows from the application to a destination (e.g., file, network).
- Example: `fs.createWriteStream`

3. **Duplex Streams**
- Allows both reading and writing (e.g., a TCP socket).
- Example: `net.Socket`

4. **Transform Streams**
- A type of duplex stream that can modify data as it is read or written (e.g.,
`zlib` for compression).
- Example: `zlib.createGzip`

#### Key Methods and Events


- **Methods**:
- `read()`: Reads data from a readable stream.
- `write(chunk)`: Writes data to a writable stream.
- `pipe(dest)`: Pipes data from one stream to another.

- **Events**:
- `data`: Emitted when a chunk of data is available.
- `end`: Emitted when no more data is available.
- `error`: Emitted when an error occurs.
- `finish`: Emitted when all data is flushed to the writable stream.

#### Example 1: Reading a File Stream


```javascript
const fs = require('fs');

const readStream = fs.createReadStream('largeFile.txt', { encoding: 'utf8' });

readStream.on('data', chunk => {


console.log('Received chunk:', chunk);
});

readStream.on('end', () => {
console.log('No more data.');
});

readStream.on('error', err => {


console.error('Error:', err);
});
```

#### Example 2: Writing to a File Stream


```javascript
const fs = require('fs');

const writeStream = fs.createWriteStream('output.txt');

writeStream.write('Hello, World!\n');
writeStream.end('Finished writing.');
writeStream.on('finish', () => console.log('Write complete.'));
```

#### Example 3: Piping Streams


```javascript
const fs = require('fs');

const readStream = fs.createReadStream('input.txt');


const writeStream = fs.createWriteStream('output.txt');

readStream.pipe(writeStream);

readStream.on('end', () => console.log('File copied successfully.'));


```

#### Example 4: Transform Stream (Compression)


```javascript
const fs = require('fs');
const zlib = require('zlib');

const readStream = fs.createReadStream('input.txt');


const gzipStream = zlib.createGzip();
const writeStream = fs.createWriteStream('output.txt.gz');

readStream.pipe(gzipStream).pipe(writeStream);
```

---

### Cluster in Node.js

The **Cluster** module allows Node.js to create child processes (workers) that
share the same server port, enabling you to utilize multi-core CPUs for better
scalability.

#### Key Concepts


1. **Master Process**: The main process that controls worker processes.
2. **Worker Processes**: Child processes created by the master to handle tasks.
3. **IPC (Inter-Process Communication)**: Workers communicate with the master using
messages.

#### How Clustering Works


- The master process distributes incoming connections or tasks to worker processes.
- Each worker runs on its own thread and handles incoming requests independently.

#### Basic Cluster Example


```javascript
const cluster = require('cluster');
const http = require('http');
const os = require('os');

if (cluster.isMaster) {
// Fork workers for each CPU core
const numCPUs = os.cpus().length;
console.log(`Master process is running (PID: ${process.pid})`);

for (let i = 0; i < numCPUs; i++) {


cluster.fork();
}

// Handle worker exit


cluster.on('exit', (worker, code, signal) => {
console.log(`Worker ${worker.process.pid} died`);
cluster.fork(); // Restart a new worker
});
} else {
// Workers can share a TCP connection
http.createServer((req, res) => {
res.writeHead(200);
res.end(`Hello from worker ${process.pid}`);
}).listen(8000);

console.log(`Worker process started (PID: ${process.pid})`);


}
```

#### Message Passing Between Master and Workers


Workers can send and receive messages from the master process using `process.send`
and `cluster.on`.

```javascript
// Master Process
if (cluster.isMaster) {
const worker = cluster.fork();

worker.on('message', message => {


console.log('Message from worker:', message);
});

worker.send('Hello Worker!');
} else {
// Worker Process
process.on('message', message => {
console.log('Message from master:', message);
process.send('Hello Master!');
});
}
```

#### Advantages of Clustering


1. **Utilizes Multi-Core CPUs**: Distributes the load across multiple cores.
2. **Improves Performance**: Each worker handles separate tasks.
3. **Resilience**: Restart workers on failure.

#### Limitations
- Workers do not share memory directly. Use shared databases, caches, or message
brokers for state sharing.
- Clustering doesn't inherently solve CPU-intensive task bottlenecks.

---

These concepts of **streaming** and **clustering** are crucial for building high-
performance, scalable Node.js applications.

You might also like