Mathias Bynens | 79e2cf0 | 2020-05-29 14:46:17 | [diff] [blame] | 1 | # fastq |
| 2 | |
| 3 | ![ci][ci-url] |
| 4 | [![npm version][npm-badge]][npm-url] |
| 5 | [![Dependency Status][david-badge]][david-url] |
| 6 | |
| 7 | Fast, in memory work queue. `fastq` is API compatible with |
| 8 | [`async.queue`](https://github.com/caolan/async#queueworker-concurrency) |
| 9 | |
| 10 | Benchmarks (1 million tasks): |
| 11 | |
| 12 | * setImmediate: 812ms |
| 13 | * fastq: 854ms |
| 14 | * async.queue: 1298ms |
| 15 | * neoAsync.queue: 1249ms |
| 16 | |
| 17 | Obtained on node 12.16.1, on a dedicated server. |
| 18 | |
| 19 | If you need zero-overhead series function call, check out |
| 20 | [fastseries](http://npm.im/fastseries). For zero-overhead parallel |
| 21 | function call, check out [fastparallel](http://npm.im/fastparallel). |
| 22 | |
| 23 | [](https://github.com/feross/standard) |
| 24 | |
| 25 | * <a href="#install">Installation</a> |
| 26 | * <a href="#usage">Usage</a> |
| 27 | * <a href="#api">API</a> |
| 28 | * <a href="#license">Licence & copyright</a> |
| 29 | |
| 30 | ## Install |
| 31 | |
| 32 | `npm i fastq --save` |
| 33 | |
| 34 | ## Usage |
| 35 | |
| 36 | ```js |
| 37 | 'use strict' |
| 38 | |
| 39 | var queue = require('fastq')(worker, 1) |
| 40 | |
| 41 | queue.push(42, function (err, result) { |
| 42 | if (err) { throw err } |
| 43 | console.log('the result is', result) |
| 44 | }) |
| 45 | |
| 46 | function worker (arg, cb) { |
| 47 | cb(null, 42 * 2) |
| 48 | } |
| 49 | ``` |
| 50 | |
| 51 | ### Setting this |
| 52 | |
| 53 | ```js |
| 54 | 'use strict' |
| 55 | |
| 56 | var that = { hello: 'world' } |
| 57 | var queue = require('fastq')(that, worker, 1) |
| 58 | |
| 59 | queue.push(42, function (err, result) { |
| 60 | if (err) { throw err } |
| 61 | console.log(this) |
| 62 | console.log('the result is', result) |
| 63 | }) |
| 64 | |
| 65 | function worker (arg, cb) { |
| 66 | console.log(this) |
| 67 | cb(null, 42 * 2) |
| 68 | } |
| 69 | ``` |
| 70 | |
| 71 | ## API |
| 72 | |
| 73 | * <a href="#fastqueue"><code>fastqueue()</code></a> |
| 74 | * <a href="#push"><code>queue#<b>push()</b></code></a> |
| 75 | * <a href="#unshift"><code>queue#<b>unshift()</b></code></a> |
| 76 | * <a href="#pause"><code>queue#<b>pause()</b></code></a> |
| 77 | * <a href="#resume"><code>queue#<b>resume()</b></code></a> |
| 78 | * <a href="#idle"><code>queue#<b>idle()</b></code></a> |
| 79 | * <a href="#length"><code>queue#<b>length()</b></code></a> |
| 80 | * <a href="#getQueue"><code>queue#<b>getQueue()</b></code></a> |
| 81 | * <a href="#kill"><code>queue#<b>kill()</b></code></a> |
| 82 | * <a href="#killAndDrain"><code>queue#<b>killAndDrain()</b></code></a> |
Tim van der Lippe | efb716a | 2020-12-01 12:54:04 | [diff] [blame] | 83 | * <a href="#error"><code>queue#<b>error()</b></code></a> |
Mathias Bynens | 79e2cf0 | 2020-05-29 14:46:17 | [diff] [blame] | 84 | * <a href="#concurrency"><code>queue#<b>concurrency</b></code></a> |
| 85 | * <a href="#drain"><code>queue#<b>drain</b></code></a> |
| 86 | * <a href="#empty"><code>queue#<b>empty</b></code></a> |
| 87 | * <a href="#saturated"><code>queue#<b>saturated</b></code></a> |
| 88 | |
| 89 | ------------------------------------------------------- |
| 90 | <a name="fastqueue"></a> |
| 91 | ### fastqueue([that], worker, concurrency) |
| 92 | |
| 93 | Creates a new queue. |
| 94 | |
| 95 | Arguments: |
| 96 | |
| 97 | * `that`, optional context of the `worker` function. |
| 98 | * `worker`, worker function, it would be called with `that` as `this`, |
| 99 | if that is specified. |
| 100 | * `concurrency`, number of concurrent tasks that could be executed in |
| 101 | parallel. |
| 102 | |
| 103 | ------------------------------------------------------- |
| 104 | <a name="push"></a> |
| 105 | ### queue.push(task, done) |
| 106 | |
| 107 | Add a task at the end of the queue. `done(err, result)` will be called |
| 108 | when the task was processed. |
| 109 | |
| 110 | ------------------------------------------------------- |
| 111 | <a name="unshift"></a> |
| 112 | ### queue.unshift(task, done) |
| 113 | |
| 114 | Add a task at the beginning of the queue. `done(err, result)` will be called |
| 115 | when the task was processed. |
| 116 | |
| 117 | ------------------------------------------------------- |
| 118 | <a name="pause"></a> |
| 119 | ### queue.pause() |
| 120 | |
| 121 | Pause the processing of tasks. Currently worked tasks are not |
| 122 | stopped. |
| 123 | |
| 124 | ------------------------------------------------------- |
| 125 | <a name="resume"></a> |
| 126 | ### queue.resume() |
| 127 | |
| 128 | Resume the processing of tasks. |
| 129 | |
| 130 | ------------------------------------------------------- |
| 131 | <a name="idle"></a> |
| 132 | ### queue.idle() |
| 133 | |
| 134 | Returns `false` if there are tasks being processed or waiting to be processed. |
| 135 | `true` otherwise. |
| 136 | |
| 137 | ------------------------------------------------------- |
| 138 | <a name="length"></a> |
| 139 | ### queue.length() |
| 140 | |
| 141 | Returns the number of tasks waiting to be processed (in the queue). |
| 142 | |
| 143 | ------------------------------------------------------- |
| 144 | <a name="getQueue"></a> |
| 145 | ### queue.getQueue() |
| 146 | |
| 147 | Returns all the tasks be processed (in the queue). Returns empty array when there are no tasks |
| 148 | |
| 149 | ------------------------------------------------------- |
| 150 | <a name="kill"></a> |
| 151 | ### queue.kill() |
| 152 | |
| 153 | Removes all tasks waiting to be processed, and reset `drain` to an empty |
| 154 | function. |
| 155 | |
| 156 | ------------------------------------------------------- |
| 157 | <a name="killAndDrain"></a> |
| 158 | ### queue.killAndDrain() |
| 159 | |
| 160 | Same than `kill` but the `drain` function will be called before reset to empty. |
| 161 | |
| 162 | ------------------------------------------------------- |
Tim van der Lippe | efb716a | 2020-12-01 12:54:04 | [diff] [blame] | 163 | <a name="error"></a> |
| 164 | ### queue.error(handler) |
| 165 | |
| 166 | Set a global error handler. `handler(err, task)` will be called |
| 167 | when any of the tasks return an error. |
| 168 | |
| 169 | ------------------------------------------------------- |
Mathias Bynens | 79e2cf0 | 2020-05-29 14:46:17 | [diff] [blame] | 170 | <a name="concurrency"></a> |
| 171 | ### queue.concurrency |
| 172 | |
| 173 | Property that returns the number of concurrent tasks that could be executed in |
| 174 | parallel. It can be altered at runtime. |
| 175 | |
| 176 | ------------------------------------------------------- |
| 177 | <a name="drain"></a> |
| 178 | ### queue.drain |
| 179 | |
| 180 | Function that will be called when the last |
| 181 | item from the queue has been processed by a worker. |
| 182 | It can be altered at runtime. |
| 183 | |
| 184 | ------------------------------------------------------- |
| 185 | <a name="empty"></a> |
| 186 | ### queue.empty |
| 187 | |
| 188 | Function that will be called when the last |
| 189 | item from the queue has been assigned to a worker. |
| 190 | It can be altered at runtime. |
| 191 | |
| 192 | ------------------------------------------------------- |
| 193 | <a name="saturated"></a> |
| 194 | ### queue.saturated |
| 195 | |
| 196 | Function that will be called when the queue hits the concurrency |
| 197 | limit. |
| 198 | It can be altered at runtime. |
| 199 | |
| 200 | ## License |
| 201 | |
| 202 | ISC |
| 203 | |
| 204 | [ci-url]: https://github.com/mcollina/fastq/workflows/ci/badge.svg |
| 205 | [npm-badge]: https://badge.fury.io/js/fastq.svg |
| 206 | [npm-url]: https://badge.fury.io/js/fastq |
| 207 | [david-badge]: https://david-dm.org/mcollina/fastq.svg |
| 208 | [david-url]: https://david-dm.org/mcollina/fastq |