cernont/pyrofetes-frontend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
CHEVALLIER Abel
2025-11-13 16:23:22 +01:00
parent de9c515a47
commit cb235644dc
34924 changed files with 3811102 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

152
node_modules/piscina/CHANGELOG.md generated vendored Normal file
View File

@@ -0,0 +1,152 @@
# Changelog
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
### [5.1.3](https://github.com/piscinajs/piscina/compare/v5.1.2...v5.1.3) (2025-07-09)
### Features
* offload drain events to microtask ([#835](https://github.com/piscinajs/piscina/issues/835)) ([76fe7a8](https://github.com/piscinajs/piscina/commit/76fe7a8138089750183836e8db2968337d8a97b0))
### [5.1.2](https://github.com/piscinajs/piscina/compare/v5.1.1...v5.1.2) (2025-06-26)
### [5.1.1](https://github.com/piscinajs/piscina/compare/v5.1.0...v5.1.1) (2025-06-19)
### Bug Fixes
* prevent race condition in idle worker cleanup ([#818](https://github.com/piscinajs/piscina/issues/818)) ([cafae5d](https://github.com/piscinajs/piscina/commit/cafae5d17340fa07e03bdb39a801fa4733dfb14f))
## [5.1.0](https://github.com/piscinajs/piscina/compare/v5.0.0...v5.1.0) (2025-06-15)
### Features
* add explicit resource management support ([#810](https://github.com/piscinajs/piscina/issues/810)) ([d625bba](https://github.com/piscinajs/piscina/commit/d625bbaf17536ba0a5654383206e4126b01db557))
### Bug Fixes
* **#805:** Fix handling of aborted tasks ([#807](https://github.com/piscinajs/piscina/issues/807)) ([5608520](https://github.com/piscinajs/piscina/commit/5608520bd7d812b253a0373d50572005ab863195))
## [5.0.0](https://github.com/piscinajs/piscina/compare/v5.0.0-alpha.2...v5.0.0) (2025-05-02)
### ⚠ BREAKING CHANGES
* drop v18 (#782)
* drop v18 ([#782](https://github.com/piscinajs/piscina/issues/782)) ([7a87d6b](https://github.com/piscinajs/piscina/commit/7a87d6b16844943ebb237de6bd8285baa81bba0b))
## [5.0.0-alpha.2](https://github.com/piscinajs/piscina/compare/v5.0.0-alpha.1...v5.0.0-alpha.2) (2025-03-16)
### Features
* Allow long-running threads ([#757](https://github.com/piscinajs/piscina/issues/757)) ([f0f4fd3](https://github.com/piscinajs/piscina/commit/f0f4fd39d50dac47b411a2a7aee1ac2d744f0e2f))
## [5.0.0-alpha.1](https://github.com/piscinajs/piscina/compare/v5.0.0-alpha.0...v5.0.0-alpha.1) (2025-01-10)
### ⚠ BREAKING CHANGES
* **#305:** Expose new `PiscinaHistogram` abstraction (#723)
### Features
* **#305:** Expose new `PiscinaHistogram` abstraction ([#723](https://github.com/piscinajs/piscina/issues/723)) ([86d736c](https://github.com/piscinajs/piscina/commit/86d736cf4c239d20e1a403d11a82b7ead0611aa8)), closes [#305](https://github.com/piscinajs/piscina/issues/305)
## [5.0.0-alpha.0](https://github.com/piscinajs/piscina/compare/v4.6.1...v5.0.0-alpha.0) (2024-12-04)
### Features
* Custom Balancer ([#590](https://github.com/piscinajs/piscina/issues/590)) ([5c42b28](https://github.com/piscinajs/piscina/commit/5c42b28942f39399ea4aad39dd1f4367959c1e8f))
* support Atomics.waitAsync ([#687](https://github.com/piscinajs/piscina/issues/687)) ([9c5a19e](https://github.com/piscinajs/piscina/commit/9c5a19ea491b159b82f23512811555a5c4aa2d3f))
* use @napi-rs/nice to support Windows ([#655](https://github.com/piscinajs/piscina/issues/655)) ([c567394](https://github.com/piscinajs/piscina/commit/c56739465000f455fcf7abc2f83501054cab22a4))
### Bug Fixes
* adjust docusaurus ([5c3d2c9](https://github.com/piscinajs/piscina/commit/5c3d2c90ff0a00f338d194b20efdc6772d8e01e3))
### [4.6.1](https://github.com/piscinajs/piscina/compare/v4.6.0...v4.6.1) (2024-06-26)
## [4.6.0](https://github.com/piscinajs/piscina/compare/v4.5.2...v4.6.0) (2024-06-18)
### Features
* expose task interface ([#565](https://github.com/piscinajs/piscina/issues/565)) ([285aa82](https://github.com/piscinajs/piscina/commit/285aa82b45cfb1f33210812c441c83a44c78ed34))
### Bug Fixes
* close pool with minThreads=0 ([#584](https://github.com/piscinajs/piscina/issues/584)) ([776bacb](https://github.com/piscinajs/piscina/commit/776bacbebbc7f3adcde767a7dfada574da58bfe6))
### [4.5.1](https://github.com/piscinajs/piscina/compare/v4.5.0...v4.5.1) (2024-05-22)
### Bug Fixes
* support nodejs v16.x again ([#572](https://github.com/piscinajs/piscina/issues/572)) ([d50391f](https://github.com/piscinajs/piscina/commit/d50391fe93a6319c2a554f34d39cce0c946564ec))
## [4.5.0](https://github.com/piscinajs/piscina/compare/v4.4.0...v4.5.0) (2024-05-20)
### Features
* allow generic when creating Piscina ([#569](https://github.com/piscinajs/piscina/issues/569)) ([108440c](https://github.com/piscinajs/piscina/commit/108440c5586bad0be376c65a56836875fce5bef9))
* Use fixed queue ([#555](https://github.com/piscinajs/piscina/issues/555)) ([8afa70f](https://github.com/piscinajs/piscina/commit/8afa70faaefeb7ed87516af06aad5924a4dbe7f0))
* use os.availableConcurrency ([#556](https://github.com/piscinajs/piscina/issues/556)) ([d1fbba2](https://github.com/piscinajs/piscina/commit/d1fbba2cae4c189b822672bb63f50b7381cbb6ab))
## [4.4.0](https://github.com/piscinajs/piscina/compare/v4.3.2...v4.4.0) (2024-02-28)
### Features
* add option to disable run/wait time recording ([#518](https://github.com/piscinajs/piscina/issues/518)) ([4a94cee](https://github.com/piscinajs/piscina/commit/4a94cee847395a0395cce68743332009214243f2))
* allow named import usage ([#517](https://github.com/piscinajs/piscina/issues/517)) ([6a7c6e1](https://github.com/piscinajs/piscina/commit/6a7c6e170b19d1c6285c0230ad02f1a259fc69a3))
### [4.3.2](https://github.com/piscinajs/piscina/compare/v4.3.1...v4.3.2) (2024-02-16)
### Bug Fixes
* **#513:** forward errors correctly to Piscina ([#514](https://github.com/piscinajs/piscina/issues/514)) ([6945d21](https://github.com/piscinajs/piscina/commit/6945d21d47b72dfa801e0309948fea9fbf708c91)), closes [#513](https://github.com/piscinajs/piscina/issues/513)
### [4.3.1](https://github.com/piscinajs/piscina/compare/v4.3.0...v4.3.1) (2024-01-30)
### Bug Fixes
* **#491:** out of bounds histogram value ([#496](https://github.com/piscinajs/piscina/issues/496)) ([0b4eada](https://github.com/piscinajs/piscina/commit/0b4eada2485a0f722f5b6d39d657fd51975df0f3)), closes [#491](https://github.com/piscinajs/piscina/issues/491)
## [4.3.0](https://github.com/piscinajs/piscina/compare/v4.2.1...v4.3.0) (2024-01-16)
### Features
* use native Node.js histogram support ([#482](https://github.com/piscinajs/piscina/issues/482)) ([aa5b140](https://github.com/piscinajs/piscina/commit/aa5b1408e33420e7c29725381d7824b0b40d26e8))
### [4.2.1](https://github.com/piscinajs/piscina/compare/v4.2.0...v4.2.1) (2023-12-13)
### Bug Fixes
* default minThreads with odd CPU count ([#457](https://github.com/piscinajs/piscina/issues/457)) ([f4edf87](https://github.com/piscinajs/piscina/commit/f4edf87c8c4883e06ab70e99a8a5050eded89c5d))
## [4.2.0](https://github.com/piscinajs/piscina/compare/v4.1.0...v4.2.0) (2023-11-19)
### Features
* Add `Piscina#close` API ([#396](https://github.com/piscinajs/piscina/issues/396)) ([5378e4c](https://github.com/piscinajs/piscina/commit/5378e4cf9143587d9457d3cef6b88aa9653749bd))
### Bug Fixes
* add signal reason support ([#403](https://github.com/piscinajs/piscina/issues/403)) ([66809f9](https://github.com/piscinajs/piscina/commit/66809f94868b4b4597401e10252e1285fabc63c2))
* do not re-create threads when calling `.destory()` ([#430](https://github.com/piscinajs/piscina/issues/430)) ([ec21ff2](https://github.com/piscinajs/piscina/commit/ec21ff28f90a4d5e001ba694fe3dcd6abec3f553))
* migrate to EventEmitterAsyncResource from core ([#433](https://github.com/piscinajs/piscina/issues/433)) ([0a539e2](https://github.com/piscinajs/piscina/commit/0a539e23e7c413cc33631f1adb32ab28b468297b))

129
node_modules/piscina/CODE_OF_CONDUCT.md generated vendored Normal file
View File

@@ -0,0 +1,129 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.
Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
jasnell@gmail.com, anna@addaleax.net, or matteo.collina@gmail.com.
All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the
reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series
of actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within
the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

27
node_modules/piscina/CONTRIBUTING generated vendored Normal file
View File

@@ -0,0 +1,27 @@
# Piscina is an OPEN Open Source Project
## What?
Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.
## Rules
There are a few basic ground-rules for contributors:
1. **No `--force` pushes** on `master` or modifying the Git history in any way after a PR has been merged.
1. **Non-master branches** ought to be used for ongoing work.
1. **External API changes and significant modifications** ought to be subject to an **internal pull-request** to solicit feedback from other contributors.
1. Internal pull-requests to solicit feedback are *encouraged* for any other non-trivial contribution but left to the discretion of the contributor.
1. Contributors should attempt to adhere to the prevailing code-style.
1. 100% code coverage
1. Semantic Versioning is used.
## Releases
Declaring formal releases remains the prerogative of the project maintainer.
## Changes to this arrangement
This document may also be subject to pull-requests or changes by contributors where you believe you have something valuable to add or change.
-----------------------------------------

24
node_modules/piscina/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,24 @@
The MIT License (MIT)
Copyright (c) 2020 James M Snell and the Piscina contributors
Piscina contributors listed at https://github.com/jasnell/piscina#the-team and
in the README file.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

941
node_modules/piscina/README.md generated vendored Normal file
View File

@@ -0,0 +1,941 @@
![Piscina Logo](https://avatars1.githubusercontent.com/u/65627548?s=200&v=4)
# piscina - the node.js worker pool
![CI](https://github.com/jasnell/piscina/workflows/CI/badge.svg)
* ✔ Fast communication between threads
* ✔ Covers both fixed-task and variable-task scenarios
* ✔ Supports flexible pool sizes
* ✔ Proper async tracking integration
* ✔ Tracking statistics for run and wait times
* ✔ Cancellation Support
* ✔ Supports enforcing memory resource limits
* ✔ Supports CommonJS, ESM, and TypeScript
* ✔ Custom task queues
* ✔ Optional CPU scheduling priorities on Linux
Written in TypeScript.
For Node.js 20.x and higher.
[MIT Licensed][].
## Documentation
- [Website](https://piscinajs.github.io/piscina/)
## Piscina API
### Example
In `main.js`:
```js
const path = require('path');
const Piscina = require('piscina');
const piscina = new Piscina({
filename: path.resolve(__dirname, 'worker.js')
});
(async function() {
const result = await piscina.run({ a: 4, b: 6 });
console.log(result); // Prints 10
})();
```
In `worker.js`:
```js
module.exports = ({ a, b }) => {
return a + b;
};
```
The worker may also be an async function or may return a Promise:
```js
const { setTimeout } = require('timers/promises');
module.exports = async ({ a, b }) => {
// Fake some async activity
await setTimeout(100);
return a + b;
};
```
ESM is also supported for both Piscina and workers:
```js
import { Piscina } from 'piscina';
const piscina = new Piscina({
// The URL must be a file:// URL
filename: new URL('./worker.mjs', import.meta.url).href
});
const result = await piscina.run({ a: 4, b: 6 });
console.log(result); // Prints 10
```
In `worker.mjs`:
```js
export default ({ a, b }) => {
return a + b;
};
```
### Exporting multiple worker functions
A single worker file may export multiple named handler functions.
```js
'use strict';
function add({ a, b }) { return a + b; }
function multiply({ a, b }) { return a * b; }
add.add = add;
add.multiply = multiply;
module.exports = add;
```
The export to target can then be specified when the task is submitted:
```js
'use strict';
const Piscina = require('piscina');
const { resolve } = require('path');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js')
});
(async function() {
const res = await Promise.all([
piscina.run({ a: 4, b: 6 }, { name: 'add' }),
piscina.run({ a: 4, b: 6 }, { name: 'multiply' })
]);
})();
```
### Cancelable Tasks
Submitted tasks may be canceled using either an `AbortController` or
an `EventEmitter`:
```js
'use strict';
const Piscina = require('piscina');
const { resolve } = require('path');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js')
});
(async function() {
const abortController = new AbortController();
try {
const { signal } = abortController;
const task = piscina.run({ a: 4, b: 6 }, { signal });
abortController.abort();
await task;
} catch (err) {
console.log('The task was canceled');
}
})();
```
Alternatively, any `EventEmitter` that emits an `'abort'` event
may be used as an abort controller:
```js
'use strict';
const Piscina = require('piscina');
const EventEmitter = require('events');
const { resolve } = require('path');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js')
});
(async function() {
const ee = new EventEmitter();
try {
const task = piscina.run({ a: 4, b: 6 }, { signal: ee });
ee.emit('abort');
await task;
} catch (err) {
console.log('The task was canceled');
}
})();
```
### Delaying Availability of Workers
A worker thread will not be made available to process tasks until Piscina
determines that it is "ready". By default, a worker is ready as soon as
Piscina loads it and acquires a reference to the exported handler function.
There may be times when the availability of a worker may need to be delayed
longer while the worker initializes any resources it may need to operate.
To support this case, the worker module may export a `Promise` that resolves
the handler function as opposed to exporting the function directly:
```js
async function initialize() {
await someAsyncInitializationActivity();
return ({ a, b }) => a + b;
}
module.exports = initialize();
```
Piscina will await the resolution of the exported Promise before marking
the worker thread available.
### Backpressure
When the `maxQueue` option is set, once the `Piscina` queue is full, no
additional tasks may be submitted until the queue size falls below the
limit. The `'drain'` event may be used to receive notification when the
queue is empty and all tasks have been submitted to workers for processing.
Example: Using a Node.js stream to feed a Piscina worker pool:
```js
'use strict';
const { resolve } = require('path');
const Pool = require('../..');
const pool = new Pool({
filename: resolve(__dirname, 'worker.js'),
maxQueue: 'auto'
});
const stream = getStreamSomehow();
stream.setEncoding('utf8');
pool.on('drain', () => {
if (stream.isPaused()) {
console.log('resuming...', counter, pool.queueSize);
stream.resume();
}
});
stream
.on('data', (data) => {
pool.run(data);
if (pool.queueSize === pool.options.maxQueue) {
console.log('pausing...', counter, pool.queueSize);
stream.pause();
}
})
.on('error', console.error)
.on('end', () => {
console.log('done');
});
```
### Out of scope asynchronous code
A worker thread is **only** active until the moment it returns a result, it can be a result of a synchronous call or a Promise that will be fulfilled/rejected in the future. Once this is done, Piscina will wait for stdout and stderr to be flushed, and then pause the worker's event-loop until the next call. If async code is scheduled without being awaited before returning since Piscina has no way of detecting this, that code execution will be resumed on the next call. Thus, it is highly recommended to properly handle all async tasks before returning a result as it could make your code unpredictable.
For example:
```js
const { setTimeout } = require('timers/promises');
module.exports = ({ a, b }) => {
// This promise should be awaited
setTimeout(1000).then(() => {
console.log('Working'); // This will **not** run during the same worker call
});
return a + b;
};
```
### Broadcast a message to all worker threads
Piscina supports broadcast communication via BroadcastChannel(Node v18+). Here is an example, the main thread sends a message, and other threads the receive message.
In `main.js`
```js
'use strict';
const { BroadcastChannel } = require('worker_threads');
const { resolve } = require('path');
const Piscina = require('piscina');
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js'),
atomics: 'disabled'
});
async function main () {
const bc = new BroadcastChannel('my_channel');
// start worker
Promise.all([
piscina.run('thread 1'),
piscina.run('thread 2')
]);
// post message in one second
setTimeout(() => {
bc.postMessage('Main thread message');
}, 1000);
}
main();
```
In `worker.js`
```js
'use strict';
const { BroadcastChannel } = require('worker_threads');
module.exports = async (thread) => {
const bc = new BroadcastChannel('my_channel');
bc.onmessage = (event) => {
console.log(thread + ' Received from:' + event.data);
};
await new Promise((resolve) => {
setTimeout(resolve, 2000);
});
};
```
### Additional Examples
Additional examples can be found in the GitHub repo at
https://github.com/piscinajs/piscina/tree/master/examples
## Class: `Piscina`
Piscina works by creating a pool of Node.js Worker Threads to which
one or more tasks may be dispatched. Each worker thread executes a
single exported function defined in a separate file. Whenever a
task is dispatched to a worker, the worker invokes the exported
function and reports the return value back to Piscina when the
function completes.
This class extends [`EventEmitter`][] from Node.js.
### Constructor: `new Piscina([options])`
* The following optional configuration is supported:
* `filename`: (`string | null`) Provides the default source for the code that
runs the tasks on Worker threads. This should be an absolute path or an
absolute `file://` URL to a file that exports a JavaScript `function` or
`async function` as its default export or `module.exports`. [ES modules][]
are supported.
* `name`: (`string | null`) Provides the name of the default exported worker
function. The default is `'default'`, indicating the default export of the
worker module.
* `minThreads`: (`number`) Sets the minimum number of threads that are always
running for this thread pool. The default is the number provided by [`os.availableParallelism`](https://nodejs.org/api/os.html#osavailableparallelism).
* `maxThreads`: (`number`) Sets the maximum number of threads that are
running for this thread pool. The default is the number provided by [`os.availableParallelism`](https://nodejs.org/api/os.html#osavailableparallelism) * 1.5.
* `idleTimeout`: (`number`) A timeout in milliseconds that specifies how long
a `Worker` is allowed to be idle, i.e. not handling any tasks, before it is
shut down. By default, this is immediate. If `Infinity` is passed as the value,
the `Worker` never shuts down. Be careful when using `Infinity`,
as it can lead to resource overuse. **Tip**: *The default `idleTimeout`
can lead to some performance loss in the application because of the overhead
involved with stopping and starting new worker threads. To improve performance,
try setting the `idleTimeout` explicitly.*
* `maxQueue`: (`number` | `string`) The maximum number of tasks that may be
scheduled to run, but not yet running due to lack of available threads, at
a given time. By default, there is no limit. The special value `'auto'`
may be used to have Piscina calculate the maximum as the square of `maxThreads`.
When `'auto'` is used, the calculated `maxQueue` value may be found by checking
the [`options.maxQueue`](#property-options-readonly) property.
* `concurrentTasksPerWorker`: (`number`) Specifies how many tasks can share
a single Worker thread simultaneously. The default is `1`. This generally
only makes sense to specify if there is some kind of asynchronous component
to the task. Keep in mind that Worker threads are generally not built for
handling I/O in parallel.
* `atomics`: (`sync` | `async` | `disabled`) Use the [`Atomics`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics) API for faster communication
between threads. This is on by default. You can disable `Atomics` globally by
setting the environment variable `PISCINA_DISABLE_ATOMICS` to `1` .
If `atomics` is `sync`, it will cause to pause threads (stoping all execution)
between tasks. Ideally, threads should wait for all operations to finish before
returning control to the main thread (avoid having open handles within a thread). If still want to have the possibility
of having open handles or handle asynchrnous tasks, you can set the environment variable `PISCINA_ENABLE_ASYNC_ATOMICS` to `1` or setting `options.atomics` to `async`.
> **Note**: The `async` mode comes with performance penalties and can lead to undesired behaviour if open handles are not tracked correctly.
* `resourceLimits`: (`object`) See [Node.js new Worker options][]
* `maxOldGenerationSizeMb`: (`number`) The maximum size of each worker threads
main heap in MB.
* `maxYoungGenerationSizeMb`: (`number`) The maximum size of a heap space for
recently created objects.
* `codeRangeSizeMb`: (`number`) The size of a pre-allocated memory range used
for generated code.
* `stackSizeMb` : (`number`) The default maximum stack size for the thread.
Small values may lead to unusable Worker instances. Default: 4
* `env`: (`object`) If set, specifies the initial value of `process.env` inside
the worker threads. See [Node.js new Worker options][] for details.
* `argv`: (`any[]`) List of arguments that will be stringified and appended to
`process.argv` in the worker. See [Node.js new Worker options][] for details.
* `execArgv`: (`string[]`) List of Node.js CLI options passed to the worker.
See [Node.js new Worker options][] for details.
* `workerData`: (`any`) Any JavaScript value that can be cloned and made
available as `require('piscina').workerData`. See [Node.js new Worker options][]
for details. Unlike regular Node.js Worker Threads, `workerData` must not
specify any value requiring a `transferList`. This is because the `workerData`
will be cloned for each pooled worker.
* `taskQueue`: (`TaskQueue`) By default, Piscina uses a first-in-first-out
queue for submitted tasks. The `taskQueue` option can be used to provide an
alternative implementation. See [Custom Task Queues][] for additional detail.
* `niceIncrement`: (`number`) An optional value that decreases priority for
the individual threads, i.e. the higher the value, the lower the priority
of the Worker threads. This value is used on Unix/Windows and requires the
optional [`@napi-rs/nice`][https://www.npmjs.com/package/@napi-rs/nice] module to be installed.
See [`nice(2)`][https://linux.die.net/man/2/nice] for more details.
* `trackUnmanagedFds`: (`boolean`) An optional setting that, when `true`, will
cause Workers to track file descriptors managed using `fs.open()` and
`fs.close()`, and will close them automatically when the Worker exits.
Defaults to `true`. (This option is only supported on Node.js 12.19+ and
all Node.js versions higher than 14.6.0).
* `closeTimeout`: (`number`) An optional time (in milliseconds) to wait for the pool to
complete all in-flight tasks when `close()` is called. The default is `30000`
* `recordTiming`: (`boolean`) By default, run and wait time will be recorded
for the pool. To disable, set to `false`.
Use caution when setting resource limits. Setting limits that are too low may
result in the `Piscina` worker threads being unusable.
### Method: `run(task[, options])`
Schedules a task to be run on a Worker thread.
* `task`: Any value. This will be passed to the function that is exported from
`filename`.
* `options`:
* `transferList`: An optional lists of objects that is passed to
[`postMessage()`] when posting `task` to the Worker, which are transferred
rather than cloned.
* `filename`: Optionally overrides the `filename` option passed to the
constructor for this task. If no `filename` was specified to the constructor,
this is mandatory.
* `name`: Optionally overrides the exported worker function used for the task.
* `signal`: An [`AbortSignal`][] instance. If passed, this can be used to
cancel a task. If the task is already running, the corresponding `Worker`
thread will be stopped.
(More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
events can be passed here.) Abortable tasks cannot share threads regardless
of the `concurrentTasksPerWorker` options.
This returns a `Promise` for the return value of the (async) function call
made to the function exported from `filename`. If the (async) function throws
an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.
### Method: `destroy()`
Stops all Workers and rejects all `Promise`s for pending tasks.
This returns a `Promise` that is fulfilled once all threads have stopped.
### Method: `close([options])`
* `options`:
* `force`: A `boolean` value that indicates whether to abort all tasks that
are enqueued but not started yet. The default is `false`.
Stops all Workers gracefully.
This returns a `Promise` that is fulfilled once all tasks that were started
have completed and all threads have stopped.
This method is similar to `destroy()`, but with the difference that `close()`
will wait for the worker tasks to finish, while `destroy()`
will abort them immediately.
### Event: `'error'`
An `'error'` event is emitted by instances of this class when:
- Uncaught exceptions occur inside Worker threads that do not currently handle
tasks.
- Unexpected messages are sent from from Worker threads.
All other errors are reported by rejecting the `Promise` returned from
`run()`, including rejections reported by the handler function
itself.
### Event: `'drain'`
A `'drain'` event is emitted when the current usage of the
pool is below the maximum capacity of the same.
The intended goal is to provide backpressure to the task source
so creating tasks that can not be executed at immediately can be avoided.
### Event: `'needsDrain'`
Similar to [`Piscina#needsDrain`](#property-needsdrain-readonly);
this event is triggered once the total capacity of the pool is exceeded
by number of tasks enqueued that are pending of execution.
### Event: `'message'`
A `'message'` event is emitted whenever a message is received from a worker thread.
### Property: `completed` (readonly)
The current number of completed tasks.
### Property: `duration` (readonly)
The length of time (in milliseconds) since this `Piscina` instance was
created.
### Property: `options` (readonly)
A copy of the options that are currently being used by this instance. This
object has the same properties as the options object passed to the constructor.
### Property: `runTime` (readonly)
A histogram summary object summarizing the collected run times of completed
tasks. All values are expressed in milliseconds.
* `runTime.average` {`number`} The average run time of all tasks
* `runTime.mean` {`number`} The mean run time of all tasks
* `runTime.stddev` {`number`} The standard deviation of collected run times
* `runTime.min` {`number`} The fastest recorded run time
* `runTime.max` {`number`} The slowest recorded run time
All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
represent the percentile distributions of run time observations. For example,
`p99` is the 99th percentile indicating that 99% of the observed run times were
faster or equal to the given value.
```js
{
average: 1880.25,
mean: 1880.25,
stddev: 1.93,
min: 1877,
max: 1882.0190887451172,
p0_001: 1877,
p0_01: 1877,
p0_1: 1877,
p1: 1877,
p2_5: 1877,
p10: 1877,
p25: 1877,
p50: 1881,
p75: 1881,
p90: 1882,
p97_5: 1882,
p99: 1882,
p99_9: 1882,
p99_99: 1882,
p99_999: 1882
}
```
### Property: `threads` (readonly)
An Array of the `Worker` instances used by this pool.
### Property: `queueSize` (readonly)
The current number of tasks waiting to be assigned to a Worker thread.
### Property: `needsDrain` (readonly)
Boolean value that specifies whether the capacity of the pool has
been exceeded by the number of tasks submitted.
This property is helpful to make decisions towards creating backpressure
over the number of tasks submitted to the pool.
### Property: `utilization` (readonly)
A point-in-time ratio comparing the approximate total mean run time
of completed tasks to the total runtime capacity of the pool.
A pools runtime capacity is determined by multiplying the `duration`
by the `options.maxThread` count. This provides an absolute theoretical
maximum aggregate compute time that the pool would be capable of.
The approximate total mean run time is determined by multiplying the
mean run time of all completed tasks by the total number of completed
tasks. This number represents the approximate amount of time the
pool as been actively processing tasks.
The utilization is then calculated by dividing the approximate total
mean run time by the capacity, yielding a fraction between `0` and `1`.
### Property: `waitTime` (readonly)
A histogram summary object summarizing the collected times tasks spent
waiting in the queue. All values are expressed in milliseconds.
* `waitTime.average` {`number`} The average wait time of all tasks
* `waitTime.mean` {`number`} The mean wait time of all tasks
* `waitTime.stddev` {`number`} The standard deviation of collected wait times
* `waitTime.min` {`number`} The fastest recorded wait time
* `waitTime.max` {`number`} The longest recorded wait time
All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
represent the percentile distributions of wait time observations. For example,
`p99` is the 99th percentile indicating that 99% of the observed wait times were
faster or equal to the given value.
```js
{
average: 1880.25,
mean: 1880.25,
stddev: 1.93,
min: 1877,
max: 1882.0190887451172,
p0_001: 1877,
p0_01: 1877,
p0_1: 1877,
p1: 1877,
p2_5: 1877,
p10: 1877,
p25: 1877,
p50: 1881,
p75: 1881,
p90: 1882,
p97_5: 1882,
p99: 1882,
p99_9: 1882,
p99_99: 1882,
p99_999: 1882
}
```
### Static property: `isWorkerThread` (readonly)
Is `true` if this code runs inside a `Piscina` threadpool as a Worker.
### Static property: `version` (readonly)
Provides the current version of this library as a semver string.
### Static method: `move(value)`
By default, any value returned by a worker function will be cloned when
returned back to the Piscina pool, even if that object is capable of
being transfered. The `Piscina.move()` method can be used to wrap and
mark transferable values such that they will by transfered rather than
cloned.
The `value` may be any object supported by Node.js to be transferable
(e.g. `ArrayBuffer`, any `TypedArray`, or `MessagePort`), or any object
implementing the `Transferable` interface.
```js
const { move } = require('piscina');
module.exports = () => {
return move(new ArrayBuffer(10));
}
```
The `move()` method will throw if the `value` is not transferable.
The object returned by the `move()` method should not be set as a
nested value in an object. If it is used, the `move()` object itself
will be cloned as opposed to transfering the object it wraps.
#### Interface: `Transferable`
Objects may implement the `Transferable` interface to create their own
custom transferable objects. This is useful when an object being
passed into or from a worker contains a deeply nested transferable
object such as an `ArrayBuffer` or `MessagePort`.
`Transferable` objects expose two properties inspected by Piscina
to determine how to transfer the object. These properties are
named using the special static `Piscina.transferableSymbol` and
`Piscina.valueSymbol` properties:
* The `Piscina.transferableSymbol` property provides the object
(or objects) that are to be included in the `transferList`.
* The `Piscina.valueSymbol` property provides a surrogate value
to transmit in place of the `Transferable` itself.
Both properties are required.
For example,
```js
const {
move,
transferableSymbol,
valueSymbol
} = require('piscina');
module.exports = () => {
const obj = {
a: { b: new Uint8Array(5); },
c: { new Uint8Array(10); },
get [transferableSymbol]() {
// Transfer the two underlying ArrayBuffers
return [this.a.b.buffer, this.c.buffer];
}
get [valueSymbol]() {
return { a: { b: this.a.b }, c: this.c };
}
};
return move(obj);
};
```
## Custom Task Queues
By default, Piscina uses a simple array-based first-in-first-out (fifo)
task queue. When a new task is submitted and there are no available
workers, tasks are pushed on to the queue until a worker becomes
available.
If the default fifo queue is not sufficient, user code may replace the
task queue implementation with a custom implementation using the
`taskQueue` option on the Piscina constructor.
Custom task queue objects *must* implement the `TaskQueue` interface,
described below using TypeScript syntax:
```ts
interface Task {
readonly [Piscina.queueOptionsSymbol] : object | null;
}
interface TaskQueue {
readonly size : number;
shift () : Task | null;
remove (task : Task) : void;
push (task : Task) : void;
}
```
An example of a custom task queue that uses a shuffled priority queue
is available in [`examples/task-queue`](./examples/task-queue/index.js);
The special symbol `Piscina.queueOptionsSymbol` may be set as a property
on tasks submitted to `run()` as a way of passing additional
options on to the custom `TaskQueue` implementation. (Note that because the
queue options are set as a property on the task, tasks with queue
options cannot be submitted as JavaScript primitives).
### Built-In Queues
Piscina also provides the `FixedQueue`, a more performant task queue implementation based on [`FixedQueue`](https://github.com/nodejs/node/blob/de7b37880f5a541d5f874c1c2362a65a4be76cd0/lib/internal/fixed_queue.js) from Node.js project.
Here are some benchmarks to compare new `FixedQueue` with `ArrayTaskQueue` (current default). The benchmarks demonstrate substantial improvements in push and shift operations, especially with larger queue sizes.
```
Queue size = 1000
┌─────────┬─────────────────────────────────────────┬───────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ns) │ Margin │ Samples │
├─────────┼─────────────────────────────────────────┼───────────┼────────────────────┼──────────┼─────────┤
│ 0 │ 'ArrayTaskQueue full push + full shift' │ '9 692' │ 103175.15463917515 │ '±0.80%' │ 970 │
│ 1 │ 'FixedQueue full push + full shift' │ '131 879' │ 7582.696390658352 │ '±1.81%' │ 13188 │
└─────────┴─────────────────────────────────────────┴───────────┴────────────────────┴──────────┴─────────┘
Queue size = 100_000
┌─────────┬─────────────────────────────────────────┬─────────┬────────────────────┬──────────┬─────────┐
│ (index) │ Task Name │ ops/sec │ Average Time (ns) │ Margin │ Samples │
├─────────┼─────────────────────────────────────────┼─────────┼────────────────────┼──────────┼─────────┤
│ 0 │ 'ArrayTaskQueue full push + full shift' │ '0' │ 1162376920.0000002 │ '±1.77%' │ 10 │
│ 1 │ 'FixedQueue full push + full shift' │ '1 026' │ 974328.1553396407 │ '±2.51%' │ 103 │
└─────────┴─────────────────────────────────────────┴─────────┴────────────────────┴──────────┴─────────┘
```
In terms of Piscina performance itself, using `FixedQueue` with a queue size of 100,000 queued tasks can result in up to 6 times faster execution times.
Users can import `FixedQueue` from the `Piscina` package and pass it as the `taskQueue` option to leverage its benefits.
#### Using FixedQueue Example
Here's an example of how to use the `FixedQueue`:
```js
const { Piscina, FixedQueue } = require('piscina');
const { resolve } = require('path');
// Create a Piscina pool with FixedQueue
const piscina = new Piscina({
filename: resolve(__dirname, 'worker.js'),
taskQueue: new FixedQueue()
});
// Submit tasks to the pool
for (let i = 0; i < 10; i++) {
piscina.run({ data: i }).then((result) => {
console.log(result);
}).catch((error) => {
console.error(error);
});
}
```
## Current Limitations (Things we're working on / would love help with)
* Improved Documentation
* Benchmarks
## Performance Notes
Workers are generally optimized for offloading synchronous,
compute-intensive operations off the main Node.js event loop thread.
While it is possible to perform asynchronous operations and I/O
within a Worker, the performance advantages of doing so will be
minimal.
Specifically, it is worth noting that asynchronous operations
within Node.js, including I/O such as file system operations
or CPU-bound tasks such as crypto operations or compression
algorithms, are already performed in parallel by Node.js and
libuv on a per-process level. This means that there will be
little performance impact on moving such async operations into
a Piscina worker (see examples/scrypt for example).
### Queue Size
Piscina provides the ability to configure the minimum and
maximum number of worker threads active in the pool, as well as
set limits on the number of tasks that may be queued up waiting
for a free worker. It is important to note that setting the
`maxQueue` size too high relative to the number of worker threads
can have a detrimental impact on performance and memory usage.
Setting the `maxQueue` size too small can also be problematic
as doing so could cause your worker threads to become idle and
be shutdown. Our testing has shown that a `maxQueue` size of
approximately the square of the maximum number of threads is
generally sufficient and performs well for many cases, but this
will vary significantly depending on your workload. It will be
important to test and benchmark your worker pools to ensure you've
effectively balanced queue wait times, memory usage, and worker
pool utilization.
### Queue Pressure and Idle Threads
The thread pool maintained by Piscina has both a minimum and maximum
limit to the number of threads that may be created. When a Piscina
instance is created, it will spawn the minimum number of threads
immediately, then create additional threads as needed up to the
limit set by `maxThreads`. Whenever a worker completes a task, a
check is made to determine if there is additional work for it to
perform. If there is no additional work, the thread is marked idle.
By default, idle threads are shutdown immediately, with Piscina
ensuring that the pool always maintains at least the minimum.
When a Piscina pool is processing a stream of tasks (for instance,
processing http server requests as in the React server-side
rendering example in examples/react-ssr), if the rate in which
new tasks are received and queued is not sufficient to keep workers
from going idle and terminating, the pool can experience a thrashing
effect -- excessively creating and terminating workers that will
cause a net performance loss. There are a couple of strategies to
avoid this churn:
Strategy 1: Ensure that the queue rate of new tasks is sufficient to
keep workers from going idle. We refer to this as "queue pressure".
If the queue pressure is too low, workers will go idle and terminate.
If the queue pressure is too high, tasks will stack up, experience
increased wait latency, and consume additional memory.
Strategy 2: Increase the `idleTimeout` configuration option. By
default, idle threads terminate immediately. The `idleTimeout` option
can be used to specify a longer period of time to wait for additional
tasks to be submitted before terminating the worker. If the queue
pressure is not maintained, this could result in workers sitting idle
but those will have less of a performance impact than the thrashing
that occurs when threads are repeatedly terminated and recreated.
Strategy 3: Increase the `minThreads` configuration option. This has
the same basic effect as increasing the `idleTimeout`. If the queue
pressure is not high enough, workers may sit idle indefinitely but
there will be less of a performance hit.
In applications using Piscina, it will be most effective to use a
combination of these three approaches and tune the various configuration
parameters to find the optimum combination both for the application
workload and the capabilities of the deployment environment. There
are no one set of options that are going to work best.
### Thread priority on Linux systems
On Linux systems that support [`nice(2)`][], Piscina is capable of setting
the priority of every worker in the pool. To use this mechanism, an additional
optional native addon dependency (`@napi-rs/nice`, `npm i @napi-rs/nice`) is required.
Once [`@napi-rs/nice`][] is installed, creating a `Piscina` instance with the
`niceIncrement` configuration option will set the priority for the pool:
```js
const Piscina = require('piscina');
const pool = new Piscina({
worker: '/absolute/path/to/worker.js',
niceIncrement: 20
});
```
The higher the `niceIncrement`, the lower the CPU scheduling priority will be
for the pooled workers which will generally extend the execution time of
CPU-bound tasks but will help prevent those threads from stealing CPU time from
the main Node.js event loop thread. Whether this is a good thing or not depends
entirely on your application and will require careful profiling to get correct.
The key metrics to pay attention to when tuning the `niceIncrement` are the
sampled run times of the tasks in the worker pool (using the [`runTime`][]
property) and the [delay of the Node.js main thread event loop][].
### Multiple Thread Pools and Embedding Piscina as a Dependency
Every `Piscina` instance creates a separate pool of threads and operates
without any awareness of the other. When multiple pools are created in a
single application the various threads may contend with one another, and
with the Node.js main event loop thread, and may cause an overall reduction
in system performance.
Modules that embed Piscina as a dependency *should* make it clear via
documentation that threads are being used. It would be ideal if those
would make it possible for users to provide an existing `Piscina` instance
as a configuration option in lieu of always creating their own.
## The Team
* James M Snell <jasnell@gmail.com>
* Anna Henningsen <anna@addaleax.net>
* Matteo Collina <matteo.collina@gmail.com>
* Rafael Gonzaga <rafael.nunu@hotmail.com>
* Robert Nagy <ronagy@icloud.com>
* Carlos Fuentes <me@metcoder.dev>
## Acknowledgements
Piscina development is sponsored by [NearForm Research][].
[`Atomics`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics
[`EventEmitter`]: https://nodejs.org/api/events.html
[`postMessage`]: https://nodejs.org/api/worker_threads.html#worker_threads_port_postmessage_value_transferlist
[`examples/task-queue`]: https://github.com/jasnell/piscina/blob/master/examples/task-queue/index.js
[`nice(2)`]: https://linux.die.net/man/2/nice
[`@napi-rs/nice`]: https://npmjs.org/package/@napi-rs/nice
[`runTime`]: #property-runtime-readonly
[Custom Task Queues]: #custom_task_queues
[ES modules]: https://nodejs.org/api/esm.html
[Node.js new Worker options]: https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options
[MIT Licensed]: LICENSE.md
[NearForm Research]: https://www.nearform.com/research/
[delay of the Node.js main thread event loop]: https://nodejs.org/dist/latest-v14.x/docs/api/perf_hooks.html#perf_hooks_perf_hooks_monitoreventloopdelay_options

2
node_modules/piscina/benchmark/fixtures/add.js generated vendored Normal file
View File

@@ -0,0 +1,2 @@
'use strict';
module.exports = ({ a, b }) => a + b;

41
node_modules/piscina/benchmark/piscina-queue-comparison.js generated vendored Normal file
View File

@@ -0,0 +1,41 @@
const { Bench } = require('tinybench');
const { Piscina, FixedQueue, ArrayTaskQueue } = require('..');
const { resolve } = require('node:path');
const QUEUE_SIZE = 100_000;
const bench = new Bench({ time: 100, warmup: true });
bench
.add('Piscina with ArrayTaskQueue', async () => {
const queue = new ArrayTaskQueue();
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/add.js'),
taskQueue: queue
});
const tasks = [];
for (let i = 0; i < QUEUE_SIZE; i++) {
tasks.push(pool.run({ a: 4, b: 6 }));
}
await Promise.all(tasks);
await pool.destroy();
})
.add('Piscina with FixedQueue', async () => {
const queue = new FixedQueue();
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/add.js'),
taskQueue: queue
});
const tasks = [];
for (let i = 0; i < QUEUE_SIZE; i++) {
tasks.push(pool.run({ a: 4, b: 6 }));
}
await Promise.all(tasks);
await pool.destroy();
});
(async () => {
await bench.run();
console.table(bench.table());
})();

32
node_modules/piscina/benchmark/queue-comparison.js generated vendored Normal file
View File

@@ -0,0 +1,32 @@
const { Bench } = require('tinybench');
const { ArrayTaskQueue, FixedQueue } = require('..');
const QUEUE_SIZE = 100_000;
const bench = new Bench({ time: 100, warmup: true });
bench
.add('ArrayTaskQueue full push + full shift', async () => {
const queue = new ArrayTaskQueue();
for (let i = 0; i < QUEUE_SIZE; i++) {
queue.push(i);
}
for (let i = 0; i < QUEUE_SIZE; i++) {
queue.shift();
}
})
.add('FixedQueue full push + full shift', async () => {
const queue = new FixedQueue();
for (let i = 0; i < QUEUE_SIZE; i++) {
queue.push(i);
}
for (let i = 0; i < QUEUE_SIZE; i++) {
queue.shift();
}
});
(async () => {
await bench.run();
console.table(bench.table());
})();

29
node_modules/piscina/benchmark/simple-benchmark-async.js generated vendored Normal file
View File

@@ -0,0 +1,29 @@
'use strict';
const { Piscina } = require('../dist');
const { resolve } = require('path');
async function simpleBenchmark ({ duration = 10000 } = {}) {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/add.js'), atomics: 'async' });
let done = 0;
const results = [];
const start = process.hrtime.bigint();
while (pool.queueSize === 0) {
results.push(scheduleTasks());
}
async function scheduleTasks () {
while ((process.hrtime.bigint() - start) / 1_000_000n < duration) {
await pool.run({ a: 4, b: 6 });
done++;
}
}
await Promise.all(results);
return done / duration * 1e3;
}
simpleBenchmark().then((opsPerSecond) => {
console.log(`opsPerSecond: ${opsPerSecond} (with default taskQueue)`);
});

33
node_modules/piscina/benchmark/simple-benchmark-fixed-queue.js generated vendored Normal file
View File

@@ -0,0 +1,33 @@
'use strict';
const { Piscina, FixedQueue } = require('..');
const { resolve } = require('path');
async function simpleBenchmark ({ duration = 10000 } = {}) {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/add.js'),
taskQueue: new FixedQueue()
});
let done = 0;
const results = [];
const start = process.hrtime.bigint();
while (pool.queueSize === 0) {
results.push(scheduleTasks());
}
async function scheduleTasks () {
while ((process.hrtime.bigint() - start) / 1_000_000n < duration) {
await pool.run({ a: 4, b: 6 });
done++;
}
}
await Promise.all(results);
return done / duration * 1e3;
}
simpleBenchmark().then((opsPerSecond) => {
console.log(`opsPerSecond: ${opsPerSecond} (with FixedQueue as taskQueue)`);
});

29
node_modules/piscina/benchmark/simple-benchmark.js generated vendored Normal file
View File

@@ -0,0 +1,29 @@
'use strict';
const { Piscina } = require('..');
const { resolve } = require('path');
async function simpleBenchmark ({ duration = 10000 } = {}) {
const pool = new Piscina({ filename: resolve(__dirname, 'fixtures/add.js') });
let done = 0;
const results = [];
const start = process.hrtime.bigint();
while (pool.queueSize === 0) {
results.push(scheduleTasks());
}
async function scheduleTasks () {
while ((process.hrtime.bigint() - start) / 1_000_000n < duration) {
await pool.run({ a: 4, b: 6 });
done++;
}
}
await Promise.all(results);
return done / duration * 1e3;
}
simpleBenchmark().then((opsPerSecond) => {
console.log(`opsPerSecond: ${opsPerSecond} (with default taskQueue)`);
});

20
node_modules/piscina/dist/abort.d.ts generated vendored Normal file
View File

@@ -0,0 +1,20 @@
interface AbortSignalEventTargetAddOptions {
once: boolean;
}
export interface AbortSignalEventTarget {
addEventListener: (name: 'abort', listener: () => void, options?: AbortSignalEventTargetAddOptions) => void;
removeEventListener: (name: 'abort', listener: () => void) => void;
aborted?: boolean;
reason?: unknown;
}
export interface AbortSignalEventEmitter {
off: (name: 'abort', listener: () => void) => void;
once: (name: 'abort', listener: () => void) => void;
}
export type AbortSignalAny = AbortSignalEventTarget | AbortSignalEventEmitter;
export declare class AbortError extends Error {
constructor(reason?: AbortSignalEventTarget['reason']);
get name(): string;
}
export declare function onabort(abortSignal: AbortSignalAny, listener: () => void): void;
export {};

24
node_modules/piscina/dist/abort.js generated vendored Normal file
View File

@@ -0,0 +1,24 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.AbortError = void 0;
exports.onabort = onabort;
class AbortError extends Error {
constructor(reason) {
// TS does not recognizes the cause clause
// @ts-expect-error
super('The task has been aborted', { cause: reason });
}
get name() {
return 'AbortError';
}
}
exports.AbortError = AbortError;
function onabort(abortSignal, listener) {
if ('addEventListener' in abortSignal) {
abortSignal.addEventListener('abort', listener, { once: true });
}
else {
abortSignal.once('abort', listener);
}
}
//# sourceMappingURL=abort.js.map

1
node_modules/piscina/dist/abort.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"abort.js","sourceRoot":"","sources":["../src/abort.ts"],"names":[],"mappings":";;;AAkCA,0BAMC;AAlBD,MAAa,UAAW,SAAQ,KAAK;IACnC,YAAa,MAAyC;QACpD,0CAA0C;QAC1C,mBAAmB;QACnB,KAAK,CAAC,2BAA2B,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,CAAC,CAAC;IACxD,CAAC;IAED,IAAI,IAAI;QACN,OAAO,YAAY,CAAC;IACtB,CAAC;CACF;AAVD,gCAUC;AAED,SAAgB,OAAO,CAAE,WAA2B,EAAE,QAAoB;IACxE,IAAI,kBAAkB,IAAI,WAAW,EAAE,CAAC;QACtC,WAAW,CAAC,gBAAgB,CAAC,OAAO,EAAE,QAAQ,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;IAClE,CAAC;SAAM,CAAC;QACN,WAAW,CAAC,IAAI,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC;IACtC,CAAC;AACH,CAAC"}

26
node_modules/piscina/dist/common.d.ts generated vendored Normal file
View File

@@ -0,0 +1,26 @@
export declare const READY = "_WORKER_READY";
/**
* True if the object implements the Transferable interface
*
* @export
* @param {unknown} value
* @return {*} {boolean}
*/
export declare function isTransferable(value: unknown): boolean;
/**
* True if object implements Transferable and has been returned
* by the Piscina.move() function
*
* TODO: narrow down the type of value
* @export
* @param {(unknown & PiscinaMovable)} value
* @return {*} {boolean}
*/
export declare function isMovable(value: any): boolean;
export declare function markMovable(value: {}): void;
export declare const commonState: {
isWorkerThread: boolean;
workerData: undefined;
};
export declare function maybeFileURLToPath(filename: string): string;
export declare function getAvailableParallelism(): number;

60
node_modules/piscina/dist/common.js generated vendored Normal file
View File

@@ -0,0 +1,60 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.commonState = exports.READY = void 0;
exports.isTransferable = isTransferable;
exports.isMovable = isMovable;
exports.markMovable = markMovable;
exports.maybeFileURLToPath = maybeFileURLToPath;
exports.getAvailableParallelism = getAvailableParallelism;
const node_url_1 = require("node:url");
const node_os_1 = require("node:os");
const symbols_1 = require("./symbols");
// States wether the worker is ready to receive tasks
exports.READY = '_WORKER_READY';
/**
* True if the object implements the Transferable interface
*
* @export
* @param {unknown} value
* @return {*} {boolean}
*/
function isTransferable(value) {
return (value != null &&
typeof value === 'object' &&
symbols_1.kTransferable in value &&
symbols_1.kValue in value);
}
/**
* True if object implements Transferable and has been returned
* by the Piscina.move() function
*
* TODO: narrow down the type of value
* @export
* @param {(unknown & PiscinaMovable)} value
* @return {*} {boolean}
*/
function isMovable(value) {
return isTransferable(value) && value[symbols_1.kMovable] === true;
}
function markMovable(value) {
Object.defineProperty(value, symbols_1.kMovable, {
enumerable: false,
configurable: true,
writable: true,
value: true
});
}
// State of Piscina pool
exports.commonState = {
isWorkerThread: false,
workerData: undefined
};
function maybeFileURLToPath(filename) {
return filename.startsWith('file:')
? (0, node_url_1.fileURLToPath)(new node_url_1.URL(filename))
: filename;
}
function getAvailableParallelism() {
return (0, node_os_1.availableParallelism)();
}
//# sourceMappingURL=common.js.map

1
node_modules/piscina/dist/common.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"common.js","sourceRoot":"","sources":["../src/common.ts"],"names":[],"mappings":";;;AAeA,wCAOC;AAWD,8BAEC;AAED,kCAOC;AAQD,gDAIC;AAED,0DAEC;AA5DD,uCAA8C;AAC9C,qCAA+C;AAE/C,uCAA4D;AAE5D,qDAAqD;AACxC,QAAA,KAAK,GAAG,eAAe,CAAC;AAErC;;;;;;GAMG;AACH,SAAgB,cAAc,CAAE,KAAc;IAC5C,OAAO,CACL,KAAK,IAAI,IAAI;QACb,OAAO,KAAK,KAAK,QAAQ;QACzB,uBAAa,IAAI,KAAK;QACtB,gBAAM,IAAI,KAAK,CAChB,CAAC;AACJ,CAAC;AAED;;;;;;;;GAQG;AACH,SAAgB,SAAS,CAAE,KAAU;IACnC,OAAO,cAAc,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,kBAAQ,CAAC,KAAK,IAAI,CAAC;AAC3D,CAAC;AAED,SAAgB,WAAW,CAAE,KAAS;IACpC,MAAM,CAAC,cAAc,CAAC,KAAK,EAAE,kBAAQ,EAAE;QACrC,UAAU,EAAE,KAAK;QACjB,YAAY,EAAE,IAAI;QAClB,QAAQ,EAAE,IAAI;QACd,KAAK,EAAE,IAAI;KACZ,CAAC,CAAC;AACL,CAAC;AAED,wBAAwB;AACX,QAAA,WAAW,GAAG;IACzB,cAAc,EAAE,KAAK;IACrB,UAAU,EAAE,SAAS;CACtB,CAAC;AAEF,SAAgB,kBAAkB,CAAE,QAAiB;IACnD,OAAO,QAAQ,CAAC,UAAU,CAAC,OAAO,CAAC;QACjC,CAAC,CAAC,IAAA,wBAAa,EAAC,IAAI,cAAG,CAAC,QAAQ,CAAC,CAAC;QAClC,CAAC,CAAC,QAAQ,CAAC;AACf,CAAC;AAED,SAAgB,uBAAuB;IACrC,OAAO,IAAA,8BAAoB,GAAE,CAAC;AAChC,CAAC"}

7
node_modules/piscina/dist/errors.d.ts generated vendored Normal file
View File

@@ -0,0 +1,7 @@
export declare const Errors: {
ThreadTermination: () => Error;
FilenameNotProvided: () => Error;
TaskQueueAtLimit: () => Error;
NoTaskQueueAvailable: () => Error;
CloseTimeout: () => Error;
};

11
node_modules/piscina/dist/errors.js generated vendored Normal file
View File

@@ -0,0 +1,11 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.Errors = void 0;
exports.Errors = {
ThreadTermination: () => new Error('Terminating worker thread'),
FilenameNotProvided: () => new Error('filename must be provided to run() or in options object'),
TaskQueueAtLimit: () => new Error('Task queue is at limit'),
NoTaskQueueAvailable: () => new Error('No task queue available and all Workers are busy'),
CloseTimeout: () => new Error('Close operation timed out')
};
//# sourceMappingURL=errors.js.map

1
node_modules/piscina/dist/errors.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"errors.js","sourceRoot":"","sources":["../src/errors.ts"],"names":[],"mappings":";;;AAAa,QAAA,MAAM,GAAG;IACpB,iBAAiB,EAAE,GAAG,EAAE,CAAC,IAAI,KAAK,CAAC,2BAA2B,CAAC;IAC/D,mBAAmB,EAAE,GAAG,EAAE,CACxB,IAAI,KAAK,CAAC,yDAAyD,CAAC;IACtE,gBAAgB,EAAE,GAAG,EAAE,CAAC,IAAI,KAAK,CAAC,wBAAwB,CAAC;IAC3D,oBAAoB,EAAE,GAAG,EAAE,CACzB,IAAI,KAAK,CAAC,kDAAkD,CAAC;IAC/D,YAAY,EAAE,GAAG,EAAE,CAAC,IAAI,KAAK,CAAC,2BAA2B,CAAC;CAC3D,CAAC"}

13
node_modules/piscina/dist/esm-wrapper.mjs generated vendored Normal file
View File

@@ -0,0 +1,13 @@
import mod from "./main.js";
export default mod;
export const ArrayTaskQueue = mod.ArrayTaskQueue;
export const FixedQueue = mod.FixedQueue;
export const Piscina = mod.Piscina;
export const isWorkerThread = mod.isWorkerThread;
export const move = mod.move;
export const queueOptionsSymbol = mod.queueOptionsSymbol;
export const transferableSymbol = mod.transferableSymbol;
export const valueSymbol = mod.valueSymbol;
export const version = mod.version;
export const workerData = mod.workerData;

42
node_modules/piscina/dist/histogram.d.ts generated vendored Normal file
View File

@@ -0,0 +1,42 @@
import { RecordableHistogram } from 'node:perf_hooks';
export type PiscinaHistogramSummary = {
average: number;
mean: number;
stddev: number;
min: number;
max: number;
p0_001: number;
p0_01: number;
p0_1: number;
p1: number;
p2_5: number;
p10: number;
p25: number;
p50: number;
p75: number;
p90: number;
p97_5: number;
p99: number;
p99_9: number;
p99_99: number;
p99_999: number;
};
export type PiscinaHistogram = {
runTime: PiscinaHistogramSummary;
waitTime: PiscinaHistogramSummary;
resetRunTime(): void;
resetWaitTime(): void;
};
export declare class PiscinaHistogramHandler {
#private;
constructor();
get runTimeSummary(): PiscinaHistogramSummary;
get waitTimeSummary(): PiscinaHistogramSummary;
get runTimeCount(): number;
recordRunTime(value: number): void;
recordWaitTime(value: number): void;
resetWaitTime(): void;
resetRunTime(): void;
static createHistogramSummary(histogram: RecordableHistogram): PiscinaHistogramSummary;
static toHistogramIntegerNano(milliseconds: number): number;
}

76
node_modules/piscina/dist/histogram.js generated vendored Normal file
View File

@@ -0,0 +1,76 @@
"use strict";
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _PiscinaHistogramHandler_runTime, _PiscinaHistogramHandler_waitTime;
Object.defineProperty(exports, "__esModule", { value: true });
exports.PiscinaHistogramHandler = void 0;
const node_perf_hooks_1 = require("node:perf_hooks");
class PiscinaHistogramHandler {
constructor() {
_PiscinaHistogramHandler_runTime.set(this, void 0);
_PiscinaHistogramHandler_waitTime.set(this, void 0);
__classPrivateFieldSet(this, _PiscinaHistogramHandler_runTime, (0, node_perf_hooks_1.createHistogram)(), "f");
__classPrivateFieldSet(this, _PiscinaHistogramHandler_waitTime, (0, node_perf_hooks_1.createHistogram)(), "f");
}
get runTimeSummary() {
return PiscinaHistogramHandler.createHistogramSummary(__classPrivateFieldGet(this, _PiscinaHistogramHandler_runTime, "f"));
}
get waitTimeSummary() {
return PiscinaHistogramHandler.createHistogramSummary(__classPrivateFieldGet(this, _PiscinaHistogramHandler_waitTime, "f"));
}
get runTimeCount() {
return __classPrivateFieldGet(this, _PiscinaHistogramHandler_runTime, "f").count;
}
recordRunTime(value) {
__classPrivateFieldGet(this, _PiscinaHistogramHandler_runTime, "f").record(PiscinaHistogramHandler.toHistogramIntegerNano(value));
}
recordWaitTime(value) {
__classPrivateFieldGet(this, _PiscinaHistogramHandler_waitTime, "f").record(PiscinaHistogramHandler.toHistogramIntegerNano(value));
}
resetWaitTime() {
__classPrivateFieldGet(this, _PiscinaHistogramHandler_waitTime, "f").reset();
}
resetRunTime() {
__classPrivateFieldGet(this, _PiscinaHistogramHandler_runTime, "f").reset();
}
static createHistogramSummary(histogram) {
const { mean, stddev, min, max } = histogram;
return {
average: mean / 1000,
mean: mean / 1000,
stddev,
min: min / 1000,
max: max / 1000,
p0_001: histogram.percentile(0.001) / 1000,
p0_01: histogram.percentile(0.01) / 1000,
p0_1: histogram.percentile(0.1) / 1000,
p1: histogram.percentile(1) / 1000,
p2_5: histogram.percentile(2.5) / 1000,
p10: histogram.percentile(10) / 1000,
p25: histogram.percentile(25) / 1000,
p50: histogram.percentile(50) / 1000,
p75: histogram.percentile(75) / 1000,
p90: histogram.percentile(90) / 1000,
p97_5: histogram.percentile(97.5) / 1000,
p99: histogram.percentile(99) / 1000,
p99_9: histogram.percentile(99.9) / 1000,
p99_99: histogram.percentile(99.99) / 1000,
p99_999: histogram.percentile(99.999) / 1000,
};
}
static toHistogramIntegerNano(milliseconds) {
return Math.max(1, Math.trunc(milliseconds * 1000));
}
}
exports.PiscinaHistogramHandler = PiscinaHistogramHandler;
_PiscinaHistogramHandler_runTime = new WeakMap(), _PiscinaHistogramHandler_waitTime = new WeakMap();
//# sourceMappingURL=histogram.js.map

1
node_modules/piscina/dist/histogram.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"histogram.js","sourceRoot":"","sources":["../src/histogram.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;AAAA,qDAAuE;AAgCvE,MAAa,uBAAuB;IAIlC;QAHA,mDAA8B;QAC9B,oDAA+B;QAG7B,uBAAA,IAAI,oCAAY,IAAA,iCAAe,GAAE,MAAA,CAAC;QAClC,uBAAA,IAAI,qCAAa,IAAA,iCAAe,GAAE,MAAA,CAAC;IACrC,CAAC;IAED,IAAI,cAAc;QAChB,OAAO,uBAAuB,CAAC,sBAAsB,CAAC,uBAAA,IAAI,wCAAS,CAAC,CAAC;IACvE,CAAC;IAED,IAAI,eAAe;QACjB,OAAO,uBAAuB,CAAC,sBAAsB,CAAC,uBAAA,IAAI,yCAAU,CAAC,CAAC;IACxE,CAAC;IAED,IAAI,YAAY;QACd,OAAO,uBAAA,IAAI,wCAAS,CAAC,KAAK,CAAC;IAC7B,CAAC;IAED,aAAa,CAAC,KAAa;QACzB,uBAAA,IAAI,wCAAS,CAAC,MAAM,CAAC,uBAAuB,CAAC,sBAAsB,CAAC,KAAK,CAAC,CAAC,CAAC;IAC9E,CAAC;IAED,cAAc,CAAC,KAAa;QAC1B,uBAAA,IAAI,yCAAU,CAAC,MAAM,CACnB,uBAAuB,CAAC,sBAAsB,CAAC,KAAK,CAAC,CACtD,CAAC;IACJ,CAAC;IAED,aAAa;QACX,uBAAA,IAAI,yCAAU,CAAC,KAAK,EAAE,CAAC;IACzB,CAAC;IAED,YAAY;QACV,uBAAA,IAAI,wCAAS,CAAC,KAAK,EAAE,CAAC;IACxB,CAAC;IAED,MAAM,CAAC,sBAAsB,CAC3B,SAA8B;QAE9B,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,SAAS,CAAC;QAE7C,OAAO;YACL,OAAO,EAAE,IAAI,GAAG,IAAI;YACpB,IAAI,EAAE,IAAI,GAAG,IAAI;YACjB,MAAM;YACN,GAAG,EAAE,GAAG,GAAG,IAAI;YACf,GAAG,EAAE,GAAG,GAAG,IAAI;YACf,MAAM,EAAE,SAAS,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,IAAI;YAC1C,KAAK,EAAE,SAAS,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,IAAI;YACxC,IAAI,EAAE,SAAS,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,IAAI;YACtC,EAAE,EAAE,SAAS,CAAC,UAAU,CAAC,CAAC,CAAC,GAAG,IAAI;YAClC,IAAI,EAAE,SAAS,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,IAAI;YACtC,GAAG,EAAE,SAAS,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,IAAI;YACpC,GAAG,EAAE,SAAS,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,IAAI;YACpC,GAAG,EAAE,SAAS,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,IAAI;YACpC,GAAG,EAAE,SAAS,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,IAAI;YACpC,GAAG,EAAE,SAAS,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,IAAI;YACpC,KAAK,EAAE,SAAS,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,IAAI;YACxC,GAAG,EAAE,SAAS,CAAC,UAAU,CAAC,EAAE,CAAC,GAAG,IAAI;YACpC,KAAK,EAAE,SAAS,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,IAAI;YACxC,MAAM,EAAE,SAAS,CAAC,UAAU,CAAC,KAAK,CAAC,GAAG,IAAI;YAC1C,OAAO,EAAE,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,GAAG,IAAI;SAC7C,CAAC;IACJ,CAAC;IAED,MAAM,CAAC,sBAAsB,CAAC,YAAoB;QAChD,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,YAAY,GAAG,IAAI,CAAC,CAAC,CAAC;IACtD,CAAC;CACF;AAvED,0DAuEC"}

88
node_modules/piscina/dist/index.d.ts generated vendored Normal file
View File

@@ -0,0 +1,88 @@
import { Worker, MessagePort } from 'node:worker_threads';
import { EventEmitterAsyncResource } from 'node:events';
import { version } from '../package.json';
import type { Transferable, ResourceLimits, EnvSpecifier } from './types';
import { kQueueOptions, kTransferable, kValue } from './symbols';
import { TaskQueue, ArrayTaskQueue, FixedQueue, PiscinaTask, TransferList, TransferListItem } from './task_queue';
import { PiscinaLoadBalancer } from './worker_pool';
import { AbortSignalAny } from './abort';
import { PiscinaHistogram } from './histogram';
interface Options {
filename?: string | null;
name?: string;
minThreads?: number;
maxThreads?: number;
idleTimeout?: number;
maxQueue?: number | 'auto';
concurrentTasksPerWorker?: number;
atomics?: 'sync' | 'async' | 'disabled';
resourceLimits?: ResourceLimits;
argv?: string[];
execArgv?: string[];
env?: EnvSpecifier;
workerData?: any;
taskQueue?: TaskQueue;
niceIncrement?: number;
trackUnmanagedFds?: boolean;
closeTimeout?: number;
recordTiming?: boolean;
loadBalancer?: PiscinaLoadBalancer;
workerHistogram?: boolean;
}
interface FilledOptions extends Options {
filename: string | null;
name: string;
minThreads: number;
maxThreads: number;
idleTimeout: number;
maxQueue: number;
concurrentTasksPerWorker: number;
atomics: Options['atomics'];
taskQueue: TaskQueue;
niceIncrement: number;
closeTimeout: number;
recordTiming: boolean;
workerHistogram: boolean;
}
interface RunOptions {
transferList?: TransferList;
filename?: string | null;
signal?: AbortSignalAny | null;
name?: string | null;
}
interface CloseOptions {
force?: boolean;
}
export default class Piscina<T = any, R = any> extends EventEmitterAsyncResource {
#private;
constructor(options?: Options);
run(task: T, options?: RunOptions): Promise<R>;
close(options?: CloseOptions): Promise<void>;
destroy(): Promise<void>;
[Symbol.dispose](): void;
[Symbol.asyncDispose](): Promise<void>;
get maxThreads(): number;
get minThreads(): number;
get options(): FilledOptions;
get threads(): Worker[];
get queueSize(): number;
get completed(): number;
get histogram(): PiscinaHistogram;
get utilization(): number;
get duration(): number;
get needsDrain(): boolean;
static get isWorkerThread(): boolean;
static get workerData(): any;
static get version(): string;
static get Piscina(): typeof Piscina;
static get FixedQueue(): typeof FixedQueue;
static get ArrayTaskQueue(): typeof ArrayTaskQueue;
static move(val: Transferable | TransferListItem | ArrayBufferView | ArrayBuffer | MessagePort): ArrayBuffer | ArrayBufferView<ArrayBufferLike> | MessagePort | Transferable;
static get transferableSymbol(): symbol;
static get valueSymbol(): symbol;
static get queueOptionsSymbol(): symbol;
}
export declare const move: typeof Piscina.move;
export declare const isWorkerThread: boolean;
export declare const workerData: any;
export { Piscina, PiscinaTask, TaskQueue, kTransferable as transferableSymbol, kValue as valueSymbol, kQueueOptions as queueOptionsSymbol, version, FixedQueue };

791
node_modules/piscina/dist/index.js generated vendored Normal file
View File

@@ -0,0 +1,791 @@
"use strict";
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var _DirectlyTransferable_value, _ArrayBufferViewTransferable_view, _Piscina_pool, _Piscina_histogram;
Object.defineProperty(exports, "__esModule", { value: true });
exports.FixedQueue = exports.version = exports.queueOptionsSymbol = exports.valueSymbol = exports.transferableSymbol = exports.Piscina = exports.workerData = exports.isWorkerThread = exports.move = void 0;
const node_worker_threads_1 = require("node:worker_threads");
const node_events_1 = require("node:events");
const node_path_1 = require("node:path");
const node_util_1 = require("node:util");
const node_perf_hooks_1 = require("node:perf_hooks");
const promises_1 = require("node:timers/promises");
const package_json_1 = require("../package.json");
Object.defineProperty(exports, "version", { enumerable: true, get: function () { return package_json_1.version; } });
const symbols_1 = require("./symbols");
Object.defineProperty(exports, "queueOptionsSymbol", { enumerable: true, get: function () { return symbols_1.kQueueOptions; } });
Object.defineProperty(exports, "transferableSymbol", { enumerable: true, get: function () { return symbols_1.kTransferable; } });
Object.defineProperty(exports, "valueSymbol", { enumerable: true, get: function () { return symbols_1.kValue; } });
const task_queue_1 = require("./task_queue");
Object.defineProperty(exports, "FixedQueue", { enumerable: true, get: function () { return task_queue_1.FixedQueue; } });
const worker_pool_1 = require("./worker_pool");
const abort_1 = require("./abort");
const histogram_1 = require("./histogram");
const errors_1 = require("./errors");
const common_1 = require("./common");
const cpuParallelism = (0, common_1.getAvailableParallelism)();
const kDefaultOptions = {
filename: null,
name: 'default',
minThreads: Math.max(Math.floor(cpuParallelism / 2), 1),
maxThreads: cpuParallelism * 1.5,
idleTimeout: 0,
maxQueue: Infinity,
concurrentTasksPerWorker: 1,
atomics: 'sync',
taskQueue: new task_queue_1.ArrayTaskQueue(),
niceIncrement: 0,
trackUnmanagedFds: true,
closeTimeout: 30000,
recordTiming: true,
workerHistogram: false
};
const kDefaultRunOptions = {
transferList: undefined,
filename: null,
signal: null,
name: null
};
const kDefaultCloseOptions = {
force: false
};
class DirectlyTransferable {
constructor(value) {
_DirectlyTransferable_value.set(this, void 0);
__classPrivateFieldSet(this, _DirectlyTransferable_value, value, "f");
}
get [(_DirectlyTransferable_value = new WeakMap(), symbols_1.kTransferable)]() { return __classPrivateFieldGet(this, _DirectlyTransferable_value, "f"); }
get [symbols_1.kValue]() { return __classPrivateFieldGet(this, _DirectlyTransferable_value, "f"); }
}
class ArrayBufferViewTransferable {
constructor(view) {
_ArrayBufferViewTransferable_view.set(this, void 0);
__classPrivateFieldSet(this, _ArrayBufferViewTransferable_view, view, "f");
}
get [(_ArrayBufferViewTransferable_view = new WeakMap(), symbols_1.kTransferable)]() { return __classPrivateFieldGet(this, _ArrayBufferViewTransferable_view, "f").buffer; }
get [symbols_1.kValue]() { return __classPrivateFieldGet(this, _ArrayBufferViewTransferable_view, "f"); }
}
class ThreadPool {
constructor(publicInterface, options) {
var _a, _b, _c;
this.skipQueue = [];
this.completed = 0;
this.histogram = null;
this.start = node_perf_hooks_1.performance.now();
this.inProcessPendingMessages = false;
this.startingUp = false;
this.closingUp = false;
this.workerFailsDuringBootstrap = false;
this.destroying = false;
this.publicInterface = publicInterface;
this.taskQueue = (_a = options.taskQueue) !== null && _a !== void 0 ? _a : new task_queue_1.FixedQueue();
const filename = options.filename ? (0, common_1.maybeFileURLToPath)(options.filename) : null;
this.options = { ...kDefaultOptions, ...options, filename, maxQueue: 0 };
if (this.options.recordTiming) {
this.histogram = new histogram_1.PiscinaHistogramHandler();
}
// The >= and <= could be > and < but this way we get 100 % coverage 🙃
if (options.maxThreads !== undefined &&
this.options.minThreads >= options.maxThreads) {
this.options.minThreads = options.maxThreads;
}
if (options.minThreads !== undefined &&
this.options.maxThreads <= options.minThreads) {
this.options.maxThreads = options.minThreads;
}
if (options.maxQueue === 'auto') {
this.options.maxQueue = this.options.maxThreads ** 2;
}
else {
this.options.maxQueue = (_b = options.maxQueue) !== null && _b !== void 0 ? _b : kDefaultOptions.maxQueue;
}
this.balancer = (_c = this.options.loadBalancer) !== null && _c !== void 0 ? _c : (0, worker_pool_1.LeastBusyBalancer)({ maximumUsage: this.options.concurrentTasksPerWorker });
this.workers = new worker_pool_1.AsynchronouslyCreatedResourcePool(this.options.concurrentTasksPerWorker);
this.workers.onTaskDone((w) => this._onWorkerTaskDone(w));
this.maxCapacity = this.options.maxThreads * this.options.concurrentTasksPerWorker;
this.startingUp = true;
this._ensureMinimumWorkers();
this.startingUp = false;
this._needsDrain = false;
}
_ensureMinimumWorkers() {
if (this.closingUp || this.destroying) {
return;
}
while (this.workers.size < this.options.minThreads) {
this._addNewWorker();
}
}
_addNewWorker() {
if (this.closingUp)
return;
const pool = this;
const worker = new node_worker_threads_1.Worker((0, node_path_1.resolve)(__dirname, 'worker.js'), {
env: this.options.env,
argv: this.options.argv,
execArgv: this.options.execArgv,
resourceLimits: this.options.resourceLimits,
workerData: this.options.workerData,
trackUnmanagedFds: this.options.trackUnmanagedFds
});
const { port1, port2 } = new node_worker_threads_1.MessageChannel();
const workerInfo = new worker_pool_1.WorkerInfo(worker, port1, onMessage, this.options.workerHistogram);
workerInfo.onDestroy(() => {
this.publicInterface.emit('workerDestroy', workerInfo.interface);
});
if (this.startingUp) {
// There is no point in waiting for the initial set of Workers to indicate
// that they are ready, we just mark them as such from the start.
workerInfo.markAsReady();
// We need to emit the event in the next microtask, so that the user can
// attach event listeners before the event is emitted.
queueMicrotask(() => {
this.publicInterface.emit('workerCreate', workerInfo.interface);
this._onWorkerReady(workerInfo);
});
}
else {
workerInfo.onReady(() => {
this.publicInterface.emit('workerCreate', workerInfo.interface);
this._onWorkerReady(workerInfo);
});
}
const message = {
filename: this.options.filename,
name: this.options.name,
port: port2,
sharedBuffer: workerInfo.sharedBuffer,
atomics: this.options.atomics,
niceIncrement: this.options.niceIncrement
};
worker.postMessage(message, [port2]);
function onMessage(message) {
const { taskId, result } = message;
// In case of success: Call the callback that was passed to `runTask`,
// remove the `TaskInfo` associated with the Worker, which marks it as
// free again.
const taskInfo = workerInfo.taskInfos.get(taskId);
workerInfo.taskInfos.delete(taskId);
// TODO: we can abstract the task info handling
// right into the pool.workers.taskDone method
pool.workers.taskDone(workerInfo);
/* istanbul ignore if */
if (taskInfo === undefined) {
const err = new Error(`Unexpected message from Worker: ${(0, node_util_1.inspect)(message)}`);
pool.publicInterface.emit('error', err);
}
else {
taskInfo.done(message.error, result);
}
pool._processPendingMessages();
}
function onReady() {
if (workerInfo.currentUsage() === 0) {
workerInfo.unref();
}
if (!workerInfo.isReady()) {
workerInfo.markAsReady();
}
}
function onEventMessage(message) {
pool.publicInterface.emit('message', message);
}
worker.on('message', (message) => {
message instanceof Object && common_1.READY in message ? onReady() : onEventMessage(message);
});
worker.on('error', (err) => {
this._onError(worker, workerInfo, err, false);
});
worker.on('exit', (exitCode) => {
if (this.destroying) {
return;
}
const err = new Error(`worker exited with code: ${exitCode}`);
// Only error unfinished tasks on process exit, since there are legitimate
// reasons to exit workers and we want to handle that gracefully when possible.
this._onError(worker, workerInfo, err, true);
});
worker.unref();
port1.on('close', () => {
// The port is only closed if the Worker stops for some reason, but we
// always .unref() the Worker itself. We want to receive e.g. 'error'
// events on it, so we ref it once we know it's going to exit anyway.
worker.ref();
});
this.workers.add(workerInfo);
}
_onError(worker, workerInfo, err, onlyErrorUnfinishedTasks) {
// Work around the bug in https://github.com/nodejs/node/pull/33394
worker.ref = () => { };
const taskInfos = [...workerInfo.taskInfos.values()];
workerInfo.taskInfos.clear();
// Remove the worker from the list and potentially start a new Worker to
// replace the current one.
this._removeWorker(workerInfo);
if (workerInfo.isReady() && !this.workerFailsDuringBootstrap) {
this._ensureMinimumWorkers();
}
else {
// Do not start new workers over and over if they already fail during
// bootstrap, there's no point.
this.workerFailsDuringBootstrap = true;
}
if (taskInfos.length > 0) {
// If there are remaining unfinished tasks, call the callback that was
// passed to `postTask` with the error
for (const taskInfo of taskInfos) {
taskInfo.done(err, null);
}
}
else if (!onlyErrorUnfinishedTasks) {
// If there are no unfinished tasks, instead emit an 'error' event
this.publicInterface.emit('error', err);
}
}
_processPendingMessages() {
if (this.inProcessPendingMessages || this.options.atomics === 'disabled') {
return;
}
this.inProcessPendingMessages = true;
try {
for (const workerInfo of this.workers) {
workerInfo.processPendingMessages();
}
}
finally {
this.inProcessPendingMessages = false;
}
}
_removeWorker(workerInfo) {
workerInfo.destroy();
this.workers.delete(workerInfo);
}
_onWorkerReady(workerInfo) {
this._onWorkerAvailable(workerInfo);
}
_onWorkerTaskDone(workerInfo) {
this._onWorkerAvailable(workerInfo);
}
_onWorkerAvailable(workerInfo) {
let workers = null;
while ((this.taskQueue.size > 0 || this.skipQueue.length > 0)) {
// The skipQueue will have tasks that we previously shifted off
// the task queue but had to skip over... we have to make sure
// we drain that before we drain the taskQueue.
const taskInfo = this.skipQueue.shift() ||
this.taskQueue.shift();
if (workers == null) {
workers = [...this.workers].map(workerInfo => workerInfo.interface);
}
const distributed = this._distributeTask(taskInfo, workers);
if (distributed) {
// If task was distributed, we should continue to distribute more tasks
continue;
}
else if (this.workers.size < this.options.maxThreads) {
// We spawn if possible
// TODO: scheduler will intercept this.
this._addNewWorker();
continue;
}
else {
// If balancer states that pool is busy, we should stop trying to distribute tasks
break;
}
}
//If Infinity was sent as a parameter, we skip setting the Timeout that clears the worker
if (this.options.idleTimeout === Infinity) {
return;
}
// If more workers than minThreads, we can remove idle workers
if (workerInfo.currentUsage() === 0 &&
this.workers.size > this.options.minThreads) {
workerInfo.idleTimeout = setTimeout(() => {
if (workerInfo.currentUsage() !== 0) {
// Exit early - we can't safely remove the worker.
return;
}
if (this.workers.size > this.options.minThreads) {
this._removeWorker(workerInfo);
}
}, this.options.idleTimeout).unref();
}
}
_distributeTask(task, workers) {
var _a;
// We need to verify if the task is aborted already or not
// otherwise we might be distributing aborted tasks to workers
if (task.aborted)
return true;
const candidate = this.balancer(task.interface, workers);
// Seeking for a real worker instead of customized one
if (candidate != null && candidate[symbols_1.kWorkerData] != null) {
const now = node_perf_hooks_1.performance.now();
(_a = this.histogram) === null || _a === void 0 ? void 0 : _a.recordWaitTime(now - task.created);
task.started = now;
candidate[symbols_1.kWorkerData].postTask(task);
queueMicrotask(() => this._maybeDrain());
// If candidate, let's try to distribute more tasks
return true;
}
if (task.abortSignal) {
this.skipQueue.push(task);
}
else {
this.taskQueue.push(task);
}
return false;
}
runTask(task, options) {
var _a;
let { filename, name } = options;
const { transferList = [] } = options;
if (filename == null) {
filename = this.options.filename;
}
if (name == null) {
name = this.options.name;
}
if (typeof filename !== 'string') {
return Promise.reject(errors_1.Errors.FilenameNotProvided());
}
filename = (0, common_1.maybeFileURLToPath)(filename);
let signal;
if (this.closingUp || this.destroying) {
const closingUpAbortController = new AbortController();
closingUpAbortController.abort('queue is being terminated');
signal = closingUpAbortController.signal;
}
else {
signal = (_a = options.signal) !== null && _a !== void 0 ? _a : null;
}
let resolve;
let reject;
// eslint-disable-next-line
const ret = new Promise((res, rej) => { resolve = res; reject = rej; });
const taskInfo = new task_queue_1.TaskInfo(task, transferList, filename, name, (err, result) => {
var _a;
this.completed++;
if (taskInfo.started) {
(_a = this.histogram) === null || _a === void 0 ? void 0 : _a.recordRunTime(node_perf_hooks_1.performance.now() - taskInfo.started);
}
if (err !== null) {
reject(err);
}
else {
resolve(result);
}
queueMicrotask(() => this._maybeDrain());
}, signal, this.publicInterface.asyncResource.asyncId());
if (signal !== null) {
// If the AbortSignal has an aborted property and it's truthy,
// reject immediately.
if (signal.aborted) {
reject(new abort_1.AbortError(signal.reason));
return ret;
}
taskInfo.abortListener = () => {
// Call reject() first to make sure we always reject with the AbortError
// if the task is aborted, not with an Error from the possible
// thread termination below.
reject(new abort_1.AbortError(signal.reason));
if (taskInfo.workerInfo !== null) {
// Already running: We cancel the Worker this is running on.
this._removeWorker(taskInfo.workerInfo);
this._ensureMinimumWorkers();
}
else {
// Not yet running: Remove it from the queue.
// Call should be idempotent
this.taskQueue.remove(taskInfo);
}
};
(0, abort_1.onabort)(signal, taskInfo.abortListener);
}
if (this.taskQueue.size > 0) {
const totalCapacity = this.options.maxQueue + this.pendingCapacity();
if (this.taskQueue.size >= totalCapacity) {
if (this.options.maxQueue === 0) {
reject(errors_1.Errors.NoTaskQueueAvailable());
}
else {
reject(errors_1.Errors.TaskQueueAtLimit());
}
}
else {
this.taskQueue.push(taskInfo);
}
queueMicrotask(() => this._maybeDrain());
return ret;
}
const workers = [...this.workers.readyItems].map(workerInfo => workerInfo.interface);
const distributed = this._distributeTask(taskInfo, workers);
if (!distributed) {
// We spawn if possible
// TODO: scheduler will intercept this.
if (this.workers.size < this.options.maxThreads) {
this._addNewWorker();
}
// We reject if no task queue set and no more pending capacity.
if (this.options.maxQueue <= 0 && this.pendingCapacity() === 0) {
reject(errors_1.Errors.NoTaskQueueAvailable());
}
}
;
queueMicrotask(() => this._maybeDrain());
return ret;
}
pendingCapacity() {
return this.workers.pendingItems.size *
this.options.concurrentTasksPerWorker;
}
_maybeDrain() {
/**
* Our goal is to make it possible for user space to use the pool
* in a way where always waiting === 0,
* since we want to avoid creating tasks that can't execute
* immediately in order to provide back pressure to the task source.
*/
const { maxCapacity } = this;
const currentUsage = this.workers.getCurrentUsage();
if (maxCapacity === currentUsage) {
this._needsDrain = true;
queueMicrotask(() => this.publicInterface.emit('needsDrain'));
}
else if (maxCapacity > currentUsage && this._needsDrain) {
this._needsDrain = false;
queueMicrotask(() => this.publicInterface.emit('drain'));
}
}
async destroy() {
this.destroying = true;
while (this.skipQueue.length > 0) {
const taskInfo = this.skipQueue.shift();
taskInfo.done(new Error('Terminating worker thread'));
}
while (this.taskQueue.size > 0) {
const taskInfo = this.taskQueue.shift();
taskInfo.done(new Error('Terminating worker thread'));
}
const exitEvents = [];
while (this.workers.size > 0) {
const [workerInfo] = this.workers;
exitEvents.push((0, node_events_1.once)(workerInfo.worker, 'exit'));
this._removeWorker(workerInfo);
}
try {
await Promise.all(exitEvents);
}
finally {
this.destroying = false;
}
}
async close(options) {
this.closingUp = true;
if (options.force) {
const skipQueueLength = this.skipQueue.length;
for (let i = 0; i < skipQueueLength; i++) {
const taskInfo = this.skipQueue.shift();
if (taskInfo.workerInfo === null) {
taskInfo.done(new abort_1.AbortError('pool is closed'));
}
else {
this.skipQueue.push(taskInfo);
}
}
const taskQueueLength = this.taskQueue.size;
for (let i = 0; i < taskQueueLength; i++) {
const taskInfo = this.taskQueue.shift();
if (taskInfo.workerInfo === null) {
taskInfo.done(new abort_1.AbortError('pool is closed'));
}
else {
this.taskQueue.push(taskInfo);
}
}
}
const onPoolFlushed = () => new Promise((resolve) => {
const numberOfWorkers = this.workers.size;
if (numberOfWorkers === 0) {
resolve();
return;
}
let numberOfWorkersDone = 0;
const checkIfWorkerIsDone = (workerInfo) => {
if (workerInfo.taskInfos.size === 0) {
numberOfWorkersDone++;
}
if (numberOfWorkers === numberOfWorkersDone) {
resolve();
}
};
for (const workerInfo of this.workers) {
checkIfWorkerIsDone(workerInfo);
this.workers.onTaskDone(checkIfWorkerIsDone);
}
});
const throwOnTimeOut = async (timeout) => {
await (0, promises_1.setTimeout)(timeout, null, { ref: false });
throw errors_1.Errors.CloseTimeout();
};
try {
await Promise.race([
onPoolFlushed(),
throwOnTimeOut(this.options.closeTimeout)
]);
}
catch (error) {
this.publicInterface.emit('error', error);
}
finally {
await this.destroy();
this.publicInterface.emit('close');
this.closingUp = false;
}
}
}
class Piscina extends node_events_1.EventEmitterAsyncResource {
constructor(options = {}) {
super({ ...options, name: 'Piscina' });
_Piscina_pool.set(this, void 0);
_Piscina_histogram.set(this, null);
if (typeof options.filename !== 'string' && options.filename != null) {
throw new TypeError('options.filename must be a string or null');
}
if (typeof options.name !== 'string' && options.name != null) {
throw new TypeError('options.name must be a string or null');
}
if (options.minThreads !== undefined &&
(typeof options.minThreads !== 'number' || options.minThreads < 0)) {
throw new TypeError('options.minThreads must be a non-negative integer');
}
if (options.maxThreads !== undefined &&
(typeof options.maxThreads !== 'number' || options.maxThreads < 1)) {
throw new TypeError('options.maxThreads must be a positive integer');
}
if (options.minThreads !== undefined && options.maxThreads !== undefined &&
options.minThreads > options.maxThreads) {
throw new RangeError('options.minThreads and options.maxThreads must not conflict');
}
if (options.idleTimeout !== undefined &&
(typeof options.idleTimeout !== 'number' || options.idleTimeout < 0)) {
throw new TypeError('options.idleTimeout must be a non-negative integer');
}
if (options.maxQueue !== undefined &&
options.maxQueue !== 'auto' &&
(typeof options.maxQueue !== 'number' || options.maxQueue < 0)) {
throw new TypeError('options.maxQueue must be a non-negative integer');
}
if (options.concurrentTasksPerWorker !== undefined &&
(typeof options.concurrentTasksPerWorker !== 'number' ||
options.concurrentTasksPerWorker < 1)) {
throw new TypeError('options.concurrentTasksPerWorker must be a positive integer');
}
if (options.atomics != null && (typeof options.atomics !== 'string' ||
!['sync', 'async', 'disabled'].includes(options.atomics))) {
throw new TypeError('options.atomics should be a value of sync, sync or disabled.');
}
if (options.resourceLimits !== undefined &&
(typeof options.resourceLimits !== 'object' ||
options.resourceLimits === null)) {
throw new TypeError('options.resourceLimits must be an object');
}
if (options.taskQueue !== undefined && !(0, task_queue_1.isTaskQueue)(options.taskQueue)) {
throw new TypeError('options.taskQueue must be a TaskQueue object');
}
if (options.niceIncrement !== undefined &&
(typeof options.niceIncrement !== 'number' || (options.niceIncrement < 0 && process.platform !== 'win32'))) {
throw new TypeError('options.niceIncrement must be a non-negative integer on Unix systems');
}
if (options.trackUnmanagedFds !== undefined &&
typeof options.trackUnmanagedFds !== 'boolean') {
throw new TypeError('options.trackUnmanagedFds must be a boolean value');
}
if (options.closeTimeout !== undefined && (typeof options.closeTimeout !== 'number' || options.closeTimeout < 0)) {
throw new TypeError('options.closeTimeout must be a non-negative integer');
}
if (options.loadBalancer !== undefined && (typeof options.loadBalancer !== 'function' || options.loadBalancer.length < 1)) {
throw new TypeError('options.loadBalancer must be a function with at least two args');
}
if (options.workerHistogram !== undefined && (typeof options.workerHistogram !== 'boolean')) {
throw new TypeError('options.workerHistogram must be a boolean');
}
__classPrivateFieldSet(this, _Piscina_pool, new ThreadPool(this, options), "f");
}
run(task, options = kDefaultRunOptions) {
if (options === null || typeof options !== 'object') {
return Promise.reject(new TypeError('options must be an object'));
}
const { transferList, filename, name, signal } = options;
if (transferList !== undefined && !Array.isArray(transferList)) {
return Promise.reject(new TypeError('transferList argument must be an Array'));
}
if (filename != null && typeof filename !== 'string') {
return Promise.reject(new TypeError('filename argument must be a string'));
}
if (name != null && typeof name !== 'string') {
return Promise.reject(new TypeError('name argument must be a string'));
}
if (signal != null && typeof signal !== 'object') {
return Promise.reject(new TypeError('signal argument must be an object'));
}
return __classPrivateFieldGet(this, _Piscina_pool, "f").runTask(task, { transferList, filename, name, signal });
}
async close(options = kDefaultCloseOptions) {
if (options === null || typeof options !== 'object') {
throw TypeError('options must be an object');
}
let { force } = options;
if (force !== undefined && typeof force !== 'boolean') {
return Promise.reject(new TypeError('force argument must be a boolean'));
}
force !== null && force !== void 0 ? force : (force = kDefaultCloseOptions.force);
return __classPrivateFieldGet(this, _Piscina_pool, "f").close({
force
});
}
destroy() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").destroy();
}
[(_Piscina_pool = new WeakMap(), _Piscina_histogram = new WeakMap(), Symbol.dispose)]() {
this.close();
}
[Symbol.asyncDispose]() {
return this.close();
}
get maxThreads() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").options.maxThreads;
}
get minThreads() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").options.minThreads;
}
get options() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").options;
}
get threads() {
const ret = [];
for (const workerInfo of __classPrivateFieldGet(this, _Piscina_pool, "f").workers) {
ret.push(workerInfo.worker);
}
return ret;
}
get queueSize() {
const pool = __classPrivateFieldGet(this, _Piscina_pool, "f");
return Math.max(pool.taskQueue.size - pool.pendingCapacity(), 0);
}
get completed() {
return __classPrivateFieldGet(this, _Piscina_pool, "f").completed;
}
get histogram() {
if (__classPrivateFieldGet(this, _Piscina_histogram, "f") == null) {
const piscinahistogram = {
// @ts-expect-error
get runTime() { var _a; return (_a = this.histogram) === null || _a === void 0 ? void 0 : _a.runTimeSummary; },
// @ts-expect-error
get waitTime() { var _a; return (_a = this.histogram) === null || _a === void 0 ? void 0 : _a.waitTimeSummary; },
resetRunTime() {
var _a;
// @ts-expect-error
(_a = this.histogram) === null || _a === void 0 ? void 0 : _a.resetRunTime();
},
resetWaitTime() {
var _a;
// @ts-expect-error
(_a = this.histogram) === null || _a === void 0 ? void 0 : _a.resetWaitTime();
},
};
Object.defineProperty(piscinahistogram, 'histogram', {
value: __classPrivateFieldGet(this, _Piscina_pool, "f").histogram,
writable: false,
enumerable: false,
configurable: false,
});
__classPrivateFieldSet(this, _Piscina_histogram, piscinahistogram, "f");
}
;
return __classPrivateFieldGet(this, _Piscina_histogram, "f");
}
get utilization() {
if (__classPrivateFieldGet(this, _Piscina_pool, "f").histogram == null) {
return 0;
}
// count is available as of Node.js v16.14.0 but not present in the types
const count = __classPrivateFieldGet(this, _Piscina_pool, "f").histogram.runTimeCount;
if (count === 0) {
return 0;
}
// The capacity is the max compute time capacity of the
// pool to this point in time as determined by the length
// of time the pool has been running multiplied by the
// maximum number of threads.
const capacity = this.duration * __classPrivateFieldGet(this, _Piscina_pool, "f").options.maxThreads;
const totalMeanRuntime = (__classPrivateFieldGet(this, _Piscina_pool, "f").histogram.runTimeSummary.mean / 1000) * count;
// We calculate the appoximate pool utilization by multiplying
// the mean run time of all tasks by the number of runtime
// samples taken and dividing that by the capacity. The
// theory here is that capacity represents the absolute upper
// limit of compute time this pool could ever attain (but
// never will for a variety of reasons. Multiplying the
// mean run time by the number of tasks sampled yields an
// approximation of the realized compute time. The utilization
// then becomes a point-in-time measure of how active the
// pool is.
return totalMeanRuntime / capacity;
}
get duration() {
return node_perf_hooks_1.performance.now() - __classPrivateFieldGet(this, _Piscina_pool, "f").start;
}
get needsDrain() {
return __classPrivateFieldGet(this, _Piscina_pool, "f")._needsDrain;
}
static get isWorkerThread() {
return common_1.commonState.isWorkerThread;
}
static get workerData() {
return common_1.commonState.workerData;
}
static get version() {
return package_json_1.version;
}
static get Piscina() {
return Piscina;
}
static get FixedQueue() {
return task_queue_1.FixedQueue;
}
static get ArrayTaskQueue() {
return task_queue_1.ArrayTaskQueue;
}
static move(val) {
if (val != null && typeof val === 'object' && typeof val !== 'function') {
if (!(0, common_1.isTransferable)(val)) {
if (node_util_1.types.isArrayBufferView(val)) {
val = new ArrayBufferViewTransferable(val);
}
else {
val = new DirectlyTransferable(val);
}
}
(0, common_1.markMovable)(val);
}
return val;
}
static get transferableSymbol() { return symbols_1.kTransferable; }
static get valueSymbol() { return symbols_1.kValue; }
static get queueOptionsSymbol() { return symbols_1.kQueueOptions; }
}
exports.default = Piscina;
exports.Piscina = Piscina;
exports.move = Piscina.move;
exports.isWorkerThread = Piscina.isWorkerThread;
exports.workerData = Piscina.workerData;
//# sourceMappingURL=index.js.map

1
node_modules/piscina/dist/index.js.map generated vendored Normal file
View File

File diff suppressed because one or more lines are too long

2
node_modules/piscina/dist/main.d.ts generated vendored Normal file
View File

@@ -0,0 +1,2 @@
import Piscina from './index';
export = Piscina;

7
node_modules/piscina/dist/main.js generated vendored Normal file
View File

@@ -0,0 +1,7 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
const index_1 = __importDefault(require("./index"));
module.exports = index_1.default;
//# sourceMappingURL=main.js.map

1
node_modules/piscina/dist/main.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"main.js","sourceRoot":"","sources":["../src/main.ts"],"names":[],"mappings":";;;;AAAA,oDAA8B;AAG9B,iBAAS,eAAO,CAAC"}

8
node_modules/piscina/dist/symbols.d.ts generated vendored Normal file
View File

@@ -0,0 +1,8 @@
export declare const kMovable: unique symbol;
export declare const kWorkerData: unique symbol;
export declare const kTransferable: unique symbol;
export declare const kValue: unique symbol;
export declare const kQueueOptions: unique symbol;
export declare const kRequestCountField = 0;
export declare const kResponseCountField = 1;
export declare const kFieldCount = 2;

14
node_modules/piscina/dist/symbols.js generated vendored Normal file
View File

@@ -0,0 +1,14 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.kFieldCount = exports.kResponseCountField = exports.kRequestCountField = exports.kQueueOptions = exports.kValue = exports.kTransferable = exports.kWorkerData = exports.kMovable = void 0;
// Internal symbol used to mark Transferable objects returned
// by the Piscina.move() function
exports.kMovable = Symbol('Piscina.kMovable');
exports.kWorkerData = Symbol('Piscina.kWorkerData');
exports.kTransferable = Symbol.for('Piscina.transferable');
exports.kValue = Symbol.for('Piscina.valueOf');
exports.kQueueOptions = Symbol.for('Piscina.queueOptions');
exports.kRequestCountField = 0;
exports.kResponseCountField = 1;
exports.kFieldCount = 2;
//# sourceMappingURL=symbols.js.map

1
node_modules/piscina/dist/symbols.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"symbols.js","sourceRoot":"","sources":["../src/symbols.ts"],"names":[],"mappings":";;;AAAA,6DAA6D;AAC7D,iCAAiC;AACpB,QAAA,QAAQ,GAAG,MAAM,CAAC,kBAAkB,CAAC,CAAC;AACtC,QAAA,WAAW,GAAG,MAAM,CAAC,qBAAqB,CAAC,CAAC;AAC5C,QAAA,aAAa,GAAG,MAAM,CAAC,GAAG,CAAC,sBAAsB,CAAC,CAAC;AACnD,QAAA,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,iBAAiB,CAAC,CAAC;AACvC,QAAA,aAAa,GAAG,MAAM,CAAC,GAAG,CAAC,sBAAsB,CAAC,CAAC;AACnD,QAAA,kBAAkB,GAAG,CAAC,CAAC;AACvB,QAAA,mBAAmB,GAAG,CAAC,CAAC;AACxB,QAAA,WAAW,GAAG,CAAC,CAAC"}

8
node_modules/piscina/dist/task_queue/array_queue.d.ts generated vendored Normal file
View File

@@ -0,0 +1,8 @@
import type { TaskQueue, Task } from '.';
export declare class ArrayTaskQueue implements TaskQueue {
tasks: Task[];
get size(): number;
shift(): Task | null;
push(task: Task): void;
remove(task: Task): void;
}

29
node_modules/piscina/dist/task_queue/array_queue.js generated vendored Normal file
View File

@@ -0,0 +1,29 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.ArrayTaskQueue = void 0;
const node_assert_1 = __importDefault(require("node:assert"));
class ArrayTaskQueue {
constructor() {
this.tasks = [];
}
get size() {
return this.tasks.length;
}
shift() {
var _a;
return (_a = this.tasks.shift()) !== null && _a !== void 0 ? _a : null;
}
push(task) {
this.tasks.push(task);
}
remove(task) {
const index = this.tasks.indexOf(task);
node_assert_1.default.notStrictEqual(index, -1);
this.tasks.splice(index, 1);
}
}
exports.ArrayTaskQueue = ArrayTaskQueue;
//# sourceMappingURL=array_queue.js.map

1
node_modules/piscina/dist/task_queue/array_queue.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"array_queue.js","sourceRoot":"","sources":["../../src/task_queue/array_queue.ts"],"names":[],"mappings":";;;;;;AAAA,8DAAiC;AAIjC,MAAa,cAAc;IAA3B;QACE,UAAK,GAAW,EAAE,CAAA;IAmBpB,CAAC;IAjBC,IAAI,IAAI;QACN,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC;IAC3B,CAAC;IAED,KAAK;;QACH,OAAO,MAAA,IAAI,CAAC,KAAK,CAAC,KAAK,EAAE,mCAAI,IAAI,CAAC;IACpC,CAAC;IAED,IAAI,CAAE,IAAU;QACd,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IACxB,CAAC;IAED,MAAM,CAAE,IAAU;QAChB,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;QACvC,qBAAM,CAAC,cAAc,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,CAAC;QACjC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;IAC9B,CAAC;CACF;AApBD,wCAoBC"}

17
node_modules/piscina/dist/task_queue/common.d.ts generated vendored Normal file
View File

@@ -0,0 +1,17 @@
import type { kQueueOptions } from '../symbols';
export interface TaskQueue {
readonly size: number;
shift(): Task | null;
remove(task: Task): void;
push(task: Task): void;
}
export interface PiscinaTask extends Task {
taskId: number;
filename: string;
name: string;
created: number;
isAbortable: boolean;
}
export interface Task {
readonly [kQueueOptions]: object | null;
}

4
node_modules/piscina/dist/task_queue/common.js generated vendored Normal file
View File

@@ -0,0 +1,4 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
;
//# sourceMappingURL=common.js.map

1
node_modules/piscina/dist/task_queue/common.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"common.js","sourceRoot":"","sources":["../../src/task_queue/common.ts"],"names":[],"mappings":";;AAoBC,CAAC"}

25
node_modules/piscina/dist/task_queue/fixed_queue.d.ts generated vendored Normal file
View File

@@ -0,0 +1,25 @@
import type { Task } from './common';
import { TaskQueue } from '.';
declare class FixedCircularBuffer {
bottom: number;
top: number;
list: Array<Task | undefined>;
next: FixedCircularBuffer | null;
isEmpty(): boolean;
isFull(): boolean;
push(data: Task): void;
shift(): Task | null;
remove(task: Task): void;
}
export declare class FixedQueue implements TaskQueue {
#private;
head: FixedCircularBuffer;
tail: FixedCircularBuffer;
constructor();
isEmpty(): boolean;
push(data: Task): void;
shift(): Task | null;
remove(task: Task): void;
get size(): number;
}
export {};

192
node_modules/piscina/dist/task_queue/fixed_queue.js generated vendored Normal file
View File

@@ -0,0 +1,192 @@
"use strict";
var __classPrivateFieldGet = (this && this.__classPrivateFieldGet) || function (receiver, state, kind, f) {
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a getter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot read private member from an object whose class did not declare it");
return kind === "m" ? f : kind === "a" ? f.call(receiver) : f ? f.value : state.get(receiver);
};
var __classPrivateFieldSet = (this && this.__classPrivateFieldSet) || function (receiver, state, value, kind, f) {
if (kind === "m") throw new TypeError("Private method is not writable");
if (kind === "a" && !f) throw new TypeError("Private accessor was defined without a setter");
if (typeof state === "function" ? receiver !== state || !f : !state.has(receiver)) throw new TypeError("Cannot write private member to an object whose class did not declare it");
return (kind === "a" ? f.call(receiver, value) : f ? f.value = value : state.set(receiver, value)), value;
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
var _FixedQueue_size;
Object.defineProperty(exports, "__esModule", { value: true });
exports.FixedQueue = void 0;
/*
* Modified Fixed Queue Implementation based on the one from Node.js Project
* License: MIT License
* Source: https://github.com/nodejs/node/blob/de7b37880f5a541d5f874c1c2362a65a4be76cd0/lib/internal/fixed_queue.js
*/
const node_assert_1 = __importDefault(require("node:assert"));
// Currently optimal queue size, tested on V8 6.0 - 6.6. Must be power of two.
const kSize = 2048;
const kMask = kSize - 1;
// The FixedQueue is implemented as a singly-linked list of fixed-size
// circular buffers. It looks something like this:
//
// head tail
// | |
// v v
// +-----------+ <-----\ +-----------+ <------\ +-----------+
// | [null] | \----- | next | \------- | next |
// +-----------+ +-----------+ +-----------+
// | item | <-- bottom | item | <-- bottom | [empty] |
// | item | | item | | [empty] |
// | item | | item | | [empty] |
// | item | | item | | [empty] |
// | item | | item | bottom --> | item |
// | item | | item | | item |
// | ... | | ... | | ... |
// | item | | item | | item |
// | item | | item | | item |
// | [empty] | <-- top | item | | item |
// | [empty] | | item | | item |
// | [empty] | | [empty] | <-- top top --> | [empty] |
// +-----------+ +-----------+ +-----------+
//
// Or, if there is only one circular buffer, it looks something
// like either of these:
//
// head tail head tail
// | | | |
// v v v v
// +-----------+ +-----------+
// | [null] | | [null] |
// +-----------+ +-----------+
// | [empty] | | item |
// | [empty] | | item |
// | item | <-- bottom top --> | [empty] |
// | item | | [empty] |
// | [empty] | <-- top bottom --> | item |
// | [empty] | | item |
// +-----------+ +-----------+
//
// Adding a value means moving `top` forward by one, removing means
// moving `bottom` forward by one. After reaching the end, the queue
// wraps around.
//
// When `top === bottom` the current queue is empty and when
// `top + 1 === bottom` it's full. This wastes a single space of storage
// but allows much quicker checks.
class FixedCircularBuffer {
constructor() {
this.bottom = 0;
this.top = 0;
this.list = new Array(kSize);
this.next = null;
}
isEmpty() {
return this.top === this.bottom;
}
isFull() {
return ((this.top + 1) & kMask) === this.bottom;
}
push(data) {
this.list[this.top] = data;
this.top = (this.top + 1) & kMask;
}
shift() {
const nextItem = this.list[this.bottom];
if (nextItem === undefined) {
return null;
}
this.list[this.bottom] = undefined;
this.bottom = (this.bottom + 1) & kMask;
return nextItem;
}
remove(task) {
const indexToRemove = this.list.indexOf(task);
node_assert_1.default.notStrictEqual(indexToRemove, -1);
let curr = indexToRemove;
while (true) {
const next = (curr + 1) & kMask;
this.list[curr] = this.list[next];
if (this.list[curr] === undefined) {
break;
}
if (next === indexToRemove) {
this.list[curr] = undefined;
break;
}
curr = next;
}
this.top = (this.top - 1) & kMask;
}
}
class FixedQueue {
constructor() {
_FixedQueue_size.set(this, 0);
this.head = this.tail = new FixedCircularBuffer();
}
isEmpty() {
return this.head.isEmpty();
}
push(data) {
var _a;
if (this.head.isFull()) {
// Head is full: Creates a new queue, sets the old queue's `.next` to it,
// and sets it as the new main queue.
this.head = this.head.next = new FixedCircularBuffer();
}
this.head.push(data);
__classPrivateFieldSet(this, _FixedQueue_size, (_a = __classPrivateFieldGet(this, _FixedQueue_size, "f"), _a++, _a), "f");
}
shift() {
var _a;
const tail = this.tail;
const next = tail.shift();
if (next !== null)
__classPrivateFieldSet(this, _FixedQueue_size, (_a = __classPrivateFieldGet(this, _FixedQueue_size, "f"), _a--, _a), "f");
if (tail.isEmpty() && tail.next !== null) {
// If there is another queue, it forms the new tail.
this.tail = tail.next;
tail.next = null;
}
return next;
}
remove(task) {
var _a;
let prev = null;
let buffer = this.tail;
while (true) {
if (buffer.list.includes(task)) {
buffer.remove(task);
__classPrivateFieldSet(this, _FixedQueue_size, (_a = __classPrivateFieldGet(this, _FixedQueue_size, "f"), _a--, _a), "f");
break;
}
if (buffer.next === null)
break;
prev = buffer;
buffer = buffer.next;
}
if (buffer.isEmpty()) {
// removing tail
if (prev === null) {
// if tail is not the last buffer
if (buffer.next !== null)
this.tail = buffer.next;
}
else {
// removing head
if (buffer.next === null) {
this.head = prev;
}
else {
// removing buffer from middle
prev.next = buffer.next;
}
}
}
}
get size() {
return __classPrivateFieldGet(this, _FixedQueue_size, "f");
}
}
exports.FixedQueue = FixedQueue;
_FixedQueue_size = new WeakMap();
;
//# sourceMappingURL=fixed_queue.js.map

1
node_modules/piscina/dist/task_queue/fixed_queue.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"fixed_queue.js","sourceRoot":"","sources":["../../src/task_queue/fixed_queue.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;AAAA;;;;GAIG;AACH,8DAAiC;AAGjC,8EAA8E;AAC9E,MAAM,KAAK,GAAG,IAAI,CAAC;AACnB,MAAM,KAAK,GAAG,KAAK,GAAG,CAAC,CAAC;AAExB,sEAAsE;AACtE,kDAAkD;AAClD,EAAE;AACF,mEAAmE;AACnE,kEAAkE;AAClE,kEAAkE;AAClE,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,2EAA2E;AAC3E,EAAE;AACF,+DAA+D;AAC/D,wBAAwB;AACxB,EAAE;AACF,2DAA2D;AAC3D,yDAAyD;AACzD,yDAAyD;AACzD,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,4DAA4D;AAC5D,EAAE;AACF,mEAAmE;AACnE,oEAAoE;AACpE,gBAAgB;AAChB,EAAE;AACF,4DAA4D;AAC5D,wEAAwE;AACxE,kCAAkC;AAElC,MAAM,mBAAmB;IAAzB;QACE,WAAM,GAAW,CAAC,CAAA;QAClB,QAAG,GAAW,CAAC,CAAA;QACf,SAAI,GAA4B,IAAI,KAAK,CAAC,KAAK,CAAC,CAAA;QAChD,SAAI,GAA+B,IAAI,CAAA;IA0CzC,CAAC;IAxCC,OAAO;QACL,OAAO,IAAI,CAAC,GAAG,KAAK,IAAI,CAAC,MAAM,CAAC;IAClC,CAAC;IAED,MAAM;QACJ,OAAO,CAAC,CAAC,IAAI,CAAC,GAAG,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,KAAK,IAAI,CAAC,MAAM,CAAC;IAClD,CAAC;IAED,IAAI,CAAE,IAAS;QACb,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC;QAC3B,IAAI,CAAC,GAAG,GAAG,CAAC,IAAI,CAAC,GAAG,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;IACpC,CAAC;IAED,KAAK;QACH,MAAM,QAAQ,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACxC,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YAAC,OAAO,IAAI,CAAC;QAAC,CAAC;QAC5C,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,GAAG,SAAS,CAAC;QACnC,IAAI,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;QACxC,OAAO,QAAQ,CAAC;IAClB,CAAC;IAED,MAAM,CAAE,IAAU;QAChB,MAAM,aAAa,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;QAE9C,qBAAM,CAAC,cAAc,CAAC,aAAa,EAAE,CAAC,CAAC,CAAC,CAAC;QACzC,IAAI,IAAI,GAAG,aAAa,CAAC;QACzB,OAAO,IAAI,EAAE,CAAC;YACZ,MAAM,IAAI,GAAG,CAAC,IAAI,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;YAChC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YAClC,IAAI,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,SAAS,EAAE,CAAC;gBAClC,MAAM;YACR,CAAC;YACD,IAAI,IAAI,KAAK,aAAa,EAAE,CAAC;gBAC3B,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,SAAS,CAAC;gBAC5B,MAAM;YACR,CAAC;YACD,IAAI,GAAG,IAAI,CAAC;QACd,CAAC;QACD,IAAI,CAAC,GAAG,GAAG,CAAC,IAAI,CAAC,GAAG,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;IACpC,CAAC;CACF;AAED,MAAa,UAAU;IAKrB;QAFA,2BAAgB,CAAC,EAAA;QAGf,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,GAAG,IAAI,mBAAmB,EAAE,CAAC;IACpD,CAAC;IAED,OAAO;QACL,OAAO,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;IAC7B,CAAC;IAED,IAAI,CAAE,IAAS;;QACb,IAAI,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,EAAE,CAAC;YACvB,yEAAyE;YACzE,qCAAqC;YACrC,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,GAAG,IAAI,mBAAmB,EAAE,CAAC;QACzD,CAAC;QACD,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACrB,+CAAA,CAAA,wDAAU,EAAV,IAAY,IAAA,CAAA,MAAA,CAAC;IACf,CAAC;IAED,KAAK;;QACH,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC;QACvB,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,EAAE,CAAC;QAC1B,IAAI,IAAI,KAAK,IAAI;YAAE,+CAAA,CAAA,wDAAU,EAAV,IAAY,IAAA,CAAA,MAAA,CAAC;QAChC,IAAI,IAAI,CAAC,OAAO,EAAE,IAAI,IAAI,CAAC,IAAI,KAAK,IAAI,EAAE,CAAC;YACzC,oDAAoD;YACpD,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC;YACtB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACnB,CAAC;QACD,OAAO,IAAI,CAAC;IACd,CAAC;IAED,MAAM,CAAE,IAAU;;QAChB,IAAI,IAAI,GAA+B,IAAI,CAAC;QAC5C,IAAI,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC;QACvB,OAAO,IAAI,EAAE,CAAC;YACZ,IAAI,MAAM,CAAC,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC/B,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;gBACpB,+CAAA,CAAA,wDAAU,EAAV,IAAY,IAAA,CAAA,MAAA,CAAC;gBACb,MAAM;YACR,CAAC;YACD,IAAI,MAAM,CAAC,IAAI,KAAK,IAAI;gBAAE,MAAM;YAChC,IAAI,GAAG,MAAM,CAAC;YACd,MAAM,GAAG,MAAM,CAAC,IAAI,CAAC;QACvB,CAAC;QACD,IAAI,MAAM,CAAC,OAAO,EAAE,EAAE,CAAC;YACrB,gBAAgB;YAChB,IAAI,IAAI,KAAK,IAAI,EAAE,CAAC;gBAClB,iCAAiC;gBACjC,IAAI,MAAM,CAAC,IAAI,KAAK,IAAI;oBAAE,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC;YACpD,CAAC;iBAAM,CAAC;gBACN,gBAAgB;gBAChB,IAAI,MAAM,CAAC,IAAI,KAAK,IAAI,EAAE,CAAC;oBACzB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;gBACnB,CAAC;qBAAM,CAAC;oBACN,8BAA8B;oBAC9B,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC;gBAC1B,CAAC;YACH,CAAC;QACH,CAAC;IACH,CAAC;IAED,IAAI,IAAI;QACN,OAAO,uBAAA,IAAI,wBAAM,CAAC;IACpB,CAAC;CACF;AApED,gCAoEC;;AAAA,CAAC"}

43
node_modules/piscina/dist/task_queue/index.d.ts generated vendored Normal file
View File

@@ -0,0 +1,43 @@
import type { MessagePort } from 'node:worker_threads';
import { AsyncResource } from 'node:async_hooks';
import type { WorkerInfo } from '../worker_pool';
import type { AbortSignalAny } from '../abort';
import { kQueueOptions } from '../symbols';
import type { Task, TaskQueue, PiscinaTask } from './common';
export { ArrayTaskQueue } from './array_queue';
export { FixedQueue } from './fixed_queue';
export type TaskCallback = (err: Error, result: any) => void;
export type TransferList = MessagePort extends {
postMessage: (value: any, transferList: infer T) => any;
} ? T : never;
export type TransferListItem = TransferList extends Array<infer T> ? T : never;
/**
* Verifies if a given TaskQueue is valid
*
* @export
* @param {*} value
* @return {*} {boolean}
*/
export declare function isTaskQueue(value: TaskQueue): boolean;
export declare class TaskInfo extends AsyncResource implements Task {
callback: TaskCallback;
task: any;
transferList: TransferList;
filename: string;
name: string;
taskId: number;
abortSignal: AbortSignalAny | null;
workerInfo: WorkerInfo | null;
created: number;
started: number;
aborted: boolean;
_abortListener: (() => void) | null;
constructor(task: any, transferList: TransferList, filename: string, name: string, callback: TaskCallback, abortSignal: AbortSignalAny | null, triggerAsyncId: number);
set abortListener(value: (() => void));
get abortListener(): (() => void) | null;
releaseTask(): any;
done(err: Error | null, result?: any): void;
get [kQueueOptions](): {} | null;
get interface(): PiscinaTask;
}
export { Task, TaskQueue, PiscinaTask };

108
node_modules/piscina/dist/task_queue/index.js generated vendored Normal file
View File

@@ -0,0 +1,108 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.TaskInfo = exports.FixedQueue = exports.ArrayTaskQueue = void 0;
exports.isTaskQueue = isTaskQueue;
const node_perf_hooks_1 = require("node:perf_hooks");
const node_async_hooks_1 = require("node:async_hooks");
const common_1 = require("../common");
const symbols_1 = require("../symbols");
var array_queue_1 = require("./array_queue");
Object.defineProperty(exports, "ArrayTaskQueue", { enumerable: true, get: function () { return array_queue_1.ArrayTaskQueue; } });
var fixed_queue_1 = require("./fixed_queue");
Object.defineProperty(exports, "FixedQueue", { enumerable: true, get: function () { return fixed_queue_1.FixedQueue; } });
/**
* Verifies if a given TaskQueue is valid
*
* @export
* @param {*} value
* @return {*} {boolean}
*/
function isTaskQueue(value) {
return (typeof value === 'object' &&
value !== null &&
'size' in value &&
typeof value.shift === 'function' &&
typeof value.remove === 'function' &&
typeof value.push === 'function');
}
let taskIdCounter = 0;
// Extend AsyncResource so that async relations between posting a task and
// receiving its result are visible to diagnostic tools.
class TaskInfo extends node_async_hooks_1.AsyncResource {
constructor(task, transferList, filename, name, callback, abortSignal, triggerAsyncId) {
super('Piscina.Task', { requireManualDestroy: true, triggerAsyncId });
// abortListener : (() => void) | null = null;
this.workerInfo = null;
this.aborted = false;
this._abortListener = null;
this.callback = callback;
this.task = task;
this.transferList = transferList;
// If the task is a Transferable returned by
// Piscina.move(), then add it to the transferList
// automatically
if ((0, common_1.isMovable)(task)) {
// This condition should never be hit but typescript
// complains if we dont do the check.
/* istanbul ignore if */
if (this.transferList == null) {
this.transferList = [];
}
this.transferList =
this.transferList.concat(task[symbols_1.kTransferable]);
this.task = task[symbols_1.kValue];
}
this.filename = filename;
this.name = name;
// TODO: This should not be global
this.taskId = taskIdCounter++;
this.abortSignal = abortSignal;
this.created = node_perf_hooks_1.performance.now();
this.started = 0;
}
// TODO: improve this handling - ideally should be extended
set abortListener(value) {
this._abortListener = () => {
this.aborted = true;
value();
};
}
get abortListener() {
return this._abortListener;
}
releaseTask() {
const ret = this.task;
this.task = null;
return ret;
}
done(err, result) {
this.runInAsyncScope(this.callback, null, err, result);
this.emitDestroy(); // `TaskInfo`s are used only once.
// If an abort signal was used, remove the listener from it when
// done to make sure we do not accidentally leak.
if (this.abortSignal && this.abortListener) {
if ('removeEventListener' in this.abortSignal && this.abortListener) {
this.abortSignal.removeEventListener('abort', this.abortListener);
}
else {
this.abortSignal.off('abort', this.abortListener);
}
}
}
get [symbols_1.kQueueOptions]() {
var _a, _b;
return (_b = (_a = this.task) === null || _a === void 0 ? void 0 : _a[symbols_1.kQueueOptions]) !== null && _b !== void 0 ? _b : null;
}
get interface() {
return {
taskId: this.taskId,
filename: this.filename,
name: this.name,
created: this.created,
isAbortable: this.abortSignal !== null,
[symbols_1.kQueueOptions]: this[symbols_1.kQueueOptions]
};
}
}
exports.TaskInfo = TaskInfo;
//# sourceMappingURL=index.js.map

1
node_modules/piscina/dist/task_queue/index.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/task_queue/index.ts"],"names":[],"mappings":";;;AAgCA,kCASC;AAxCD,qDAA8C;AAC9C,uDAAiD;AAIjD,sCAAsC;AACtC,wCAAkE;AAIlE,6CAA+C;AAAtC,6GAAA,cAAc,OAAA;AACvB,6CAA2C;AAAlC,yGAAA,UAAU,OAAA;AAanB;;;;;;GAMG;AACH,SAAgB,WAAW,CAAE,KAAgB;IAC3C,OAAO,CACL,OAAO,KAAK,KAAK,QAAQ;QACzB,KAAK,KAAK,IAAI;QACd,MAAM,IAAI,KAAK;QACf,OAAO,KAAK,CAAC,KAAK,KAAK,UAAU;QACjC,OAAO,KAAK,CAAC,MAAM,KAAK,UAAU;QAClC,OAAO,KAAK,CAAC,IAAI,KAAK,UAAU,CACjC,CAAC;AACJ,CAAC;AAED,IAAI,aAAa,GAAG,CAAC,CAAC;AACtB,0EAA0E;AAC1E,wDAAwD;AACxD,MAAa,QAAS,SAAQ,gCAAa;IAevC,YACE,IAAU,EACV,YAA2B,EAC3B,QAAiB,EACjB,IAAa,EACb,QAAuB,EACvB,WAAmC,EACnC,cAAuB;QACvB,KAAK,CAAC,cAAc,EAAE,EAAE,oBAAoB,EAAE,IAAI,EAAE,cAAc,EAAE,CAAC,CAAC;QAfxE,8CAA8C;QAC9C,eAAU,GAAuB,IAAI,CAAC;QAGtC,YAAO,GAAG,KAAK,CAAC;QAChB,mBAAc,GAAwB,IAAI,CAAC;QAWzC,IAAI,CAAC,QAAQ,GAAG,QAAQ,CAAC;QACzB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,YAAY,GAAG,YAAY,CAAC;QAEjC,4CAA4C;QAC5C,kDAAkD;QAClD,gBAAgB;QAChB,IAAI,IAAA,kBAAS,EAAC,IAAI,CAAC,EAAE,CAAC;YACpB,oDAAoD;YACpD,qCAAqC;YACrC,wBAAwB;YACxB,IAAI,IAAI,CAAC,YAAY,IAAI,IAAI,EAAE,CAAC;gBAC9B,IAAI,CAAC,YAAY,GAAG,EAAE,CAAC;YACzB,CAAC;YACD,IAAI,CAAC,YAAY;gBACf,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,uBAAa,CAAC,CAAC,CAAC;YAChD,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC,gBAAM,CAAC,CAAC;QAC3B,CAAC;QAED,IAAI,CAAC,QAAQ,GAAG,QAAQ,CAAC;QACzB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,kCAAkC;QAClC,IAAI,CAAC,MAAM,GAAG,aAAa,EAAE,CAAC;QAC9B,IAAI,CAAC,WAAW,GAAG,WAAW,CAAC;QAC/B,IAAI,CAAC,OAAO,GAAG,6BAAW,CAAC,GAAG,EAAE,CAAC;QACjC,IAAI,CAAC,OAAO,GAAG,CAAC,CAAC;IACnB,CAAC;IAED,2DAA2D;IAC3D,IAAI,aAAa,CAAE,KAAmB;QACpC,IAAI,CAAC,cAAc,GAAG,GAAG,EAAE;YACzB,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC;YACpB,KAAK,EAAE,CAAC;QACV,CAAC,CAAC;IACJ,CAAC;IAED,IAAI,aAAa;QACf,OAAO,IAAI,CAAC,cAAc,CAAC;IAC7B,CAAC;IAED,WAAW;QACT,MAAM,GAAG,GAAG,IAAI,CAAC,IAAI,CAAC;QACtB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,OAAO,GAAG,CAAC;IACb,CAAC;IAED,IAAI,CAAE,GAAkB,EAAE,MAAa;QACrC,IAAI,CAAC,eAAe,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC;QACvD,IAAI,CAAC,WAAW,EAAE,CAAC,CAAC,kCAAkC;QACtD,gEAAgE;QAChE,iDAAiD;QACjD,IAAI,IAAI,CAAC,WAAW,IAAI,IAAI,CAAC,aAAa,EAAE,CAAC;YAC3C,IAAI,qBAAqB,IAAI,IAAI,CAAC,WAAW,IAAI,IAAI,CAAC,aAAa,EAAE,CAAC;gBACpE,IAAI,CAAC,WAAW,CAAC,mBAAmB,CAAC,OAAO,EAAE,IAAI,CAAC,aAAa,CAAC,CAAC;YACpE,CAAC;iBAAM,CAAC;gBACL,IAAI,CAAC,WAAuC,CAAC,GAAG,CAC/C,OAAO,EAAE,IAAI,CAAC,aAAa,CAAC,CAAC;YACjC,CAAC;QACH,CAAC;IACH,CAAC;IAED,IAAI,CAAC,uBAAa,CAAC;;QACjB,OAAO,MAAA,MAAA,IAAI,CAAC,IAAI,0CAAG,uBAAa,CAAC,mCAAI,IAAI,CAAC;IAC5C,CAAC;IAED,IAAI,SAAS;QACX,OAAO;YACL,MAAM,EAAE,IAAI,CAAC,MAAM;YACnB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,IAAI,EAAE,IAAI,CAAC,IAAI;YACf,OAAO,EAAE,IAAI,CAAC,OAAO;YACrB,WAAW,EAAE,IAAI,CAAC,WAAW,KAAK,IAAI;YACtC,CAAC,uBAAa,CAAC,EAAE,IAAI,CAAC,uBAAa,CAAC;SACrC,CAAC;IACJ,CAAC;CACJ;AAnGD,4BAmGC"}

43
node_modules/piscina/dist/types.d.ts generated vendored Normal file
View File

@@ -0,0 +1,43 @@
import type { MessagePort, Worker } from 'node:worker_threads';
import type { READY } from './common';
import type { kTransferable, kValue } from './symbols';
export interface StartupMessage {
filename: string | null;
name: string;
port: MessagePort;
sharedBuffer: Int32Array;
atomics: 'async' | 'sync' | 'disabled';
niceIncrement: number;
}
export interface RequestMessage {
taskId: number;
task: any;
filename: string;
name: string;
histogramEnabled: number;
}
export interface ReadyMessage {
[READY]: true;
}
export interface ResponseMessage {
taskId: number;
result: any;
error: Error | null;
time: number | null;
}
export declare const commonState: {
isWorkerThread: boolean;
workerData: undefined;
};
export interface Transferable {
readonly [kTransferable]: object;
readonly [kValue]: object;
}
export type ResourceLimits = Worker extends {
resourceLimits?: infer T;
} ? T : {};
export type EnvSpecifier = typeof Worker extends {
new (filename: never, options?: {
env: infer T;
}): Worker;
} ? T : never;

8
node_modules/piscina/dist/types.js generated vendored Normal file
View File

@@ -0,0 +1,8 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.commonState = void 0;
exports.commonState = {
isWorkerThread: false,
workerData: undefined
};
//# sourceMappingURL=types.js.map

1
node_modules/piscina/dist/types.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"types.js","sourceRoot":"","sources":["../src/types.ts"],"names":[],"mappings":";;;AAgCa,QAAA,WAAW,GAAG;IACzB,cAAc,EAAE,KAAK;IACrB,UAAU,EAAE,SAAS;CACtB,CAAC"}

1
node_modules/piscina/dist/worker.d.ts generated vendored Normal file
View File

@@ -0,0 +1 @@
export {};

234
node_modules/piscina/dist/worker.js generated vendored Normal file
View File

@@ -0,0 +1,234 @@
"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
Object.defineProperty(o, "default", { enumerable: true, value: v });
}) : function(o, v) {
o["default"] = v;
});
var __importStar = (this && this.__importStar) || (function () {
var ownKeys = function(o) {
ownKeys = Object.getOwnPropertyNames || function (o) {
var ar = [];
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
return ar;
};
return ownKeys(o);
};
return function (mod) {
if (mod && mod.__esModule) return mod;
var result = {};
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
__setModuleDefault(result, mod);
return result;
};
})();
Object.defineProperty(exports, "__esModule", { value: true });
const node_worker_threads_1 = require("node:worker_threads");
const node_url_1 = require("node:url");
const node_perf_hooks_1 = require("node:perf_hooks");
const symbols_1 = require("./symbols");
const common_1 = require("./common");
common_1.commonState.isWorkerThread = true;
common_1.commonState.workerData = node_worker_threads_1.workerData;
/* c8 ignore next*/
function noop() { }
const handlerCache = new Map();
let useAtomics = process.env.PISCINA_DISABLE_ATOMICS !== '1';
let useAsyncAtomics = process.env.PISCINA_ENABLE_ASYNC_ATOMICS === '1';
// Get `import(x)` as a function that isn't transpiled to `require(x)` by
// TypeScript for dual ESM/CJS support.
// Load this lazily, so that there is no warning about the ESM loader being
// experimental (on Node v12.x) until we actually try to use it.
let importESMCached;
function getImportESM() {
if (importESMCached === undefined) {
// eslint-disable-next-line no-new-func
importESMCached = new Function('specifier', 'return import(specifier)');
}
return importESMCached;
}
// Look up the handler function that we call when a task is posted.
// This is either going to be "the" export from a file, or the default export.
async function getHandler(filename, name) {
let handler = handlerCache.get(`${filename}/${name}`);
if (handler != null) {
return handler;
}
try {
// With our current set of TypeScript options, this is transpiled to
// `require(filename)`.
handler = await Promise.resolve(`${filename}`).then(s => __importStar(require(s)));
if (typeof handler !== 'function') {
handler = await (handler[name]);
}
}
catch { }
if (typeof handler !== 'function') {
handler = await getImportESM()((0, node_url_1.pathToFileURL)(filename).href);
if (typeof handler !== 'function') {
handler = await (handler[name]);
}
}
if (typeof handler !== 'function') {
return null;
}
// Limit the handler cache size. This should not usually be an issue and is
// only provided for pathological cases.
/* c8 ignore next */
if (handlerCache.size > 1000) {
const [[key]] = handlerCache;
handlerCache.delete(key);
}
handlerCache.set(`${filename}/${name}`, handler);
return handler;
}
// We should only receive this message once, when the Worker starts. It gives
// us the MessagePort used for receiving tasks, a SharedArrayBuffer for fast
// communication using Atomics, and the name of the default filename for tasks
// (so we can pre-load and cache the handler).
node_worker_threads_1.parentPort.on('message', async (message) => {
var _a;
const { port, sharedBuffer, filename, name, niceIncrement } = message;
if (niceIncrement !== 0) {
(_a = (await Promise.resolve().then(() => __importStar(require('@napi-rs/nice'))).catch(noop))) === null || _a === void 0 ? void 0 : _a.nice(niceIncrement);
}
try {
if (filename != null) {
await getHandler(filename, name);
}
const readyMessage = { [common_1.READY]: true };
useAtomics = useAtomics !== false && message.atomics !== 'disabled';
useAsyncAtomics = useAtomics !== false && (useAsyncAtomics || message.atomics === 'async');
node_worker_threads_1.parentPort.postMessage(readyMessage);
port.on('message', onMessage.bind(null, port, sharedBuffer));
if (useAtomics) {
const res = atomicsWaitLoop(port, sharedBuffer);
if ((res === null || res === void 0 ? void 0 : res.then) != null)
await res;
}
}
catch (error) {
throwInNextTick(error);
}
});
let currentTasks = 0;
let lastSeenRequestCount = 0;
function atomicsWaitLoop(port, sharedBuffer) {
// This function is entered either after receiving the startup message, or
// when we are done with a task. In those situations, the *only* thing we
// expect to happen next is a 'message' on `port`.
// That call would come with the overhead of a C++ → JS boundary crossing,
// including async tracking. So, instead, if there is no task currently
// running, we wait for a signal from the parent thread using Atomics.wait(),
// and read the message from the port instead of generating an event,
// in order to avoid that overhead.
// The one catch is that this stops asynchronous operations that are still
// running from proceeding. Generally, tasks should not spawn asynchronous
// operations without waiting for them to finish, though.
if (useAsyncAtomics === true) {
// @ts-expect-error - for some reason not supported by TS
const { async, value } = Atomics.waitAsync(sharedBuffer, symbols_1.kRequestCountField, lastSeenRequestCount);
// We do not check for result
/* c8 ignore start */
return async === true && value.then(() => {
lastSeenRequestCount = Atomics.load(sharedBuffer, symbols_1.kRequestCountField);
// We have to read messages *after* updating lastSeenRequestCount in order
// to avoid race conditions.
let entry;
while ((entry = (0, node_worker_threads_1.receiveMessageOnPort)(port)) !== undefined) {
onMessage(port, sharedBuffer, entry.message);
}
});
/* c8 ignore stop */
}
while (currentTasks === 0) {
// Check whether there are new messages by testing whether the current
// number of requests posted by the parent thread matches the number of
// requests received.
// We do not check for result
Atomics.wait(sharedBuffer, symbols_1.kRequestCountField, lastSeenRequestCount);
lastSeenRequestCount = Atomics.load(sharedBuffer, symbols_1.kRequestCountField);
// We have to read messages *after* updating lastSeenRequestCount in order
// to avoid race conditions.
let entry;
while ((entry = (0, node_worker_threads_1.receiveMessageOnPort)(port)) !== undefined) {
onMessage(port, sharedBuffer, entry.message);
}
}
}
async function onMessage(port, sharedBuffer, message) {
currentTasks++;
const { taskId, task, filename, name } = message;
let response;
let transferList = [];
const start = message.histogramEnabled === 1 ? node_perf_hooks_1.performance.now() : null;
try {
const handler = await getHandler(filename, name);
if (handler === null) {
throw new Error(`No handler function exported from ${filename}`);
}
let result = await handler(task);
if ((0, common_1.isMovable)(result)) {
transferList = transferList.concat(result[symbols_1.kTransferable]);
result = result[symbols_1.kValue];
}
response = {
taskId,
result,
error: null,
time: start == null ? null : Math.round(node_perf_hooks_1.performance.now() - start)
};
if (useAtomics && !useAsyncAtomics) {
// If the task used e.g. console.log(), wait for the stream to drain
// before potentially entering the `Atomics.wait()` loop, and before
// returning the result so that messages will always be printed even
// if the process would otherwise be ready to exit.
if (process.stdout.writableLength > 0) {
await new Promise((resolve) => process.stdout.write('', resolve));
}
if (process.stderr.writableLength > 0) {
await new Promise((resolve) => process.stderr.write('', resolve));
}
}
}
catch (error) {
response = {
taskId,
result: null,
// It may be worth taking a look at the error cloning algorithm we
// use in Node.js core here, it's quite a bit more flexible
error: error,
time: start == null ? null : Math.round(node_perf_hooks_1.performance.now() - start)
};
}
currentTasks--;
try {
// Post the response to the parent thread, and let it know that we have
// an additional message available. If possible, use Atomics.wait()
// to wait for the next message.
port.postMessage(response, transferList);
Atomics.add(sharedBuffer, symbols_1.kResponseCountField, 1);
if (useAtomics) {
const res = atomicsWaitLoop(port, sharedBuffer);
if ((res === null || res === void 0 ? void 0 : res.then) != null)
await res;
}
}
catch (error) {
throwInNextTick(error);
}
}
function throwInNextTick(error) {
queueMicrotask(() => { throw error; });
}
//# sourceMappingURL=worker.js.map

1
node_modules/piscina/dist/worker.js.map generated vendored Normal file
View File

File diff suppressed because one or more lines are too long

7
node_modules/piscina/dist/worker_pool/balancer/index.d.ts generated vendored Normal file
View File

@@ -0,0 +1,7 @@
import type { PiscinaTask } from '../../task_queue';
import type { PiscinaWorker } from '..';
export type PiscinaLoadBalancer = (task: PiscinaTask, workers: PiscinaWorker[]) => PiscinaWorker | null;
export type LeastBusyBalancerOptions = {
maximumUsage: number;
};
export declare function LeastBusyBalancer(opts: LeastBusyBalancerOptions): PiscinaLoadBalancer;

25
node_modules/piscina/dist/worker_pool/balancer/index.js generated vendored Normal file
View File

@@ -0,0 +1,25 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.LeastBusyBalancer = LeastBusyBalancer;
function LeastBusyBalancer(opts) {
const { maximumUsage } = opts;
return (task, workers) => {
let candidate = null;
let checkpoint = maximumUsage;
for (const worker of workers) {
if (worker.currentUsage === 0) {
candidate = worker;
break;
}
if (worker.isRunningAbortableTask)
continue;
if (!task.isAbortable &&
(worker.currentUsage < checkpoint)) {
candidate = worker;
checkpoint = worker.currentUsage;
}
}
return candidate;
};
}
//# sourceMappingURL=index.js.map

1
node_modules/piscina/dist/worker_pool/balancer/index.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/worker_pool/balancer/index.ts"],"names":[],"mappings":";;AAWA,8CA2BC;AA3BD,SAAgB,iBAAiB,CAC/B,IAA8B;IAE9B,MAAM,EAAE,YAAY,EAAE,GAAG,IAAI,CAAC;IAE9B,OAAO,CAAC,IAAI,EAAE,OAAO,EAAE,EAAE;QACvB,IAAI,SAAS,GAAyB,IAAI,CAAC;QAC3C,IAAI,UAAU,GAAG,YAAY,CAAC;QAC9B,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;YAC7B,IAAI,MAAM,CAAC,YAAY,KAAK,CAAC,EAAE,CAAC;gBAC9B,SAAS,GAAG,MAAM,CAAC;gBACnB,MAAM;YACR,CAAC;YAED,IAAI,MAAM,CAAC,sBAAsB;gBAAE,SAAS;YAE5C,IACE,CAAC,IAAI,CAAC,WAAW;gBACjB,CAAC,MAAM,CAAC,YAAY,GAAG,UAAU,CAAC,EAClC,CAAC;gBACD,SAAS,GAAG,MAAM,CAAC;gBACnB,UAAU,GAAG,MAAM,CAAC,YAAY,CAAC;YACnC,CAAC;QACH,CAAC;QAED,OAAO,SAAS,CAAC;IACnB,CAAC,CAAC;AACJ,CAAC"}

28
node_modules/piscina/dist/worker_pool/base.d.ts generated vendored Normal file
View File

@@ -0,0 +1,28 @@
export declare abstract class AsynchronouslyCreatedResource {
onreadyListeners: (() => void)[] | null;
ondestroyListeners: (() => void)[] | null;
markAsReady(): void;
isReady(): boolean;
onReady(fn: () => void): void;
onDestroy(fn: () => void): void;
markAsDestroyed(): void;
isDestroyed(): boolean;
abstract currentUsage(): number;
}
export declare class AsynchronouslyCreatedResourcePool<T extends AsynchronouslyCreatedResource> {
pendingItems: Set<T>;
readyItems: Set<T>;
maximumUsage: number;
onAvailableListeners: ((item: T) => void)[];
onTaskDoneListeners: ((item: T) => void)[];
constructor(maximumUsage: number);
add(item: T): void;
delete(item: T): void;
[Symbol.iterator](): Generator<T, void, unknown>;
get size(): number;
maybeAvailable(item: T): void;
onAvailable(fn: (item: T) => void): void;
taskDone(item: T): void;
onTaskDone(fn: (item: T) => void): void;
getCurrentUsage(): number;
}

111
node_modules/piscina/dist/worker_pool/base.js generated vendored Normal file
View File

@@ -0,0 +1,111 @@
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.AsynchronouslyCreatedResourcePool = exports.AsynchronouslyCreatedResource = void 0;
const node_assert_1 = __importDefault(require("node:assert"));
class AsynchronouslyCreatedResource {
constructor() {
this.onreadyListeners = [];
this.ondestroyListeners = [];
}
markAsReady() {
const listeners = this.onreadyListeners;
(0, node_assert_1.default)(listeners !== null);
this.onreadyListeners = null;
for (const listener of listeners) {
listener();
}
}
isReady() {
return this.onreadyListeners === null;
}
onReady(fn) {
if (this.onreadyListeners === null) {
fn(); // Zalgo is okay here.
return;
}
this.onreadyListeners.push(fn);
}
onDestroy(fn) {
if (this.ondestroyListeners === null) {
return;
}
this.ondestroyListeners.push(fn);
}
markAsDestroyed() {
const listeners = this.ondestroyListeners;
(0, node_assert_1.default)(listeners !== null);
this.ondestroyListeners = null;
for (const listener of listeners) {
listener();
}
}
isDestroyed() {
return this.ondestroyListeners === null;
}
}
exports.AsynchronouslyCreatedResource = AsynchronouslyCreatedResource;
// TODO: this will eventually become an scheduler
class AsynchronouslyCreatedResourcePool {
constructor(maximumUsage) {
this.pendingItems = new Set();
this.readyItems = new Set();
this.maximumUsage = maximumUsage;
this.onAvailableListeners = [];
this.onTaskDoneListeners = [];
}
add(item) {
this.pendingItems.add(item);
item.onReady(() => {
/* istanbul ignore else */
if (this.pendingItems.has(item)) {
this.pendingItems.delete(item);
this.readyItems.add(item);
this.maybeAvailable(item);
}
});
}
delete(item) {
this.pendingItems.delete(item);
this.readyItems.delete(item);
}
*[Symbol.iterator]() {
yield* this.pendingItems;
yield* this.readyItems;
}
get size() {
return this.pendingItems.size + this.readyItems.size;
}
maybeAvailable(item) {
/* istanbul ignore else */
if (item.currentUsage() < this.maximumUsage) {
for (const listener of this.onAvailableListeners) {
listener(item);
}
}
}
onAvailable(fn) {
this.onAvailableListeners.push(fn);
}
taskDone(item) {
for (let i = 0; i < this.onTaskDoneListeners.length; i++) {
this.onTaskDoneListeners[i](item);
}
}
onTaskDone(fn) {
this.onTaskDoneListeners.push(fn);
}
getCurrentUsage() {
let inFlight = 0;
for (const worker of this.readyItems) {
const currentUsage = worker.currentUsage();
if (Number.isFinite(currentUsage))
inFlight += currentUsage;
}
return inFlight;
}
}
exports.AsynchronouslyCreatedResourcePool = AsynchronouslyCreatedResourcePool;
//# sourceMappingURL=base.js.map

1
node_modules/piscina/dist/worker_pool/base.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"base.js","sourceRoot":"","sources":["../../src/worker_pool/base.ts"],"names":[],"mappings":";;;;;;AAAA,8DAAiC;AAEjC,MAAsB,6BAA6B;IAAnD;QACI,qBAAgB,GAA2B,EAAE,CAAC;QAC9C,uBAAkB,GAA2B,EAAE,CAAC;IA6CpD,CAAC;IA3CG,WAAW;QACT,MAAM,SAAS,GAAG,IAAI,CAAC,gBAAgB,CAAC;QACxC,IAAA,qBAAM,EAAC,SAAS,KAAK,IAAI,CAAC,CAAC;QAC3B,IAAI,CAAC,gBAAgB,GAAG,IAAI,CAAC;QAC7B,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,QAAQ,EAAE,CAAC;QACb,CAAC;IACH,CAAC;IAED,OAAO;QACL,OAAO,IAAI,CAAC,gBAAgB,KAAK,IAAI,CAAC;IACxC,CAAC;IAED,OAAO,CAAE,EAAe;QACtB,IAAI,IAAI,CAAC,gBAAgB,KAAK,IAAI,EAAE,CAAC;YACnC,EAAE,EAAE,CAAC,CAAC,sBAAsB;YAC5B,OAAO;QACT,CAAC;QACD,IAAI,CAAC,gBAAgB,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACjC,CAAC;IAED,SAAS,CAAE,EAAe;QACxB,IAAI,IAAI,CAAC,kBAAkB,KAAK,IAAI,EAAE,CAAC;YACrC,OAAO;QACT,CAAC;QAED,IAAI,CAAC,kBAAkB,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACnC,CAAC;IAED,eAAe;QACb,MAAM,SAAS,GAAG,IAAI,CAAC,kBAAkB,CAAC;QAC1C,IAAA,qBAAM,EAAC,SAAS,KAAK,IAAI,CAAC,CAAC;QAC3B,IAAI,CAAC,kBAAkB,GAAG,IAAI,CAAC;QAC/B,KAAK,MAAM,QAAQ,IAAI,SAAS,EAAE,CAAC;YACjC,QAAQ,EAAE,CAAC;QACb,CAAC;IACH,CAAC;IAED,WAAW;QACT,OAAO,IAAI,CAAC,kBAAkB,KAAK,IAAI,CAAC;IAC1C,CAAC;CAGJ;AA/CD,sEA+CC;AAED,iDAAiD;AACjD,MAAa,iCAAiC;IAQ5C,YAAa,YAAqB;QANlC,iBAAY,GAAG,IAAI,GAAG,EAAK,CAAC;QAC5B,eAAU,GAAG,IAAI,GAAG,EAAK,CAAC;QAMxB,IAAI,CAAC,YAAY,GAAG,YAAY,CAAC;QACjC,IAAI,CAAC,oBAAoB,GAAG,EAAE,CAAC;QAC/B,IAAI,CAAC,mBAAmB,GAAG,EAAE,CAAC;IAChC,CAAC;IAED,GAAG,CAAE,IAAQ;QACX,IAAI,CAAC,YAAY,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAC5B,IAAI,CAAC,OAAO,CAAC,GAAG,EAAE;YAChB,0BAA0B;YAC1B,IAAI,IAAI,CAAC,YAAY,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC;gBAChC,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;gBAC/B,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;gBAC1B,IAAI,CAAC,cAAc,CAAC,IAAI,CAAC,CAAC;YAC5B,CAAC;QACH,CAAC,CAAC,CAAC;IACL,CAAC;IAED,MAAM,CAAE,IAAQ;QACd,IAAI,CAAC,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QAC/B,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;IAC/B,CAAC;IAED,CAAE,CAAC,MAAM,CAAC,QAAQ,CAAC;QACjB,KAAM,CAAC,CAAC,IAAI,CAAC,YAAY,CAAC;QAC1B,KAAM,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC;IAC1B,CAAC;IAED,IAAI,IAAI;QACN,OAAO,IAAI,CAAC,YAAY,CAAC,IAAI,GAAG,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;IACvD,CAAC;IAED,cAAc,CAAE,IAAQ;QACtB,0BAA0B;QAC1B,IAAI,IAAI,CAAC,YAAY,EAAE,GAAG,IAAI,CAAC,YAAY,EAAE,CAAC;YAC5C,KAAK,MAAM,QAAQ,IAAI,IAAI,CAAC,oBAAoB,EAAE,CAAC;gBACjD,QAAQ,CAAC,IAAI,CAAC,CAAC;YACjB,CAAC;QACH,CAAC;IACH,CAAC;IAED,WAAW,CAAE,EAAuB;QAClC,IAAI,CAAC,oBAAoB,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACrC,CAAC;IAED,QAAQ,CAAE,IAAQ;QAChB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,mBAAmB,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACzD,IAAI,CAAC,mBAAmB,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QACpC,CAAC;IACH,CAAC;IAED,UAAU,CAAE,EAAuB;QACjC,IAAI,CAAC,mBAAmB,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACpC,CAAC;IAED,eAAe;QACb,IAAI,QAAQ,GAAG,CAAC,CAAC;QACjB,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,UAAU,EAAE,CAAC;YACrC,MAAM,YAAY,GAAG,MAAM,CAAC,YAAY,EAAE,CAAC;YAE3C,IAAI,MAAM,CAAC,QAAQ,CAAC,YAAY,CAAC;gBAAE,QAAQ,IAAI,YAAY,CAAC;QAC9D,CAAC;QAED,OAAO,QAAQ,CAAC;IAClB,CAAC;CACF;AAzED,8EAyEC"}

43
node_modules/piscina/dist/worker_pool/index.d.ts generated vendored Normal file
View File

@@ -0,0 +1,43 @@
import { Worker, MessagePort } from 'node:worker_threads';
import { RecordableHistogram } from 'node:perf_hooks';
import { ResponseMessage } from '../types';
import { TaskInfo } from '../task_queue';
import { kWorkerData } from '../symbols';
import { PiscinaHistogramSummary } from '../histogram';
import { AsynchronouslyCreatedResource, AsynchronouslyCreatedResourcePool } from './base';
export * from './balancer';
type ResponseCallback = (response: ResponseMessage) => void;
export type PiscinaWorker = {
id: number;
currentUsage: number;
isRunningAbortableTask: boolean;
histogram: PiscinaHistogramSummary | null;
terminating: boolean;
destroyed: boolean;
[kWorkerData]: WorkerInfo;
};
export declare class WorkerInfo extends AsynchronouslyCreatedResource {
worker: Worker;
taskInfos: Map<number, TaskInfo>;
idleTimeout: NodeJS.Timeout | null;
port: MessagePort;
sharedBuffer: Int32Array;
lastSeenResponseCount: number;
onMessage: ResponseCallback;
histogram: RecordableHistogram | null;
terminating: boolean;
destroyed: boolean;
constructor(worker: Worker, port: MessagePort, onMessage: ResponseCallback, enableHistogram: boolean);
get id(): number;
destroy(): void;
clearIdleTimeout(): void;
ref(): WorkerInfo;
unref(): WorkerInfo;
_handleResponse(message: ResponseMessage): void;
postTask(taskInfo: TaskInfo): void;
processPendingMessages(): void;
isRunningAbortableTask(): boolean;
currentUsage(): number;
get interface(): PiscinaWorker;
}
export { AsynchronouslyCreatedResourcePool };

174
node_modules/piscina/dist/worker_pool/index.js generated vendored Normal file
View File

@@ -0,0 +1,174 @@
"use strict";
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
var desc = Object.getOwnPropertyDescriptor(m, k);
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
desc = { enumerable: true, get: function() { return m[k]; } };
}
Object.defineProperty(o, k2, desc);
}) : (function(o, m, k, k2) {
if (k2 === undefined) k2 = k;
o[k2] = m[k];
}));
var __exportStar = (this && this.__exportStar) || function(m, exports) {
for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
};
var __importDefault = (this && this.__importDefault) || function (mod) {
return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.AsynchronouslyCreatedResourcePool = exports.WorkerInfo = void 0;
const node_worker_threads_1 = require("node:worker_threads");
const node_perf_hooks_1 = require("node:perf_hooks");
const node_assert_1 = __importDefault(require("node:assert"));
const errors_1 = require("../errors");
const symbols_1 = require("../symbols");
const histogram_1 = require("../histogram");
const base_1 = require("./base");
Object.defineProperty(exports, "AsynchronouslyCreatedResourcePool", { enumerable: true, get: function () { return base_1.AsynchronouslyCreatedResourcePool; } });
__exportStar(require("./balancer"), exports);
class WorkerInfo extends base_1.AsynchronouslyCreatedResource {
constructor(worker, port, onMessage, enableHistogram) {
super();
this.idleTimeout = null;
this.lastSeenResponseCount = 0;
this.terminating = false;
this.destroyed = false;
this.worker = worker;
this.port = port;
this.port.on('message', (message) => this._handleResponse(message));
this.onMessage = onMessage;
this.taskInfos = new Map();
this.sharedBuffer = new Int32Array(new SharedArrayBuffer(symbols_1.kFieldCount * Int32Array.BYTES_PER_ELEMENT));
this.histogram = enableHistogram ? (0, node_perf_hooks_1.createHistogram)() : null;
}
get id() {
return this.worker.threadId;
}
destroy() {
if (this.terminating || this.destroyed)
return;
this.terminating = true;
this.clearIdleTimeout();
this.worker.terminate();
this.port.close();
for (const taskInfo of this.taskInfos.values()) {
taskInfo.done(errors_1.Errors.ThreadTermination());
}
this.taskInfos.clear();
this.terminating = false;
this.destroyed = true;
this.markAsDestroyed();
}
clearIdleTimeout() {
if (this.idleTimeout != null) {
clearTimeout(this.idleTimeout);
this.idleTimeout = null;
}
}
ref() {
this.port.ref();
return this;
}
unref() {
// Note: Do not call ref()/unref() on the Worker itself since that may cause
// a hard crash, see https://github.com/nodejs/node/pull/33394.
this.port.unref();
return this;
}
_handleResponse(message) {
var _a;
if (message.time != null) {
(_a = this.histogram) === null || _a === void 0 ? void 0 : _a.record(histogram_1.PiscinaHistogramHandler.toHistogramIntegerNano(message.time));
}
this.onMessage(message);
if (this.taskInfos.size === 0) {
// No more tasks running on this Worker means it should not keep the
// process running.
this.unref();
}
}
postTask(taskInfo) {
(0, node_assert_1.default)(!this.taskInfos.has(taskInfo.taskId));
(0, node_assert_1.default)(!this.terminating && !this.destroyed);
const message = {
task: taskInfo.releaseTask(),
taskId: taskInfo.taskId,
filename: taskInfo.filename,
name: taskInfo.name,
histogramEnabled: this.histogram != null ? 1 : 0
};
try {
this.port.postMessage(message, taskInfo.transferList);
}
catch (err) {
// This would mostly happen if e.g. message contains unserializable data
// or transferList is invalid.
taskInfo.done(err);
return;
}
taskInfo.workerInfo = this;
this.taskInfos.set(taskInfo.taskId, taskInfo);
queueMicrotask(() => this.clearIdleTimeout());
this.ref();
// Inform the worker that there are new messages posted, and wake it up
// if it is waiting for one.
Atomics.add(this.sharedBuffer, symbols_1.kRequestCountField, 1);
Atomics.notify(this.sharedBuffer, symbols_1.kRequestCountField, 1);
}
processPendingMessages() {
if (this.destroyed)
return;
// If we *know* that there are more messages than we have received using
// 'message' events yet, then try to load and handle them synchronously,
// without the need to wait for more expensive events on the event loop.
// This would usually break async tracking, but in our case, we already have
// the extra TaskInfo/AsyncResource layer that rectifies that situation.
const actualResponseCount = Atomics.load(this.sharedBuffer, symbols_1.kResponseCountField);
if (actualResponseCount !== this.lastSeenResponseCount) {
this.lastSeenResponseCount = actualResponseCount;
let entry;
while ((entry = (0, node_worker_threads_1.receiveMessageOnPort)(this.port)) !== undefined) {
this._handleResponse(entry.message);
}
}
}
isRunningAbortableTask() {
// If there are abortable tasks, we are running one at most per Worker.
if (this.taskInfos.size !== 1)
return false;
const [[, task]] = this.taskInfos;
return task.abortSignal !== null;
}
currentUsage() {
if (this.isRunningAbortableTask())
return Infinity;
return this.taskInfos.size;
}
get interface() {
const worker = this;
return {
get id() {
return worker.worker.threadId;
},
get currentUsage() {
return worker.currentUsage();
},
get isRunningAbortableTask() {
return worker.isRunningAbortableTask();
},
get histogram() {
return worker.histogram != null ? histogram_1.PiscinaHistogramHandler.createHistogramSummary(worker.histogram) : null;
},
get terminating() {
return worker.terminating;
},
get destroyed() {
return worker.destroyed;
},
[symbols_1.kWorkerData]: worker
};
}
}
exports.WorkerInfo = WorkerInfo;
//# sourceMappingURL=index.js.map

1
node_modules/piscina/dist/worker_pool/index.js.map generated vendored Normal file
View File

@@ -0,0 +1 @@
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/worker_pool/index.ts"],"names":[],"mappings":";;;;;;;;;;;;;;;;;;;;AAAA,6DAAgF;AAChF,qDAAuE;AACvE,8DAAiC;AAGjC,sCAAmC;AAGnC,wCAA+F;AAC/F,4CAAgF;AAEhF,iCAA0F;AA4LjF,kHA5L+B,wCAAiC,OA4L/B;AA3L1C,6CAA2B;AAc3B,MAAa,UAAW,SAAQ,oCAA6B;IAYzD,YACE,MAAe,EACf,IAAkB,EAClB,SAA4B,EAC5B,eAAwB;QAExB,KAAK,EAAE,CAAC;QAfV,gBAAW,GAA2B,IAAI,CAAC;QAG3C,0BAAqB,GAAY,CAAC,CAAC;QAGnC,gBAAW,GAAG,KAAK,CAAC;QACpB,cAAS,GAAG,KAAK,CAAC;QAShB,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;QACrB,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,SAAS,EACpB,CAAC,OAAyB,EAAE,EAAE,CAAC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,CAAC,CAAC;QAChE,IAAI,CAAC,SAAS,GAAG,SAAS,CAAC;QAC3B,IAAI,CAAC,SAAS,GAAG,IAAI,GAAG,EAAE,CAAC;QAC3B,IAAI,CAAC,YAAY,GAAG,IAAI,UAAU,CAChC,IAAI,iBAAiB,CAAC,qBAAW,GAAG,UAAU,CAAC,iBAAiB,CAAC,CAAC,CAAC;QACrE,IAAI,CAAC,SAAS,GAAG,eAAe,CAAC,CAAC,CAAC,IAAA,iCAAe,GAAE,CAAC,CAAC,CAAC,IAAI,CAAC;IAC9D,CAAC;IAED,IAAI,EAAE;QACJ,OAAO,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC;IAC9B,CAAC;IAED,OAAO;QACL,IAAI,IAAI,CAAC,WAAW,IAAI,IAAI,CAAC,SAAS;YAAE,OAAO;QAE/C,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC;QACxB,IAAI,CAAC,gBAAgB,EAAE,CAAC;QACxB,IAAI,CAAC,MAAM,CAAC,SAAS,EAAE,CAAC;QACxB,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;QAClB,KAAK,MAAM,QAAQ,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,EAAE,CAAC;YAC/C,QAAQ,CAAC,IAAI,CAAC,eAAM,CAAC,iBAAiB,EAAE,CAAC,CAAC;QAC5C,CAAC;QACD,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,CAAC;QAEvB,IAAI,CAAC,WAAW,GAAG,KAAK,CAAC;QACzB,IAAI,CAAC,SAAS,GAAG,IAAI,CAAC;QACtB,IAAI,CAAC,eAAe,EAAE,CAAC;IACzB,CAAC;IAED,gBAAgB;QACd,IAAI,IAAI,CAAC,WAAW,IAAI,IAAI,EAAE,CAAC;YAC7B,YAAY,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC;YAC/B,IAAI,CAAC,WAAW,GAAG,IAAI,CAAC;QAC1B,CAAC;IACH,CAAC;IAED,GAAG;QACD,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC;QAChB,OAAO,IAAI,CAAC;IACd,CAAC;IAED,KAAK;QACH,4EAA4E;QAC5E,+DAA+D;QAC/D,IAAI,CAAC,IAAI,CAAC,KAAK,EAAE,CAAC;QAClB,OAAO,IAAI,CAAC;IACd,CAAC;IAED,eAAe,CAAE,OAAyB;;QACxC,IAAI,OAAO,CAAC,IAAI,IAAI,IAAI,EAAE,CAAC;YACzB,MAAA,IAAI,CAAC,SAAS,0CAAE,MAAM,CAAC,mCAAuB,CAAC,sBAAsB,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;QACvF,CAAC;QAED,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC;QAExB,IAAI,IAAI,CAAC,SAAS,CAAC,IAAI,KAAK,CAAC,EAAE,CAAC;YAC9B,oEAAoE;YACpE,mBAAmB;YACnB,IAAI,CAAC,KAAK,EAAE,CAAC;QACf,CAAC;IACH,CAAC;IAED,QAAQ,CAAE,QAAmB;QAC3B,IAAA,qBAAM,EAAC,CAAC,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC;QAC7C,IAAA,qBAAM,EAAC,CAAC,IAAI,CAAC,WAAW,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QAE7C,MAAM,OAAO,GAAoB;YAC/B,IAAI,EAAE,QAAQ,CAAC,WAAW,EAAE;YAC5B,MAAM,EAAE,QAAQ,CAAC,MAAM;YACvB,QAAQ,EAAE,QAAQ,CAAC,QAAQ;YAC3B,IAAI,EAAE,QAAQ,CAAC,IAAI;YACnB,gBAAgB,EAAE,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;SACjD,CAAC;QAEF,IAAI,CAAC;YACH,IAAI,CAAC,IAAI,CAAC,WAAW,CAAC,OAAO,EAAE,QAAQ,CAAC,YAAY,CAAC,CAAC;QACxD,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,wEAAwE;YACxE,8BAA8B;YAC9B,QAAQ,CAAC,IAAI,CAAQ,GAAG,CAAC,CAAC;YAC1B,OAAO;QACT,CAAC;QAED,QAAQ,CAAC,UAAU,GAAG,IAAI,CAAC;QAC3B,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,QAAQ,CAAC,MAAM,EAAE,QAAQ,CAAC,CAAC;QAC9C,cAAc,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,gBAAgB,EAAE,CAAC,CAAA;QAC7C,IAAI,CAAC,GAAG,EAAE,CAAC;QAEX,uEAAuE;QACvE,4BAA4B;QAC5B,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,YAAY,EAAE,4BAAkB,EAAE,CAAC,CAAC,CAAC;QACtD,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,YAAY,EAAE,4BAAkB,EAAE,CAAC,CAAC,CAAC;IAC3D,CAAC;IAED,sBAAsB;QACpB,IAAI,IAAI,CAAC,SAAS;YAAE,OAAO;QAC3B,wEAAwE;QACxE,wEAAwE;QACxE,wEAAwE;QACxE,4EAA4E;QAC5E,wEAAwE;QACxE,MAAM,mBAAmB,GACvB,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,YAAY,EAAE,6BAAmB,CAAC,CAAC;QACvD,IAAI,mBAAmB,KAAK,IAAI,CAAC,qBAAqB,EAAE,CAAC;YACvD,IAAI,CAAC,qBAAqB,GAAG,mBAAmB,CAAC;YAEjD,IAAI,KAAK,CAAC;YACV,OAAO,CAAC,KAAK,GAAG,IAAA,0CAAoB,EAAC,IAAI,CAAC,IAAI,CAAC,CAAC,KAAK,SAAS,EAAE,CAAC;gBAC/D,IAAI,CAAC,eAAe,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;YACtC,CAAC;QACH,CAAC;IACH,CAAC;IAED,sBAAsB;QACpB,uEAAuE;QACvE,IAAI,IAAI,CAAC,SAAS,CAAC,IAAI,KAAK,CAAC;YAAE,OAAO,KAAK,CAAC;QAC5C,MAAM,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC;QAClC,OAAO,IAAI,CAAC,WAAW,KAAK,IAAI,CAAC;IACnC,CAAC;IAED,YAAY;QACV,IAAI,IAAI,CAAC,sBAAsB,EAAE;YAAE,OAAO,QAAQ,CAAC;QACnD,OAAO,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;IAC7B,CAAC;IAED,IAAI,SAAS;QACX,MAAM,MAAM,GAAG,IAAI,CAAC;QACpB,OAAO;YACL,IAAI,EAAE;gBACJ,OAAO,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAC;YAChC,CAAC;YACD,IAAI,YAAY;gBACd,OAAO,MAAM,CAAC,YAAY,EAAE,CAAC;YAC/B,CAAC;YACD,IAAI,sBAAsB;gBACxB,OAAO,MAAM,CAAC,sBAAsB,EAAE,CAAC;YACzC,CAAC;YACD,IAAI,SAAS;gBACX,OAAO,MAAM,CAAC,SAAS,IAAI,IAAI,CAAC,CAAC,CAAC,mCAAuB,CAAC,sBAAsB,CAAC,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;YAC5G,CAAC;YACD,IAAI,WAAW;gBACb,OAAO,MAAM,CAAC,WAAW,CAAC;YAC5B,CAAC;YACD,IAAI,SAAS;gBACX,OAAO,MAAM,CAAC,SAAS,CAAC;YAC1B,CAAC;YACD,CAAC,qBAAW,CAAC,EAAE,MAAM;SACtB,CAAC;IACJ,CAAC;CACJ;AA3KD,gCA2KC"}

31
node_modules/piscina/docs/docs/Introduction.md generated vendored Normal file
View File

@@ -0,0 +1,31 @@
---
sidebar_position: 1
slug: /
---
# Introduction
Piscina.js is a powerful Node.js worker pool library that allows you to efficiently run CPU-intensive tasks in parallel using worker threads. It provides a simple API for offloading computationally expensive tasks to a pool of worker threads, thereby improving the performance and scalability of your Node.js applications.
## Why Piscina?
In the early days of worker threads, the Node.js core team encountered an issue where a user's application was spinning up thousands of concurrent worker threads, leading to performance issues. While this specific issue helped identify a minor memory leak in the worker implementation, it highlighted a broader problem: the misuse of worker threads due to a lack of understanding.
While worker threads have matured and their usage has become more widespread, there is still a need for better examples and education around their correct usage. This realization led to the creation of Piscina, an open-source project sponsored by [NearForm Research](https://www.nearform.com/), focused on providing guidance and best practices for using worker threads in Node.js applications.
With worker threads now a well-established feature in Node.js, Piscina aims to bridge the gap between the potential of worker threads and their practical implementation.
## Key features
✔ Fast communication between threads\
✔ Covers both fixed-task and variable-task scenarios\
✔ Supports flexible pool sizes\
✔ Proper async tracking integration\
✔ Tracking statistics for run and wait times\
✔ Cancellation Support\
✔ Supports enforcing memory resource limits\
✔ Supports CommonJS, ESM, and TypeScript\
✔ Custom task queues\
✔ Optional CPU scheduling priorities on Linux

30
node_modules/piscina/docs/docs/api-reference/api-overview.md generated vendored Normal file
View File

@@ -0,0 +1,30 @@
---
id: API Overview
sidebar_position: 1
---
| API | Description |
| ---------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Class: `Piscina`** | This class extends [`EventEmitter`](https://nodejs.org/api/events.html#events_class_eventemitter) from Node.js. |
| **Constructor: `new Piscina([options])`** | Creates a new instance of the `Piscina` class. [List of available options](./class.md) |
| **Method: `run(task[, options])`** | Schedules a task to be run on a Worker thread. |
| **Method: `runTask(task[, transferList][, filename][, abortSignal])`** | **Deprecated** -- Use `run(task, options)` instead. |
| **Method: `destroy()`** | Stops all Workers and rejects all `Promises` for pending tasks. |
| **Method: `close([options])`** | Stops all Workers gracefully. |
| **Event: `'error'`** | Emitted when uncaught exceptions occur inside Worker threads or unexpected messages are sent from Worker threads. |
| **Event: `'drain'`** | Emitted whenever the `queueSize` reaches `0`. |
| **Event: `'needsDrain'`** | Triggered once the total capacity of the pool is exceeded by the number of tasks enqueued that are pending execution. |
| **Event: `'message'`** | Emitted whenever a message is received from a worker thread. |
| **Property: `completed` (readonly)** | The current number of completed tasks. |
| **Property: `duration` (readonly)** | The length of time (in milliseconds) since this `Piscina` instance was created. |
| **Property: `options` (readonly)** | A copy of the options that are currently being used by this instance. |
| **Property: `runTime` (readonly)** | A histogram summary object summarizing the collected run times of completed tasks. |
| **Property: `threads` (readonly)** | An Array of the `Worker` instances used by this pool. |
| **Property: `queueSize` (readonly)** | The current number of tasks waiting to be assigned to a Worker thread. |
| **Property: `needsDrain` (readonly)** | Boolean value that specifies whether the capacity of the pool has been exceeded by the number of tasks submitted. |
| **Property: `utilization` (readonly)** | A point-in-time ratio comparing the approximate total mean run time of completed tasks to the total runtime capacity of the pool. |
| **Property: `waitTime` (readonly)** | A histogram summary object summarizing the collected times tasks spent waiting in the queue. |
| **Static Property: `isWorkerThread` (readonly)** | Is `true` if this code runs inside a `Piscina` threadpool as a Worker. |
| **Static Property: `version` (readonly)** | Provides the current version of this library as a semver string. |
| **Static Method: `move(value)`** | By default, any value returned by a worker function will be cloned when returned back to the Piscina pool, even if that object is capable of being transferred. The `Piscina.move()` method can be used to wrap and mark transferable values such that they will be transferred rather than cloned. |
| **Interface: `Transferable`** | Objects may implement the `Transferable` interface to create their own custom transferable objects. |

332
node_modules/piscina/docs/docs/api-reference/class.md generated vendored Normal file
View File

@@ -0,0 +1,332 @@
---
id: Instance
sidebar_position: 2
---
## Class: `Piscina`
Piscina works by creating a pool of Node.js Worker Threads to which
one or more tasks may be dispatched. Each worker thread executes a
single exported function defined in a separate file. Whenever a
task is dispatched to a worker, the worker invokes the exported
function and reports the return value back to Piscina when the
function completes.
This class extends [`EventEmitter`](https://nodejs.org/api/events.html) from Node.js.
### Constructor: `new Piscina([options])`
- The following optional configuration is supported:
- `filename`: (`string | null`) Provides the default source for the code that
runs the tasks on Worker threads. This should be an absolute path or an
absolute `file://` URL to a file that exports a JavaScript `function` or
`async function` as its default export or `module.exports`. [ES modules](https://nodejs.org/api/esm.html)
are supported.
- `name`: (`string | null`) Provides the name of the default exported worker
function. The default is `'default'`, indicating the default export of the
worker module.
- `minThreads`: (`number`) Sets the minimum number of threads that are always
running for this thread pool. The default is the number provided by [`os.availableParallelism`](https://nodejs.org/api/os.html#osavailableparallelism).
- `maxThreads`: (`number`) Sets the maximum number of threads that are
running for this thread pool. The default is the number provided by [`os.availableParallelism`](https://nodejs.org/api/os.html#osavailableparallelism) \* 1.5.
- `idleTimeout`: (`number`) A timeout in milliseconds that specifies how long
a `Worker` is allowed to be idle, i.e. not handling any tasks, before it is
shut down. By default, this is immediate. If `Infinity` is passed as the value,
the `Worker` never shuts down.
:::info
The default `idleTimeout` can lead to some performance loss in the application because of the overhead involved with stopping and starting new worker threads. To improve performance, try setting the `idleTimeout` explicitly.
:::
:::info
Be careful when when setting `idleTimeout` to `Infinity`, as this will prevent the worker from shutting down, even when idle, potentially leading to resource overuse.
:::
- `maxQueue`: (`number` | `string`) The maximum number of tasks that may be
scheduled to run, but not yet running due to lack of available threads, at
a given time. By default, there is no limit. The special value `'auto'`
may be used to have Piscina calculate the maximum as the square of `maxThreads`.
When `'auto'` is used, the calculated `maxQueue` value may be found by checking
the [`options.maxQueue`](#property-options-readonly) property.
- `concurrentTasksPerWorker`: (`number`) Specifies how many tasks can share
a single Worker thread simultaneously. The default is `1`. This generally
only makes sense to specify if there is some kind of asynchronous component
to the task. Keep in mind that Worker threads are generally not built for
handling I/O in parallel.
- `atomics`: (`sync` | `async` | `disabled`) Use the [`Atomics`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Atomics) API for faster communication
between threads. This is on by default. You can disable `Atomics` globally by
setting the environment variable `PISCINA_DISABLE_ATOMICS` to `1` .
If `atomics` is `sync`, it will cause to pause threads (stoping all execution)
between tasks. Ideally, threads should wait for all operations to finish before
returning control to the main thread (avoid having open handles within a thread). If still want to have the possibility
of having open handles or handle asynchrnous tasks, you can set the environment variable `PISCINA_ENABLE_ASYNC_ATOMICS` to `1` or setting `options.atomics` to `async`.
:::info
**Note**: The `async` mode comes with performance penalties and can lead to undesired behaviour if open handles are not tracked correctly.
Workers should be designed to wait for all operations to finish before returning control to the main thread, if any background operations are still running
`async` can be of help (e.g. for cache warming, etc).
:::
- `resourceLimits`: (`object`) See [Node.js new Worker options](https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options)
- `maxOldGenerationSizeMb`: (`number`) The maximum size of each worker threads
main heap in MB.
- `maxYoungGenerationSizeMb`: (`number`) The maximum size of a heap space for
recently created objects.
- `codeRangeSizeMb`: (`number`) The size of a pre-allocated memory range used
for generated code.
- `stackSizeMb` : (`number`) The default maximum stack size for the thread.
Small values may lead to unusable Worker instances. Default: 4
- `env`: (`object`) If set, specifies the initial value of `process.env` inside
the worker threads. See [Node.js new Worker options](https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options) for details.
- `argv`: (`any[]`) List of arguments that will be stringified and appended to
`process.argv` in the worker. See [Node.js new Worker options](https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options) for details.
- `execArgv`: (`string[]`) List of Node.js CLI options passed to the worker.
See [Node.js new Worker options](https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options) for details.
- `workerData`: (`any`) Any JavaScript value that can be cloned and made
available as `require('piscina').workerData`. See [Node.js new Worker options](https://nodejs.org/api/worker_threads.html#worker_threads_new_worker_filename_options)
for details. Unlike regular Node.js Worker Threads, `workerData` must not
specify any value requiring a `transferList`. This is because the `workerData`
will be cloned for each pooled worker.
- `taskQueue`: (`TaskQueue`) By default, Piscina uses a first-in-first-out
queue for submitted tasks. The `taskQueue` option can be used to provide an
alternative implementation. See [Custom Task Queues](https://github.com/piscinajs/piscina#custom_task_queues) for additional detail.
- `niceIncrement`: (`number`) An optional value that decreases priority for
the individual threads, i.e. the higher the value, the lower the priority
of the Worker threads. This value is used on Unix/Windows and requires the
optional [`@napi-rs/nice`](https://npmjs.org/package/@napi-rs/nice) module to be installed.
See [`nice(2)`](https://linux.die.net/man/2/nice) and [`SetThreadPriority`](https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-setthreadpriority) for more details.
- `trackUnmanagedFds`: (`boolean`) An optional setting that, when `true`, will
cause Workers to track file descriptors managed using `fs.open()` and
`fs.close()`, and will close them automatically when the Worker exits.
Defaults to `true`. (This option is only supported on Node.js 12.19+ and
all Node.js versions higher than 14.6.0).
- `closeTimeout`: (`number`) An optional time (in milliseconds) to wait for the pool to
complete all in-flight tasks when `close()` is called. The default is `30000`
- `recordTiming`: (`boolean`) By default, run and wait time will be recorded
for the pool. To disable, set to `false`.
- `workerHistogram`: (`boolean`) By default `false`. It will hint the Worker pool to record statistics for each individual Worker
- `loadBalancer`: ([`PiscinaLoadBalancer`](#piscinaloadbalancer)) By default, Piscina uses a least-busy algorithm. The `loadBalancer`
option can be used to provide an alternative implementation. See [Custom Load Balancers](../advanced-topics/loadbalancer.mdx) for additional detail.
- `workerHistogram`: (`boolean`) By default `false`. It will hint the Worker pool to record statistics for each individual Worker
- `loadBalancer`: ([`PiscinaLoadBalancer`](#piscinaloadbalancer)) By default, Piscina uses a least-busy algorithm. The `loadBalancer`
option can be used to provide an alternative implementation. See [Custom Load Balancers](../advanced-topics/loadbalancer.mdx) for additional detail.
:::caution
Use caution when setting resource limits. Setting limits that are too low may
result in the `Piscina` worker threads being unusable.
:::
:::info
**Note on Explicit Resource Management**: Piscina does has support for `Symbol.dispose` and `Symbol.asyncDispose` for explicit resource management for its usage with the `using` keyword.
This is only avaiable on Node.js 24 and higher.
For more information, see the [Explicit Resource Management](https://github.com/tc39/proposal-explicit-resource-management).
:::
## `PiscinaHistogram`
The `PiscinaHistogram` allows you to access the histogram data for the pool of worker threads.
It can be reset upon request in case of a need to clear the data.
**Example**:
```js
import { Piscina } from 'piscina';
const pool = new Piscina({
filename: resolve(__dirname, 'path/to/worker.js'),
});
const firstBatch = [];
for (let n = 0; n < 10; n++) {
firstBatch.push(pool.run('42'));
}
await Promise.all(firstBatch);
console.log(pool.histogram.runTime); // Print run time histogram summary
console.log(pool.histogram.waitTime); // Print wait time histogram summary
// If in need to reset the histogram data for a new set of tasks
pool.histogram.resetRunTime();
pool.histogram.resetWaitTime();
const secondBatch = [];
for (let n = 0; n < 10; n++) {
secondBatch.push(pool.run('42'));
}
await Promise.all(secondBatch);
// The histogram data will only contain the data for the second batch of tasks
console.log(pool.histogram.runTime);
console.log(pool.histogram.waitTime);
```
### Interface: `PiscinaLoadBalancer`
- `runTime`: (`PiscinaHistogramSummary`) Run Time Histogram Summary. Time taken to execute a task.
- `waitTime`: (`PiscinaHistogramSummary`) Wait Time Histogram Summary. Time between a task being submitted and the task starting to run.
> **Note**: The histogram data is only available if `recordTiming` is set to `true`.
```ts
type PiscinaHistogram = {
runTime: PiscinaHistogramSummary;
waitTime: PiscinaHistogramSummary;
resetRunTime(): void; // Reset Run Time Histogram
resetWaitTime(): void; // Reset Wait Time Histogram
```
### Interface: `PiscinaHistogramSummary`
```ts
type PiscinaHistogramSummary = {
average: number;
mean: number;
stddev: number;
min: number;
max: number;
p0_001: number;
p0_01: number;
p0_1: number;
p1: number;
p2_5: number;
p10: number;
p25: number;
p50: number;
p75: number;
p90: number;
p97_5: number;
p99: number;
p99_9: number;
p99_99: number;
p99_999: number;
}
```
## `PiscinaLoadBalancer`
The `PiscinaLoadBalancer` interface is used to implement custom load balancing algorithm that determines which worker thread should be assigned a task.
> For more information, see [Custom Load Balancers](../advanced-topics/loadbalancer.mdx).
### Interface: `PiscinaLoadBalancer`
```ts
type PiscinaLoadBalancer = (
task: PiscinaTask, // Task to be distributed
workers: PiscinaWorker[] // Array of Worker instances
) => PiscinaWorker | null; // Worker instance to be assigned the task
```
If the `PiscinaLoadBalancer` returns `null`, `Piscina` will attempt to spawn a new worker, otherwise the task will be queued until a worker is available.
### Interface: `PiscinaTask`
```ts
interface PiscinaTask {
taskId: number; // Unique identifier for the task
filename: string; // Filename of the worker module
name: string; // Name of the worker function
created: number; // Timestamp when the task was created
isAbortable: boolean; // Indicates if the task can be aborted through AbortSignal
}
```
### Interface: `PiscinaWorker`
```ts
interface PiscinaWorker {
id: number; // Unique identifier for the worker
currentUsage: number; // Number of tasks currently running on the worker
isRunningAbortableTask: boolean; // Indicates if the worker is running an abortable task
histogram: HistogramSummary | null; // Worker histogram
terminating: boolean; // Indicates if the worker is terminating
destroyed: boolean; // Indicates if the worker has been destroyed
}
```
### Example: Custom Load Balancer
#### JavaScript
<a id="custom-load-balancer-example-js"> </a>
```js
const { Piscina } = require('piscina');
function LeastBusyBalancer(opts) {
const { maximumUsage } = opts;
return (task, workers) => {
let candidate = null;
let checkpoint = maximumUsage;
for (const worker of workers) {
if (worker.currentUsage === 0) {
candidate = worker;
break;
}
if (worker.isRunningAbortableTask) continue;
if (!task.isAbortable && worker.currentUsage < checkpoint) {
candidate = worker;
checkpoint = worker.currentUsage;
}
}
return candidate;
};
}
const piscina = new Piscina({
loadBalancer: LeastBusyBalancer({ maximumUsage: 2 }),
});
piscina
.runTask({ filename: 'worker.js', name: 'default' })
.then((result) => console.log(result))
.catch((err) => console.error(err));
```
#### TypeScript
<a id="custom-load-balancer-example-ts"> </a>
```ts
import { Piscina } from 'piscina';
function LeastBusyBalancer(
opts: LeastBusyBalancerOptions
): PiscinaLoadBalancer {
const { maximumUsage } = opts;
return (task, workers) => {
let candidate: PiscinaWorker | null = null;
let checkpoint = maximumUsage;
for (const worker of workers) {
if (worker.currentUsage === 0) {
candidate = worker;
break;
}
if (worker.isRunningAbortableTask) continue;
if (!task.isAbortable && worker.currentUsage < checkpoint) {
candidate = worker;
checkpoint = worker.currentUsage;
}
}
return candidate;
};
}
const piscina = new Piscina({
loadBalancer: LeastBusyBalancer({ maximumUsage: 2 }),
});
piscina
.runTask({ filename: 'worker.js', name: 'default' })
.then((result) => console.log(result))
.catch((err) => console.error(err));
```

42
node_modules/piscina/docs/docs/api-reference/event.md generated vendored Normal file
View File

@@ -0,0 +1,42 @@
---
id: Events
sidebar_position: 4
---
## Event: `'error'`
An `'error'` event is emitted by instances of this class when:
- Uncaught exceptions occur inside Worker threads that do not currently handle
tasks.
- Unexpected messages are sent from from Worker threads.
All other errors are reported by rejecting the `Promise` returned from
`run()` or `runTask()`, including rejections reported by the handler function
itself.
## Event: `'drain'`
A `'drain'` event is emitted whenever the `queueSize` reaches `0`.
## Event: `'needsDrain'`
Similar to [`Piscina#needsDrain`](https://github.com/piscinajs/piscina#property-needsdrain-readonly);
this event is triggered once the total capacity of the pool is exceeded
by number of tasks enqueued that are pending of execution.
## Event: `'message'`
A `'message'` event is emitted whenever a message is received from a worker thread.
## Event: `'workerCreate'`
Event that is triggered when a new worker is created.
As argument, it receives the worker instance.
## Event: `'workerDestroy'`
Event that is triggered when a worker is destroyed.
As argument, it receives the worker instance that has been destroyed.

51
node_modules/piscina/docs/docs/api-reference/interface.md generated vendored Normal file
View File

@@ -0,0 +1,51 @@
---
id: Interface
sidebar_position: 7
---
## Interface: `Transferable`
Objects may implement the `Transferable` interface to create their own
custom transferable objects. This is useful when an object being
passed into or from a worker contains a deeply nested transferable
object such as an `ArrayBuffer` or `MessagePort`.
`Transferable` objects expose two properties inspected by Piscina
to determine how to transfer the object. These properties are
named using the special static `Piscina.transferableSymbol` and
`Piscina.valueSymbol` properties:
* The `Piscina.transferableSymbol` property provides the object
(or objects) that are to be included in the `transferList`.
* The `Piscina.valueSymbol` property provides a surrogate value
to transmit in place of the `Transferable` itself.
Both properties are required.
For example:
```js
const {
move,
transferableSymbol,
valueSymbol
} = require('piscina');
module.exports = () => {
const obj = {
a: { b: new Uint8Array(5); },
c: { new Uint8Array(10); },
get [transferableSymbol]() {
// Transfer the two underlying ArrayBuffers
return [this.a.b.buffer, this.c.buffer];
}
get [valueSymbol]() {
return { a: { b: this.a.b }, c: this.c };
}
};
return move(obj);
};
```

78
node_modules/piscina/docs/docs/api-reference/method.md generated vendored Normal file
View File

@@ -0,0 +1,78 @@
---
id: Methods
sidebar_position: 3
---
## Method: `run(task[, options])`
Schedules a task to be run on a Worker thread.
* `task`: Any value. This will be passed to the function that is exported from
`filename`.
* `options`:
* `transferList`: An optional lists of objects that is passed to
[`postMessage()`] when posting `task` to the Worker, which are transferred
rather than cloned.
* `filename`: Optionally overrides the `filename` option passed to the
constructor for this task. If no `filename` was specified to the constructor,
this is mandatory.
* `name`: Optionally overrides the exported worker function used for the task.
* `abortSignal`: An `AbortSignal` instance. If passed, this can be used to
cancel a task. If the task is already running, the corresponding `Worker`
thread will be stopped.
(More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
events can be passed here.) Abortable tasks cannot share threads regardless
of the `concurrentTasksPerWorker` options.
This returns a `Promise` for the return value of the (async) function call
made to the function exported from `filename`. If the (async) function throws
an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.
## Method: `runTask(task[, transferList][, filename][, abortSignal])`
**Deprecated** -- Use `run(task, options)` instead.
Schedules a task to be run on a Worker thread.
* `task`: Any value. This will be passed to the function that is exported from
`filename`.
* `transferList`: An optional lists of objects that is passed to
[`postMessage()`] when posting `task` to the Worker, which are transferred
rather than cloned.
* `filename`: Optionally overrides the `filename` option passed to the
constructor for this task. If no `filename` was specified to the constructor,
this is mandatory.
* `signal`: An [`AbortSignal`][] instance. If passed, this can be used to
cancel a task. If the task is already running, the corresponding `Worker`
thread will be stopped.
(More generally, any `EventEmitter` or `EventTarget` that emits `'abort'`
events can be passed here.) Abortable tasks cannot share threads regardless
of the `concurrentTasksPerWorker` options.
This returns a `Promise` for the return value of the (async) function call
made to the function exported from `filename`. If the (async) function throws
an error, the returned `Promise` will be rejected with that error.
If the task is aborted, the returned `Promise` is rejected with an error
as well.
## Method: `destroy()`
Stops all Workers and rejects all `Promise`s for pending tasks.
This returns a `Promise` that is fulfilled once all threads have stopped.
## Method: `close([options])`
* `options`:
* `force`: A `boolean` value that indicates whether to abort all tasks that
are enqueued but not started yet. The default is `false`.
It stops all Workers gracefully.
This returns a `Promise` that is fulfilled once all tasks that were started
have completed and all threads have stopped.
This method is similar to `destroy()`, but with the difference that `close()`
will wait for the worker tasks to finish, while `destroy()`
will abort them immediately.

133
node_modules/piscina/docs/docs/api-reference/property.md generated vendored Normal file
View File

@@ -0,0 +1,133 @@
---
id: Properties
sidebar_position: 5
---
## Property: `completed` (readonly)
The current number of completed tasks.
## Property: `duration` (readonly)
The length of time (in milliseconds) since this `Piscina` instance was
created.
## Property: `options` (readonly)
A copy of the options that are currently being used by this instance. This
object has the same properties as the options object passed to the constructor.
## Property: `runTime` (readonly)
A histogram summary object summarizing the collected run times of completed
tasks. All values are expressed in milliseconds.
* `runTime.average` {`number`} The average run time of all tasks
* `runTime.mean` {`number`} The mean run time of all tasks
* `runTime.stddev` {`number`} The standard deviation of collected run times
* `runTime.min` {`number`} The fastest recorded run time
* `runTime.max` {`number`} The slowest recorded run time
All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
represent the percentile distributions of run time observations. For example,
`p99` is the 99th percentile indicating that 99% of the observed run times were
faster or equal to the given value.
```js
{
average: 1880.25,
mean: 1880.25,
stddev: 1.93,
min: 1877,
max: 1882.0190887451172,
p0_001: 1877,
p0_01: 1877,
p0_1: 1877,
p1: 1877,
p2_5: 1877,
p10: 1877,
p25: 1877,
p50: 1881,
p75: 1881,
p90: 1882,
p97_5: 1882,
p99: 1882,
p99_9: 1882,
p99_99: 1882,
p99_999: 1882
}
```
## Property: `threads` (readonly)
An Array of the `Worker` instances used by this pool.
## Property: `queueSize` (readonly)
The current number of tasks waiting to be assigned to a Worker thread.
## Property: `needsDrain` (readonly)
Boolean value that specifies whether the capacity of the pool has
been exceeded by the number of tasks submitted.
This property is helpful to make decisions towards creating backpressure
over the number of tasks submitted to the pool.
## Property: `utilization` (readonly)
A point-in-time ratio comparing the approximate total mean run time
of completed tasks to the total runtime capacity of the pool.
A pools runtime capacity is determined by multiplying the `duration`
by the `options.maxThread` count. This provides an absolute theoretical
maximum aggregate compute time that the pool would be capable of.
The approximate total mean run time is determined by multiplying the
mean run time of all completed tasks by the total number of completed
tasks. This number represents the approximate amount of time the
pool as been actively processing tasks.
The utilization is then calculated by dividing the approximate total
mean run time by the capacity, yielding a fraction between `0` and `1`.
## Property: `waitTime` (readonly)
A histogram summary object summarizing the collected times tasks spent
waiting in the queue. All values are expressed in milliseconds.
* `waitTime.average` {`number`} The average wait time of all tasks
* `waitTime.mean` {`number`} The mean wait time of all tasks
* `waitTime.stddev` {`number`} The standard deviation of collected wait times
* `waitTime.min` {`number`} The fastest recorded wait time
* `waitTime.max` {`number`} The longest recorded wait time
All properties following the pattern `p{N}` where N is a number (e.g. `p1`, `p99`)
represent the percentile distributions of wait time observations. For example,
`p99` is the 99th percentile indicating that 99% of the observed wait times were
faster or equal to the given value.
```js
{
average: 1880.25,
mean: 1880.25,
stddev: 1.93,
min: 1877,
max: 1882.0190887451172,
p0_001: 1877,
p0_01: 1877,
p0_1: 1877,
p1: 1877,
p2_5: 1877,
p10: 1877,
p25: 1877,
p50: 1881,
p75: 1881,
p90: 1882,
p97_5: 1882,
p99: 1882,
p99_9: 1882,
p99_99: 1882,
p99_999: 1882
}
```

38
node_modules/piscina/docs/docs/api-reference/static-property.md generated vendored Normal file
View File

@@ -0,0 +1,38 @@
---
id: Static Properties and Methods
sidebar_position: 6
---
## Static property: `isWorkerThread` (readonly)
Is `true` if this code runs inside a `Piscina` threadpool as a Worker.
## Static property: `version` (readonly)
Provides the current version of this library as a semver string.
## Static method: `move(value)`
By default, any value returned by a worker function will be cloned when
returned back to the Piscina pool, even if that object is capable of
being transfered. The `Piscina.move()` method can be used to wrap and
mark transferable values such that they will by transfered rather than
cloned.
The `value` may be any object supported by Node.js to be transferable
(e.g. `ArrayBuffer`, any `TypedArray`, or `MessagePort`), or any object
implementing the `Transferable` interface.
```js
const { move } = require('piscina');
module.exports = () => {
return move(new ArrayBuffer(10));
}
```
The `move()` method will throw if the `value` is not transferable.
The object returned by the `move()` method should not be set as a
nested value in an object. If it is used, the `move()` object itself
will be cloned as opposed to transfering the object it wraps.

9
node_modules/piscina/docs/docs/update-log/changelog.md generated vendored Normal file
View File

@@ -0,0 +1,9 @@
---
title: "ChangeLog"
sidebar_position: 1
---
import Changelog from '../../../CHANGELOG.md'
<Changelog/>

97
node_modules/piscina/docs/docs/update-log/release-notes.md generated vendored Normal file
View File

@@ -0,0 +1,97 @@
---
id: Release Notes
sidebar_position: 1
---
### 4.1.0
#### Features
* add `needsDrain` property ([#368](https://github.com/piscinajs/piscina/issues/368)) ([2d49b63](https://github.com/piscinajs/piscina/commit/2d49b63368116c172a52e2019648049b4d280162))
* correctly handle process.exit calls outside of a task ([#361](https://github.com/piscinajs/piscina/issues/361)) ([8e6d16e](https://github.com/piscinajs/piscina/commit/8e6d16e1dc23f8bb39772ed954f6689852ad435f))
#### Bug Fixes
* Fix types for TypeScript 4.7 ([#239](https://github.com/piscinajs/piscina/issues/239)) ([a38fb29](https://github.com/piscinajs/piscina/commit/a38fb292e8fcc45cc20abab8668f82d908a24dc0))
* use CJS imports ([#374](https://github.com/piscinajs/piscina/issues/374)) ([edf8dc4](https://github.com/piscinajs/piscina/commit/edf8dc4f1a19e9b49e266109cdb70d9acc86f3ca))
### 4.0.0
* Drop Node.js 14.x support
* Add Node.js 20.x to CI
### 3.2.0
* Adds a new `PISCINA_DISABLE_ATOMICS` environment variable as an alternative way of
disabling Piscina's internal use of the `Atomics` API. (https://github.com/piscinajs/piscina/pull/163)
* Fixes a bug with transferable objects. (https://github.com/piscinajs/piscina/pull/155)
* Fixes CI issues with TypeScript. (https://github.com/piscinajs/piscina/pull/161)
### 3.1.0
* Deprecates `piscina.runTask()`; adds `piscina.run()` as an alternative.
https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
* Allows multiple exported handler functions from a single file.
https://github.com/piscinajs/piscina/commit/d7fa24d7515789001f7237ad6ae9ad42d582fc75
### 3.0.0
* Drops Node.js 10.x support
* Updates minimum TypeScript target to ES2019
### 2.1.0
* Adds name property to indicate `AbortError` when tasks are
canceled using an `AbortController` (or similar)
* More examples
### 2.0.0
* Added unmanaged file descriptor tracking
* Updated dependencies
### 1.6.1
* Bug fix: Reject if AbortSignal is already aborted
* Bug Fix: Use once listener for abort event
### 1.6.0
* Add the `niceIncrement` configuration parameter.
### 1.5.1
* Bug fixes around abortable task selection.
### 1.5.0
* Added `Piscina.move()`
* Added Custom Task Queues
* Added utilization metric
* Wait for workers to be ready before considering them as candidates
* Additional examples
### 1.4.0
* Added `maxQueue = 'auto'` to autocalculate the maximum queue size.
* Added more examples, including an example of implementing a worker
as a Node.js native addon.
### 1.3.0
* Added the `'drain'` event
### 1.2.0
* Added support for ESM and file:// URLs
* Added `env`, `argv`, `execArgv`, and `workerData` options
* More examples
### 1.1.0
* Added support for Worker Thread `resourceLimits`
### 1.0.0
* Initial release!

71
node_modules/piscina/package.json generated vendored Normal file
View File

@@ -0,0 +1,71 @@
{
"name": "piscina",
"version": "5.1.3",
"description": "A fast, efficient Node.js Worker Thread Pool implementation",
"main": "./dist/main.js",
"types": "./dist/index.d.ts",
"exports": {
"types": "./dist/index.d.ts",
"import": "./dist/esm-wrapper.mjs",
"require": "./dist/main.js"
},
"engines": {
"node": ">=20.x"
},
"scripts": {
"build": "tsc && gen-esm-wrapper . dist/esm-wrapper.mjs",
"lint": "eslint",
"test": "node scripts/run-tests.js --pattern='test/**/*test.ts'",
"test:ci": "npm run lint && npm run build && npm run test:coverage",
"test:coverage": "node scripts/run-tests.js --coverage --pattern='test/**/*test.ts'",
"prepack": "npm run build",
"benchmark": "npm run benchmark:queue && npm run benchmark:piscina",
"benchmark:piscina": "npm run benchmark:default && npm run benchmark:queue:fixed && npm run benchmark:default:comparison",
"benchmark:default": "node benchmark/simple-benchmark.js",
"benchmark:default:async": "node benchmark/simple-benchmark.js",
"benchmark:default:comparison": "node benchmark/piscina-queue-comparison.js",
"benchmark:queue": "npm run benchmark:queue:comparison",
"benchmark:queue:fixed": "node benchmark/simple-benchmark-fixed-queue.js",
"benchmark:queue:comparison": "node benchmark/queue-comparison.js"
},
"repository": {
"type": "git",
"url": "git+https://github.com/piscinajs/piscina.git"
},
"keywords": [
"fast",
"worker threads",
"thread pool",
"wade wilson"
],
"author": "James M Snell <jasnell@gmail.com>",
"contributors": [
"Anna Henningsen <anna@addaleax.net>",
"Matteo Collina <matteo.collina@gmail.com>",
"Carlos Fuentes <me@metcoder.dev>"
],
"license": "MIT",
"devDependencies": {
"@types/node": "^22.4.1",
"abort-controller": "^3.0.0",
"concat-stream": "^2.0.0",
"eslint": "^9.16.0",
"gen-esm-wrapper": "^1.1.1",
"glob": "^11.0.2",
"neostandard": "^0.12.0",
"tinybench": "^4.0.1",
"tsx": "^4.20.3",
"typescript": "5.8.3"
},
"optionalDependencies": {
"@napi-rs/nice": "^1.0.4"
},
"bugs": {
"url": "https://github.com/piscinajs/piscina/issues"
},
"homepage": "https://github.com/piscinajs/piscina#readme",
"directories": {
"example": "examples",
"test": "test"
}
}

64
node_modules/piscina/scripts/run-tests.js generated vendored Normal file
View File

@@ -0,0 +1,64 @@
'use strict';
// As node 20 test runner does not support glob patterns in input
// and considering that we could have multiple OS we manually
// resolve the test files and pass them to the test runner
const { spawnSync } = require('node:child_process');
const { parseArgs } = require('node:util');
const { globSync } = require('glob');
const options = {
pattern: {
type: 'string',
short: 'p',
description: 'Glob pattern to match test files',
default: 'test/**/*test.ts',
},
coverage: {
type: 'boolean',
short: 'c',
description: 'Run tests with coverage',
default: false,
},
};
const {
values,
} = parseArgs({ args: process.argv.slice(2), options });
const pattern = values.pattern;
const isCoverage = values.coverage;
const testFiles = globSync(pattern, { absolute: true });
const args = [
'--enable-source-maps',
'--import=tsx',
'--test',
...testFiles
];
let result;
// we skip coverage for node 20
// because this issuse happen https://github.com/nodejs/node/pull/53315
if (isCoverage && !process.version.startsWith('v20.')) {
result = spawnSync(
'node',
[
'--experimental-test-coverage',
'--test-reporter=lcov',
'--test-reporter-destination=lcov.info',
'--test-reporter=spec',
'--test-reporter-destination=stdout',
...args
],
{ stdio: 'inherit' }
);
} else {
result = spawnSync('node', args, { stdio: 'inherit' });
}
process.exit(result.status);

41
node_modules/piscina/src/abort.ts generated vendored Normal file
View File

@@ -0,0 +1,41 @@
interface AbortSignalEventTargetAddOptions {
once: boolean;
}
export interface AbortSignalEventTarget {
addEventListener: (
name: 'abort',
listener: () => void,
options?: AbortSignalEventTargetAddOptions
) => void;
removeEventListener: (name: 'abort', listener: () => void) => void;
aborted?: boolean;
reason?: unknown;
}
export interface AbortSignalEventEmitter {
off: (name: 'abort', listener: () => void) => void;
once: (name: 'abort', listener: () => void) => void;
}
export type AbortSignalAny = AbortSignalEventTarget | AbortSignalEventEmitter;
export class AbortError extends Error {
constructor (reason?: AbortSignalEventTarget['reason']) {
// TS does not recognizes the cause clause
// @ts-expect-error
super('The task has been aborted', { cause: reason });
}
get name () {
return 'AbortError';
}
}
export function onabort (abortSignal: AbortSignalAny, listener: () => void) {
if ('addEventListener' in abortSignal) {
abortSignal.addEventListener('abort', listener, { once: true });
} else {
abortSignal.once('abort', listener);
}
}

61
node_modules/piscina/src/common.ts generated vendored Normal file
View File

@@ -0,0 +1,61 @@
import { fileURLToPath, URL } from 'node:url';
import { availableParallelism } from 'node:os';
import { kMovable, kTransferable, kValue } from './symbols';
// States wether the worker is ready to receive tasks
export const READY = '_WORKER_READY';
/**
* True if the object implements the Transferable interface
*
* @export
* @param {unknown} value
* @return {*} {boolean}
*/
export function isTransferable (value: unknown): boolean {
return (
value != null &&
typeof value === 'object' &&
kTransferable in value &&
kValue in value
);
}
/**
* True if object implements Transferable and has been returned
* by the Piscina.move() function
*
* TODO: narrow down the type of value
* @export
* @param {(unknown & PiscinaMovable)} value
* @return {*} {boolean}
*/
export function isMovable (value: any): boolean {
return isTransferable(value) && value[kMovable] === true;
}
export function markMovable (value: {}): void {
Object.defineProperty(value, kMovable, {
enumerable: false,
configurable: true,
writable: true,
value: true
});
}
// State of Piscina pool
export const commonState = {
isWorkerThread: false,
workerData: undefined
};
export function maybeFileURLToPath (filename : string) : string {
return filename.startsWith('file:')
? fileURLToPath(new URL(filename))
: filename;
}
export function getAvailableParallelism () : number {
return availableParallelism();
}

9
node_modules/piscina/src/errors.ts generated vendored Normal file
View File

@@ -0,0 +1,9 @@
export const Errors = {
ThreadTermination: () => new Error('Terminating worker thread'),
FilenameNotProvided: () =>
new Error('filename must be provided to run() or in options object'),
TaskQueueAtLimit: () => new Error('Task queue is at limit'),
NoTaskQueueAvailable: () =>
new Error('No task queue available and all Workers are busy'),
CloseTimeout: () => new Error('Close operation timed out')
};

104
node_modules/piscina/src/histogram.ts generated vendored Normal file
View File

@@ -0,0 +1,104 @@
import { RecordableHistogram, createHistogram } from 'node:perf_hooks';
export type PiscinaHistogramSummary = {
average: number;
mean: number;
stddev: number;
min: number;
max: number;
p0_001: number;
p0_01: number;
p0_1: number;
p1: number;
p2_5: number;
p10: number;
p25: number;
p50: number;
p75: number;
p90: number;
p97_5: number;
p99: number;
p99_9: number;
p99_99: number;
p99_999: number;
};
export type PiscinaHistogram = {
runTime: PiscinaHistogramSummary;
waitTime: PiscinaHistogramSummary;
resetRunTime(): void;
resetWaitTime(): void;
};
export class PiscinaHistogramHandler {
#runTime: RecordableHistogram;
#waitTime: RecordableHistogram;
constructor() {
this.#runTime = createHistogram();
this.#waitTime = createHistogram();
}
get runTimeSummary(): PiscinaHistogramSummary {
return PiscinaHistogramHandler.createHistogramSummary(this.#runTime);
}
get waitTimeSummary(): PiscinaHistogramSummary {
return PiscinaHistogramHandler.createHistogramSummary(this.#waitTime);
}
get runTimeCount(): number {
return this.#runTime.count;
}
recordRunTime(value: number) {
this.#runTime.record(PiscinaHistogramHandler.toHistogramIntegerNano(value));
}
recordWaitTime(value: number) {
this.#waitTime.record(
PiscinaHistogramHandler.toHistogramIntegerNano(value)
);
}
resetWaitTime(): void {
this.#waitTime.reset();
}
resetRunTime(): void {
this.#runTime.reset();
}
static createHistogramSummary(
histogram: RecordableHistogram
): PiscinaHistogramSummary {
const { mean, stddev, min, max } = histogram;
return {
average: mean / 1000,
mean: mean / 1000,
stddev,
min: min / 1000,
max: max / 1000,
p0_001: histogram.percentile(0.001) / 1000,
p0_01: histogram.percentile(0.01) / 1000,
p0_1: histogram.percentile(0.1) / 1000,
p1: histogram.percentile(1) / 1000,
p2_5: histogram.percentile(2.5) / 1000,
p10: histogram.percentile(10) / 1000,
p25: histogram.percentile(25) / 1000,
p50: histogram.percentile(50) / 1000,
p75: histogram.percentile(75) / 1000,
p90: histogram.percentile(90) / 1000,
p97_5: histogram.percentile(97.5) / 1000,
p99: histogram.percentile(99) / 1000,
p99_9: histogram.percentile(99.9) / 1000,
p99_99: histogram.percentile(99.99) / 1000,
p99_999: histogram.percentile(99.999) / 1000,
};
}
static toHistogramIntegerNano(milliseconds: number): number {
return Math.max(1, Math.trunc(milliseconds * 1000));
}
}

1013
node_modules/piscina/src/index.ts generated vendored Normal file
View File

File diff suppressed because it is too large Load Diff

4
node_modules/piscina/src/main.ts generated vendored Normal file
View File

@@ -0,0 +1,4 @@
import Piscina from './index';
// Used as the require() entry point to maintain existing behavior
export = Piscina;

10
node_modules/piscina/src/symbols.ts generated vendored Normal file
View File

@@ -0,0 +1,10 @@
// Internal symbol used to mark Transferable objects returned
// by the Piscina.move() function
export const kMovable = Symbol('Piscina.kMovable');
export const kWorkerData = Symbol('Piscina.kWorkerData');
export const kTransferable = Symbol.for('Piscina.transferable');
export const kValue = Symbol.for('Piscina.valueOf');
export const kQueueOptions = Symbol.for('Piscina.queueOptions');
export const kRequestCountField = 0;
export const kResponseCountField = 1;
export const kFieldCount = 2;

25
node_modules/piscina/src/task_queue/array_queue.ts generated vendored Normal file
View File

@@ -0,0 +1,25 @@
import assert from 'node:assert';
import type { TaskQueue, Task } from '.';
export class ArrayTaskQueue implements TaskQueue {
tasks: Task[] = []
get size () {
return this.tasks.length;
}
shift (): Task | null {
return this.tasks.shift() ?? null;
}
push (task: Task): void {
this.tasks.push(task);
}
remove (task: Task): void {
const index = this.tasks.indexOf(task);
assert.notStrictEqual(index, -1);
this.tasks.splice(index, 1);
}
}

21
node_modules/piscina/src/task_queue/common.ts generated vendored Normal file
View File

@@ -0,0 +1,21 @@
import type { kQueueOptions } from '../symbols';
export interface TaskQueue {
readonly size: number;
shift(): Task | null;
remove(task: Task): void;
push(task: Task): void;
}
// Public Interface
export interface PiscinaTask extends Task {
taskId: number;
filename: string;
name: string;
created: number;
isAbortable: boolean;
}
export interface Task {
readonly [kQueueOptions]: object | null
};

177
node_modules/piscina/src/task_queue/fixed_queue.ts generated vendored Normal file
View File

@@ -0,0 +1,177 @@
/*
* Modified Fixed Queue Implementation based on the one from Node.js Project
* License: MIT License
* Source: https://github.com/nodejs/node/blob/de7b37880f5a541d5f874c1c2362a65a4be76cd0/lib/internal/fixed_queue.js
*/
import assert from 'node:assert';
import type { Task } from './common';
import { TaskQueue } from '.';
// Currently optimal queue size, tested on V8 6.0 - 6.6. Must be power of two.
const kSize = 2048;
const kMask = kSize - 1;
// The FixedQueue is implemented as a singly-linked list of fixed-size
// circular buffers. It looks something like this:
//
// head tail
// | |
// v v
// +-----------+ <-----\ +-----------+ <------\ +-----------+
// | [null] | \----- | next | \------- | next |
// +-----------+ +-----------+ +-----------+
// | item | <-- bottom | item | <-- bottom | [empty] |
// | item | | item | | [empty] |
// | item | | item | | [empty] |
// | item | | item | | [empty] |
// | item | | item | bottom --> | item |
// | item | | item | | item |
// | ... | | ... | | ... |
// | item | | item | | item |
// | item | | item | | item |
// | [empty] | <-- top | item | | item |
// | [empty] | | item | | item |
// | [empty] | | [empty] | <-- top top --> | [empty] |
// +-----------+ +-----------+ +-----------+
//
// Or, if there is only one circular buffer, it looks something
// like either of these:
//
// head tail head tail
// | | | |
// v v v v
// +-----------+ +-----------+
// | [null] | | [null] |
// +-----------+ +-----------+
// | [empty] | | item |
// | [empty] | | item |
// | item | <-- bottom top --> | [empty] |
// | item | | [empty] |
// | [empty] | <-- top bottom --> | item |
// | [empty] | | item |
// +-----------+ +-----------+
//
// Adding a value means moving `top` forward by one, removing means
// moving `bottom` forward by one. After reaching the end, the queue
// wraps around.
//
// When `top === bottom` the current queue is empty and when
// `top + 1 === bottom` it's full. This wastes a single space of storage
// but allows much quicker checks.
class FixedCircularBuffer {
bottom: number = 0
top: number = 0
list: Array<Task | undefined> = new Array(kSize)
next: FixedCircularBuffer | null = null
isEmpty () {
return this.top === this.bottom;
}
isFull () {
return ((this.top + 1) & kMask) === this.bottom;
}
push (data:Task) {
this.list[this.top] = data;
this.top = (this.top + 1) & kMask;
}
shift () {
const nextItem = this.list[this.bottom];
if (nextItem === undefined) { return null; }
this.list[this.bottom] = undefined;
this.bottom = (this.bottom + 1) & kMask;
return nextItem;
}
remove (task: Task) {
const indexToRemove = this.list.indexOf(task);
assert.notStrictEqual(indexToRemove, -1);
let curr = indexToRemove;
while (true) {
const next = (curr + 1) & kMask;
this.list[curr] = this.list[next];
if (this.list[curr] === undefined) {
break;
}
if (next === indexToRemove) {
this.list[curr] = undefined;
break;
}
curr = next;
}
this.top = (this.top - 1) & kMask;
}
}
export class FixedQueue implements TaskQueue {
head: FixedCircularBuffer
tail: FixedCircularBuffer
#size: number = 0
constructor () {
this.head = this.tail = new FixedCircularBuffer();
}
isEmpty () {
return this.head.isEmpty();
}
push (data:Task) {
if (this.head.isFull()) {
// Head is full: Creates a new queue, sets the old queue's `.next` to it,
// and sets it as the new main queue.
this.head = this.head.next = new FixedCircularBuffer();
}
this.head.push(data);
this.#size++;
}
shift (): Task | null {
const tail = this.tail;
const next = tail.shift();
if (next !== null) this.#size--;
if (tail.isEmpty() && tail.next !== null) {
// If there is another queue, it forms the new tail.
this.tail = tail.next;
tail.next = null;
}
return next;
}
remove (task: Task) {
let prev: FixedCircularBuffer | null = null;
let buffer = this.tail;
while (true) {
if (buffer.list.includes(task)) {
buffer.remove(task);
this.#size--;
break;
}
if (buffer.next === null) break;
prev = buffer;
buffer = buffer.next;
}
if (buffer.isEmpty()) {
// removing tail
if (prev === null) {
// if tail is not the last buffer
if (buffer.next !== null) this.tail = buffer.next;
} else {
// removing head
if (buffer.next === null) {
this.head = prev;
} else {
// removing buffer from middle
prev.next = buffer.next;
}
}
}
}
get size () {
return this.#size;
}
};

148
node_modules/piscina/src/task_queue/index.ts generated vendored Normal file
View File

@@ -0,0 +1,148 @@
import type { MessagePort } from 'node:worker_threads';
import { performance } from 'node:perf_hooks';
import { AsyncResource } from 'node:async_hooks';
import type { WorkerInfo } from '../worker_pool';
import type { AbortSignalAny, AbortSignalEventEmitter } from '../abort';
import { isMovable } from '../common';
import { kTransferable, kValue, kQueueOptions } from '../symbols';
import type { Task, TaskQueue, PiscinaTask } from './common';
export { ArrayTaskQueue } from './array_queue';
export { FixedQueue } from './fixed_queue';
export type TaskCallback = (err: Error, result: any) => void
// Grab the type of `transferList` off `MessagePort`. At the time of writing,
// only ArrayBuffer and MessagePort are valid, but let's avoid having to update
// our types here every time Node.js adds support for more objects.
export type TransferList = MessagePort extends {
postMessage: (value: any, transferList: infer T) => any
}
? T
: never
export type TransferListItem = TransferList extends Array<infer T> ? T : never
/**
* Verifies if a given TaskQueue is valid
*
* @export
* @param {*} value
* @return {*} {boolean}
*/
export function isTaskQueue (value: TaskQueue): boolean {
return (
typeof value === 'object' &&
value !== null &&
'size' in value &&
typeof value.shift === 'function' &&
typeof value.remove === 'function' &&
typeof value.push === 'function'
);
}
let taskIdCounter = 0;
// Extend AsyncResource so that async relations between posting a task and
// receiving its result are visible to diagnostic tools.
export class TaskInfo extends AsyncResource implements Task {
callback : TaskCallback;
task : any;
transferList : TransferList;
filename : string;
name : string;
taskId : number;
abortSignal : AbortSignalAny | null;
// abortListener : (() => void) | null = null;
workerInfo : WorkerInfo | null = null;
created : number;
started : number;
aborted = false;
_abortListener: (() => void) | null = null;
constructor (
task : any,
transferList : TransferList,
filename : string,
name : string,
callback : TaskCallback,
abortSignal : AbortSignalAny | null,
triggerAsyncId : number) {
super('Piscina.Task', { requireManualDestroy: true, triggerAsyncId });
this.callback = callback;
this.task = task;
this.transferList = transferList;
// If the task is a Transferable returned by
// Piscina.move(), then add it to the transferList
// automatically
if (isMovable(task)) {
// This condition should never be hit but typescript
// complains if we dont do the check.
/* istanbul ignore if */
if (this.transferList == null) {
this.transferList = [];
}
this.transferList =
this.transferList.concat(task[kTransferable]);
this.task = task[kValue];
}
this.filename = filename;
this.name = name;
// TODO: This should not be global
this.taskId = taskIdCounter++;
this.abortSignal = abortSignal;
this.created = performance.now();
this.started = 0;
}
// TODO: improve this handling - ideally should be extended
set abortListener (value: (() => void)) {
this._abortListener = () => {
this.aborted = true;
value();
};
}
get abortListener (): (() => void) | null {
return this._abortListener;
}
releaseTask () : any {
const ret = this.task;
this.task = null;
return ret;
}
done (err : Error | null, result? : any) : void {
this.runInAsyncScope(this.callback, null, err, result);
this.emitDestroy(); // `TaskInfo`s are used only once.
// If an abort signal was used, remove the listener from it when
// done to make sure we do not accidentally leak.
if (this.abortSignal && this.abortListener) {
if ('removeEventListener' in this.abortSignal && this.abortListener) {
this.abortSignal.removeEventListener('abort', this.abortListener);
} else {
(this.abortSignal as AbortSignalEventEmitter).off(
'abort', this.abortListener);
}
}
}
get [kQueueOptions] () : {} | null {
return this.task?.[kQueueOptions] ?? null;
}
get interface (): PiscinaTask {
return {
taskId: this.taskId,
filename: this.filename,
name: this.name,
created: this.created,
isAbortable: this.abortSignal !== null,
[kQueueOptions]: this[kQueueOptions]
};
}
}
export { Task, TaskQueue, PiscinaTask };

52
node_modules/piscina/src/types.ts generated vendored Normal file
View File

@@ -0,0 +1,52 @@
import type { MessagePort, Worker } from 'node:worker_threads';
import type { READY } from './common';
import type { kTransferable, kValue } from './symbols';
export interface StartupMessage {
filename: string | null
name: string
port: MessagePort
sharedBuffer: Int32Array
atomics: 'async' | 'sync' | 'disabled'
niceIncrement: number
}
export interface RequestMessage {
taskId: number
task: any
filename: string
name: string
histogramEnabled: number
}
export interface ReadyMessage {
[READY]: true
}
export interface ResponseMessage {
taskId: number
result: any
error: Error | null
time: number | null
}
export const commonState = {
isWorkerThread: false,
workerData: undefined
};
export interface Transferable {
readonly [kTransferable]: object;
readonly [kValue]: object;
}
export type ResourceLimits = Worker extends {
resourceLimits?: infer T;
}
? T
: {};
export type EnvSpecifier = typeof Worker extends {
new (filename: never, options?: { env: infer T }): Worker;
}
? T
: never;

239
node_modules/piscina/src/worker.ts generated vendored Normal file
View File

@@ -0,0 +1,239 @@
import { parentPort, MessagePort, receiveMessageOnPort, workerData } from 'node:worker_threads';
import { pathToFileURL } from 'node:url';
import { performance } from 'node:perf_hooks';
import type {
ReadyMessage,
RequestMessage,
ResponseMessage,
StartupMessage
} from './types';
import {
kResponseCountField,
kRequestCountField,
kTransferable,
kValue
} from './symbols';
import {
READY,
commonState,
isMovable
} from './common';
commonState.isWorkerThread = true;
commonState.workerData = workerData;
/* c8 ignore next*/
function noop (): void {}
const handlerCache : Map<string, Function> = new Map();
let useAtomics : boolean = process.env.PISCINA_DISABLE_ATOMICS !== '1';
let useAsyncAtomics : boolean = process.env.PISCINA_ENABLE_ASYNC_ATOMICS === '1';
// Get `import(x)` as a function that isn't transpiled to `require(x)` by
// TypeScript for dual ESM/CJS support.
// Load this lazily, so that there is no warning about the ESM loader being
// experimental (on Node v12.x) until we actually try to use it.
let importESMCached : (specifier : string) => Promise<any> | undefined;
function getImportESM () {
if (importESMCached === undefined) {
// eslint-disable-next-line no-new-func
importESMCached = new Function('specifier', 'return import(specifier)') as typeof importESMCached;
}
return importESMCached;
}
// Look up the handler function that we call when a task is posted.
// This is either going to be "the" export from a file, or the default export.
async function getHandler (filename : string, name : string) : Promise<Function | null> {
let handler = handlerCache.get(`${filename}/${name}`);
if (handler != null) {
return handler;
}
try {
// With our current set of TypeScript options, this is transpiled to
// `require(filename)`.
handler = await import(filename);
if (typeof handler !== 'function') {
handler = await ((handler as any)[name]);
}
} catch {}
if (typeof handler !== 'function') {
handler = await getImportESM()(pathToFileURL(filename).href);
if (typeof handler !== 'function') {
handler = await ((handler as any)[name]);
}
}
if (typeof handler !== 'function') {
return null;
}
// Limit the handler cache size. This should not usually be an issue and is
// only provided for pathological cases.
/* c8 ignore next */
if (handlerCache.size > 1000) {
const [[key]] = handlerCache;
handlerCache.delete(key);
}
handlerCache.set(`${filename}/${name}`, handler);
return handler;
}
// We should only receive this message once, when the Worker starts. It gives
// us the MessagePort used for receiving tasks, a SharedArrayBuffer for fast
// communication using Atomics, and the name of the default filename for tasks
// (so we can pre-load and cache the handler).
parentPort!.on('message', async (message: StartupMessage) => {
const { port, sharedBuffer, filename, name, niceIncrement } = message;
if (niceIncrement !== 0) {
(await import('@napi-rs/nice').catch(noop))?.nice(niceIncrement);
}
try {
if (filename != null) {
await getHandler(filename, name);
}
const readyMessage : ReadyMessage = { [READY]: true };
useAtomics = useAtomics !== false && message.atomics !== 'disabled';
useAsyncAtomics = useAtomics !== false && (useAsyncAtomics || message.atomics === 'async');
parentPort!.postMessage(readyMessage);
port.on('message', onMessage.bind(null, port, sharedBuffer));
if (useAtomics) {
const res = atomicsWaitLoop(port, sharedBuffer);
if (res?.then != null) await res;
}
} catch (error) {
throwInNextTick(error as Error);
}
});
let currentTasks : number = 0;
let lastSeenRequestCount : number = 0;
function atomicsWaitLoop (port : MessagePort, sharedBuffer : Int32Array) {
// This function is entered either after receiving the startup message, or
// when we are done with a task. In those situations, the *only* thing we
// expect to happen next is a 'message' on `port`.
// That call would come with the overhead of a C++ → JS boundary crossing,
// including async tracking. So, instead, if there is no task currently
// running, we wait for a signal from the parent thread using Atomics.wait(),
// and read the message from the port instead of generating an event,
// in order to avoid that overhead.
// The one catch is that this stops asynchronous operations that are still
// running from proceeding. Generally, tasks should not spawn asynchronous
// operations without waiting for them to finish, though.
if (useAsyncAtomics === true) {
// @ts-expect-error - for some reason not supported by TS
const { async, value } = Atomics.waitAsync(sharedBuffer, kRequestCountField, lastSeenRequestCount);
// We do not check for result
/* c8 ignore start */
return async === true && value.then(() => {
lastSeenRequestCount = Atomics.load(sharedBuffer, kRequestCountField);
// We have to read messages *after* updating lastSeenRequestCount in order
// to avoid race conditions.
let entry;
while ((entry = receiveMessageOnPort(port)) !== undefined) {
onMessage(port, sharedBuffer, entry.message);
}
});
/* c8 ignore stop */
}
while (currentTasks === 0) {
// Check whether there are new messages by testing whether the current
// number of requests posted by the parent thread matches the number of
// requests received.
// We do not check for result
Atomics.wait(sharedBuffer, kRequestCountField, lastSeenRequestCount);
lastSeenRequestCount = Atomics.load(sharedBuffer, kRequestCountField);
// We have to read messages *after* updating lastSeenRequestCount in order
// to avoid race conditions.
let entry;
while ((entry = receiveMessageOnPort(port)) !== undefined) {
onMessage(port, sharedBuffer, entry.message);
}
}
}
async function onMessage (
port : MessagePort,
sharedBuffer : Int32Array,
message : RequestMessage) {
currentTasks++;
const { taskId, task, filename, name } = message;
let response : ResponseMessage;
let transferList : any[] = [];
const start = message.histogramEnabled === 1 ? performance.now() : null;
try {
const handler = await getHandler(filename, name);
if (handler === null) {
throw new Error(`No handler function exported from ${filename}`);
}
let result = await handler(task);
if (isMovable(result)) {
transferList = transferList.concat(result[kTransferable]);
result = result[kValue];
}
response = {
taskId,
result,
error: null,
time: start == null ? null : Math.round(performance.now() - start)
};
if (useAtomics && !useAsyncAtomics) {
// If the task used e.g. console.log(), wait for the stream to drain
// before potentially entering the `Atomics.wait()` loop, and before
// returning the result so that messages will always be printed even
// if the process would otherwise be ready to exit.
if (process.stdout.writableLength > 0) {
await new Promise((resolve) => process.stdout.write('', resolve));
}
if (process.stderr.writableLength > 0) {
await new Promise((resolve) => process.stderr.write('', resolve));
}
}
} catch (error) {
response = {
taskId,
result: null,
// It may be worth taking a look at the error cloning algorithm we
// use in Node.js core here, it's quite a bit more flexible
error: <Error>error,
time: start == null ? null : Math.round(performance.now() - start)
};
}
currentTasks--;
try {
// Post the response to the parent thread, and let it know that we have
// an additional message available. If possible, use Atomics.wait()
// to wait for the next message.
port.postMessage(response, transferList);
Atomics.add(sharedBuffer, kResponseCountField, 1);
if (useAtomics) {
const res = atomicsWaitLoop(port, sharedBuffer);
if (res?.then != null) await res;
}
} catch (error) {
throwInNextTick(error as Error);
}
}
function throwInNextTick (error : Error) {
queueMicrotask(() => { throw error; });
}

39
node_modules/piscina/src/worker_pool/balancer/index.ts generated vendored Normal file
View File

@@ -0,0 +1,39 @@
import type { PiscinaTask } from '../../task_queue';
import type { PiscinaWorker } from '..';
export type PiscinaLoadBalancer = (
task: PiscinaTask,
workers: PiscinaWorker[]
) => PiscinaWorker | null; // If candidate is passed, it will be used as the result of the load balancer and ingore the command;
export type LeastBusyBalancerOptions = {
maximumUsage: number;
};
export function LeastBusyBalancer (
opts: LeastBusyBalancerOptions
): PiscinaLoadBalancer {
const { maximumUsage } = opts;
return (task, workers) => {
let candidate: PiscinaWorker | null = null;
let checkpoint = maximumUsage;
for (const worker of workers) {
if (worker.currentUsage === 0) {
candidate = worker;
break;
}
if (worker.isRunningAbortableTask) continue;
if (
!task.isAbortable &&
(worker.currentUsage < checkpoint)
) {
candidate = worker;
checkpoint = worker.currentUsage;
}
}
return candidate;
};
}

126
node_modules/piscina/src/worker_pool/base.ts generated vendored Normal file
View File

@@ -0,0 +1,126 @@
import assert from 'node:assert';
export abstract class AsynchronouslyCreatedResource {
onreadyListeners : (() => void)[] | null = [];
ondestroyListeners : (() => void)[] | null = [];
markAsReady () : void {
const listeners = this.onreadyListeners;
assert(listeners !== null);
this.onreadyListeners = null;
for (const listener of listeners) {
listener();
}
}
isReady () : boolean {
return this.onreadyListeners === null;
}
onReady (fn : () => void) {
if (this.onreadyListeners === null) {
fn(); // Zalgo is okay here.
return;
}
this.onreadyListeners.push(fn);
}
onDestroy (fn : () => void) {
if (this.ondestroyListeners === null) {
return;
}
this.ondestroyListeners.push(fn);
}
markAsDestroyed () {
const listeners = this.ondestroyListeners;
assert(listeners !== null);
this.ondestroyListeners = null;
for (const listener of listeners) {
listener();
}
}
isDestroyed () {
return this.ondestroyListeners === null;
}
abstract currentUsage() : number;
}
// TODO: this will eventually become an scheduler
export class AsynchronouslyCreatedResourcePool<
T extends AsynchronouslyCreatedResource> {
pendingItems = new Set<T>();
readyItems = new Set<T>();
maximumUsage : number;
onAvailableListeners : ((item : T) => void)[];
onTaskDoneListeners : ((item : T) => void)[];
constructor (maximumUsage : number) {
this.maximumUsage = maximumUsage;
this.onAvailableListeners = [];
this.onTaskDoneListeners = [];
}
add (item : T) {
this.pendingItems.add(item);
item.onReady(() => {
/* istanbul ignore else */
if (this.pendingItems.has(item)) {
this.pendingItems.delete(item);
this.readyItems.add(item);
this.maybeAvailable(item);
}
});
}
delete (item : T) {
this.pendingItems.delete(item);
this.readyItems.delete(item);
}
* [Symbol.iterator] () {
yield * this.pendingItems;
yield * this.readyItems;
}
get size () {
return this.pendingItems.size + this.readyItems.size;
}
maybeAvailable (item : T) {
/* istanbul ignore else */
if (item.currentUsage() < this.maximumUsage) {
for (const listener of this.onAvailableListeners) {
listener(item);
}
}
}
onAvailable (fn : (item : T) => void) {
this.onAvailableListeners.push(fn);
}
taskDone (item : T) {
for (let i = 0; i < this.onTaskDoneListeners.length; i++) {
this.onTaskDoneListeners[i](item);
}
}
onTaskDone (fn : (item : T) => void) {
this.onTaskDoneListeners.push(fn);
}
getCurrentUsage (): number {
let inFlight = 0;
for (const worker of this.readyItems) {
const currentUsage = worker.currentUsage();
if (Number.isFinite(currentUsage)) inFlight += currentUsage;
}
return inFlight;
}
}

200
node_modules/piscina/src/worker_pool/index.ts generated vendored Normal file
View File

@@ -0,0 +1,200 @@
import { Worker, MessagePort, receiveMessageOnPort } from 'node:worker_threads';
import { createHistogram, RecordableHistogram } from 'node:perf_hooks';
import assert from 'node:assert';
import { RequestMessage, ResponseMessage } from '../types';
import { Errors } from '../errors';
import { TaskInfo } from '../task_queue';
import { kFieldCount, kRequestCountField, kResponseCountField, kWorkerData } from '../symbols';
import { PiscinaHistogramHandler, PiscinaHistogramSummary } from '../histogram';
import { AsynchronouslyCreatedResource, AsynchronouslyCreatedResourcePool } from './base';
export * from './balancer';
type ResponseCallback = (response : ResponseMessage) => void;
export type PiscinaWorker = {
id: number;
currentUsage: number;
isRunningAbortableTask: boolean;
histogram: PiscinaHistogramSummary | null;
terminating: boolean;
destroyed: boolean;
[kWorkerData]: WorkerInfo;
}
export class WorkerInfo extends AsynchronouslyCreatedResource {
worker : Worker;
taskInfos : Map<number, TaskInfo>;
idleTimeout : NodeJS.Timeout | null = null;
port : MessagePort;
sharedBuffer : Int32Array;
lastSeenResponseCount : number = 0;
onMessage : ResponseCallback;
histogram: RecordableHistogram | null;
terminating = false;
destroyed = false;
constructor (
worker : Worker,
port : MessagePort,
onMessage : ResponseCallback,
enableHistogram: boolean
) {
super();
this.worker = worker;
this.port = port;
this.port.on('message',
(message : ResponseMessage) => this._handleResponse(message));
this.onMessage = onMessage;
this.taskInfos = new Map();
this.sharedBuffer = new Int32Array(
new SharedArrayBuffer(kFieldCount * Int32Array.BYTES_PER_ELEMENT));
this.histogram = enableHistogram ? createHistogram() : null;
}
get id (): number {
return this.worker.threadId;
}
destroy () : void {
if (this.terminating || this.destroyed) return;
this.terminating = true;
this.clearIdleTimeout();
this.worker.terminate();
this.port.close();
for (const taskInfo of this.taskInfos.values()) {
taskInfo.done(Errors.ThreadTermination());
}
this.taskInfos.clear();
this.terminating = false;
this.destroyed = true;
this.markAsDestroyed();
}
clearIdleTimeout () : void {
if (this.idleTimeout != null) {
clearTimeout(this.idleTimeout);
this.idleTimeout = null;
}
}
ref () : WorkerInfo {
this.port.ref();
return this;
}
unref () : WorkerInfo {
// Note: Do not call ref()/unref() on the Worker itself since that may cause
// a hard crash, see https://github.com/nodejs/node/pull/33394.
this.port.unref();
return this;
}
_handleResponse (message : ResponseMessage) : void {
if (message.time != null) {
this.histogram?.record(PiscinaHistogramHandler.toHistogramIntegerNano(message.time));
}
this.onMessage(message);
if (this.taskInfos.size === 0) {
// No more tasks running on this Worker means it should not keep the
// process running.
this.unref();
}
}
postTask (taskInfo : TaskInfo) {
assert(!this.taskInfos.has(taskInfo.taskId));
assert(!this.terminating && !this.destroyed);
const message : RequestMessage = {
task: taskInfo.releaseTask(),
taskId: taskInfo.taskId,
filename: taskInfo.filename,
name: taskInfo.name,
histogramEnabled: this.histogram != null ? 1 : 0
};
try {
this.port.postMessage(message, taskInfo.transferList);
} catch (err) {
// This would mostly happen if e.g. message contains unserializable data
// or transferList is invalid.
taskInfo.done(<Error>err);
return;
}
taskInfo.workerInfo = this;
this.taskInfos.set(taskInfo.taskId, taskInfo);
queueMicrotask(() => this.clearIdleTimeout())
this.ref();
// Inform the worker that there are new messages posted, and wake it up
// if it is waiting for one.
Atomics.add(this.sharedBuffer, kRequestCountField, 1);
Atomics.notify(this.sharedBuffer, kRequestCountField, 1);
}
processPendingMessages () {
if (this.destroyed) return;
// If we *know* that there are more messages than we have received using
// 'message' events yet, then try to load and handle them synchronously,
// without the need to wait for more expensive events on the event loop.
// This would usually break async tracking, but in our case, we already have
// the extra TaskInfo/AsyncResource layer that rectifies that situation.
const actualResponseCount =
Atomics.load(this.sharedBuffer, kResponseCountField);
if (actualResponseCount !== this.lastSeenResponseCount) {
this.lastSeenResponseCount = actualResponseCount;
let entry;
while ((entry = receiveMessageOnPort(this.port)) !== undefined) {
this._handleResponse(entry.message);
}
}
}
isRunningAbortableTask () : boolean {
// If there are abortable tasks, we are running one at most per Worker.
if (this.taskInfos.size !== 1) return false;
const [[, task]] = this.taskInfos;
return task.abortSignal !== null;
}
currentUsage () : number {
if (this.isRunningAbortableTask()) return Infinity;
return this.taskInfos.size;
}
get interface (): PiscinaWorker {
const worker = this;
return {
get id () {
return worker.worker.threadId;
},
get currentUsage () {
return worker.currentUsage();
},
get isRunningAbortableTask () {
return worker.isRunningAbortableTask();
},
get histogram () {
return worker.histogram != null ? PiscinaHistogramHandler.createHistogramSummary(worker.histogram) : null;
},
get terminating () {
return worker.terminating;
},
get destroyed () {
return worker.destroyed;
},
[kWorkerData]: worker
};
}
}
export { AsynchronouslyCreatedResourcePool };

284
node_modules/piscina/test/abort-task.test.ts generated vendored Normal file
View File

@@ -0,0 +1,284 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { EventEmitter } from 'node:events';
import Piscina from '..';
import { resolve } from 'path';
const TIMEOUT_MAX = 2 ** 31 - 1;
test('tasks can be aborted through AbortController while running', () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const buf = new Int32Array(new SharedArrayBuffer(4));
const abortController = new AbortController();
assert.rejects(pool.run(buf, { signal: abortController.signal }),
/The task has been aborted/);
Atomics.wait(buf, 0, 0);
assert.strictEqual(Atomics.load(buf, 0), 1);
abortController.abort();
});
test('tasks can be aborted through EventEmitter while running', () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const buf = new Int32Array(new SharedArrayBuffer(4));
const ee = new EventEmitter();
assert.rejects(pool.run(buf, { signal: ee }), /The task has been aborted/);
Atomics.wait(buf, 0, 0);
assert.strictEqual(Atomics.load(buf, 0), 1);
ee.emit('abort');
});
test('tasks can be aborted through EventEmitter before running', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.js'),
maxThreads: 1
});
const bufs = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
const ee = new EventEmitter();
const task1 = pool.run(bufs[0]);
const abortable = pool.run(bufs[1], { signal: ee });
assert.strictEqual(pool.queueSize, 0); // Means it's running
assert.rejects(abortable, /The task has been aborted/);
ee.emit('abort');
// Wake up the thread handling the first task.
Atomics.store(bufs[0], 0, 1);
Atomics.notify(bufs[0], 0, 1);
await task1;
});
test('abortable tasks will not share workers (abortable posted second)', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/wait-for-notify.ts'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
const bufs = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
const task1 = pool.run(bufs[0]);
const ee = new EventEmitter();
assert.rejects(pool.run(bufs[1], { signal: ee }), /The task has been aborted/);
assert.strictEqual(pool.queueSize, 0);
ee.emit('abort');
// Wake up the thread handling the first task.
Atomics.store(bufs[0], 0, 1);
Atomics.notify(bufs[0], 0, 1);
await task1;
});
// TODO: move to testing balancer
test('abortable tasks will not share workers (abortable posted first)', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
const ee = new EventEmitter();
assert.rejects(pool.run('while(true);', { signal: ee }), /The task has been aborted/);
const task2 = pool.run('42');
assert.strictEqual(pool.queueSize, 1);
ee.emit('abort');
// Wake up the thread handling the second task.
assert.strictEqual(await task2, 42);
});
// TODO: move to testing balancer
test('abortable tasks will not share workers (on worker available)', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/sleep.js'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
// Task 1 will sleep 100 ms then complete,
// Task 2 will sleep 300 ms then complete.
// Abortable task 3 should still be in the queue
// when Task 1 completes, but should not be selected
// until after Task 2 completes because it is abortable.
const ret = await Promise.all([
pool.run({ time: 100, a: 1 }),
pool.run({ time: 300, a: 2 }),
pool.run({ time: 100, a: 3 }, { signal: new EventEmitter() })
]);
assert.strictEqual(ret[0], 0);
assert.strictEqual(ret[1], 1);
assert.strictEqual(ret[2], 2);
});
test('abortable tasks will not share workers (destroy workers)', () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/sleep.js'),
maxThreads: 1,
concurrentTasksPerWorker: 2
});
// Task 1 will sleep 0 ms then complete,
// Task 2 will sleep indefinitely (TIMEOUT_MAX) then complete.
// Abortable task 3 should still be in the queue
// when Task 1 completes, but should not be selected
// until after Task 2 completes because it is abortable.
pool.run({ time: 0, a: 1 }).then(() => {
pool.destroy();
});
assert.rejects(pool.run({ time: TIMEOUT_MAX, a: 2 }), /Terminating worker thread/);
assert.rejects(pool.run({ time: TIMEOUT_MAX, a: 3 }, { signal: new EventEmitter() }),
/Terminating worker thread/);
});
test('aborted AbortSignal rejects task immediately', () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/move.ts')
});
const controller = new AbortController();
// Abort the controller early
controller.abort();
assert.strictEqual(controller.signal.aborted, true);
// The data won't be moved because the task will abort immediately.
const data = new Uint8Array(new SharedArrayBuffer(4));
assert.rejects(pool.run(data, { signal: controller.signal, transferList: [data.buffer] }),
/The task has been aborted/);
assert.strictEqual(data.length, 4);
});
test('task with AbortSignal cleans up properly', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
const ee = new EventEmitter();
await pool.run('1+1', { signal: ee });
const { getEventListeners } = EventEmitter as any;
if (typeof getEventListeners === 'function') {
assert.strictEqual(getEventListeners(ee, 'abort').length, 0);
}
const controller = new AbortController();
await pool.run('1+1', { signal: controller.signal });
});
test('aborted AbortSignal rejects task immediately (with reason)', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/move.ts')
});
const customReason = new Error('custom reason');
const controller = new AbortController();
controller.abort(customReason);
assert.strictEqual(controller.signal.aborted, true);
assert.strictEqual(controller.signal.reason, customReason);
// The data won't be moved because the task will abort immediately.
const data = new Uint8Array(new SharedArrayBuffer(4));
try {
await pool.run(data, { transferList: [data.buffer], signal: controller.signal });
} catch (error) {
assert.strictEqual(error.message, 'The task has been aborted');
assert.strictEqual(error.cause, customReason);
}
assert.strictEqual(data.length, 4);
});
test('tasks can be aborted through AbortController while running', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const reason = new Error('custom reason');
const buf = new Int32Array(new SharedArrayBuffer(4));
const abortController = new AbortController();
try {
const promise = pool.run(buf, { signal: abortController.signal });
Atomics.wait(buf, 0, 0);
assert.strictEqual(Atomics.load(buf, 0), 1);
abortController.abort(reason);
await promise;
} catch (error) {
assert.strictEqual(error.message, 'The task has been aborted');
assert.strictEqual(error.cause, reason);
}
});
test('aborted AbortSignal rejects task immediately (with reason)', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/move.ts')
});
const customReason = new Error('custom reason');
const controller = new AbortController();
controller.abort(customReason);
assert.strictEqual(controller.signal.aborted, true);
assert.strictEqual(controller.signal.reason, customReason);
// The data won't be moved because the task will abort immediately.
const data = new Uint8Array(new SharedArrayBuffer(4));
try {
await pool.run(data, { transferList: [data.buffer], signal: controller.signal });
} catch (error) {
assert.strictEqual(error.message, 'The task has been aborted');
assert.strictEqual(error.cause, customReason);
}
assert.strictEqual(data.length, 4);
});
test('tasks can be aborted through AbortController while running', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep.ts')
});
const reason = new Error('custom reason');
const buf = new Int32Array(new SharedArrayBuffer(4));
const abortController = new AbortController();
try {
const promise = pool.run(buf, { signal: abortController.signal });
Atomics.wait(buf, 0, 0);
assert.strictEqual(Atomics.load(buf, 0), 1);
abortController.abort(reason);
await promise;
} catch (error) {
assert.strictEqual(error.message, 'The task has been aborted');
assert.strictEqual(error.cause, reason);
}
});

45
node_modules/piscina/test/async-context.test.ts generated vendored Normal file
View File

@@ -0,0 +1,45 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { resolve } from 'node:path';
import { createHook, executionAsyncId } from 'node:async_hooks';
import Piscina from '..';
test('postTask() calls the correct async hooks', async () => {
let taskId;
let initCalls = 0;
let beforeCalls = 0;
let afterCalls = 0;
let resolveCalls = 0;
const hook = createHook({
init (id, type) {
if (type === 'Piscina.Task') {
initCalls++;
taskId = id;
}
},
before (id) {
if (id === taskId) beforeCalls++;
},
after (id) {
if (id === taskId) afterCalls++;
},
promiseResolve () {
if (executionAsyncId() === taskId) resolveCalls++;
}
});
hook.enable();
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval.js')
});
await pool.run('42');
hook.disable();
assert.strictEqual(initCalls, 1);
assert.strictEqual(beforeCalls, 1);
assert.strictEqual(afterCalls, 1);
assert.strictEqual(resolveCalls, 1);
});

119
node_modules/piscina/test/atomics-optimization.test.ts generated vendored Normal file
View File

@@ -0,0 +1,119 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { resolve } from 'node:path';
import Piscina from '..';
test('coverage test for Atomics optimization (sync mode)', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/notify-then-sleep-or.js'),
minThreads: 2,
maxThreads: 2,
concurrentTasksPerWorker: 2,
atomics: 'sync'
});
const tasks = [];
let v: number;
// Post 4 tasks, and wait for all of them to be ready.
const i32array = new Int32Array(new SharedArrayBuffer(4));
for (let index = 0; index < 4; index++) {
tasks.push(pool.run({ i32array, index }));
}
// Wait for 2 tasks to enter 'wait' state.
do {
v = Atomics.load(i32array, 0);
if (popcount8(v) >= 2) break;
Atomics.wait(i32array, 0, v);
} while (true);
// The check above could also be !== 2 but it's hard to get things right
// sometimes and this gives us a nice assertion. Basically, at this point
// exactly 2 tasks should be in Atomics.wait() state.
assert.strictEqual(popcount8(v), 2);
// Wake both tasks up as simultaneously as possible. The other 2 tasks should
// then start executing.
Atomics.store(i32array, 0, 0);
Atomics.notify(i32array, 0, Infinity);
// Wait for the other 2 tasks to enter 'wait' state.
do {
v = Atomics.load(i32array, 0);
if (popcount8(v) >= 2) break;
Atomics.wait(i32array, 0, v);
} while (true);
// At this point, the first two tasks are definitely finished and have
// definitely posted results back to the main thread, and the main thread
// has definitely not received them yet, meaning that the Atomics check will
// be used. Making sure that that works is the point of this test.
// Wake up the remaining 2 tasks in order to make sure that the test finishes.
// Do the same consistency check beforehand as above.
assert.strictEqual(popcount8(v), 2);
Atomics.store(i32array, 0, 0);
Atomics.notify(i32array, 0, Infinity);
await Promise.all(tasks);
});
// Inefficient but straightforward 8-bit popcount
function popcount8 (v : number) : number {
v &= 0xff;
if (v & 0b11110000) return popcount8(v >>> 4) + popcount8(v & 0xb00001111);
if (v & 0b00001100) return popcount8(v >>> 2) + popcount8(v & 0xb00000011);
if (v & 0b00000010) return popcount8(v >>> 1) + popcount8(v & 0xb00000001);
return v;
}
test('avoids unbounded recursion', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread.ts'),
minThreads: 2,
maxThreads: 2,
atomics: 'sync'
});
const tasks = [];
for (let i = 1; i <= 10000; i++) {
tasks.push(pool.run(null));
}
await Promise.all(tasks);
});
test('enable async mode', async () => {
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval-params.js'),
minThreads: 1,
maxThreads: 1,
atomics: 'async'
});
const bufs = [
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4)),
new Int32Array(new SharedArrayBuffer(4))
];
const script = `
setTimeout(() => { Atomics.add(input.shared[0], 0, 1); Atomics.notify(input.shared[0], 0, Infinity); }, 100);
setTimeout(() => { Atomics.add(input.shared[1], 0, 1); Atomics.notify(input.shared[1], 0, Infinity); }, 300);
setTimeout(() => { Atomics.add(input.shared[2], 0, 1); Atomics.notify(input.shared[2], 0, Infinity); }, 500);
true
`;
const promise = pool.run({
code: script,
shared: bufs
});
const atResult1 = Atomics.wait(bufs[0], 0, 0);
const atResult2 = Atomics.wait(bufs[1], 0, 0);
const atResult3 = Atomics.wait(bufs[2], 0, 0);
assert.deepStrictEqual([atResult1, atResult2, atResult3], ['ok', 'ok', 'ok']);
assert.strictEqual(await promise, true);
});

25
node_modules/piscina/test/console-log.test.ts generated vendored Normal file
View File

@@ -0,0 +1,25 @@
import assert from 'node:assert/strict';
import { resolve } from 'node:path';
import { spawn } from 'node:child_process';
import { test } from 'node:test';
import concat from 'concat-stream';
test('console.log() calls are not blocked by Atomics.wait() (sync mode)', async () => {
const proc = spawn(process.execPath, [
...process.execArgv, resolve(__dirname, 'fixtures/console-log.ts')
], {
stdio: ['inherit', 'pipe', 'pipe'],
env: {
PISCINA_ENABLE_ASYNC_ATOMICS: '0'
}
});
const dataStdout = await new Promise((resolve) => {
proc.stdout.setEncoding('utf8').pipe(concat(resolve));
});
const dataStderr = await new Promise((resolve) => {
proc.stderr.setEncoding('utf8').pipe(concat(resolve));
});
assert.strictEqual(dataStdout, 'A\n');
assert.strictEqual(dataStderr, 'B\n');
});

211
node_modules/piscina/test/fixed-queue.test.ts generated vendored Normal file
View File

@@ -0,0 +1,211 @@
import assert from 'node:assert';
import { test } from 'node:test';
import { resolve } from 'node:path';
import { kQueueOptions } from '../dist/symbols';
import { Piscina, FixedQueue, PiscinaTask as Task } from '..';
// @ts-expect-error - it misses several properties, but it's enough for the test
class QueueTask implements Task {
get [kQueueOptions] () {
return null;
}
}
test('queue length', () => {
const queue = new FixedQueue();
assert.strictEqual(queue.size, 0);
queue.push(new QueueTask());
assert.strictEqual(queue.size, 1);
queue.shift();
assert.strictEqual(queue.size, 0);
});
test('queue length should not become negative', () => {
const queue = new FixedQueue();
assert.strictEqual(queue.size, 0);
queue.shift();
assert.strictEqual(queue.size, 0);
});
test('queue remove', () => {
const queue = new FixedQueue();
const task = new QueueTask();
assert.strictEqual(queue.size, 0, 'should be empty on start');
queue.push(task);
assert.strictEqual(queue.size, 1, 'should contain single task after push');
queue.remove(task);
assert.strictEqual(queue.size, 0, 'should be empty after task removal');
});
test('remove not queued task should not lead to errors', () => {
const queue = new FixedQueue();
const task = new QueueTask();
assert.strictEqual(queue.size, 0, 'should be empty on start');
queue.remove(task);
assert.strictEqual(queue.size, 0, 'should be empty after task removal');
});
test('removing elements from intermediate CircularBuffer should not lead to issues', () => {
/*
The test intends to check following scenario:
1) We fill the queue with 3 full circular buffers amount of items.
2) Empty the middle circular buffer with remove().
3) This should lead to the removal of the middle buffer from the queue:
- Before emptying: tail buffer -> middle buffer -> head buffer.
- After emptying: tail buffer -> head buffer.
*/
const queue = new FixedQueue();
// size of single circular buffer
const batchSize = 2047;
const firstBatch = Array.from({ length: batchSize }, () => new QueueTask());
const secondBatch = Array.from({ length: batchSize }, () => new QueueTask());
const thirdBatch = Array.from({ length: batchSize }, () => new QueueTask());
const tasks = firstBatch.concat(secondBatch, thirdBatch);
for (const task of tasks) {
queue.push(task);
}
assert.strictEqual(queue.size, tasks.length, `should contain ${batchSize} * 3 items`);
let size = queue.size;
for (const task of secondBatch) {
queue.remove(task);
assert.strictEqual(queue.size, --size, `should contain ${size} items`);
}
const expected = firstBatch.concat(thirdBatch);
const actual = [];
while (!queue.isEmpty()) {
const task = queue.shift();
actual.push(task);
}
assert.deepEqual(actual, expected);
});
test('removing elements from first CircularBuffer should not lead to issues', () => {
/*
The test intends to check following scenario:
1) We fill the queue with 3 full circular buffers amount of items.
2) Empty the first circular buffer with remove().
3) This should lead to the removal of the tail buffer from the queue:
- Before emptying: tail buffer -> middle buffer -> head buffer.
- After emptying: tail buffer (previously middle) -> head buffer.
*/
const queue = new FixedQueue();
// size of single circular buffer
const batchSize = 2047;
const firstBatch = Array.from({ length: batchSize }, () => new QueueTask());
const secondBatch = Array.from({ length: batchSize }, () => new QueueTask());
const thirdBatch = Array.from({ length: batchSize }, () => new QueueTask());
const tasks = firstBatch.concat(secondBatch, thirdBatch);
for (const task of tasks) {
queue.push(task);
}
assert.strictEqual(queue.size, tasks.length, `should contain ${batchSize} * 3 items`);
let size = queue.size;
for (const task of firstBatch) {
queue.remove(task);
assert.strictEqual(queue.size, --size, `should contain ${size} items`);
}
const expected = secondBatch.concat(thirdBatch);
const actual = [];
while (!queue.isEmpty()) {
const task = queue.shift();
actual.push(task);
}
assert.deepEqual(actual, expected);
});
test('removing elements from last CircularBuffer should not lead to issues', async () => {
/*
The test intends to check following scenario:
1) We fill the queue with 3 full circular buffers amount of items.
2) Empty the last circular buffer with remove().
3) This should lead to the removal of the head buffer from the queue:
- Before emptying: tail buffer -> middle buffer -> head buffer.
- After emptying: tail buffer -> head buffer (previously middle).
*/
const queue = new FixedQueue();
// size of single circular buffer
const batchSize = 2047;
const firstBatch = Array.from({ length: batchSize }, () => new QueueTask());
const secondBatch = Array.from({ length: batchSize }, () => new QueueTask());
const thirdBatch = Array.from({ length: batchSize }, () => new QueueTask());
const tasks = firstBatch.concat(secondBatch, thirdBatch);
for (const task of tasks) {
queue.push(task);
}
assert.strictEqual(queue.size, tasks.length, `should contain ${batchSize} * 3 items`);
let size = queue.size;
for (const task of thirdBatch) {
queue.remove(task);
assert.strictEqual(queue.size, --size, `should contain ${size} items`);
}
const expected = firstBatch.concat(secondBatch);
const actual = [];
while (!queue.isEmpty()) {
const task = queue.shift();
actual.push(task);
}
assert.deepEqual(actual, expected);
});
test('simple integraion with Piscina', async () => {
const queue = new FixedQueue();
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/simple-isworkerthread-named-import.ts'),
taskQueue: queue
});
const result = await pool.run(null);
assert.strictEqual(result, 'done');
});
test('concurrent calls with Piscina', async () => {
const queue = new FixedQueue();
const pool = new Piscina({
filename: resolve(__dirname, 'fixtures/eval-async.js'),
taskQueue: queue
});
const tasks = ['1+1', '2+2', '3+3'];
const results = await Promise.all(tasks.map((task) => pool.run(task)));
// eslint-disable-next-line
const expected = tasks.map(eval);
assert.deepEqual(results, expected);
});

12
node_modules/piscina/test/fixtures/console-log.ts generated vendored Normal file
View File

@@ -0,0 +1,12 @@
import { resolve } from 'node:path';
import Piscina from '../..';
const pool = new Piscina({
filename: resolve(__dirname, 'eval.js'),
maxThreads: 1,
env: {
PISCINA_ENABLE_ASYNC_ATOMICS: process.env.PISCINA_ENABLE_ASYNC_ATOMICS
}
});
pool.run('console.log("A"); console.error("B");');

13
node_modules/piscina/test/fixtures/esm-async.mjs generated vendored Normal file
View File

@@ -0,0 +1,13 @@
import { promisify } from 'node:util';
const sleep = promisify(setTimeout);
// eslint-disable-next-line no-eval
function handler (code) { return eval(code); }
async function load () {
await sleep(5);
return handler;
}
export default load();

2
node_modules/piscina/test/fixtures/esm-export.mjs generated vendored Normal file
View File

@@ -0,0 +1,2 @@
// eslint-disable-next-line no-eval
export default function (code) { return eval(code); };

15
node_modules/piscina/test/fixtures/eval-async.js generated vendored Normal file
View File

@@ -0,0 +1,15 @@
'use strict';
const { promisify } = require('node:util');
const sleep = promisify(setTimeout);
// eslint-disable-next-line no-eval
function handler (code) { return eval(code); }
async function load () {
await sleep(5);
return handler;
}
module.exports = load();

2
node_modules/piscina/test/fixtures/eval-params.js generated vendored Normal file
View File

@@ -0,0 +1,2 @@
// eslint-disable-next-line no-eval
module.exports = function (input) { return eval(input.code); };

2
node_modules/piscina/test/fixtures/eval.js generated vendored Normal file
View File

@@ -0,0 +1,2 @@
// eslint-disable-next-line no-eval
module.exports = function (code) { return eval(code); };

10
node_modules/piscina/test/fixtures/move.ts generated vendored Normal file
View File

@@ -0,0 +1,10 @@
import assert from 'node:assert';
import { types } from 'node:util';
import Piscina from '../..';
export default function (moved) {
if (moved !== undefined) {
assert(types.isAnyArrayBuffer(moved));
}
return Piscina.move(new ArrayBuffer(10));
}

Some files were not shown because too many files have changed in this diff Show More