Unoptimized implementation of pMapIterable preserveOrder option
#79
Conversation
…fer and `promiseEmitter` - move `promise` inner `try-catch` logic to `promise.catch` (now also catches when `iterator.next()` throws) - when `returnValue === pMapSkip`, `popRandomPromise` unconditionally (skip `popNextPromise` in main `while` loop instead) - add tests for `preserveOrder: false`, sync throwing mappers/iterables, more intricate `pMapSkip` scenarios
index.js
Outdated
|
|
||
| while (promises.length > 0) { | ||
| const {error, done, value} = await promises[0]; // eslint-disable-line no-await-in-loop | ||
| const {promise, result: {error, done, value}} = await nextPromise(); // eslint-disable-line no-await-in-loop |
There was a problem hiding this comment.
👍 I had not done so to avoid checking preserveOrder on every yield loop iteration, but now that you mention it, I imagine the overhead of a function call is much more than that of a ternary.
index.js
Outdated
| .then(result => ({result, promise})); | ||
|
|
||
| promises.push(promise); | ||
| promise.then(p => { |
There was a problem hiding this comment.
Use try/catch/async/await here. Also, would be better to use a deferred promise and we call it after anything resolves, instead of racing? https://github.com/sindresorhus/p-defer
There was a problem hiding this comment.
Deferring the promise is a neat technique! I had not encountered that before. It can definitely replace my use of EventTarget. However, I notice this library currently has 0 runtime dependencies - would it be acceptable to introduce p-defer as a runtime dependency or should I just copy the code over? I've done the latter for now.
However, I don't think it can replace Promise.race altogether, with the reason being that multiple promises can resolve between yield loop runs, before we have a chance to reset the somePromiseHasSettled deferred promise (since a promise can only hold one resolved value, the value of the second resolved promise would be lost). So, promises is needed as a buffer for resolved promises we have not gotten a chance to yield yet, and Promise.race is needed to check if any of those have resolved yet.
I've also used a deferred promise in conjunction with a new helper function to eliminate use of .then/.catch.
index.js
Outdated
| // This occurs when `concurrency > 1`: the first `promise` will `trySpawn` and `promises.push` another promise before `await`ing the mapper, | ||
| // but the ongoing `Promise.race(promises)` call from `nextPromise` is oblivious to this new promise as it was not present in `promises` | ||
| // when the race began. | ||
| new Promise(resolve => { |
There was a problem hiding this comment.
Less "thinking" and more "debugging after things didn't work as expected" 😅
|
So, will we get this published? |
…cificPromise, done condition continue vs return explanation, comment on non-use of spread
|
@artsiommiksiuk thanks for the ping! Last time I worked on this, I had been waiting to see if #78 or #76 would drastically change the implementation here before I continued with my own changes. Since no such changed have materialized, I'll pick this back up. |
…eserveOrder: false)
3 participants
Fixes #72.
Spun off from #74 in light of
pMapIterablepreserveOrderoption #74 (comment)pMapIterablepreserveOrderoption #74 (comment)Summary
Adds a
preserveOrderoption topMapIterable, which indicatesImplementation considerations
promisemust return reference to itself sopopRandomPromisecan findindexOf.preserveOrder: false, racing thepromisespool is not enough, sincetrySpawnmay be add additional promises to the buffer after the race has begun: so we race an event listener alongside (promisespool is still needed to buffer multiple promises finishing simultaneously.returnValue === pMapSkip && preserveOrder: false, unlike whenpreserveOrder: falsewe have no way to know if returning will cause ourpromiseto win an ongoing race, and so have no way of knowing whether to skip popping ourselves (to allow the main loop to pop us instead). So, whenreturnValue === pMapSkip, we insteadpopRandomPromiseourselves unconditionally and instead skippopNextPromisein mainwhileloop.promiseinnertry-catchlogic topromise.catch(which now also catches wheniterator.next()throws)preserveOrder: false && result.done, some promises in the pool may still be pending, socontinue(as a matter of fact, we could probablycontinuein all cases since the current implementation does not run concurrentiterator.nextcalls, so ifpreserveOrder: true && result.donethere should be no remainingpromisesin the queue - but hopefully that will change soon as we think about concurrentiterator.nextcalls!