Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KRNL-221 add callChunked array function #41

Merged
merged 9 commits into from
Mar 23, 2023
Merged

Conversation

Dosexe
Copy link
Contributor

@Dosexe Dosexe commented Mar 22, 2023

Changes

Add callChunked function

Checklist

  • Apply one of following labels; major, minor, patch or skip-release
  • I've updated the documentation, or no changes were necessary
  • I've updated the tests, or no changes were necessary

docs/array-utils.md Outdated Show resolved Hide resolved
src/utils/arrayUtils.ts Outdated Show resolved Hide resolved
src/utils/arrayUtils.ts Outdated Show resolved Hide resolved
@Dosexe Dosexe requested a review from kibertoad March 22, 2023 09:28
processFn: (arrayChunk: T[]) => Promise<void>,
): Promise<void> {
for (let i = 0; i < array.length; i += chunkSize) {
const arrayChunk = array.slice(i, i + chunkSize)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔶 Very surprised that you don't use the chunk function directly above. If you did you could have:

const chunks = chunk(array, chunkSize)
for (const arrayChunk of chunks) {
  await processFn(arrayChunk)
} 

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can also be done as a promise reducer:

chunk(array, chunkSize)
  .reduce(
    (promiseChain, chunk) => promiseChain.then(() => callback(chunk)),
    Promise.resolve()
  )

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for of is slower than old-school looping, so for a low-level library I would recommend keeping the original approach.
see https://github.com/kibertoad/nodejs-benchmark-tournament/blob/master/loops/_results/results.md

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I'm not sure how I feel about using reduce here, sounds like an overcomplication. Original version is pretty simple.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, what should I do at the end?:)

Copy link
Collaborator

@kibertoad kibertoad Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I see no reason to change this code, for me it is as simple as it needs to be, I see no advantages, performance or complexity-wise in suggested alternatives.
@rupert-mckay This is a yellow, non-blocking-but-please-explain-your-position comment, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. Happy for the PR to merge as-is. Still a bit surprised that the chunk helper isn't used, but it's not important.

@@ -14,3 +14,14 @@ export function chunk<T>(array: T[], chunkSize: number): T[][] {
// eslint-disable-next-line @typescript-eslint/no-unsafe-return
Copy link
Contributor

@rupert-mckay rupert-mckay Mar 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Feel free to ignore this comment as beyond the scope of this PR... but I really don't like what's going on in the chunk function. Ignoring no-unsafe-return is dangerous, and the implementation is especially fiddly and prone to error.

Chunking can be expressed rather naturally with a generator function which yields one chunk at a time. We can gather them into an array with Array.from. Here's an example alternative implementation:

function* chunkGenerator<Item>(
  array: readonly Item[],
  chunkSize = 1,
) {
  let index = 0;

  while (index < array.length) {
    yield array.slice(index, index + chunkSize);
    index += chunkSize;
  }
}

const chunk = <Item>(
  array: readonly Item[],
  chunkSize = 1,
) => Array.from(chunkGenerator(array, chunkSize));

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to ignore this comment

This seems to contradict the RED flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. My instinct had been "this is something I feel very strongly about"... but it is naturally contradicted by "... but it might not at all be relevant to this PR".

I'll edit to 🟢

@Dosexe Dosexe merged commit de79eed into main Mar 23, 2023
@Dosexe Dosexe deleted the feature/KRNL-221_call_chunked branch March 23, 2023 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants