Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: migrate to picoquery #3606

Closed
wants to merge 1 commit into from
Closed

feat: migrate to picoquery #3606

wants to merge 1 commit into from

Conversation

43081j
Copy link

@43081j 43081j commented Jul 28, 2024

Migrates to using picoquery for query string parsing, a much faster and lighter library.

Note that we have to decode the stringified query in some cases since we were previously double encoding.

PQ lets you configure the nesting syntax style, but it seems we don't actually care about that much since we basically dump a JSON object in a single parameter rather than encoding it as query params.

Motivation and Context

As part of the ongoing ecosystem cleanup, we are migrating various projects to use lighter/faster packages.

In this case, moving from qs to picoquery - primarily because it is much faster.

How Has This Been Tested?

Existing tests should cover it. Though if there are any gaps or places you think we should increase test coverage, please do shout up.

Types of changes

  • No code changes (changes to documentation, CI, metadata, etc)
  • Dependency changes (any modification to dependencies in package.json)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Migrates to using picoquery for query string parsing, a much faster and
lighter library.

Note that we have to decode the stringified query in some cases since we
were previously double encoding.
@43081j
Copy link
Author

43081j commented Aug 9, 2024

@char0n any chance you or another maintainer can take a look at this?

the performance gain is around 6-7x the speed of qs

@char0n
Copy link
Member

char0n commented Aug 11, 2024

Hi,

It's being considered. There is also a proposal to just use native URLSearchParams here - #3585.

@char0n
Copy link
Member

char0n commented Aug 11, 2024

I've issued #3633 where we eliminate code that don't really need qs library at all.

@43081j
Copy link
Author

43081j commented Aug 11, 2024

that works for me!

i would push for using URLSearchParams too but assumed you needed nesting support

note your other PR still has the dependency. we should be able to drop it still

@char0n
Copy link
Member

char0n commented Aug 11, 2024

i would push for using URLSearchParams too but assumed you needed nesting support

I'm still researching how exactly we use qs and what we really need from it. Ideal situation is to just use URLSearchParams.

note your other PR still has the dependency. we should be able to drop it still

There are two location where qs is used. My PR addressed one of those locations, where qs is not needed at all, and pure URLSearchParams just do the job.


Related to picoquery - just by looking at the library, it is very young which means, not really battle tested and have no downloads. It's a bit risky integrating it, given it has no history. Please don't take this the wrong way, I'm just stating objective facts. But...it's doesn't mean picoquery is not being still considered (the tests do pass when integrating it). I'm trying to come up with the simplest and most maintainable solution for this project.

@43081j
Copy link
Author

43081j commented Aug 11, 2024

Related to picoquery - just by looking at the library, it is very young which means, not really battle tested and have no downloads

temporarily true while we land the various PRs across the ecosystem to introduce it (starting with storybook for example)

you are right though, it is fairly new (though it is a fork on fast-querystring which certainly is battle tested)

we have to start somewhere. this is the start of a larger migration (part of the ecosystem-cleanup)

you have these options really:

  • do nothing (not great, qs is slow and bloated)
  • switch to picoquery (happening elsewhere across the ecosystem but is a fairly new library. very actively developed so even if you do find problems, they will be fixed)
  • switch to URLSearchParams and stop relying on nested query strings (ideal, but depends what you're using it for currently)
  • switch to fast-querystring (same as picoquery but without nesting support, only supports arrays like foo=1&foo=2)

also be aware i'm not just trying to push some random new library here. the cleanup project is a huge job, and a query string library was one of the major gaps we've now filled by publishing a new library. we will be recommending this library and contributing it to many other projects as time goes on.

that said, it is new regardless. so if you're not up for that, we should wait or switch to URLSearchParams and remove all nesting

@char0n
Copy link
Member

char0n commented Aug 11, 2024

@43081j, thanks for explaining the full context in #3606 (comment) and I fully support your effort.

I've managed to construct a simple algorithm that just replicates what we originally needed from qs in #3634 just using URLSearchParams. It's just 15 lines of code and entire test suite is reporting green. Please have a look if you can.

@char0n char0n self-assigned this Aug 11, 2024
@char0n
Copy link
Member

char0n commented Aug 12, 2024

Superseded by #3634. Thank you again for opening this topic and all the ecosystem-cleanup effort

@char0n char0n closed this Aug 12, 2024
char0n added a commit that referenced this pull request Aug 12, 2024
@43081j 43081j deleted the pq branch August 12, 2024 08:24
char0n added a commit that referenced this pull request Aug 12, 2024
swagger-bot pushed a commit that referenced this pull request Aug 12, 2024
## [3.29.1](v3.29.0...v3.29.1) (2024-08-12)

### Bug Fixes

* **deps:** replace qs with native URLSearchParams ([b2f4c38](b2f4c38)), closes [#3606](#3606) [#3585](#3585)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants