Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Browser environments #4

Closed
MarcoPolo opened this issue Apr 29, 2024 · 5 comments · Fixed by #5
Closed

Support Browser environments #4

MarcoPolo opened this issue Apr 29, 2024 · 5 comments · Fixed by #5

Comments

@MarcoPolo
Copy link
Contributor

MarcoPolo commented Apr 29, 2024

Hey Paolo, thanks for making this. It's just what I needed for the js-libp2p implementation of libp2p+HTTP (tl;dr - partial summary - run HTTP request/response on top of libp2p streams).

I got milo working as the HTTP parser in the browser, with a couple of changes to use TypedArrays instead of the Node-only Buffer. It's not too much work.

Would you be open to merging a change to support browser (and non-node) environments?

I see a couple of options:

  1. Add a small polyfill for the methods we use. Buffer.from and check for typeof Buffer !== 'undefined'.
  2. Just use TypedArrays everywhere instead of Buffer.
@ShogunPanda
Copy link
Owner

Hi!
Having Milo running on browsers it was already on my roadmap, but I was currently focusing on bringing Milo into Node internals without sacrificing perfomance.
Anyway I think your PR looked fine, thanks a lot for this!

@CxRes
Copy link

CxRes commented Jul 6, 2024

@MarcoPolo @ShogunPanda Can you provide some instructions for using this, say when the incoming stream is Uint8Array. Thanks in advance!

@MarcoPolo
Copy link
Contributor Author

Here's how I use it to read an HTTP message from a Duplex<Uint8Array> which, for reading, is basically a generator that returns Uint8Arrays. https://github.com/libp2p/js-libp2p-http-fetch/blob/main/src/fetch/index.ts#L97

@ShogunPanda
Copy link
Owner

ShogunPanda commented Jul 7, 2024

@CxRes The example @MarcoPolo already contains that.

If you want a even more isolated example, you think about something this for Node streams:

const milo = require('@perseveranza-pets/milo')

// Get the stream somehow
const stream = getStream()

// Declare a chunk pointer. We need in callbacks
let lastChunk

// Create a parser
const parser = milo.create()

// Declare our callbacks
milo.setOnData(parser, (p, from, size) => {
  console.log(`Pos=${milo.getPosition(p)} Body: ${lastChunk.slice(from, from + size).toString()}`)
})

// Here we're passing the chunk directly, which will copy the data. 
// In the real world it's better to use WASM backed buffers allocated with milo.alloc if possible.
// Unless specify differently, a stream is always a sequence of Uint8.
stream.on('data', chunk => {
  lastChunk = chunk
  const consumed = milo.parse(parser, chunk, chunk.length)

  // You should check here if all bytes were consumed.
})

I think you can easily adapt this example to WebStream.
Is that enough?

@CxRes
Copy link

CxRes commented Jul 7, 2024

@MarcoPolo @ShogunPanda Thank a bunch for your responses!

I think you can easily adapt this example to WebStream.
Is that enough?

Not sure, though adapting it to a webstream is not a problem, I will have to give it a try when I get a chance. What I am trying to do is read a multipart (or possibly application/http in a later iteration) stream and would want to it break up in sequence of Fetch Response objects for each part body (which are little http like messages). So I want to be able to parse headers (don't need trailers) and pass on the little bodies as streams to the consumer. The important thing is I do not want chunk bodies forcing the consumer to wait for an entire part.

I am doing this for a new protocol I am developing on HTTP for notifications called Per Resource Events. I am coming out with libraries for it and would be better able to show you what I mean once they online.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants