Suggestion: Allow individual limits per m3u stream.

[shorty789]

Can I provide a suggestion? It would be absolutely amazing if you could set a max simultaneous stream limit per playlist.

An example would be the ability to give one of the streams a maximum of 3 users, while another one a maximum of one user, it would then be great to be able to merge those playlists into a single one that can support 4 simultaneous streams, with the 5th attempt to connect either redirecting to some 'catch all' stream, or just refusing to connect entirely. Not sure how easy it would be to implement though to be honest.

[sonroyaalmerol]

This is still a WIP but do you mind helping me test the dev build with this image: sonroyaalmerol/m3u-stream-merger-proxy:pr-7?

You should be able to set the maximum stream limits with the M3U_MAX_CONCURRENCY_X (M3U_MAX_CONCURRENCY_1, M3U_MAX_CONCURRENCY_2, etc. corresponding to M3U_URL_X) env variables. There are some breaking changes as well behind the scenes so expect that the databases made by the previous versions won't be compatible anymore.

[shorty789]

Awesome, I will aim to have a look at this before the weekend is over, do you have a discord? might make comms a little easier.

[shorty789]

I have tested it out, setting each stream to have a max concurrency of 1 for the moment, only using streams that support a max of 1 user per m3u.

if I load channel A on device A, it works perfectly fine, I then load channel B on device B, but when I do, it behaves almost like it is still trying to play from the same playlist. To help with testing, could you add a log output for what stream number is being selected? I can then confirm for sure if it is only loading from a m3u.

[sonroyaalmerol]

Can you try sonroyaalmerol/m3u-stream-merger-proxy:pr-12 for me? I also added more log outputs as requested. I'll wait for your response before I change anything else.

[shorty789]

OK, so I tried this, took a little while longer as it had a log entry for every channel being added, but the final log entries appeared to be a crash. i think there may be a few too many logs being generated now though as they rapidly got lost in a wall of text.

Would there be any chance of removing the log entries when adding a url to the database for the moment?

[sonroyaalmerol]

Try it again now. Added a DEBUG env var to toggle the said log entries. Should be turned off by default. Do show me the logs if you still get the crash afterwards.

You should be getting something like Background process: Updated M3U #X from http://****.** when it's done parsing a url. You should also get a Background process: Updated M3U database. after all the urls have been parsed.

[shorty789]

panic: runtime error: index out of range [1] with length 1

goroutine 12 [running]:
m3u-stream-merger/m3u.ParseM3UFromURL(0xc00010f1e0, {0xc00002806a, 0x20}, 0x2, 0x1)
/app/m3u/parser.go:49 +0xe7c
main.updateSource(0xc00010f1e0, {0xc00002806a, 0x20}, 0x2, 0x1)
/app/main.go:67 +0x136
main.updateSources.func1(0x0?, {0xc00002806a?, 0x0?}, 0x0?, 0x0?)
/app/main.go:112 +0x4e
created by main.updateSources in goroutine 18
/app/main.go:110 +0x4a8

This is the error that I received.

[sonroyaalmerol]

Seems like a bug within the parser. Can you try it again? I did some workarounds that might work.

If it still won't work, I'll need to see the format of the problematic line in the m3u file. At that point, it will be better for you to set DEBUG to true to see which url it's having trouble with.

[shorty789]

OK, so it would appear that the issue is with an unexpected EOF.
I will let you know how it goes once I have finished testing further.

[sonroyaalmerol]

I added auto-retry when unexpected EOF is detected with MAX_RETRIES (default: 10) if it helps in case you're hitting some sort of rate limiting.

[sonroyaalmerol]

I've had multiple small releases since :pr-12 was merged. Please use the :latest tag if you want to test it again.

[shorty789]

I have just tried the most recent version and the m3u is now coming back as blank, pr-12 did work but the counters were a bit off, my aim was to update and test further.

[shorty789]

I reverted back to pr-12 and i have a working merged m3u again.

[sonroyaalmerol]

My bad! I messed up the previous release. The :latest tag should be fine now. Do test further with the latest version.

[shorty789]

Thank you, I have just tested, the m3u works now, but I get the following error when I try to play anything

2024/03/06 22:19:01 Received request from :50128 for URL: /stream/VUsgLSBCQkMgMSBVSEQ=.mp4
2024/03/06 22:19:01 http: panic serving :50128: runtime error: invalid memory address or nil pointer dereference
goroutine 33592 [running]:
net/http.(*conn).serve.func1()
/usr/local/go/src/net/http/server.go:1898 +0xbe
panic({0x8b5cc0?, 0xcc5cf0?})
/usr/local/go/src/runtime/panic.go:770 +0x132
github.com/hashicorp/go-memdb.(*MemDB).getRoot(...)
/go/pkg/mod/github.com/hashicorp/[email protected]/memdb.go:65
github.com/hashicorp/go-memdb.(*MemDB).Txn(0x0, 0x0)
/go/pkg/mod/github.com/hashicorp/[email protected]/memdb.go:78 +0x4d
m3u-stream-merger/database.GetConcurrency(0x4)
/app/database/memdb.go:45 +0x51
main.checkConcurrency(0x4)
/app/mp4_handler.go:145 +0x9b
main.loadBalancer({0x38, {0xc000812460, 0xe}, {0xc000812497, 0x7}, {0xc000b94210, 0x2e}, {0xc0008124e0, 0xc}, {0xc000b80180, ...}})
/app/mp4_handler.go:21 +0xbc
main.mp4Handler({0x9caf90, 0xc0000ac2a0}, 0xc000296360, 0xc00018c5b0)
/app/mp4_handler.go:100 +0x5c5
main.main.func2({0x9caf90?, 0xc0000ac2a0?}, 0xc000307b30?)
/app/main.go:165 +0x25
net/http.HandlerFunc.ServeHTTP(0xcd6770?, {0x9caf90?, 0xc0000ac2a0?}, 0x84bb1a?)
/usr/local/go/src/net/http/server.go:2166 +0x29
net/http.(*ServeMux).ServeHTTP(0x4662d9?, {0x9caf90, 0xc0000ac2a0}, 0xc000296360)
/usr/local/go/src/net/http/server.go:2683 +0x1ad
net/http.serverHandler.ServeHTTP({0xc000b822d0?}, {0x9caf90?, 0xc0000ac2a0?}, 0x6?)
/usr/local/go/src/net/http/server.go:3137 +0x8e
net/http.(*conn).serve(0xc000b842d0, {0x9cb548, 0xc000198210})
/usr/local/go/src/net/http/server.go:2039 +0x5e8
created by net/http.(*Server).Serve in goroutine 1

[sonroyaalmerol]

Another build has been released. The memdb panic should be fixed now.

[shorty789]

Thank you, it seems to be playing now,

2024/03/06 23:01:55 Received request from :46488 for URL: /stream/VUsgLSBCQkMgMSBVSEQ=.mp4
2024/03/06 23:01:55 Current concurrent connections for M3U_4: 0
2024/03/06 23:01:56 Proxying :46488 to xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2024/03/06 23:01:56 Sent MP4 stream to :46488

I will keep you posted as i try it out.

[shorty789]

OK so I have 4 playlists configured.

Connected first device and started playing, everything seemed ok.
Connected second device, everything was ok and playing.
When I stopped playback on the second device, it also stopped on the first and will not play anything else.

2024/03/06 23:01:55 Received request from :46488 for URL: /stream/VUsgLSBCQkMgMSBVSEQ=.mp4
2024/03/06 23:01:55 Current concurrent connections for M3U_4: 0
2024/03/06 23:01:56 Proxying :46488 to xxxxxxxxxxxxxxxxx
2024/03/06 23:01:56 Sent MP4 stream to :46488
2024/03/06 23:05:07 Received request from :33798 for URL: /stream/VUsgLSBDSEFOTkVMIDQgRkhE.mp4
2024/03/06 23:05:07 Current concurrent connections for M3U_4: 1
2024/03/06 23:05:07 Concurrency limit reached (1): xxxxxxxxxxxxxxxxx
2024/03/06 23:05:07 Current concurrent connections for M3U_2: 0
2024/03/06 23:05:07 Proxying :33798 to xxxxxxxxxxxxxxxxx
2024/03/06 23:05:07 Sent MP4 stream to :33798
2024/03/06 23:06:07 Client disconnected after fetching MP4 stream
2024/03/06 23:06:31 Client disconnected after fetching MP4 stream

[sonroyaalmerol]

Can you try it with the :dev tag?

[shorty789]

OK, from what I have tried with the dev tag, that seems much better, it seems to be using each playlist accordingly.

One thing i did notice initially was that it would always cycle through 4, 3, 2, 1 in that order when selecting the m3u to load from.

Say I had a max concurrency of 3 on playlist 4, and a max concurrency of 1 on the other 3 playlists, given the way it cycles, would it be the case that it would use up all 3 streams on playlist 4 before trying the other 3 playlists?

The other thing it might be handy to have currenly is a log output at regular timed intervals just printing how many active connections the app believes it has, as I will be able to more accurately test if the counter is working as expected.

[sonroyaalmerol]

One thing i did notice initially was that it would always cycle through 4, 3, 2, 1 in that order when selecting the m3u to load from.

Good catch. That is definitely not by design. I'll have to fix that.

Say I had a max concurrency of 3 on playlist 4, and a max concurrency of 1 on the other 3 playlists, given the way it cycles, would it be the case that it would use up all 3 streams on playlist 4 before trying the other 3 playlists?

Currently, yes. I was thinking of implementing round robin load balancing but wanted to make sure the concurrency limit feature worked first. I'll have it on the next release.

The other thing it might be handy to have currenly is a log output at regular timed intervals just printing how many active connections the app believes it has, as I will be able to more accurately test if the counter is working as expected.

Sure. I'll add this as another env var option.

[shorty789]

Awesome, thank you, it would also be nice to be able to specify the time for updating the playlist rather than have it happen at a timed interval based on when the container is started, shall i submit a separate suggestion for this? I am more than happy to continue testing too, keep up the awesome work.

[sonroyaalmerol]

No problem! I'd appreciate it if you create a different issue for each of the suggestions, including the bugs you found (reversed order, etc.) just so I can track things better.

On that note, I'll be closing this issue now.