Replies: 26 comments 4 replies
-
|
here is some more logs. it looks like it is suck however i just confirmed the two groups are present in the db/m3u so we good there. now the only problem is when the the db is big |
Beta Was this translation helpful? Give feedback.
-
|
please let me know if am leaking info in this screenshot and if u can a way to prevent info leak when using the debug will be awesome because manually making sure and removing the URL is a pain. |
Beta Was this translation helpful? Give feedback.
-
|
Can you test and see if #124 results to faster M3U returns? I've reworked the caching behind the scenes. It still should have a similar behavior where the cache needs to be built completely on first request. Also
|
Beta Was this translation helpful? Give feedback.
-
|
base url is where the proxy server lives? |
Beta Was this translation helpful? Give feedback.
-
|
Yep! Not the base URL of the source streams. |
Beta Was this translation helpful? Give feedback.
-
|
ok let me test and report back |
Beta Was this translation helpful? Give feedback.
-
|
every time u mentions a specific pr does it means it is also available in dev tag as well? |
Beta Was this translation helpful? Give feedback.
-
|
Only if the PR has been merged. There are specific PRs where I try not to merge them immediately as it changes a lot of components. |
Beta Was this translation helpful? Give feedback.
-
|
i have tried all possible combinations such as |
Beta Was this translation helpful? Give feedback.
-
|
Well, the fact that you're seeing that log means that you're not using the image of the PR I mentioned. PRs that I mention will have a comment from a bot containing the image URL for that specific PR. For #124, it should be this comment. Use that image URL instead of the usual
To make things simpler for you, the base URL can be derived from the URL you use to access the generated M3U. For example, if you access the M3U with this URL: |
Beta Was this translation helpful? Give feedback.
-
|
rest assured am trying the correct image and those ip are from the docker network on my debain 12 linux machine. i will share screenshot of everything once i get back home, thank you. |
Beta Was this translation helpful? Give feedback.
-
|
Sure. Do double check as it is actually impossible for that PR image to return that log line. It doesn't exist in the code of that PR. |
Beta Was this translation helpful? Give feedback.
-
|
current image being use and docker ip/network. |
Beta Was this translation helpful? Give feedback.
-
|
I guess it's the workflow not doing what it's supposed not building the right image anymore in PRs for some reason. 🤦 I'll merge it to dev. You can test it from there instead. |
Beta Was this translation helpful? Give feedback.
-
|
ok thanks |
Beta Was this translation helpful? Give feedback.
-
|
unfortunately, the issue has not been fixed. here is some screenshot of what it has been doping. db is around 300mb btw. i left it all night because i had to go to sleep. |
Beta Was this translation helpful? Give feedback.
-
|
Can you give me more context: what makes you say it doesn't work? What is the output when you request for the M3U url? |
Beta Was this translation helpful? Give feedback.
-
|
it keeps loading in the browser or if i try to download the m3u via idm it never downloads. there is a little of cpu spike on the db container and proxy container (I believe) but that is about it. for example, the small channel sample am sing for debugging (~38 channels) works as expected, sync is quick and same goes for export but when u give it multiple url/providers that comes with vod, etc which all together is around 300mb in size then export doesn't seem to work. i dont the sync taking time the firs t time but the exports after that i would like to be quick. |
Beta Was this translation helpful? Give feedback.
-
|
I did more work for this issue. I've merged the changes to the dev build if you want to try it out again. |
Beta Was this translation helpful? Give feedback.
This comment has been hidden.
This comment has been hidden.
-
|
could the slowness be caused by the sorting? it still doing a lot of this and there is no progress bar so idk how much time is left for this to finish |
Beta Was this translation helpful? Give feedback.
-
|
Please be patient and wait for everything to be processed to cache. We're talking about ~300mb worth of strings being processed. How I wish it is as easy as simply setting the "amount of CPU" to be used. That is just not how software works. Most of the process required for the proxy to work are single-threaded and cannot be parallelized. Also, do disable debugging mode if you want the maximum performance possible. Logging affects performance more than you might expect especially in cases like these. Logging is a single-threaded process which pretty much forces even the parallelized processes to wait for the log to be printed to the terminal before the next job is executed. The sorting is index-based and does not add any time complexity at all. Once everything is processed, the M3U will be stored in plain-text, both in memory and as a file. At that point, the only bottleneck would be your RAM speed and disk i/o. The function for time complexity will always be directly proportional to the amount of ingress data. |
Beta Was this translation helpful? Give feedback.
-
|
I've just merged to dev more optimizations. This will probably be something that will improve over time as I fix other issues as a side effect. I won't be focusing on this anymore in the near future. Converting this to a discussion instead. |
Beta Was this translation helpful? Give feedback.
-
|
it didn't work in the end even after disabling debugging, and yes i agree with you and i thank u for working and trying so hard on this. i will try again later on with no vod and only channels to see if that makes a difference. |
Beta Was this translation helpful? Give feedback.
-
|
@aniel300 I've done some major refactoring and optimization with #144 which will decrease the number of queries the proxy does with the database by at least half. I'm not sure how that would translate in an actual set of streams. Test it out in the dev build with |
Beta Was this translation helpful? Give feedback.
-
|
will do. am also eager to test with vod out of the equation to see the size of the db and see if that makes a difference, thank you for continue giving this a try. if this happens to be a vod/ db being to big i can settle for channels only 😉🙏 |
Beta Was this translation helpful? Give feedback.
-
|
getting some go crashes i think and db still at 1kb. not sure if am supposed to keep waiting sync.runtime_SemacquireMutex(0x0?, 0x0?, 0xc0008024d0?) |
Beta Was this translation helpful? Give feedback.
-
|
with CLEAR_ON_BOOT off am getting this logs m3u-stream-merger/m3u.ParseM3UFromURL.func1() |
Beta Was this translation helpful? Give feedback.
-
|
i just notice that with |
Beta Was this translation helpful? Give feedback.
-
|
where can i get this image? #143 as latest dev broke something form me. i will make a new container tomorrow for testing and debugging that way i don't have to be going back fort. |
Beta Was this translation helpful? Give feedback.
-
this is in reference to #72 (comment) I was trying to capture an interesting line of the log that day but i could not do it, but i think i just bumped into by mistake the line is this
2024/08/23 22:57:17 [DEBUG] Cache miss. Retrieving streams from Redis...not sure if this will be helpful or not.Beta Was this translation helpful? Give feedback.
All reactions