getting this error #72
Comments
|
got it to work with v0.6 but now is kinda stuck here |
|
i also cant open or download the m3u. it takes forever |
|
Hi @aniel300, can you try deleting the data folder then restarting the container? Give it a few minutes to download the original M3Us before trying to access/download the proxy M3U. I'd like to see the entire logs when you do that. If it is long, attach it as a file. Remove parts that contain your actual IPTV URLs. |
|
i would love to do that. do u have a discord where we can debug this? |
|
Sorry, I don't have one set up yet. |
|
ok that is fine. right now i cant give u the logs now but i have try deleting the folder and deleting the container and then starting again and the same issue still happening. i will provide logs as soon i can |
|
i cant get to that point, for some reason is taking long time to download and parse just 1 m3u. here are some logs. i found after testing a little that v0.6 does not have this issue but that version adds ".mp4" to live channels making them vod runtime.goexit({}) |
|
it finished but only few channels were processed when it is supposed to process the entire m3u |
|
do u know if this can be fix or should i make a new gh issue? 2024/08/13 02:56:58 Received request from xxx:2590 for URL: /stream/TEE6IFRlbGVtdW5kbyBGb3J0IE15ZXJzIEZsb3JpZGEgV1dEVCBDRA== |
|
Hi @aniel300, can you try the latest release, Take note that Redis is now required for |
|
will test and report back, thank u |
am getting "exec /gomain: no such file or directory" with latest and dev ` m3uproxy-redis: |
|
As mentioned in the other issue, the exec /gomain error should be fixed on the latest release. Let me know the test results for your original issue thanks! |
|
yes the "exec /gomain" was fixed however am getting this error now 2024/08/18 12:45:51 SYNC_ON_BOOT enabled. Starting initial M3U update. |
|
here are some more logs with a different m3u 2024/08/18 12:58:12 SYNC_ON_BOOT enabled. Starting initial M3U update. |
|
Can you list out the tags used by your m3u? Examples are: tvg-id, tvg-name, tvg-logo, group-title, etc. Just the tags, no need for the values. |
|
here is just an example of one of my m3u #EXTINF:-1 tvg-id="" tvg-name="AF: Vox Africa" tvg-logo="http://s3.i3ns.net:2052/portal/picon/2022-07/f9feb90f6ce99f55806bfd57d7b2bc51.png" group-title="Africa",AF: Vox Africa |
|
I see the problem now. It might be because of your m3u's tvg-id being empty. I'll see what I can do. |
|
@sonroyaalmerol thank u bud, keep up the good work. |
|
Can you try the dev build? The existence of tvg-id should not matter anymore by default. |
|
did u also implemented it for the other tvgs? |
|
still getting this on one of my m3u just fyi 2024/08/18 14:12:42 Uncaught attribute: t=E/x² - 2020 |
|
Yes, none of the tags are required for the default sorting logic. The "Uncaught attribute" error just means that the parser was able to detect an m3u tag that I haven't implemented yet. However, that doesn't really look like an actual tag so probably an edge case with the parser that I missed. Either way, the streams should still work just fine. If you don't mind, can you search up that specific string ("E/x² - 2020") within the source M3U? I'd like to see the entire line if possible (without the urls). |
|
#EXTINF:-1 tvg-id="" tvg-name="DE - t=E/x² - 2020" tvg-logo="https://image.tmdb.org/t/p/w600_and_h900_bestv2/yYo70zJYNfmP3PcAOhCclqBDZlP.jpg" group-title="VOD | GERMAN | DEUTSCH",DE - t=E/x² - 2020 |
|
umm got this error 2024/08/18 14:13:28 Background process: Updated M3U database. here is how it looks inside the m3u: #EXTINF:-1 tvg-id=".AMAZON.PRIME.1.-.LEAGUE.2" tvg-name="FR - DAZN LIGUE 1 - LIGUE 2 : " tvg-logo="http://go4.pw/FRANCE/DAZN.png" group-title="EU | FRANCE DAZN",FR - DAZN LIGUE 1 - LIGUE 2 : |
|
am not sure if it has finished but am getting this error when trying to download or open the end result m3u 2024/08/18 14:37:55 SYNC_ON_BOOT enabled. Starting initial M3U update. |
|
i dont think it has finished yet 2024/08/18 14:45:02 GetStreams error: stream not found: (My) Truth |
|
I've optimized the redis queries in the dev build just now. Can you try it for me? |
|
is kinda stuck now (i gave it another playlist, a more light one but it is stuck i think) 😭 |
|
Hmm, maybe you need a multithreaded database. Can you monitor your resource usage (CPU, RAM, etc.) while the background process is ongoing? You can try replacing Redis with KeyDB. It should just be a drop-in replacement. It should be as simple as this sample: version: '3'
services:
m3u-stream-merger-proxy:
image: sonroyaalmerol/m3u-stream-merger-proxy:dev
ports:
- "8080:8080"
environment:
- REDIS_ADDR=keydb:6379
...
restart: always
keydb:
image: eqalpha/keydb
restart: always |
|
i got it to work with a small amount of channel using keydb and then i decided to give it everything and i got this error and am not sure if it worked, got interrupted or if it is going to work. resource usage is light in all senses. 2024/08/18 20:30:52 Background process: Updated M3U #4 from xx |
|
@sonroyaalmerol any update on this. no rush, just wondering 🙏 |
|
I've increased the timeouts from 3 seconds (default) to 1 minute in the latest dev build. That should help with the i/o timeout errors. |
|
ok trying right now. thank you for you hard work. I shall buy you a beer some time in the near future 🙏 |
|
ok this fixed it. however now the export process is what is taking long time. here is a screenshot of the resources usage. |
|
|
|
How many channels does your M3U have in total? I just need some estimates. I'll probably have to implement some sort of chunking for the queries to be done in parallel by KeyDB. |
|
it is channels and vod...the db is about 300mb in size. not sure if that answers ur question. |
|
I've added some chunking, parallelism, and caching to the queries in the dev build. If it is still slow to export, the cache should build up in the first request and will be faster on the subsequent requests. |
|
nice. let me try it. thank you for u hard work |
|
it didnt work. still taking forever and it never starts downloading :( |
|
Can you attach the last few lines of the log? Just want to make sure which part is taking long |
|
i don't think there is logs in regard to this but here is what portainer says 2024/08/20 18:02:45 Uncaught attribute: timeshift=1 |
|
I've added more debug logs to help us. It will now generate A LOT of logs with |
|
it is not getting stuck as i can see it it is moving in the logs, it just taking some time. here are some logs and as well am trying to get more. hopefully there is no personal info leaking here 2024/08/20 19:06:27 [DEBUG] Processing stream: {princess-multi-sub-s01-e05-en-tis-time-for-torture-princess-multi-sub-s01-e05 " Princess [MULTI-SUB] S01 E05" ,EN - 'Tis Time for "Torture," Princess [MULTI-SUB] S01 E05 https://image.tmdb.org/t/p/w600_and_h900_bestv2/dQuqH4ukyfQv6k45AxwxKwMSXHH.jpg SRS | ANIME [EN] map[]} |
|
I've added |
|
ok let me try that and yes thank u for all ur time and effort, i really appreciate it |
|
there is an error pretending to CACHE_ON_SYNC but am having a hard time catching it with portainer |
|
can u remove all the url stuff from the debug? |
|
Closing this issue as the initial problem seems to be resolved at this point and has gone off to multiple different problems. Feel free to create new separate issues if needed. |
can u revert this to how it was originally, or not needed? |
m3u-stream-merger/database.(*Instance).GetStreamByTitle(0xc00018a000, {0xc014aef8fe, 0x1a})
/app/database/db.go:331 +0xdc fp=0xc036563c88 sp=0xc036563af0 pc=0x52c29c
m3u-stream-merger/m3u.insertStreamToDb(0xc00018a000, {0x0, {0xc014aef8fe, 0x1a}, {0x0, 0x0}, {0xc00d0a1d7b, 0x43}, {0xc00d0a1dcd, 0x9}, ...})
/app/m3u/parser.go:67 +0x68 fp=0xc036563eb0 sp=0xc036563c88 pc=0x6bc3a8
m3u-stream-merger/m3u.ParseM3UFromURL.func1(0x3132453430532066?)
/app/m3u/parser.go:185 +0xdd fp=0xc036563fc8 sp=0xc036563eb0 pc=0x6bdd9d
m3u-stream-merger/m3u.ParseM3UFromURL.gowrap1()
/app/m3u/parser.go:186 +0x24 fp=0xc036563fe0 sp=0xc036563fc8 pc=0x6bdc84
runtime.goexit({})
/usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc036563fe8 sp=0xc036563fe0 pc=0x46dd81
created by m3u-stream-merger/m3u.ParseM3UFromURL in goroutine 37
/app/m3u/parser.go:183 +0x566
goroutine 130620 gp=0xc03094b500 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc02e391ec0 sp=0xc02e391ea0 pc=0x43c38e
runtime.chanrecv(0xc0232b61e0, 0xc02e391fc0, 0x1)
/usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc02e391f38 sp=0xc02e391ec0 pc=0x40871f
runtime.chanrecv1(0x16?, 0x0?)
/usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc02e391f60 sp=0xc02e391f38 pc=0x408352
m3u-stream-merger/m3u.ParseM3UFromURL.func3()
/app/m3u/parser.go:196 +0x26 fp=0xc02e391fe0 sp=0xc02e391f60 pc=0x6bd946
runtime.goexit({})
/usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc02e391fe8 sp=0xc02e391fe0 pc=0x46dd81
created by m3u-stream-merger/m3u.ParseM3UFromURL in goroutine 37
/app/m3u/parser.go:195 +0x687
goroutine 130621 gp=0xc03094b6c0 m=nil [sync.Mutex.Lock]:
runtime.gopark(0xc035930f80?, 0x4448ed?, 0xa0?, 0xf?, 0x4696e6?)
/usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc035930f20 sp=0xc035930f00 pc=0x43c38e
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:408
runtime.semacquire1(0xb700c4, 0x0, 0x3, 0x1, 0x15)
/usr/local/go/src/runtime/sema.go:160 +0x225 fp=0xc035930f88 sp=0xc035930f20 pc=0x44e745
sync.runtime_SemacquireMutex(0xc0be249000?, 0x78?, 0xc01a6df808?)
/usr/local/go/src/runtime/sema.go:77 +0x25 fp=0xc035930fc0 sp=0xc035930f88 pc=0x46a465
sync.(*Mutex).lockSlow(0xb700c0)
/usr/local/go/src/sync/mutex.go:171 +0x15d fp=0xc035931010 sp=0xc035930fc0 pc=0x476dfd
sync.(*Mutex).Lock(...)
/usr/local/go/src/sync/mutex.go:90
github.com/mattn/go-sqlite3.newHandle(0xc0200c00c0, {0x7f23c0, 0xc000d7f340})
/go/pkg/mod/github.com/mattn/[email protected]/callback.go:110 +0x5e fp=0xc035931068 sp=0xc035931010 pc=0x5128be
github.com/mattn/go-sqlite3.(*SQLiteConn).RegisterFunc(0xc0200c00c0, {0x85157f, 0xc}, {0x7ef220, 0xc00d1a4d90}, 0x1)
/go/pkg/mod/github.com/mattn/[email protected]/sqlite3.go:676 +0x659 fp=0xc035931158 sp=0xc035931068 pc=0x516b59
github.com/mattn/go-sqlite3.(*SQLiteDriver).Open(0xc0000d0080, {0xc000192000, 0x14})
/go/pkg/mod/github.com/mattn/[email protected]/sqlite3.go:1589 +0x32fb fp=0xc035931788 sp=0xc035931158 pc=0x51b55b
database/sql.dsnConnector.Connect(...)
/usr/local/go/src/database/sql/sql.go:791
database/sql.(*dsnConnector).Connect(0x20?, {0x0?, 0xc034fbc700?})
:1 +0x31 fp=0xc0359317b0 sp=0xc035931788 pc=0x4f0271
database/sql.(*DB).conn(0xc0001901a0, {0x8def10, 0xb6fe60}, 0x1)
/usr/local/go/src/database/sql/sql.go:1415 +0x71e fp=0xc035931938 sp=0xc0359317b0 pc=0x4e8d7e
database/sql.(*DB).query(0xc0001901a0, {0x8def10, 0xb6fe60}, {0x8665fb, 0x4b}, {0xc035931bc8, 0x1, 0x1}, 0xe0?)
/usr/local/go/src/database/sql/sql.go:1749 +0x57 fp=0xc0359319b0 sp=0xc035931938 pc=0x4eb0f7
database/sql.(*DB).QueryContext.func1(0x18?)
/usr/local/go/src/database/sql/sql.go:1732 +0x4f fp=0xc035931a18 sp=0xc0359319b0 pc=0x4eb02f
database/sql.(*DB).retry(0x417d18?, 0xc035931a88)
/usr/local/go/src/database/sql/sql.go:1566 +0x42 fp=0xc035931a60 sp=0xc035931a18 pc=0x4e9b62
database/sql.(*DB).QueryContext(0xc00cf61ab8?, {0x8def10?, 0xb6fe60?}, {0x8665fb?, 0xc01d166001?}, {0xc02c0cdbc8?, 0x0?, 0x0?})
/usr/local/go/src/database/sql/sql.go:1731 +0xc5 fp=0xc035931af0 sp=0xc035931a60 pc=0x4eaf45
database/sql.(*DB).Query(...)
/usr/local/go/src/database/sql/sql.go:1745
m3u-stream-merger/database.(*Instance).GetStreamByTitle(0xc00018a000, {0xc00cf61ab8, 0xf})
/app/database/db.go:331 +0xdc fp=0xc035931c88 sp=0xc035931af0 pc=0x52c29c
m3u-stream-merger/m3u.insertStreamToDb(0xc00018a000, {0x0, {0xc00cf61ab8, 0xf}, {0x0, 0x0}, {0xc023fd6d5a, 0x23}, {0xc023fd6d8c, 0x18}, ...})
/app/m3u/parser.go:67 +0x68 fp=0xc035931eb0 sp=0xc035931c88 pc=0x6bc3a8
m3u-stream-merger/m3u.ParseM3UFromURL.func1(0x0?)
/app/m3u/parser.go:185 +0xdd fp=0xc035931fc8 sp=0xc035931eb0 pc=0x6bdd9d
m3u-stream-merger/m3u.ParseM3UFromURL.gowrap1()
/app/m3u/parser.go:186 +0x24 fp=0xc035931fe0 sp=0xc035931fc8 pc=0x6bdc84
runtime.goexit({})
/usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc035931fe8 sp=0xc035931fe0 pc=0x46dd81
created by m3u-stream-merger/m3u.ParseM3UFromURL in goroutine 37
/app/m3u/parser.go:183 +0x566
goroutine 133873 gp=0xc03094b880 m=nil [sync.Mutex.Lock]:
runtime.gopark(0xc01e626f80?, 0x4448ed?, 0xa0?, 0x6f?, 0x4696e6?)
/usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc01e626f20 sp=0xc01e626f00 pc=0x43c38e
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:408
runtime.semacquire1(0xb700c4, 0x0, 0x3, 0x1, 0x15)
/usr/local/go/src/runtime/sema.go:160 +0x225 fp=0xc01e626f88 sp=0xc01e626f20 pc=0x44e745
sync.runtime_SemacquireMutex(0xc0bb2e5e50?, 0x80?, 0xc01a2ba808?)
/usr/local/go/src/runtime/sema.go:77 +0x25 fp=0xc01e626fc0 sp=0xc01e626f88 pc=0x46a465
sync.(*Mutex).lockSlow(0xb700c0)
/usr/local/go/src/sync/mutex.go:171 +0x15d fp=0xc01e627010 sp=0xc01e626fc0 pc=0x476dfd
sync.(*Mutex).Lock(...)
/usr/local/go/src/sync/mutex.go:90
github.com/mattn/go-sqlite3.newHandle(0xc036bc7260, {0x7f23c0, 0xc007170c40})
/go/pkg/mod/github.com/mattn/[email protected]/callback.go:110 +0x5e fp=0xc01e627068 sp=0xc01e627010 pc=0x5128be
github.com/mattn/go-sqlite3.(*SQLiteConn).RegisterFunc(0xc036bc7260, {0x85157f, 0xc}, {0x7ef220, 0xc01a1be1d0}, 0x1)
/go/pkg/mod/github.com/mattn/[email protected]/sqlite3.go:676 +0x659 fp=0xc01e627158 sp=0xc01e627068 pc=0x516b59
github.com/mattn/go-sqlite3.(*SQLiteDriver).Open(0xc0000d0080, {0xc000192000, 0x14})
/go/pkg/mod/github.com/mattn/[email protected]/sqlite3.go:1589 +0x32fb fp=0xc01e627788 sp=0xc01e627158 pc=0x51b55b
database/sql.dsnConnector.Connect(...)
/usr/local/go/src/database/sql/sql.go:791
database/sql.(*dsnConnector).Connect(0x0?, {0x0?, 0x8dc4c0?})
:1 +0x31 fp=0xc01e6277b0 sp=0xc01e627788 pc=0x4f0271
database/sql.(*DB).conn(0xc0001901a0, {0x8def10, 0xb6fe60}, 0x1)
/usr/local/go/src/database/sql/sql.go:1415 +0x71e fp=0xc01e627938 sp=0xc01e6277b0 pc=0x4e8d7e
The text was updated successfully, but these errors were encountered: