Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Few errors with new build #21

Closed
ZetsApocNow opened this issue Jun 28, 2024 · 30 comments
Closed

Few errors with new build #21

ZetsApocNow opened this issue Jun 28, 2024 · 30 comments
Assignees
Labels
bug Something isn't working enhancement New feature or request

Comments

@ZetsApocNow
Copy link

Select type of series:
1: Anime (Gogoanime)
2: Anime (Animepahe)
3: Drama (Myasiantv)
4: Movies & TV Shows (Vidsrc)
5: Movies & TV Shows (Superembed)

Enter your choice: 4
C:\udb\Clients\BaseClient.py:182: SyntaxWarning: invalid escape sequence '\d'
resolutions = _regex_list(master_m3u8_data, 'RESOLUTION=(\d+x\d+)', 1)
C:\udb\Clients\BaseClient.py:684: SyntaxWarning: invalid escape sequence '\d'
is_match = lambda word: re.search('\d+.\d+.\d+.\d+', word)
Error occurred: No module named 'distutils'. Check log for more details.

Ready for one more? Reload UDB (y|n)?

Select type of series:
1: Anime (Gogoanime)
2: Anime (Animepahe)
3: Drama (Myasiantv)
4: Movies & TV Shows (Vidsrc)
5: Movies & TV Shows (Superembed)

Enter your choice: 5
Error occurred: No module named 'distutils'. Check log for more details.

Ready for one more? Reload UDB (y|n)?

@Prudhvi-pln
Copy link
Owner

That's strange. It looks more like an installation issue. Did you change / update your python ?
Try this: Uberi/speech_recognition#732

FYI, It is working fine for me:
image

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Jun 28, 2024

Alright, I applied the fix from the link, tried again, said Superembed requires Chrome, installed that. Now, on 4, it just says "Invalid vidsrc keys" - which I guess is to be expected as per your explanation of the cause, but on 5 I'm getting no luck either - shown below

Select type of series:
1: Anime (Gogoanime)
2: Anime (Animepahe)
3: Drama (Myasiantv)
4: Movies & TV Shows (Vidsrc)
5: Movies & TV Shows (Superembed)

Enter your choice: 5

Enter series/movie name: charmed

Search Results:
1: Charmed | Language: English | Genre: Comedy, Drama, Mystery, Sci-Fi & Fantasy
   | Type: TV Show | User Score: 82% | Released: 1998 | Status: Ended
   | Seasons: 8 | Total Episodes: 178
2: Charmed | Language: English | Genre: Drama, Sci-Fi & Fantasy
   | Type: TV Show | User Score: 73% | Released: 2018 | Status: Canceled
   | Seasons: 4 | Total Episodes: 72
3: Beautiful Charm (Yahşi Cazibe) | Language: Turkish | Genre: Comedy, Family
   | Type: TV Show | User Score: 53% | Released: 2010 | Status: Ended
   | Seasons: 2 | Total Episodes: 93
4: Second Time is a Charm (第二次也很美) | Language: Chinese | Genre:
   | Type: TV Show | User Score: 30% | Released: 2019 | Status: Ended
   | Seasons: 1 | Total Episodes: 57
5: Charmsukh | Language: Hindi | Genre: Drama
   | Type: TV Show | User Score: 60% | Released: 2019 | Status: Returning Series
   | Seasons: 1 | Total Episodes: 56

Enter 0 to search with different key word

Select one of the above: 1

Available Episodes Details: 178 episodes found.
Enter seasons range to display (ex: 1-16) [default=1-8]: 3
Showing episodes for seasons [3 - 3]:
-------------- Season: 3 --------------
Season: 03 | Charmed - S03E01
Season: 03 | Charmed - S03E02
Season: 03 | Charmed - S03E03
Season: 03 | Charmed - S03E04
Season: 03 | Charmed - S03E05
Season: 03 | Charmed - S03E06
Season: 03 | Charmed - S03E07
Season: 03 | Charmed - S03E08
Season: 03 | Charmed - S03E09
Season: 03 | Charmed - S03E10
Season: 03 | Charmed - S03E11
Season: 03 | Charmed - S03E12
Season: 03 | Charmed - S03E13
Season: 03 | Charmed - S03E14
Season: 03 | Charmed - S03E15
Season: 03 | Charmed - S03E16
Season: 03 | Charmed - S03E17
Season: 03 | Charmed - S03E18
Season: 03 | Charmed - S03E19
Season: 03 | Charmed - S03E20
Season: 03 | Charmed - S03E21
Season: 03 | Charmed - S03E22

Enter seasons to download (ex: 1-16) [default=1-8]: 3

Download entire season(s) (y|n)? y

Fetching Episodes & Available Resolutions:
-------------- Season: 3 --------------
Error occurred: too many values to unpack (expected 2). Check log for more details.

Ready for one more? Reload UDB (y|n)?

In the log it states

2024-06-29 00:10:50,521 - INFO - <module>:373 - -------------------------------- NEW UDB INSTANCE v2.13.0 --------------------------------
2024-06-29 00:10:50,522 - INFO - <module>:375 - CLI options: Namespace(conf='config_udb.yaml', version=False, series_type=None, series_name=None, series_year=None, seasons=None, episodes=None, resolution=None, start_download=False, disable_colors=False, hls_size_accuracy=0, update=False)
2024-06-29 00:10:51,543 - INFO - <module>:379 - Selected Series type: Movies & TV Shows (Superembed)
2024-06-29 00:10:51,884 - INFO - <module>:383 - Client: <Clients.SuperembedClient.SuperembedClient object at 0x000001B10343B620>
2024-06-29 00:10:54,072 - INFO - search_and_select_series:106 - Searching with keyword: charmed
2024-06-29 00:10:58,057 - INFO - search_and_select_series:108 - Search Results Found
2024-06-29 00:10:58,770 - INFO - <module>:397 - Selected series: {'title': 'Charmed', 'link': 'https://www.themoviedb.org//tv/1981-charmed?language=en-US', 'Status': 'Ended', 'Network': 'Network', 'Type': 'Scripted', 'Original Language': 'English', 'genre': 'Comedy, Drama, Mystery, Sci-Fi & Fantasy', 'score': '82', 'year': '1998', 'show_id': '1981', 'type': 'tv', 'seasons': 8, 'episodes': 178, 'episodes_per_season': {'Specials': '14', 'Season 1': '22', 'Season 2': '22', 'Season 3': '22', 'Season 4': '22', 'Season 5': '23', 'Season 6': '23', 'Season 7': '22', 'Season 8': '22'}}
2024-06-29 00:10:58,771 - INFO - <module>:400 - Fetching episodes list
2024-06-29 00:11:36,299 - INFO - <module>:409 - Displaying episodes list
2024-06-29 00:11:48,085 - INFO - <module>:419 - Fetching episodes based on selected_eps = {3: {'start': 1, 'end': 22}}
2024-06-29 00:11:49,285 - INFO - patch_exe:346 - patching driver executable C:\Users\theda\appdata\roaming\undetected_chromedriver\undetected_chromedriver.exe
2024-06-29 00:11:50,200 - INFO - _configure_headless:493 - setting properties for headless
2024-06-29 00:11:52,657 - ERROR - <module>:512 - Error occurred: too many values to unpack (expected 2). Check log for more details.
2024-06-29 00:11:52,658 - WARNING - <module>:513 - Stacktrace: Traceback (most recent call last):
  File "C:\udb\Clients\SuperembedClient.py", line 333, in fetch_episode_links
    m3u8_links, subtitles = self._resolve_vipstream_source(link)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\udb\Clients\SuperembedClient.py", line 207, in _resolve_vipstream_source
    lang, url = sub.split(']')
    ^^^^^^^^^
ValueError: too many values to unpack (expected 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\udb\udb.py", line 421, in <module>
    target_ep_links = client.fetch_episode_links(episodes, selected_eps)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\udb\Clients\SuperembedClient.py", line 344, in fetch_episode_links
    raise Exception(e)          # propagate the exception as-is
    ^^^^^^^^^^^^^^^^^^
Exception: too many values to unpack (expected 2)


@Prudhvi-pln Prudhvi-pln self-assigned this Jun 29, 2024
@Prudhvi-pln Prudhvi-pln added the bug Something isn't working label Jun 29, 2024
@Prudhvi-pln
Copy link
Owner

Prudhvi-pln commented Jun 29, 2024

Updated the code with required fix. Also, updated Vidsrc client with reliable keys source (which updates every hour).
Please check and kindly let me know if you see any other issues.

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Jun 29, 2024

Was getting errors, decided to delete all I had and redownload fully, errors still present. Intermittent regex issues present with /d and /g

Option 1 seems to work well and completes
Option 2 seems to work well and completes
Option 3 again has regex issues but works through it and completes
Option 4 seems to work well and completes, tested multiple times, regex issues show sometimes
Option 5 errors out

Enter seasons to download (ex: 1-16) [default=1-3]: 1

Download entire season(s) (y|n)? n

Enter Season-1 episodes to download (ex: 1-16) [default=1-24]: 1-5

Fetching Episodes & Available Resolutions:
-------------- Season: 1 --------------
Failed to extract stream source link for source: vipstream. Error: 'NoneType' object has no attribute 'get'
Failed to extract stream source link for source: vipstream. Error: 'NoneType' object has no attribute 'get'
Failed to extract stream source link for source: vipstream. Error: 'NoneType' object has no attribute 'get'
Failed to extract stream source link for source: vipstream. Error: 'NoneType' object has no attribute 'get'
Failed to extract stream source link for source: vipstream. Error: 'NoneType' object has no attribute 'get'
No episodes are available for download!

Also I want to mention that a lot of errors surface if using multiple instances of the script, either writing to log issues or other. I prevented them before by removing some logger tidbits in the scripts and could run 4/5 at a time without issue.

For example, running 2 starts giving me errors like this and they appear on 3 obviously

I made changes involving this I think in udb.py, but I'm sure there's a better way to do things.

image

--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 74, in emit
    self.doRollover()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 179, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 115, in rotate
    os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\log\\udb.log' -> 'P:\\udb\\log\\udb.log.1'
Call stack:
  File "P:\udb\udb.py", line 396, in <module>
    target_series = search_and_select_series(series_name_predef, series_year_predef)
  File "P:\udb\udb.py", line 108, in search_and_select_series
    logger.info('Search Results Found')
Message: 'Search Results Found'
Arguments: ()

From my understanding it fails when trying to write/read the log simultaneously with another instance, preventing the write stops all issues but it isn't exactly a safe way to do things as it gains the inability to debug. So hopefully you can come up with a fix for it as this script really shines when you have 3/4 going at once for full efficiency.

@Prudhvi-pln
Copy link
Owner

Prudhvi-pln commented Jun 29, 2024

Oh, thanks for the detailed information 🙂.

So, there are 3 issues:

  1. Regex warnings
  2. Failures in Superembed
  3. Ability to run multiple instances of udb

For issue-1, it is not happening for me. Can you share the python & pip versions you are using?
For issue-2, I could not recreate it. But my guess is that there is a captcha which pops-up intermittently, which might be causing it. I'll try to recreate it and find a solution.
For issue-3, yes, current UDB is meant to run as one instance (becoz it internally use parallel processing during downloads, so your cpu and internet bandwidth is highly utilized) and uses one log file. In order to run multiple instances, will need to modify the code a bit. I'll consider this as a new feature.

While I investigate 2, please use option-4 (vidsrc), as it is more reliable now. In case you see the same invalid keys error, just retry after an hour.

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Jun 29, 2024

pip 24.0
python 3.12

@Prudhvi-pln
Copy link
Owner

Fixed issue-1, it is depreciation warning with python > 3.11.
Added feature for issue-3.

Still no luck with issue-2, as I could not recreate it.

@Prudhvi-pln Prudhvi-pln added the enhancement New feature or request label Jul 12, 2024
@ZetsApocNow
Copy link
Author

Working brilliantly thanks! I'll let you know if I come across any other issues from use. Good work,

@ZetsApocNow
Copy link
Author

Still get the logging issues, even after using different log names with -l, which doesn't make much sense. Like I'd have one set up for -l multi1 and another -l multi2 and they'd both say their seperate logs are in use by the other one.

Variations of this

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\logs\\multi1' -> 'P:\\udb\\logs\\multi1.1'
Call stack:
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\thread.py", line 92, in _worker
    work_item.run()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "P:\udb\udb.py", line 281, in call_downloader
    return download_fn(link, dl_config)
  File "P:\udb\udb.py", line 262, in downloader
    status, msg = dlClient.start_download(ep_details['downloadLink'])
  File "P:\udb\Utils\HLSDownloader.py", line 162, in start_download
    self._download_subtitles()
  File "P:\udb\Utils\HLSDownloader.py", line 106, in _download_subtitles
    self.logger.warning(f'Failed to download {sub_name} subtitle with error: {e}')
Message: 'Failed to download Farsi/Persian subtitle with error: [Errno 2] No such file or directory: "S:\\\\Jellyfin\\\\All Files\\\\dls\\\\The Vampire Diaries (2009)\\\\Season-8\\\\temp_dir\\\\EP 15 - We\'re Planning a June Wedding - 720P\\\\Farsi/Persian_14283574.vtt"'
Arguments: ()

@Prudhvi-pln
Copy link
Owner

Hmm.. this looks like an issue with Rotating File logging (it is reaching a deadlock-like scenario). I've added a delay option for logger. Please take a new pull. (Note: I didn't release it as version, so you need to use git pull). Let me know if the issue is still occurring.

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Jul 14, 2024

Updated, and now I'm running 3 concurrently and there's no logging issues as before, so looks good

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Jul 14, 2024

Hit the logging error after download complete and on the further stages. Got multi1, multi2, multi3 open with their own logs, only the first one has the errors. Edit, now the others have it too.

Ready for one more? Reload UDB (y|n)?
--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 74, in emit
    self.doRollover()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 179, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 115, in rotate
    os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\logs\\multi1' -> 'P:\\udb\\logs\\multi1.1'
Call stack:
  File "P:\udb\udb.py", line 379, in <module>
    logger.info(f'-------------------------------- NEW UDB INSTANCE v{__version__} --------------------------------')
Message: '-------------------------------- NEW UDB INSTANCE v2.13.2 --------------------------------'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 74, in emit
    self.doRollover()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 179, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 115, in rotate
    os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\logs\\multi1' -> 'P:\\udb\\logs\\multi1.1'
Call stack:
  File "P:\udb\udb.py", line 381, in <module>
    logger.info(f'CLI options: {args}')
Message: "CLI options: Namespace(conf='config_udb.yaml', log_file='multi1', version=False, series_type=None, series_name=None, series_year=None, seasons=None, episodes=None, resolution=None, start_download=False, disable_colors=False, hls_size_accuracy=0, update=False)"
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 74, in emit
    self.doRollover()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 179, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 115, in rotate
    os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\logs\\multi1' -> 'P:\\udb\\logs\\multi1.1'
Call stack:
  File "P:\udb\udb.py", line 384, in <module>
    delete_old_logs(config['LoggerConfig']['log_dir'], config['LoggerConfig'].get('log_retention_days', 7))
  File "P:\udb\Utils\commons.py", line 379, in delete_old_logs
    logging.info(f'Found {len(files_to_delete)} files to delete!')
Message: 'Found 0 files to delete!'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 74, in emit
    self.doRollover()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 179, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 115, in rotate
    os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\logs\\multi1' -> 'P:\\udb\\logs\\multi1.1'
Call stack:
  File "P:\udb\udb.py", line 384, in <module>
    delete_old_logs(config['LoggerConfig']['log_dir'], config['LoggerConfig'].get('log_retention_days', 7))
  File "P:\udb\Utils\commons.py", line 391, in delete_old_logs
    logging.info(f'Deleted {len(files_to_delete)} files.')
Message: 'Deleted 0 files.'
Arguments: ()
-- Logging error --- 64%|█████████████████████████████████░░░░░░░░░░░░░░░░░░░| 516/805 [00:51<00:21, 13.16seg/s, R/F: 0/0]
Traceback (most recent call last):███░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░| 208/887 [00:20<00:48, 13.98seg/s, R/F: 0/0]
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 313, in _put_conn
    self.pool.put(conn, block=False)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\queue.py", line 137, in put
    raise Full
queue.Full

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 74, in emit
    self.doRollover()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 179, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\logging\handlers.py", line 115, in rotate
    os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'P:\\udb\\logs\\multi3' -> 'P:\\udb\\logs\\multi3.1'
Call stack:
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\thread.py", line 92, in _worker
    work_item.run()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "P:\udb\Utils\commons.py", line 243, in wrapper
    return_status = func(*args, **kwargs)
  File "P:\udb\Utils\HLSDownloader.py", line 67, in _download_segment
    ts_file.write(self._get_stream_data(ts_url))
  File "P:\udb\Utils\BaseDownloader.py", line 69, in _get_stream_data
    response = self._get_raw_stream_data(url, stream)
  File "P:\udb\Utils\BaseDownloader.py", line 60, in _get_raw_stream_data
    response = self.req_session.get(url, stream=stream, timeout=self.request_timeout)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 746, in send
    r.content
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\models.py", line 902, in content
    self._content = b"".join(self.iter_content(CONTENT_CHUNK_SIZE)) or b""
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\models.py", line 820, in generate
    yield from self.raw.stream(chunk_size, decode_content=True)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\response.py", line 1040, in stream
    yield from self.read_chunked(amt, decode_content=decode_content)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\response.py", line 1172, in read_chunked
    with self._error_catcher():
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\contextlib.py", line 144, in __exit__
    next(self.gen)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\response.py", line 788, in _error_catcher
    self.release_conn()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\response.py", line 628, in release_conn
    self._pool._put_conn(self._connection)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 330, in _put_conn
    log.warning(
Message: 'Connection pool is full, discarding connection: %s. Connection pool size: %s'
Arguments: ('xqaw.v421c6e485f.site', 10)

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Jul 16, 2024

Failed with code: 404. Page not found for https://github.com/KillerDogeEmpire/vidplay-keys/blob/keys/keys.json
Failed with code: 404. Page not found for https://github.com/KillerDogeEmpire/vidplay-keys/pulls
Error occurred: 'NoneType' object has no attribute 'select'. Check log for more details.

His repository has been taken down

also superembed has still never worked

Episode: 01 | Stream link not found for source: vipstream
Episode: 02 | Stream link not found for source: vipstream

etc etc, on anything

@Prudhvi-pln
Copy link
Owner

Oops..! That's bad. I guess he made it private or removed it. Typical problems of open-source 😞.
I need to do some research for an alternative or create my own repo.

And.. for Superembed, I'm working on a logic to crack the captcha. Unfortunately, I couldn't spend time on this due to my work. Hopefully, I will push an update this weekend.

@Prudhvi-pln
Copy link
Owner

While I'm still trying to decode new vidsrc logic. I've fixed Superembed Client. Note: It is a beta release, so may not be 100% accurate

@rdwxth
Copy link

rdwxth commented Jul 24, 2024

@rdwxth
Copy link

rdwxth commented Jul 24, 2024

Superembed also uses a simple image based captcha which wants you to identify if the person is male of female, maybe this could help

https://huggingface.co/spaces/Yuqi/Gender_Classifier/tree/main

or you can reverse https://huggingface.co/spaces/bewil1990/GenderClassifier and use that

@Prudhvi-pln
Copy link
Owner

@rdwxth, thanks for the info. I'm working on vidsrc client using a similar reference: https://github.com/Inside4ndroid/vidsrc-api-js. and what you have shared is really helpful!

For Superembed, I'm planning to keep the repo simple without any ML model, so I used an external API. But the link you have shared is a good alternative.

Also, did you see any issues with current gender identification api used in Superembed Client ? If so, I'll incorporate the hugging face API (hoping there is no rate limit on it 😅)

@rdwxth
Copy link

rdwxth commented Jul 24, 2024

It would be better to scrape all the images of males and females and classify them. Then, we can solve the CAPTCHA using these classifications, since I've realized there is only a limited number of images and sometimes images repeat. As a fallback, we can use the ML model API, but scraping all the images and manually classifying them sounds like a better main option to me. It would be more work at the start but later on would be time saving and cost effective.

@Prudhvi-pln
Copy link
Owner

@ZetsApocNow, Restored support for Vidsrc client in v2.13.4.

@harry-odinson-369
Copy link

harry-odinson-369 commented Jul 27, 2024

Hi @Prudhvi-pln, it would be great if you could update this project with flask or fastapi.

@Prudhvi-pln
Copy link
Owner

Hi @harry-odinson-369 , thanks for using UDB. I'm more of a CLI guy, so I didn't plan to expose this over API. But I'll surely update you if I plan to add API support.

@Prudhvi-pln
Copy link
Owner

Closing this issue, assuming there are no other issues. In case of the same issue, please reopen this.

For any new issues / feature request, open a new one.

@ZetsApocNow
Copy link
Author

ZetsApocNow commented Aug 3, 2024

4 (tried with different shows also)

Available Episodes Details: Error occurred: HTTPSConnectionPool(host='vidsrc.to', port=443): Max retries exceeded with url: /embed/tv/1818 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000002157A9D68A0>: Failed to resolve 'vidsrc.to' ([Errno 11001] getaddrinfo failed)")). Check log for more details.

5

ERROR: Segment download failed [seg-895-v1-a1.jpg] due to: HTTPSConnectionPool(host='dsas.extensionsdns.net', port=443): Read timed out. (read timeout=30)

and in log for 4

2024-08-03 10:49:41,989 - INFO - <module>:390 - -------------------------------- NEW UDB INSTANCE v2.13.4 --------------------------------
2024-08-03 10:49:42,027 - INFO - <module>:392 - CLI options: Namespace(conf='config_udb.yaml', log_file='multi.log', version=False, series_type=None, series_name=None, series_year=None, seasons=None, episodes=None, resolution=None, start_download=False, disable_colors=False, hls_size_accuracy=0, update=False)
2024-08-03 10:49:42,028 - INFO - delete_old_logs:373 - Deleting log files older than 7 days and greater than 3...
2024-08-03 10:49:42,028 - INFO - delete_old_logs:380 - Found 0 files to delete!
2024-08-03 10:49:42,028 - INFO - delete_old_logs:392 - Deleted 0 files.
2024-08-03 10:49:45,966 - INFO - <module>:399 - Selected Series type: Movies & TV Shows (Vidsrc)
2024-08-03 10:49:47,241 - INFO - <module>:403 - Client: <Clients.VidSrcClient.VidSrcClient object at 0x0000021578FDF4A0>
2024-08-03 10:49:50,831 - INFO - search_and_select_series:107 - Searching with keyword: lexx
2024-08-03 10:49:53,071 - INFO - search_and_select_series:109 - Search Results Found
2024-08-03 10:49:55,547 - INFO - <module>:417 - Selected series: {'title': 'Lexx', 'link': 'https://www.themoviedb.org//tv/1818-lexx?language=en-US', 'show_id': '1818', 'year': '1997', 'type': 'tv', 'Status': 'Ended', 'Networks': 'Networks', 'Type': 'Scripted', 'Original Language': 'English', 'genre': 'Comedy, Drama, Sci-Fi & Fantasy', 'score': '69', 'seasons': 4, 'episodes': 61, 'episodes_per_season': {'Specials': '31', 'Season 1': '4', 'Season 2': '20', 'Season 3': '13', 'Season 4': '24'}}
2024-08-03 10:49:55,547 - INFO - <module>:420 - Fetching episodes list
2024-08-03 10:50:09,591 - ERROR - <module>:536 - Error occurred: HTTPSConnectionPool(host='vidsrc.to', port=443): Max retries exceeded with url: /embed/tv/1818 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000002157A9D68A0>: Failed to resolve 'vidsrc.to' ([Errno 11001] getaddrinfo failed)")). Check log for more details.
2024-08-03 10:50:09,782 - WARNING - <module>:537 - Stacktrace: Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\socket.py", line 963, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 491, in _make_request
    raise new_e
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connection.py", line 205, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x000002157A9D68A0>: Failed to resolve 'vidsrc.to' ([Errno 11001] getaddrinfo failed)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='vidsrc.to', port=443): Max retries exceeded with url: /embed/tv/1818 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000002157A9D68A0>: Failed to resolve 'vidsrc.to' ([Errno 11001] getaddrinfo failed)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "P:\udb\udb.py", line 422, in <module>
    episodes = client.fetch_episodes_list(target_series)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "P:\udb\Clients\VidSrcClient.py", line 111, in fetch_episodes_list
    soup = self._get_bsoup(list_episodes_url, silent=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "P:\udb\Clients\BaseClient.py", line 133, in _get_bsoup
    html_content = self._send_request(search_url, referer=referer, request_type=request_type, extra_headers=extra_headers, cookies=cookies, return_type='text', post_data=post_data, upload_data=upload_data, silent=silent)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "P:\udb\Utils\commons.py", line 254, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "P:\udb\Clients\BaseClient.py", line 97, in _send_request
    response = self.req_session.get(url, timeout=self.request_timeout, headers=header, cookies=cookies)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\theda\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='vidsrc.to', port=443): Max retries exceeded with url: /embed/tv/1818 (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000002157A9D68A0>: Failed to resolve 'vidsrc.to' ([Errno 11001] getaddrinfo failed)"))

@rdwxth
Copy link

rdwxth commented Aug 3, 2024

That's because the host vidsrc.to has been closed down rendering the scraper useless unless we can find an alternate domain with the automated API.

@Prudhvi-pln
Copy link
Owner

Yep. That's right. Vidsrc changed their domain and their backend logic tooo again...💢

@ZetsApocNow
For Superembed (5), it looks like a mere connection issue. HTTPSConnectionPool(host='dsas.extensionsdns.net', port=443): Read timed out. (read timeout=30)
You can try below:

  • change your network
  • increase the timeout value in your config file: request_timeout
  • just run udb to download again, as it has the capability to resume downloads 🙂

@rdwxth
Copy link

rdwxth commented Aug 3, 2024

Any idea what the new domain is?

@Prudhvi-pln
Copy link
Owner

Any idea what the new domain is?

It is changed to vidsrc.cc

@rdwxth
Copy link

rdwxth commented Aug 4, 2024

That isnt the actual vidsrc.to just a copy, they yeeted the ui and ye. They scrape vidsrc.me and rabbitstream btw and scraping them is relatively easy.

@ZetsApocNow
Copy link
Author

@ZetsApocNow For Superembed (5), it looks like a mere connection issue. HTTPSConnectionPool(host='dsas.extensionsdns.net', port=443): Read timed out. (read timeout=30)

increased to 60 then to 120

get variations of this spammed and it grinds to a halt, have to restart multiple times to get it to continue a few % at a time, eventually downloads don't pass 98/99%

ERROR: Segment download failed [seg-846-v1-a1.js] due to: ('Connection broken: IncompleteRead(81216 bytes read, 445560 more expected)', IncompleteRead(81216 bytes read, 445560 more expected))|

waited a long long time, never finishes, connection is good, no disconnects elsewhere

can't currently use the tool, was great while it lasted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants