Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tgcrypto & pyaesni for faster work #3778

Closed
wants to merge 7 commits into from

Conversation

BlackCatDevel0per
Copy link

Hello, I am coding my telegram bot for downloading videos from YouTube and Instagram and i encountered with slow video uploads, later I looked for an analogue of cryptg for faster work and i found tgcrypto and pyaesni (does not work on all cpu) and after files uploading a little faster. Can you please add support for these libraries?

@Lonami
Copy link
Member

Lonami commented Mar 23, 2022

Thanks for the PR. tgcrypto was removed after some "events" with the library author, and I do not plan on adding it back. There's a different library which does just that and is external to Telethon (can be used if needed, I do not remember it). cryptg, which is currently the supported one, also uses AES-NI if available, so I don't think there's a need for pyaesni. I just need to finish publishing / the CI job for cryptg.

In short, cryptg is the one that needs fixing (CI and publish), not support for more libraries.

@Lonami Lonami closed this Mar 23, 2022
@BlackCatDevel0per
Copy link
Author

I also thought to improve cryptg, but I don't have enough knowledge in Rust and cryptography for this, but thanks for the answer!

@Lonami
Copy link
Member

Lonami commented Mar 24, 2022

The only thing holding me back right now is my lack of GitHub Actions knowledge. One can help forking and playing around with CI until it works (not much Rust knowledge needed).

@BlackCatDevel0per
Copy link
Author

BlackCatDevel0per commented Mar 25, 2022

I tried to build with cibuildwheel, but it didn't work. I made a simple workflow, but it doesn't build the manylinux wheels that PyPi needs. I haven't worked much with CI either (mostly I had no experience with cross-building)

I hope this helps)

wheels build:

name: wheels-def

on:
  push:
    tags:
      - 'v*'

jobs:
  build_wheels:
    name: Build wheels on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-20.04, windows-2019, macos-10.15]

    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2

      - name: Build wheels
        run: python -m pip install -r requirements.txt && python setup.py bdist_wheel
      
      - uses: actions/upload-artifact@v2
        with:
          path: ./dist/*.whl

  build_sdist:
    name: Build source distribution
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
      - name: Build sdist
        run: python -m pip install -r requirements.txt && python setup.py sdist
      - uses: actions/upload-artifact@v2
        with:
          path: dist/*.tar.gz

  upload_pypi:
      needs: [build_wheels, build_sdist]
      runs-on: ubuntu-20.04
      steps:
        - uses: actions/download-artifact@v2
          with:
            name: artifact
            path: dist

        - uses: pypa/[email protected]
          with:
            user: __token__
            password: ${{ secrets.pypi_token }}
            repository_url: https://test.pypi.org/legacy/

cibuildwheel (error from yum)

  http://mirror.es.its.nyu.edu/epel/7/x86_64/repodata/repomd.xml: [Errno 12] Timeout on http://mirror.es.its.nyu.edu/epel/7/x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds')

@Lonami
Copy link
Member

Lonami commented Mar 26, 2022

I took a look at your repository and this is the error of the above script:

cibuildwheel 2.x no longer supports Python < 3.6. Please use the 1.x series or update CIBW_SKIP

The error of the source build baffles me though:

Run python setup.py sdist
Traceback (most recent call last):
  File "setup.py", line 61, in <module>
    main(sys.argv)
  File "setup.py", line 56, in main
    zip_safe=False,
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/__init__.py", line 76, in setup
    _install_setup_requires(attrs)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/__init__.py", line 65, in _install_setup_requires
    dist = MinimalDistribution(attrs)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/__init__.py", line 55, in __init__
    super().__init__(filtered)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/dist.py", line 460, in __init__
    for ep in metadata.entry_points(group='distutils.setup_keywords'):
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 999, in entry_points
    return SelectableGroups.load(eps).select(**params)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 449, in load
    ordered = sorted(eps, key=by_group)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 997, in <genexpr>
    dist.entry_points for dist in unique(distributions())
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/_itertools.py", line 16, in unique_everseen
    k = key(element)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 931, in _normalized_name
    return self._name_from_stem(stem) or super()._normalized_name
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 600, in _normalized_name
    return Prepared.normalize(self.name)
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 855, in normalize
    return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/re.py", line 194, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

@Lonami
Copy link
Member

Lonami commented Mar 26, 2022

It can't be this StackOverflow question, python setup.py sdist bdist_wheel raises TypeError: expected string or bytes-like object, because our code doesn't do that:

        packages=find_packages(),

Perhaps Twine is out of date somehow? Not sure...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants