Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YouTube] Refactor JavaScript usage and fix extraction of obfuscated signature deobfuscation function #1108

Merged
merged 7 commits into from
Sep 22, 2023
Merged

[YouTube] Refactor JavaScript usage and fix extraction of obfuscated signature deobfuscation function #1108

merged 7 commits into from
Sep 22, 2023

Commits on Sep 21, 2023

  1. [YouTube] Add utility class around signatures and fix signature deobf… 

    …uscation function extraction

The goal of this class is to decouple the extraction of signature timestamp and
signature deobfuscation function from YoutubeStreamExtractor.

The extraction of the signature deobfuscation function has been also adapted to
support the latest YouTube player versions.

This new class, YoutubeSignatureUtils, doens't store anything temporary such as
a copy of the player code, which has to be passed where required. It is not
public, as it will be used by a JavaScript player manager class in the future,
in order to handle in a better way fetching, caching and resetting cache of the
player code.
    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    6884d19 View commit details
    Browse the repository at this point in the history

  2. [YouTube] Refactor JavaScript player management API 

    This commit is introducing breaking changes.

For clients, everything is managed in a new class called
YoutubeJavaScriptPlayerManager:
- caching JavaScript base player code and its extracted code (functions and
variables);
- getting player signature timestamp;
- getting deobfuscated signatures of streaming URLs;
- getting streaming URLs with a throttling parameter deobfuscated, if
applicable.

The class delegates the extraction parts to external package-private classes:
- YoutubeJavaScriptExtractor, to extract and download YouTube's JavaScript base
player code: it always already present before and has been edited to mainly
remove the previous caching system and made it package-private;
- YoutubeSignatureUtils, for player signature timestamp and signature
deobfuscation function of streaming URLs, added in a recent commit;
- YoutubeThrottlingParameterUtils, which was originally
YoutubeThrottlingDecrypter, for throttling parameter of streaming URLs
deobfuscation function and checking whether this parameter is in a streaming
URL.

YoutubeJavaScriptPlayerManager caches and then runs the extracted code if it
has been executed successfully. The cache system of throttling parameters
deobfuscated values has been kept, its size can be get using the
getThrottlingParametersCacheSize method and can be cleared independently using
the clearThrottlingParametersCache method.

If an exception occurs during the extraction or the parsing of a function
property which is not related to JavaScript base player code fetching, it is
stored until caches are cleared, making subsequent failing extraction calls of
the requested function or property faster and consuming less resources, as the
result should be the same until the base player code changes.

All caches can be reset using the clearAllCaches method of
YoutubeJavaScriptPlayerManager.

Classes using JavaScript base player code and utilities directly (in the code
and its tests) have been also updated in this commit.
    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7de3753 View commit details
    Browse the repository at this point in the history

  3. [YouTube] Convert signature timestamp to integer 

    The signature timestamp is used as a number by HTML5 clients, so it should be
used in the same way by the extractor too instead of being a string.

As the timestamp doesn't seem to exceed 5 digits, an integer is used to store
its value.
    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    a04bc32 View commit details
    Browse the repository at this point in the history

  4. [YouTube] Add tests for signature timestamp extraction and signature … 

    …deobfuscation function extraction and execution
    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    1fa85ec View commit details
    Browse the repository at this point in the history

  5. [YouTube] Quote signature deobfuscation function name and add semicol… 

    …on only where needed
    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    588c6a8 View commit details
    Browse the repository at this point in the history

  6. [YouTube] Catch any exception when extracting something from JavaScri… 

    …pt's base player
    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    714b141 View commit details
    Browse the repository at this point in the history

  7. [YouTube] Update stream mocks

    @AudricV
    AudricV committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    6ed2209 View commit details
    Browse the repository at this point in the history