Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/downloading-youtube-videos #169

Conversation

Shak2000
Copy link
Contributor

Link to Relevant Issue

This pull request resolves #161

Description of Changes

Make it possible for the program to download a YouTube video embedded onto a website. This is needed for the Portland scraper.

@codecov
Copy link

codecov bot commented Feb 21, 2022

Codecov Report

Merging #169 (d007270) into main (290d573) will decrease coverage by 0.01%.
The diff coverage is 96.42%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #169      +/-   ##
==========================================
- Coverage   94.56%   94.54%   -0.02%     
==========================================
  Files          50       50              
  Lines        2558     2587      +29     
==========================================
+ Hits         2419     2446      +27     
- Misses        139      141       +2     
Impacted Files Coverage Δ
cdp_backend/utils/file_utils.py 93.95% <92.85%> (+0.57%) ⬆️
cdp_backend/tests/conftest.py 100.00% <100.00%> (ø)
cdp_backend/tests/utils/test_file_utils.py 100.00% <100.00%> (ø)
cdp_backend/utils/string_utils.py 81.39% <0.00%> (-3.98%) ⬇️
cdp_backend/database/validators.py 84.48% <0.00%> (ø)
cdp_backend/pipeline/event_index_pipeline.py 85.71% <0.00%> (ø)
cdp_backend/tests/utils/test_string_utils.py 100.00% <0.00%> (ø)
cdp_backend/tests/database/test_validators.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 290d573...d007270. Read the comment docs.

Copy link
Collaborator

@isaacna isaacna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! Just had a few comments about naming the file and accepting more url formats

@evamaxfield evamaxfield changed the title Feature/downloading youtube videos feature/downloading-youtube-videos Feb 22, 2022
cdp_backend/tests/utils/test_file_utils.py Show resolved Hide resolved
cdp_backend/utils/file_utils.py Outdated Show resolved Hide resolved
cdp_backend/utils/file_utils.py Outdated Show resolved Hide resolved
cdp_backend/utils/file_utils.py Outdated Show resolved Hide resolved
cdp_backend/utils/file_utils.py Outdated Show resolved Hide resolved
Copy link
Member

@evamaxfield evamaxfield left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor nitpick but everything else looks perfect. Thanks for making the changes!

cdp_backend/utils/file_utils.py Outdated Show resolved Hide resolved
dst: str
The location of the downloaded file.
"""
dst = Path(str(dst) + ".mp4")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was about to merge but one last look I think catches this.

This will always overwrite the supplied dst correct?

I think there simply needs to be an if dst is not None before this statement.

Additionally the docstring / parameter typing should be updated to dst: Optional[Path] = None

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I assumed that dst is not empty since if dst is None, it would be a assigned a value, as seen here. I can throw an exception if dst is None though. WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohhhh i see what you are doing. Yes. My mistake. Okay so then only nitpick would be instead of doing this str(dst) + ".mp4" string addition. Path has a function called with_suffix that is "safer" imo.

Path(str(dst)).with_suffix(".mp4")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But yes, you are correct. My mistake on on the overwrite vs simply adding the mp4

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed a new change based on your feedback.

@evamaxfield evamaxfield merged commit 9f1ede6 into CouncilDataProject:main Mar 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a function for downloading youtube videos
3 participants