Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add regex to url before scraping #4174

Merged

Conversation

jlssmt
Copy link
Contributor

@jlssmt jlssmt commented Sep 8, 2024

What type of PR is this?

  • bug
  • feature

What this PR does / why we need it:

every url string will be formatted before imported
chefkoch.de app for example generates this kind of string when pressing the "share dish" button:

Hunger? Rezept auf Chefkoch.de:
Kürbis-Linsen-Lasagne

vegetarisch, Resteverwertung

https://www.chefkoch.de/rezepte/3104901463397655/Kuerbis-Linsen-Lasagne.html?utm_medium=sharing&utm_source=chefkoch_android_app&utm_campaign=sharing_rds_ck_android

yes. it generates the whole multiline string.
amazon app also adds text before the url when sharing...
so i bet there are other recipe apps also doing this.

Which issue(s) this PR fixes:

the issue I have in my brain, but not on github ;-)

Testing

manual

@jlssmt jlssmt force-pushed the add-regex-to-url-before-scraping branch 5 times, most recently from d6794b7 to bdd6200 Compare September 12, 2024 14:55
@jlssmt jlssmt changed the title add regex to url before scraping feat: add regex to url before scraping Sep 12, 2024
@Kuchenpirat
Copy link
Collaborator

Hey @jlssmt thanks for your work on this so far.

There seems to be a small linting error:

  • mealie/services/scraper/scraper.py:1:1: I001 [*] Import block is un-sorted or un-formatted

Please fix that before we can continue with a proper review.

@jlssmt jlssmt force-pushed the add-regex-to-url-before-scraping branch from 973edbf to 9dca4f3 Compare September 24, 2024 13:41
@jlssmt
Copy link
Contributor Author

jlssmt commented Sep 24, 2024

Hey @Kuchenpirat
I'm not a Python developer and I didn't figure out how to setup the dev environment (to run the tests locally).
But maybe it's fixed now.

@jlssmt jlssmt force-pushed the add-regex-to-url-before-scraping branch from 9dca4f3 to 4e134e5 Compare September 24, 2024 15:58
Copy link
Collaborator

@Kuchenpirat Kuchenpirat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, sorry it took me so long to come back to this. And thanks for your contribution!🚀👍Always nice to see new faces around.
This will certainly help make scraping more seamless.

@Kuchenpirat Kuchenpirat merged commit 38502e8 into mealie-recipes:mealie-next Sep 27, 2024
13 checks passed
boc-the-git pushed a commit to boc-the-git/mealie that referenced this pull request Sep 28, 2024
boc-the-git pushed a commit to boc-the-git/mealie that referenced this pull request Sep 28, 2024
Choromanski pushed a commit to Choromanski/mealie that referenced this pull request Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants