Filter out irrelevant feed URLs

[Eakam1007]

Issue This PR Addresses

Resolves #3688

Type of Change

• Bugfix: Change which fixes an issue
• New Feature: Change which adds functionality
• Documentation Update: Change which improves documentation
• UI: Change which improves UI

Description

The feed discovery services can return feed URLs that are not relevant. One such example is the wordpress.com's comments feed: https://blog.wordpress.com/comments/feed.
We want to filter out the irrelevant URLs for four hosts - dev.to, medium.com, wordpress.com, blogspot.com.
This PR adds filters to remove the following:

• wordpress comments feed
• wordpress json oembed feed - Wordpress already returns an rss+xml feed url. So, we skip the json+oembed feed url
• blogspot.com service.post url - There are two other feed urls for atom+xml and rss+xml which are returned

Steps to test the PR

• Ensure you have login component running locally
• Go through the sign up process until you reach the blog URL form
• Test different blog URLs from different hosts and check if the feed URLs mentioned above are filtered out

Checklist

• Quality: This PR builds and passes our npm test and works locally
• Tests: This PR includes thorough tests or an explanation of why it does not
• Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not (if applicable)
• Documentation: This PR includes updated/added documentation to user exposed functionality or configuration variables are added/changed or an explanation of why it does not(if applicable)

[manekenpix]

LGTM 👍

Blogspot URLs still return an extra feedUrl that I'm not sure if we need.
Maybe cc @humphd can clarify this.

• Wordpress:
  Before:

{
    "feedUrls": [
        {
            "feedUrl": "https://neilong31.wordpress.com/feed/",
            "type": "blog"
        },
        {
            "feedUrl": "https://neilong31.wordpress.com/comments/feed/",
            "type": "blog"
        },
        {
            "feedUrl": "https://public-api.wordpress.com/oembed/?format=json&url=https%3A%2F%2Fneilong31.wordpress.com%2F&for=wpcom-auto-discovery",
            "type": "blog"
        },
        {
            "feedUrl": "https://public-api.wordpress.com/oembed/?format=xml&url=https%3A%2F%2Fneilong31.wordpress.com%2F&for=wpcom-auto-discovery",
            "type": "blog"
        }
    ]
}

After

{
    "feedUrls": [
        {
            "feedUrl": "https://neilong31.wordpress.com/feed/",
            "type": "blog"
        }
    ]
}

• Blogspot
  Before

{
    "feedUrls": [
        {
            "feedUrl": "https://whataboutopensource.blogspot.com/feeds/posts/default",
            "type": "blog"
        },
        {
            "feedUrl": "https://whataboutopensource.blogspot.com/feeds/posts/default?alt=rss",
            "type": "blog"
        },
        {
            "feedUrl": "https://www.blogger.com/feeds/6561714874239648778/posts/default",
            "type": "blog"
        }
    ]
}

After

{
    "feedUrls": [
        {
            "feedUrl": "https://whataboutopensource.blogspot.com/feeds/posts/default",
            "type": "blog"
        },
        {
            "feedUrl": "https://whataboutopensource.blogspot.com/feeds/posts/default?alt=rss",
            "type": "blog"
        }
    ]
}

[humphd]

So it's giving both Atom and RSS versions of the feed. I guess we could filter out the ?alt=rss version if there are two?

[Eakam1007]

Yea, I ended up leaving the atom or rss versions in. I can add another filter to remove the ?alt=rss version

[Eakam1007]

I have added a filter to skip the ?alt=rss feed url

[humphd]

@manekenpix let me know if you want to review this, otherwise I'll merge this week.