Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove logic for guessing slug from an unregistered domain #4730

Closed

Conversation

stsewd
Copy link
Member

@stsewd stsewd commented Oct 4, 2018

Close #1991

self.middleware.process_request(request)
self.assertEqual(request.urlconf, self.urlconf_subdomain)
self.assertEqual(request.cname, True)
self.assertEqual(request.rtdheader, True)
self.assertEqual(request.slug, 'pip')

@override_settings(USE_SUBDOMAIN=True)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already put this on the class

@stsewd stsewd changed the title Remove logic for guessing slug from a unregistered domain Remove logic for guessing slug from an unregistered domain Oct 4, 2018
@stsewd stsewd mentioned this pull request Oct 4, 2018
@ericholscher
Copy link
Member

@stsewd I'm a bit worried this might break our users CNAME's if they aren't properly configured. Can we put together a PR that upgrades the debug statement to a warning, so we can check in our logs to see if this code is actively being used?

@ericholscher
Copy link
Member

(It can be another PR, so we can merge this when we're ready)

@stsewd
Copy link
Member Author

stsewd commented Oct 16, 2018

I opened #4769

@stsewd stsewd closed this Oct 22, 2018
@stsewd stsewd reopened this Oct 22, 2018
@stsewd
Copy link
Member Author

stsewd commented Oct 22, 2018

Looks like travis is still busy p:

@codecov
Copy link

codecov bot commented Oct 22, 2018

Codecov Report

Merging #4730 into master will decrease coverage by 0.55%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4730      +/-   ##
==========================================
- Coverage   76.85%   76.29%   -0.56%     
==========================================
  Files         158      158              
  Lines        9989    10007      +18     
  Branches     1254     1262       +8     
==========================================
- Hits         7677     7635      -42     
- Misses       1978     2029      +51     
- Partials      334      343       +9
Impacted Files Coverage Δ
readthedocs/core/utils/__init__.py 77.77% <ø> (-2.64%) ⬇️
readthedocs/core/middleware.py 88.99% <100%> (+4.43%) ⬆️
readthedocs/vcs_support/backends/svn.py 27.69% <0%> (-11.71%) ⬇️
readthedocs/builds/forms.py 87.87% <0%> (-7.96%) ⬇️
readthedocs/restapi/client.py 82.35% <0%> (-5.53%) ⬇️
readthedocs/vcs_support/backends/hg.py 59.09% <0%> (-5.43%) ⬇️
readthedocs/builds/views.py 85.71% <0%> (-5.03%) ⬇️
readthedocs/restapi/views/integrations.py 87.23% <0%> (-4.72%) ⬇️
readthedocs/projects/utils.py 54.54% <0%> (-4.55%) ⬇️
... and 44 more

Copy link
Member

@humitos humitos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes are good!

@humitos
Copy link
Member

humitos commented Oct 23, 2018

@stsewd I'm a bit worried this might break our users CNAME's if they aren't properly configured

I agree with @ericholscher that we need to do something with these people before deploying this.

It's fine to have a good documentation saying how users should configure their domains, but we know there are many people that didn't follow those instructions properly (from the Cloudflare feature) and now we should probably create their Domain objects for their projects or at least warn these users.

On the other hand, I'm personally interested in merging this PR because it removes the cname_to_slug call which is producing a lot of SPAM in Sentry (https://github.com/rtfd/readthedocs-corporate/issues/410) and killing our quota. We may attack that in a different PR, though.

I'm marking this PR as blocked because of this.

@humitos humitos added the Status: blocked Issue is blocked on another issue label Oct 23, 2018
@humitos
Copy link
Member

humitos commented Oct 23, 2018

We were talking about it would be good to make a plan to deprecate this:

  1. Create a feature flag that it's enabled only for existing projects (not new ones)
  2. If the middleware finds that they are CNAMEing us without a Domain object created, we automatically create a new one for them.

So, in the near future, we will have all these projects properly configured and also the new ones, because our new docs are correct but also because they will fail otherwise.

At the moment, after the deploy, Eric already found at least one project with this issue.

@stsewd
Copy link
Member Author

stsewd commented Oct 23, 2018

Looks like codecov isn't happy with the coverage from the diff 😢 I'll need to figure out how to increase the coverage here...

@agjohnson
Copy link
Contributor

We hit issues with producing Domain objects for requests for projects without a Domain setup in the past. It lead to projects overtaking known slugs. For example, I think the case we hit was a well known project didn't have a Domain, and some rando domain was resolving to the project. We then assigned the rando Domain to the project as canonical.

👍 on adding a feature flag with migration that changes the behavior going forward. I'm 👎 on adding magic logic to continue adding new domains for domain-less projects. New projects should have a domain set up correctly, so I'm not worried about these projects being misconfigured. I do think we should get away from using magic to set up projects that are not configured well. Explicit is better than magic from a code maintenance perspective.

@agjohnson agjohnson added this to the 2.9 milestone Nov 10, 2018
@humitos
Copy link
Member

humitos commented Jan 17, 2019

If I understand correctly, the actionable part of this PR is this #4730 (comment) together with #4730 (comment). Can we unblock it?

@stsewd
Copy link
Member Author

stsewd commented Jan 17, 2019

We should have some logs from this actually #4769

@humitos
Copy link
Member

humitos commented Jan 17, 2019

A quick update here:

  • 33 projects with CNAME detected
  • 49 projects with CNAME cached
  • 272 CNAME 404

@stsewd
Copy link
Member Author

stsewd commented Jan 17, 2019

49 projects with CNAME cached

Those are new projects

33 projects with CNAME detected

Should we take manual action?

  • Create the domain for them?
  • Send them an email?

@ericholscher
Copy link
Member

What timeframe are those stats for? I think if it's <100, we can probably just create them manually. I do think we'll still end up breaking some subset of users, but if we use the last week or so of queries, that should get everything in real use.

@humitos
Copy link
Member

humitos commented Jan 21, 2019

Those numbers come from parsing logs from web02. We save logs for 10 days only.

@ericholscher
Copy link
Member

So that's from the last 10 days on web02?

@humitos
Copy link
Member

humitos commented Jan 21, 2019

Yes

@stsewd
Copy link
Member Author

stsewd commented Jan 22, 2019

I messed up trying to solve the conflicts (wrong option in the merge command). I opened #5143. Closing here.

@stsewd stsewd closed this Jan 22, 2019
@stsewd stsewd deleted the remove-cname-from-unregister-domain branch February 27, 2019 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: blocked Issue is blocked on another issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants