v1 - Contact Us form not working on live site

[adamkendis]

Description

We need to fix the Contact Us form on the v1 website because it allows users to communicate with the team directly.

After filling in all form fields and clicking on the Submit Button, the action does not complete loading. It appears that the process is stuck in an infinite loop. I let for the loading for about 10 minutes with no change.

Action Items

• Investigate why the Contact Us form on 311-data.org (v1) is not working.
• Implement fix

Resources

We received a direct email from a researcher indicating the contact us form on the live site (311-data.org) didn't appear to be working.

Image of Submit button infinite loading:

311-data.org/contact

[mattyweb]

hey @adamkendis just saw this. you need something from me here?

[adamkendis]

Status: Blocked by work the past week. Getting CORS error on POST to /feedback but it also looks like the error isn't being properly handled in the client, causing the button to spin endlessly without any useful feedback for the user. Will be resolving the CORS error this Saturday, 10/2. Will also see if there's a low LOE improvement/fix for providing feedback in the UI on a failed Contact Us form submittal.

[ExperimentsInHonesty]

@adamkendis Please provide update

1. Progress
2. Blockers
3. Availability
4. ETA

[adamkendis]

Need to dig into AWS. Will look into this before next Thursday's meeting.

[adamkendis]

I haven't been able to resolve this yet. Thoughts on a band-aid fix by replacing the v1 form with "send us an email at [email protected]" or something along these lines?

[adamkendis]

Just had a thought: this might be failing due to a change in the Github API. Will investigate over the weekend.

[ExperimentsInHonesty]

@adamkendis Please provide update

1. Progress
2. Blockers
3. Availability
4. ETA

[ExperimentsInHonesty]

@adamkendis Please provide update

Progress
Blockers
Availability
ETA

[EchoProject]

@nichhk This is the issue with the contact form on the v1 site not working. I just tried testing again, and when the submit button is clicked, there is a circle-loading image that does not resolve. Fixing this is a priority because we have users that still use the v1. Let me know if you have any questions.

From previous comments:
"Getting CORS error on POST to /feedback but it also looks like the error isn't being properly handled in the client, causing the button to spin endlessly without any useful feedback for the user. Will be resolving the CORS error this Saturday, 10/2. Will also see if there's a low LOE improvement/fix for providing feedback in the UI on a failed Contact Us form submittal."

"this might be failing due to a change in the Github API."

[nichhk]

I repro'ed this on the prod site, and the server logs showed a NoneTypeError here. I think it's saying that ISSUES_URL is not being provided. This makes sense--it looks like the ".env" config file is not being properly created in Continuous_Deployment_Backend_Prod.yml--I believe we'd need to copy ".env.example" to ".env" first. Right now, it seems like we're just creating an ".env" file that contains only the GITHUB_SHA.

I've tried to set up the client and server locally, but I'm having several issues with dependency mismatches. I'll keep working through those. Also, I'm running everything from the master branch; I'm assuming that the prod client/server are ran from master.

The CORS error still exists. We should just be able to specify that CORS is okay, somewhere. I'll look into that after I can get a repro running locally.

[nichhk]

I was able to bring up the master versions of the client and the server, after resolving a dependency version issue (see PR). The Contact Us functionality works as expected--the UI shows an error message saying that it wasn't able to send the message, which is expected because the GITHUB_SHA is not real.

This makes me question whether the prod server is actually running using master code. Does anyone know whether the prod server is running from master or dev?

[nichhk]

I'm still not sure, but it seems possible that the prod server is running off dev, based on the GitHub action create_release_dev.yml: you can see that we are buillding/pushing new docker images based off the dev branch, and restarting the prod ECS tasks. I'm not sure why this set up--making the prod client live in master, and making the prod server live in dev--but it's rather unintuitive to me. I think we'll need to talk to Matty, or have more access to the AWS account, to really be sure though.

I think the hypothesis that the ".env" file is not being created properly (from the earlier comment) still applies here though; create_release_dev.yml does not copy the ".env.example" to ".env". I can try testing this locally.

[nichhk]

The prod server is indeed running off the 'dev' branch, AFAICT. The config for the AWS ECS task prod-la-311-data-server shows that we are executing a gunicorn command to bring up the server, and gunicorn is only used in 'dev', not 'master'. This is very confusing, TBH--we should have merged the 'dev' server changes into 'master' if we are using it for the prod server. I imagine there are a lot of other config and GitHub Action changes we'd need to merge into 'master' as well. Performing this merge will require more comprehensive understanding of the system, so I'll hold off on that.

Besides that, I'm not actually sure how the prod server was deployed. I see that we have a 'create_release_dev.yml' Action that will build and push a new server API image to Docker Hub. Then, the ECS service for dev-la-311-data-svc is updated, presumably using that new server API image on Docker Hub. But I don't see any analog of 'create_release_dev.yml' for prod. Maybe someone performed the deployment based on local changes. So I will work on creating a 'create_release_prod.yml'; it will be very similar to 'create_release_dev.yml', but I'll make sure to copy the .env file.

Before merging 'create_release_prod.yml', I'll need to manually test the interaction between the 'master' client and the 'dev' server while submitting feedback.

[mattyweb]

Hey, I'm the last person to release the API in production so I can hopefully clarify things a bit. The last API release to production as far as I know was a year ago in January from the release/v1.5.0 branch.

That was branched from dev since we've got a monorepo and the frontend code was getting repeatedly redesigned. There's some shims in the 1.5 API that let it work with the 1.0 frontend. There have been releases to the report server and the prefect image since then but I don't think the API image has changed much. Maybe some configs were modified. You can look at the images on Docker Hub for the exact dates.

The dev code (which was intended to be the 2.0 version) hasn't been updated in more than a year so there are undoubtably some dependencies that need to be updated. That said, the critical path is going to be getting the frontend working on the Dev site. If y'all are able to get the Dev site working I'm happy to find some time to help deploy the backend to Prod when you're ready.

[nichhk]

Thanks for chiming in Matthew! I'm new to this project, but I believe we want to fix the feedback mechanism on v1 before we fix the dev site--we still have users on v1, and we want to make sure to collect their feedback. In any case, I don't think one depends on the other--e.g., fixing the dev site won't fix the feedback mechanism on v1.

Good to learn about v1.5.0. I took a look, and I don't see something like 'create_release_prod.yml' in the github workflows directory. Would that be the correct way to deploy a new API version to prod? Also, do you have any opinion on copying .env.example to .env in the workflow?

[mattyweb]

The create_release_dev workflow creates and publishes a new Docker image for the API from the dev branch then refreshes the ECS cluster. The production API is just using the dev Docker image. The actual task definition for the cluster are all in Terraform.

I think I deployed using some version of the last line of the action like this (it was more than a year ago though so I forget):

aws ecs update-service --cluster prod-la-311-data-cluster --service dev-la-311-data-svc --force-new-deployment

Anyway, there really won't be a need to create a new GHA until you want to deploy from a different branch (ie. once everything is on master).

Separately, not sure who is prioritizing tickets but fixing the v1 feedback feels like a waste of time to me. First, I'd be surprised if anyone visiting the site wasn't working on the project. Second, the v1 feedback was kind of a hack--it creates a GitHub issue. I think GitHub changed the API for that at some point and that it would need to swapped out. That all predated me so I couldn't say what that would entail (it was done 2+ years ago).

[nichhk]

Thanks for the clarification regarding the API versions.

I've been able to get the v1 feedback mechanism to mostly work locally. It fails on the last step, which is sending an email to the user using SendGrid, and it's failing because the SendGrid key isn't provided (it's not on the 1pass, so I don't know what it is). But it is able to make an issue and add it to the project.

All I had to do was update the '.env' file to have the right GH keys. I imagine it will work completely fine if we have the SendGrid key. So I guess I just need to update 'create_release_dev.yml' to echo more keys to '.env', and we should be good. In any case, we still need to fix the v1 frontend to handle errors gracefully. I'm looking into that now.

[nichhk]

I do feel like a separate GHA action would be helpful here. I think we need to have different .env configurations for prod and dev, at the very least for API_ALLOWED_ORIGINS. And since the .env is built into each Docker image, we'd need to have separate Docker images for prod and dev.

[nichhk]

So the deployments worked as expected, but once the new API images were live, CORS was not working on either v1 or v2. Apparently this is a common bug with FastAPI, with many people commenting that it doesn't work as expected. I submitted several PRs trying to fix the CORS issues. The two that mattered:

1. Make middleware factory #1215: For whatever reason, we can't use app.add_middleware--we need to create the App first, and then wrap it in each Middleware we'd like to use.
2. add www origin #1216: Once that was checked in, I saw that CORS error was saying that https://www.311-data.org was not an allowed origin, so I added that.

So now Contact Us is working on v1! As mentioned in #1206, I think that this only fixes the happy path (i.e., if any of the steps fail, the UI will still show a spinning submit button). But I think fixing the unhappy path is a low priority right now.