From a990c3dc593fc2bc94f41ada5f73c46efb60d61e Mon Sep 17 00:00:00 2001 From: James Kunstle Date: Wed, 17 Apr 2024 16:46:07 -0400 Subject: [PATCH 1/6] points the postgres pod to the correct pvc Signed-off-by: James Kunstle --- openshift/base/8k-postgres-cache.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openshift/base/8k-postgres-cache.yaml b/openshift/base/8k-postgres-cache.yaml index fb3aa312..60653c4b 100644 --- a/openshift/base/8k-postgres-cache.yaml +++ b/openshift/base/8k-postgres-cache.yaml @@ -26,7 +26,7 @@ spec: ports: - containerPort: 5432 volumeMounts: - - mountPath: /var/lib/postgresql/data + - mountPath: /var/lib/pgql/data name: postgres-cache-data envFrom: - secretRef: From 0aa4baa0ff74101925de7637d0a347eeae6f3fb8 Mon Sep 17 00:00:00 2001 From: James Kunstle Date: Mon, 22 Apr 2024 10:58:18 -0400 Subject: [PATCH 2/6] adds docs about oauth implementation in 8Knot Signed-off-by: James Kunstle --- .wordlist-md | 10 +++++++++ README.md | 2 ++ docs/user-accounts-in-8knot.md | 38 ++++++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) create mode 100644 docs/user-accounts-in-8knot.md diff --git a/.wordlist-md b/.wordlist-md index 2c407053..41d0423f 100644 --- a/.wordlist-md +++ b/.wordlist-md @@ -80,3 +80,13 @@ filesystem Podman credsStore credStore +codebase +oauth +FlaskLogin +JSON +UI +UUID +backend +href's +TLS +OAuth's diff --git a/README.md b/README.md index 0102af7d..3e54e6af 100644 --- a/README.md +++ b/README.md @@ -204,6 +204,8 @@ you're using. Note: You'll have to manually fill in the \ in the AUGUR_USER_AUTH_ENDPOINT environment variable. In-depth instructions for enabling 8Knot + Augur integration is available in [AUGUR_LOGIN.md](docs/AUGUR_LOGIN.md). +An overview of OAuth's implementation in 8Knot can be found here: [user-accounts-in-8knot.md](docs/user-accounts-in-8knot.md). + ### Runtime diff --git a/docs/user-accounts-in-8knot.md b/docs/user-accounts-in-8knot.md new file mode 100644 index 00000000..0b248b1b --- /dev/null +++ b/docs/user-accounts-in-8knot.md @@ -0,0 +1,38 @@ +# How do user accounts work in 8Knot (OAuth and session handling) + +8Knot has user accounts- there are log in / log out buttons, and user groups are persist across logins. +However, 8Knot doesn't directly handle these accounts. Instead, Augur manages the user's accounts and 8Knot connects as an OAuth client. + +This document describes how this flow works, what parts of the codebase are relevant, and where there are currently (4/22/24) gaps in the implementation. + +## Notes +- `access_token` and `bearer_token` are the same thing + +## How the flow works + +Let's assume that we've got a functional 8Knot instance running that's been configured as an oauth client for Augur. We've documented how to connect an application to the Augur frontend as an oauth client (specifically, you can find it on the 8Knot welcome page) but let's dig deeper into how this works on the application server. + +Assume: +1. Your application is configured with a valid `application_id` and `client_secret`. e.g. login, refresh, and manage groups all work as expected. + +Then the following happens when the user clicks `Augur Log in / Sign up` in the UI: +1. The frontend application href's to `http:///login/`. +2. `/login/` is a Flask route that the backend serves. It's defined in the file `8Knot/8Knot/_login.py`. +3. The backend route gets the OAuth provider route from the environment (`OAUTH_AUTHORIZE_URL`) and redirects to that host with the URL format: `http:///?client_id=&response_type=code/`. +4. The user should be routed to the Augur frontend where they can log in and authorize 8Knot to use their account. +5. When the user authorizes 8Knot's use of their account, Augur will redirect them back to the registered 8Knot application route `http:///authorize/?code=`. +6. The 8Knot backend `/authorize/` route receives this code as a query parameter in the URL and uses it, along with the application's client secret, to post a request to the `OAUTH_TOKEN_URL` intending to receive a `bearer token` and `refresh token` from the Augur frontend. +7. The Augur frontend, if all values are acceptable, returns a `bearer token`, a `refresh token`, a `token expiration` and a `username`. +8. 8Knot creates a random `id_number = str(uuid.uuid1())` for the user and stores a JSON payload `{username, access_token, refresh_token, expiration}` in Redis under that UUID. +9. Finally, `login_user(User(id_number))` is called, setting an HTTP-only `session` cookie in the client's browser that will be used by FlaskLogin to handle the user's session in the future. + +## Topics out of scope for this document: + +- oauth2.0 flow implementation details: [an overview](https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2) +- Flask Login: [docs](https://flask-login.readthedocs.io/en/latest/#flask_login.login_user) + +## Current implementation gaps: + +1. The use of the user's `access_token` is zero-shot. If authentication fails, it fails silently without notifying the user that they should log in again. +2. We don't currently user the refresh token even though it's available. +3. Session cookies are `HTTP-only` but not `Secure` (require TLS) in general, but this should only be the case in development. From 02beda3bd23d398c66eecbb662a7120a887aabc7 Mon Sep 17 00:00:00 2001 From: cdolfi Date: Thu, 9 May 2024 17:12:18 -0400 Subject: [PATCH 3/6] graph button fix --- .../pages/contributions/visualizations/pr_first_response.py | 6 +----- .../contributions/visualizations/pr_review_response.py | 6 +----- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/8Knot/pages/contributions/visualizations/pr_first_response.py b/8Knot/pages/contributions/visualizations/pr_first_response.py index d1ba712a..cc6b8e12 100644 --- a/8Knot/pages/contributions/visualizations/pr_first_response.py +++ b/8Knot/pages/contributions/visualizations/pr_first_response.py @@ -64,12 +64,9 @@ step=1, value=2, size="sm", + style={"width": "100px"}, ), className="me-2", - width=2, - ), - dbc.Col( - width=6, ), dbc.Col( dbc.Button( @@ -83,7 +80,6 @@ ), ], align="center", - justify="between", ), ] ), diff --git a/8Knot/pages/contributions/visualizations/pr_review_response.py b/8Knot/pages/contributions/visualizations/pr_review_response.py index 077b007a..e97d4acd 100644 --- a/8Knot/pages/contributions/visualizations/pr_review_response.py +++ b/8Knot/pages/contributions/visualizations/pr_review_response.py @@ -63,12 +63,9 @@ step=1, value=2, size="sm", + style={"width": "100px"}, ), className="me-2", - width=2, - ), - dbc.Col( - width=6, ), dbc.Col( dbc.Button( @@ -82,7 +79,6 @@ ), ], align="center", - justify="between", ), ] ), From f73f3be48d6cfd56ae6d3105d9a2f1b2d342f059 Mon Sep 17 00:00:00 2001 From: cdolfi Date: Thu, 9 May 2024 18:01:45 -0400 Subject: [PATCH 4/6] xaxis update --- .../visualizations/contrib_importance_over_time.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/8Knot/pages/contributors/visualizations/contrib_importance_over_time.py b/8Knot/pages/contributors/visualizations/contrib_importance_over_time.py index 63ce11cf..ca92273e 100644 --- a/8Knot/pages/contributors/visualizations/contrib_importance_over_time.py +++ b/8Knot/pages/contributors/visualizations/contrib_importance_over_time.py @@ -375,10 +375,12 @@ def create_figure(df_final, threshold, step_size): hovertemplate="%{y} people contributing to
%{customdata[0]}% of %{text} from
%{customdata[1]}
", ) + # update xaxes to show only the year + fig.update_xaxes(showgrid=True, ticklabelmode="period", dtick="M12", tickformat="%Y") + # layout styling fig.update_layout( xaxis_title=f"Timeline (stepsize = {step_size} months)", - xaxis=dict(tick0=start_date), yaxis_title="Lottery Factor", font=dict(size=14), margin_b=40, From b7be62caa9175498ff03c30babc1f5a7c2657fb4 Mon Sep 17 00:00:00 2001 From: cdolfi Date: Fri, 10 May 2024 16:24:59 -0400 Subject: [PATCH 5/6] assignment negative fix --- .../contributions/visualizations/cntrb_pr_assignment.py | 9 +++++++-- .../visualizations/cntrib_issue_assignment.py | 9 +++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/8Knot/pages/contributions/visualizations/cntrb_pr_assignment.py b/8Knot/pages/contributions/visualizations/cntrb_pr_assignment.py index 0376975d..b290ec12 100644 --- a/8Knot/pages/contributions/visualizations/cntrb_pr_assignment.py +++ b/8Knot/pages/contributions/visualizations/cntrb_pr_assignment.py @@ -372,5 +372,10 @@ def pr_assignment(df, start_date, end_date, contrib): (df_in_range["assignment_action"] == "assigned") & (df_in_range["assign_date"] <= end_date) ] - # return the different of assignments and unassignments - return df_assigned.shape[0] - df_unassign.shape[0] + # the different of assignments and unassignments + assign_value = df_assigned.shape[0] - df_unassign.shape[0] + + # prevent negative assignments + assign_value = 0 if assign_value < 0 else assign_value + + return assign_value diff --git a/8Knot/pages/contributions/visualizations/cntrib_issue_assignment.py b/8Knot/pages/contributions/visualizations/cntrib_issue_assignment.py index eb7b33b6..1addc2ef 100644 --- a/8Knot/pages/contributions/visualizations/cntrib_issue_assignment.py +++ b/8Knot/pages/contributions/visualizations/cntrib_issue_assignment.py @@ -369,5 +369,10 @@ def issue_assignment(df, start_date, end_date, contrib): (df_in_range["assignment_action"] == "assigned") & (df_in_range["assign_date"] <= end_date) ] - # return the different of assignments and unassignments - return df_assigned.shape[0] - df_unassign.shape[0] + # the different of assignments and unassignments + assign_value = df_assigned.shape[0] - df_unassign.shape[0] + + # prevent negative assignments + assign_value = 0 if assign_value < 0 else assign_value + + return assign_value From 1724ef01570a6bd431f815619eb41dafbc4ced05 Mon Sep 17 00:00:00 2001 From: cdolfi Date: Thu, 9 May 2024 17:26:30 -0400 Subject: [PATCH 6/6] normalized email case --- 8Knot/pages/affiliation/visualizations/commit_domains.py | 4 ++-- .../affiliation/visualizations/org_associated_activity.py | 4 ++-- .../pages/affiliation/visualizations/org_core_contributors.py | 4 ++-- 8Knot/pages/affiliation/visualizations/unqiue_domains.py | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/8Knot/pages/affiliation/visualizations/commit_domains.py b/8Knot/pages/affiliation/visualizations/commit_domains.py index 7cdcdeca..0d7c8262 100644 --- a/8Knot/pages/affiliation/visualizations/commit_domains.py +++ b/8Knot/pages/affiliation/visualizations/commit_domains.py @@ -178,8 +178,8 @@ def process_data(df: pd.DataFrame, num, start_date, end_date): # creates list of emails for each contribution and flattens list result emails = df.author_email.tolist() - # remove any entries not in email format - emails = [x for x in emails if "@" in x] + # remove any entries not in email format and put all emails in lowercase + emails = [x.lower() for x in emails if "@" in x] # creates list of email domains from the emails list email_domains = [x[x.rindex("@") + 1 :] for x in emails] diff --git a/8Knot/pages/affiliation/visualizations/org_associated_activity.py b/8Knot/pages/affiliation/visualizations/org_associated_activity.py index f9778d0f..bd62170a 100644 --- a/8Knot/pages/affiliation/visualizations/org_associated_activity.py +++ b/8Knot/pages/affiliation/visualizations/org_associated_activity.py @@ -221,8 +221,8 @@ def process_data(df: pd.DataFrame, num, start_date, end_date, email_filter): # creates list of emails for each contribution and flattens list result emails = df.email_list.str.split(" , ").explode("email_list").tolist() - # remove any entries not in email format - emails = [x for x in emails if "@" in x] + # remove any entries not in email format and flattens list result + emails = [x.lower() for x in emails if "@" in x] # creates list of email domains from the emails list email_domains = [x[x.rindex("@") + 1 :] for x in emails] diff --git a/8Knot/pages/affiliation/visualizations/org_core_contributors.py b/8Knot/pages/affiliation/visualizations/org_core_contributors.py index e0b512a0..6fcecb54 100644 --- a/8Knot/pages/affiliation/visualizations/org_core_contributors.py +++ b/8Knot/pages/affiliation/visualizations/org_core_contributors.py @@ -234,8 +234,8 @@ def process_data(df: pd.DataFrame, contributions, contributors, start_date, end_ # creates list of unique emails and flattens list result emails = df.email_list.str.split(" , ").explode("email_list").tolist() - # remove any entries not in email format - emails = [x for x in emails if "@" in x] + # remove any entries not in email format and flattens list result + emails = [x.lower() for x in emails if "@" in x] # creates list of email domains from the emails list email_domains = [x[x.rindex("@") + 1 :] for x in emails] diff --git a/8Knot/pages/affiliation/visualizations/unqiue_domains.py b/8Knot/pages/affiliation/visualizations/unqiue_domains.py index 702feece..95670fb1 100644 --- a/8Knot/pages/affiliation/visualizations/unqiue_domains.py +++ b/8Knot/pages/affiliation/visualizations/unqiue_domains.py @@ -179,8 +179,8 @@ def process_data(df: pd.DataFrame, num, start_date, end_date): # creates list of unique emails and flattens list result emails = df.email_list.str.split(" , ").explode("email_list").unique().tolist() - # remove any entries not in email format - emails = [x for x in emails if "@" in x] + # remove any entries not in email format and put all emails in lowercase + emails = [x.lower() for x in emails if "@" in x] # creates list of email domains from the emails list email_domains = [x[x.rindex("@") + 1 :] for x in emails]