Update codeql.yml to automatically create new CodeQL issues (#6503)

* Create Check for CodeQL alerts step in codeql.yml * Make create-codeql-issue folder with issue-body.md file * Change branch name for testing * Change branch back to gh-pages and token secrets back to GITHUB_TOKEN * Fix issueBodyTemplatePath * Move create-codeql-issues folder into trigger-issue folder * Comment out error message for createIssueResponse * Refactor codeql.yml and move scripts to separate js files * Create fetch-alerts.js * Create check-existing-issues.js * Create create-new-issues.js * Replace branch and token for testing * Move secrets to codeql.yml and pass as argument to functions * Require core in fetch-alerts.js and check-existing-issues.js * Remove require core and pass as argument to js files * Swap core with setOutput * Remove require core * Remove core from setOutput invocation * Add comma * Require in core and remove from arguments * Remove core require * Remove return statements * Add console log for testing * Add more console logs for testing * Add env to yml steps and use process.env in token * Require in core * Add Set up Node.js step to enable core * Add step to install actions/core module * Declare alerts and alertId in yml file * Add comma to headers in POST request * Replace TEAMS with H4LA_TOKEN * Update node version * Revert H4LA_TOKEN to TEAMS * Revert H4LA_TOKEN to GITHUB_TOKEN * Remove console logs * Update codeql.yml file * Update fetch-alerts.js file * Update check-existing-issues.js file * Update create-new-issues.js file * Replace missing curly braces * Replace response with fetchAlertsResponse * Replace listAlertsForRepo with GET request * Replace ok with 200 * Add comments and console log for testing * Remove .json() * Replace ok with 200 * Update comments and POST request syntax * Replace ok with 200 on create-new-issues.js and change POST to GET on check-existing-issues.js * Replace .json() with .data * Update POST request syntax * Update comments and change secrets * Revert secret name * Add comments * Batch API requests to avoid hitting rate limit * Add comments * Adjust alertIdsWithoutIssues.push logic * Add console logs * Reduce batches from 10 to 5 due to GitHub limit * Adjust createIssueResponse query * Add comment * Add template literals to query url and add comment for testing * Change 200 to 201 * Update comment * Batch issue creation requests to avoid rate limit * Adjust batching of new issue requests * Revert changes * Remove comment * Move console log * Update console log * Update console log * Update console log * Replaced test branch with gh-pages * Update console logs and comments * Removed fs and updated issueTitle * Removed fs * Add fs back in * Remove fs * Add HACKFORLA_ADMIN_TOKEN in fetch-alerts secret * Revert secret back to HACKFORLA_BOT_PA_TOKEN * Swap bot token for GITHUB_TOKEN * Replace GITHUB_TOKEN with HACKFORLA_BOT_PA_TOKEN on Create New Issues step * Revert token on Create New Issues * Add How to manage CodeQL alerts to issue template * Add workflow_dispatch for manual retries * Update put request to use create function * Refactor body variable * Change token * Changed GITHUB_TOKEN to HACKFORLA_ADMIN_TOKEN * Add issues: write to permissions * Change permissions to write-all * Move permissions up above jobs * Move permissions back to original location * Updated branch to test * Change token and branch * Change branch back to gh-pages * Change branch to test * Change branch back to gh-pages * Add if statement to new yml steps * Change branch for testing * Change branch back to gh-pages * Add create-new-issues id and reorder id and if conditions to be consistent * Change branch to test * Added unused variable to see if CodeQL picks it up * Changed branch to gh-pages * Removed unused variable
hackforla · Apr 17, 2024 · 452149a · 452149a
1 parent 6d05915
commit 452149a
Show file tree

Hide file tree

Showing 5 changed files with 219 additions and 3 deletions.
diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
@@ -13,12 +13,13 @@ name: "CodeQL"
 
 on:
   push:
-    branches: [ "gh-pages" ]
+    branches: [ 'gh-pages' ]
   pull_request:
     # The branches below must be a subset of the branches above
-    branches: [ "gh-pages" ]
+    branches: [ 'gh-pages' ]
   schedule:
     - cron: '30 5 * * 5'
+  workflow_dispatch:
 
 jobs:
   analyze:
@@ -29,6 +30,7 @@ jobs:
       actions: read
       contents: read
       security-events: write
+      issues: write
 
     strategy:
       fail-fast: false
@@ -75,4 +77,42 @@ jobs:
     - name: Perform CodeQL Analysis
       uses: github/codeql-action/analyze@v3
       with:
-        category: "/language:${{matrix.language}}"
+        category: "/language:${{matrix.language}}"
+
+    # Fetch Alerts
+    - name: Fetch Alerts
+      id: fetch-alerts
+      if: github.event_name != 'pull_request'
+      uses: actions/github-script@v7
+      with:
+        github-token: ${{ secrets.GITHUB_TOKEN }}
+        script: |
+          const script = require('./github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js');
+          const fetchAlerts = script({ g: github, c: context });
+          return fetchAlerts
+
+    # Check Existing Issues
+    - name: Check Existing Issues
+      id: check-existing-issues
+      if: github.event_name != 'pull_request'
+      uses: actions/github-script@v7
+      with:
+        github-token: ${{ secrets.GITHUB_TOKEN }}
+        script: |
+          const script = require('./github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js');
+          const alerts = ${{ steps.fetch-alerts.outputs.result }};
+          const checkExistingIssues = script({ g: github, c: context, alerts});
+          return checkExistingIssues
+
+    # Create New Issues
+    - name: Create New Issues
+      id: create-new-issues
+      if: github.event_name != 'pull_request'
+      uses: actions/github-script@v7
+      with:
+        github-token: ${{ secrets.HACKFORLA_ADMIN_TOKEN }}
+        script: |
+          const script = require('./github-actions/trigger-issue/create-codeql-issues/create-new-issues.js');
+          const alertIds = ${{ steps.check-existing-issues.outputs.result }};
+          const newIssues = script({ g: github, c: context, alertIds});
+
diff --git a/github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js b/github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js
@@ -0,0 +1,62 @@
+// Global variables
+var github;
+var context;
+
+/**
+ * Fetches existing issues for each alert and sets the output for alerts without existing issues.
+ * @param {Object} options - The options object.
+ * @param {string} options.g - The GitHub access token.
+ * @param {Object} options.c - The context object.
+ * @param {Array<Object>} options.alerts - The array of alerts to check.
+ * @returns {Promise<Array<number>>} An array of alert IDs without existing issues.
+ * @throws {Error} If the GET request fails.
+ */
+const checkExistingIssues = async ({ g, c, alerts }) => {
+  // Rename parameters
+  github = g;
+  context = c;
+
+  // Initialize empty array to store alertIds
+  let alertIdsWithoutIssues = [];
+
+  // Batch alerts into groups of 5 for each request to avoid rate limit
+  const batchedAlertIds = alerts.reduce((acc, alert, index) => {
+    // For indexes 0 to 4, batchIndex == 0
+    // For indexes 5 to 9, batchIndex == 1
+    // For indexes 10 to 14, batchIndex == 2
+    // Etc.
+    const batchIndex = Math.floor(index / 5);
+    // if acc[batchIndex] == undefined, a new array is created before pushing the alert number
+    acc[batchIndex] = acc[batchIndex] || [];
+    // Push alert.number to inner array
+    acc[batchIndex].push(alert.number);
+    // Returns array of arrays
+    return acc;
+  }, []);
+
+  // Loop through each batch of alerts
+  for (const fiveAlertIds of batchedAlertIds) {
+    // Creates one query for multiple alertIds
+    const q = fiveAlertIds.map(alertId => `repo:${context.repo.owner}/${context.repo.repo}+state:open+"${alertId}"+in:title`).join('+OR+');
+
+    // Query GitHub API in batches
+    const searchResponse = await github.request('GET /search/issues', { q });
+
+    // Throw error if GET request fails
+    if (searchResponse.status !== 200) {
+      throw new Error(`Failed to search for issues: ${searchResponse.status} - ${searchResponse.statusText}`);
+    }
+
+    // Store the response data in a variable for easy access
+    const searchResult = searchResponse.data;
+
+    // Push alertIds that do not have existing issues in searchResult to output array
+    alertIdsWithoutIssues.push(...fiveAlertIds.filter(alertId => !searchResult.items.some(item => item.title.includes(alertId))));
+  };
+
+  // Return flat array of alertIds that do not have issues
+  console.log('alertIds without issues: ', alertIdsWithoutIssues);
+  return alertIdsWithoutIssues;
+};
+
+module.exports = checkExistingIssues
diff --git a/github-actions/trigger-issue/create-codeql-issues/create-new-issues.js b/github-actions/trigger-issue/create-codeql-issues/create-new-issues.js
@@ -0,0 +1,52 @@
+const fs = require('fs');
+
+// Global variables
+var github;
+var context;
+
+/**
+ * Creates new GitHub issues for each alert that doesn't have an existing issue.
+ * @param {Object} options - The options object.
+ * @param {string} options.g - The GitHub access token.
+ * @param {Object} options.c - The context object.
+ * @param {Array<number>} options.alertIds - The array of alert IDs to create issues for.
+ * @returns {Promise<void>}
+ * @throws {Error} If the POST request fails.
+ */
+const createNewIssues = async ({ g, c, alertIds }) => {
+  // Rename parameters
+  github = g;
+  context = c;
+
+  // Loop through each alertId
+  for (const alertId of alertIds) {
+    // Create the issue title
+    const title = `Resolve CodeQL Alert #${alertId} - Generated by GHA`;
+
+    // Read the issue body template file
+    const issueBodyTemplatePath = 'github-actions/trigger-issue/create-codeql-issues/issue-body.md';
+    let issueBodyTemplate = fs.readFileSync(issueBodyTemplatePath, 'utf8');
+
+    // Replace placeholders with actual values in the issue body template
+    const body = issueBodyTemplate.replace(/\${alertId}/g, alertId);
+
+    // Create a new GitHub issue
+    const createIssueResponse = await github.rest.issues.create({
+      owner: context.repo.owner,
+      repo: context.repo.repo,
+      title,
+      body,
+      labels: ['ready for dev lead']
+    });      
+
+    // Log issue titles and links in GHA workflow
+    console.log('Issue Created:', createIssueResponse.data.title, createIssueResponse.data.html_url);
+
+    // Throw error if POST request fails (201 not created)
+    if (createIssueResponse.status !== 201) {
+      throw new Error(`Failed to create issue for alert ${alertId}: ${createIssueResponse.status} - ${createIssueResponse.statusText}`);
+    }
+  }
+};
+
+module.exports = createNewIssues;
diff --git a/github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js b/github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js
@@ -0,0 +1,36 @@
+// Global variables
+var github;
+var context;
+
+/**
+ * Fetches a list of open CodeQL alerts from the GitHub API.
+ * @param {Object} params - The parameters for the fetch operation.
+ * @param {Object} params.g - The GitHub object for making API requests.
+ * @param {Object} params.c - The context object containing repository information.
+ * @returns {Promise<Array>} A promise that resolves with an array of alerts when the fetch is successful.
+ * @throws {Error} If the GET request fails.
+ */
+const fetchAlerts = async ({ g, c }) => {
+  // Rename parameters
+  github = g;
+  context = c; 
+
+  // Get a list of open CodeQL alerts
+  const fetchAlertsResponse = await github.request('GET /repos/{owner}/{repo}/code-scanning/alerts', {
+    owner: context.repo.owner,
+    repo: context.repo.repo,
+    state: 'open',
+    per_page: 100,
+    page: 1
+  });
+
+  // Throw error if GET request fails
+  if (fetchAlertsResponse.status !== 200) {
+    throw new Error(`Failed to fetch alerts: ${fetchAlertsResponse.status} - ${fetchAlertsResponse.statusText}`);
+  }
+
+  // Return alerts
+  return fetchAlertsResponse.data
+};
+
+module.exports = fetchAlerts
diff --git a/github-actions/trigger-issue/create-codeql-issues/issue-body.md b/github-actions/trigger-issue/create-codeql-issues/issue-body.md
@@ -0,0 +1,26 @@
+### Prerequisite
+
+1. Be a member of Hack for LA. (There are no fees to join.) If you have not joined yet, please follow the steps on our [Getting Started page](https://www.hackforla.org/getting-started).
+2. Before you claim or start working on an issue, please make sure you have read our [How to Contribute to Hack for LA Guide](https://github.com/hackforla/website/blob/7f0c132c96f71230b8935759e1f8711ccb340c0f/CONTRIBUTING.md).
+
+### Overview
+We need to resolve the new alert [(${alertId})](https://github.com/hackforla/website/security/code-scanning/${alertId}) and either recommend dismissal of the alert or update the code files to resolve the alert.
+
+### Action Items
+- [ ] The following action item serves to "link" this issue as the "tracking issue" for the CodeQL alert and to provide more details regarding the alert: https://github.com/hackforla/website/security/code-scanning/${alertId}
+- [ ] In a comment in this issue, add your analysis and recommendations.  The recommendation can be one of the following: `dismiss as test`, `dismiss as false positive`, `dismiss as won't fix`, or `update code`.  An example of a `false positive` is a report of a JavaScript syntax error that is caused by markdown or liquid symbols such as `---` or `{%`
+- [ ] **If the recommendation is to dismiss the alert:**
+  - [ ] Apply the label `ready for dev lead` 
+  - [ ] Move the issue to `Questions/In Review`
+- [ ] **If the recommendation is to update code:**
+  - [ ] Create an issue branch and proceed with the code update
+  - [ ] Test using docker to ensure that there are no changes to any affected webpage(s)
+  - [ ] Proceed with pull request in the usual manner
+
+### Resources/Instructions
+- [HfLA website: CodeQL scan alert audits - issue 5005](https://docs.google.com/spreadsheets/d/1B3R-fI8OW0LcYuwZICQZ2fB8sjlE3VsfyGIXoReNBIs/edit#gid=193401043)
+- [Code scanning results page](https://github.com/hackforla/website/security/code-scanning)
+- [CodeQL query help for JavaScript](https://codeql.github.com/codeql-query-help/javascript/)
+- [How to manage CodeQL alerts](https://github.com/hackforla/website/issues/6463#issuecomment-2002573270 )
+
+This issue was automatically generated from the codeql.yml workflow