From 452149a4fc7879a909f6419b5c12d4852d6adc75 Mon Sep 17 00:00:00 2001 From: Gayle Martin <76500899+gaylem@users.noreply.github.com> Date: Wed, 17 Apr 2024 18:38:04 -0500 Subject: [PATCH] Update codeql.yml to automatically create new CodeQL issues (#6503) * Create Check for CodeQL alerts step in codeql.yml * Make create-codeql-issue folder with issue-body.md file * Change branch name for testing * Change branch back to gh-pages and token secrets back to GITHUB_TOKEN * Fix issueBodyTemplatePath * Move create-codeql-issues folder into trigger-issue folder * Comment out error message for createIssueResponse * Refactor codeql.yml and move scripts to separate js files * Create fetch-alerts.js * Create check-existing-issues.js * Create create-new-issues.js * Replace branch and token for testing * Move secrets to codeql.yml and pass as argument to functions * Require core in fetch-alerts.js and check-existing-issues.js * Remove require core and pass as argument to js files * Swap core with setOutput * Remove require core * Remove core from setOutput invocation * Add comma * Require in core and remove from arguments * Remove core require * Remove return statements * Add console log for testing * Add more console logs for testing * Add env to yml steps and use process.env in token * Require in core * Add Set up Node.js step to enable core * Add step to install actions/core module * Declare alerts and alertId in yml file * Add comma to headers in POST request * Replace TEAMS with H4LA_TOKEN * Update node version * Revert H4LA_TOKEN to TEAMS * Revert H4LA_TOKEN to GITHUB_TOKEN * Remove console logs * Update codeql.yml file * Update fetch-alerts.js file * Update check-existing-issues.js file * Update create-new-issues.js file * Replace missing curly braces * Replace response with fetchAlertsResponse * Replace listAlertsForRepo with GET request * Replace ok with 200 * Add comments and console log for testing * Remove .json() * Replace ok with 200 * Update comments and POST request syntax * Replace ok with 200 on create-new-issues.js and change POST to GET on check-existing-issues.js * Replace .json() with .data * Update POST request syntax * Update comments and change secrets * Revert secret name * Add comments * Batch API requests to avoid hitting rate limit * Add comments * Adjust alertIdsWithoutIssues.push logic * Add console logs * Reduce batches from 10 to 5 due to GitHub limit * Adjust createIssueResponse query * Add comment * Add template literals to query url and add comment for testing * Change 200 to 201 * Update comment * Batch issue creation requests to avoid rate limit * Adjust batching of new issue requests * Revert changes * Remove comment * Move console log * Update console log * Update console log * Update console log * Replaced test branch with gh-pages * Update console logs and comments * Removed fs and updated issueTitle * Removed fs * Add fs back in * Remove fs * Add HACKFORLA_ADMIN_TOKEN in fetch-alerts secret * Revert secret back to HACKFORLA_BOT_PA_TOKEN * Swap bot token for GITHUB_TOKEN * Replace GITHUB_TOKEN with HACKFORLA_BOT_PA_TOKEN on Create New Issues step * Revert token on Create New Issues * Add How to manage CodeQL alerts to issue template * Add workflow_dispatch for manual retries * Update put request to use create function * Refactor body variable * Change token * Changed GITHUB_TOKEN to HACKFORLA_ADMIN_TOKEN * Add issues: write to permissions * Change permissions to write-all * Move permissions up above jobs * Move permissions back to original location * Updated branch to test * Change token and branch * Change branch back to gh-pages * Change branch to test * Change branch back to gh-pages * Add if statement to new yml steps * Change branch for testing * Change branch back to gh-pages * Add create-new-issues id and reorder id and if conditions to be consistent * Change branch to test * Added unused variable to see if CodeQL picks it up * Changed branch to gh-pages * Removed unused variable --- .github/workflows/codeql.yml | 46 +++++++++++++- .../check-existing-issues.js | 62 +++++++++++++++++++ .../create-codeql-issues/create-new-issues.js | 52 ++++++++++++++++ .../create-codeql-issues/fetch-alerts.js | 36 +++++++++++ .../create-codeql-issues/issue-body.md | 26 ++++++++ 5 files changed, 219 insertions(+), 3 deletions(-) create mode 100644 github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js create mode 100644 github-actions/trigger-issue/create-codeql-issues/create-new-issues.js create mode 100644 github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js create mode 100644 github-actions/trigger-issue/create-codeql-issues/issue-body.md diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 85c3f1127d..851f4b675f 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -13,12 +13,13 @@ name: "CodeQL" on: push: - branches: [ "gh-pages" ] + branches: [ 'gh-pages' ] pull_request: # The branches below must be a subset of the branches above - branches: [ "gh-pages" ] + branches: [ 'gh-pages' ] schedule: - cron: '30 5 * * 5' + workflow_dispatch: jobs: analyze: @@ -29,6 +30,7 @@ jobs: actions: read contents: read security-events: write + issues: write strategy: fail-fast: false @@ -75,4 +77,42 @@ jobs: - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v3 with: - category: "/language:${{matrix.language}}" \ No newline at end of file + category: "/language:${{matrix.language}}" + + # Fetch Alerts + - name: Fetch Alerts + id: fetch-alerts + if: github.event_name != 'pull_request' + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const script = require('./github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js'); + const fetchAlerts = script({ g: github, c: context }); + return fetchAlerts + + # Check Existing Issues + - name: Check Existing Issues + id: check-existing-issues + if: github.event_name != 'pull_request' + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const script = require('./github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js'); + const alerts = ${{ steps.fetch-alerts.outputs.result }}; + const checkExistingIssues = script({ g: github, c: context, alerts}); + return checkExistingIssues + + # Create New Issues + - name: Create New Issues + id: create-new-issues + if: github.event_name != 'pull_request' + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.HACKFORLA_ADMIN_TOKEN }} + script: | + const script = require('./github-actions/trigger-issue/create-codeql-issues/create-new-issues.js'); + const alertIds = ${{ steps.check-existing-issues.outputs.result }}; + const newIssues = script({ g: github, c: context, alertIds}); + diff --git a/github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js b/github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js new file mode 100644 index 0000000000..d3e18f8547 --- /dev/null +++ b/github-actions/trigger-issue/create-codeql-issues/check-existing-issues.js @@ -0,0 +1,62 @@ +// Global variables +var github; +var context; + +/** + * Fetches existing issues for each alert and sets the output for alerts without existing issues. + * @param {Object} options - The options object. + * @param {string} options.g - The GitHub access token. + * @param {Object} options.c - The context object. + * @param {Array} options.alerts - The array of alerts to check. + * @returns {Promise>} An array of alert IDs without existing issues. + * @throws {Error} If the GET request fails. + */ +const checkExistingIssues = async ({ g, c, alerts }) => { + // Rename parameters + github = g; + context = c; + + // Initialize empty array to store alertIds + let alertIdsWithoutIssues = []; + + // Batch alerts into groups of 5 for each request to avoid rate limit + const batchedAlertIds = alerts.reduce((acc, alert, index) => { + // For indexes 0 to 4, batchIndex == 0 + // For indexes 5 to 9, batchIndex == 1 + // For indexes 10 to 14, batchIndex == 2 + // Etc. + const batchIndex = Math.floor(index / 5); + // if acc[batchIndex] == undefined, a new array is created before pushing the alert number + acc[batchIndex] = acc[batchIndex] || []; + // Push alert.number to inner array + acc[batchIndex].push(alert.number); + // Returns array of arrays + return acc; + }, []); + + // Loop through each batch of alerts + for (const fiveAlertIds of batchedAlertIds) { + // Creates one query for multiple alertIds + const q = fiveAlertIds.map(alertId => `repo:${context.repo.owner}/${context.repo.repo}+state:open+"${alertId}"+in:title`).join('+OR+'); + + // Query GitHub API in batches + const searchResponse = await github.request('GET /search/issues', { q }); + + // Throw error if GET request fails + if (searchResponse.status !== 200) { + throw new Error(`Failed to search for issues: ${searchResponse.status} - ${searchResponse.statusText}`); + } + + // Store the response data in a variable for easy access + const searchResult = searchResponse.data; + + // Push alertIds that do not have existing issues in searchResult to output array + alertIdsWithoutIssues.push(...fiveAlertIds.filter(alertId => !searchResult.items.some(item => item.title.includes(alertId)))); + }; + + // Return flat array of alertIds that do not have issues + console.log('alertIds without issues: ', alertIdsWithoutIssues); + return alertIdsWithoutIssues; +}; + +module.exports = checkExistingIssues diff --git a/github-actions/trigger-issue/create-codeql-issues/create-new-issues.js b/github-actions/trigger-issue/create-codeql-issues/create-new-issues.js new file mode 100644 index 0000000000..a6ab6681fc --- /dev/null +++ b/github-actions/trigger-issue/create-codeql-issues/create-new-issues.js @@ -0,0 +1,52 @@ +const fs = require('fs'); + +// Global variables +var github; +var context; + +/** + * Creates new GitHub issues for each alert that doesn't have an existing issue. + * @param {Object} options - The options object. + * @param {string} options.g - The GitHub access token. + * @param {Object} options.c - The context object. + * @param {Array} options.alertIds - The array of alert IDs to create issues for. + * @returns {Promise} + * @throws {Error} If the POST request fails. + */ +const createNewIssues = async ({ g, c, alertIds }) => { + // Rename parameters + github = g; + context = c; + + // Loop through each alertId + for (const alertId of alertIds) { + // Create the issue title + const title = `Resolve CodeQL Alert #${alertId} - Generated by GHA`; + + // Read the issue body template file + const issueBodyTemplatePath = 'github-actions/trigger-issue/create-codeql-issues/issue-body.md'; + let issueBodyTemplate = fs.readFileSync(issueBodyTemplatePath, 'utf8'); + + // Replace placeholders with actual values in the issue body template + const body = issueBodyTemplate.replace(/\${alertId}/g, alertId); + + // Create a new GitHub issue + const createIssueResponse = await github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title, + body, + labels: ['ready for dev lead'] + }); + + // Log issue titles and links in GHA workflow + console.log('Issue Created:', createIssueResponse.data.title, createIssueResponse.data.html_url); + + // Throw error if POST request fails (201 not created) + if (createIssueResponse.status !== 201) { + throw new Error(`Failed to create issue for alert ${alertId}: ${createIssueResponse.status} - ${createIssueResponse.statusText}`); + } + } +}; + +module.exports = createNewIssues; diff --git a/github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js b/github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js new file mode 100644 index 0000000000..94c119d0f3 --- /dev/null +++ b/github-actions/trigger-issue/create-codeql-issues/fetch-alerts.js @@ -0,0 +1,36 @@ +// Global variables +var github; +var context; + +/** + * Fetches a list of open CodeQL alerts from the GitHub API. + * @param {Object} params - The parameters for the fetch operation. + * @param {Object} params.g - The GitHub object for making API requests. + * @param {Object} params.c - The context object containing repository information. + * @returns {Promise} A promise that resolves with an array of alerts when the fetch is successful. + * @throws {Error} If the GET request fails. + */ +const fetchAlerts = async ({ g, c }) => { + // Rename parameters + github = g; + context = c; + + // Get a list of open CodeQL alerts + const fetchAlertsResponse = await github.request('GET /repos/{owner}/{repo}/code-scanning/alerts', { + owner: context.repo.owner, + repo: context.repo.repo, + state: 'open', + per_page: 100, + page: 1 + }); + + // Throw error if GET request fails + if (fetchAlertsResponse.status !== 200) { + throw new Error(`Failed to fetch alerts: ${fetchAlertsResponse.status} - ${fetchAlertsResponse.statusText}`); + } + + // Return alerts + return fetchAlertsResponse.data +}; + +module.exports = fetchAlerts diff --git a/github-actions/trigger-issue/create-codeql-issues/issue-body.md b/github-actions/trigger-issue/create-codeql-issues/issue-body.md new file mode 100644 index 0000000000..06e4b12a52 --- /dev/null +++ b/github-actions/trigger-issue/create-codeql-issues/issue-body.md @@ -0,0 +1,26 @@ +### Prerequisite + +1. Be a member of Hack for LA. (There are no fees to join.) If you have not joined yet, please follow the steps on our [Getting Started page](https://www.hackforla.org/getting-started). +2. Before you claim or start working on an issue, please make sure you have read our [How to Contribute to Hack for LA Guide](https://github.com/hackforla/website/blob/7f0c132c96f71230b8935759e1f8711ccb340c0f/CONTRIBUTING.md). + +### Overview +We need to resolve the new alert [(${alertId})](https://github.com/hackforla/website/security/code-scanning/${alertId}) and either recommend dismissal of the alert or update the code files to resolve the alert. + +### Action Items +- [ ] The following action item serves to "link" this issue as the "tracking issue" for the CodeQL alert and to provide more details regarding the alert: https://github.com/hackforla/website/security/code-scanning/${alertId} +- [ ] In a comment in this issue, add your analysis and recommendations. The recommendation can be one of the following: `dismiss as test`, `dismiss as false positive`, `dismiss as won't fix`, or `update code`. An example of a `false positive` is a report of a JavaScript syntax error that is caused by markdown or liquid symbols such as `---` or `{%` +- [ ] **If the recommendation is to dismiss the alert:** + - [ ] Apply the label `ready for dev lead` + - [ ] Move the issue to `Questions/In Review` +- [ ] **If the recommendation is to update code:** + - [ ] Create an issue branch and proceed with the code update + - [ ] Test using docker to ensure that there are no changes to any affected webpage(s) + - [ ] Proceed with pull request in the usual manner + +### Resources/Instructions +- [HfLA website: CodeQL scan alert audits - issue 5005](https://docs.google.com/spreadsheets/d/1B3R-fI8OW0LcYuwZICQZ2fB8sjlE3VsfyGIXoReNBIs/edit#gid=193401043) +- [Code scanning results page](https://github.com/hackforla/website/security/code-scanning) +- [CodeQL query help for JavaScript](https://codeql.github.com/codeql-query-help/javascript/) +- [How to manage CodeQL alerts](https://github.com/hackforla/website/issues/6463#issuecomment-2002573270 ) + +This issue was automatically generated from the codeql.yml workflow \ No newline at end of file