Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update logging & memory management, retry certain HTTP requests, catch out of memory situations #258

Merged
merged 39 commits into from
Mar 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
143eb35
Move garbage collection to main.php, set memory limit, log URLs to PRs.
gudmdharalds Mar 8, 2022
c7640e8
Rename class to match file
gudmdharalds Mar 8, 2022
1f23d1b
Encoding message logged for more safety, updating tests accordingly.
gudmdharalds Mar 8, 2022
ed34df9
SVG PHP runtime version displayed in scan-details.
gudmdharalds Mar 8, 2022
b203446
Log when PHP version is determined.
gudmdharalds Mar 8, 2022
c9bc198
Garbage collection moved to main.php, stop logging auto-approvals.
gudmdharalds Mar 8, 2022
9fb2d6c
Adding log message when starting linting.
gudmdharalds Mar 8, 2022
cf746bd
Adding logging, garbage collection has moved to main.php, moved remov…
gudmdharalds Mar 8, 2022
b79b7cf
Add function to output URLs to PRs, change usage in main.php
gudmdharalds Mar 8, 2022
8be58b2
Fixing whitespacing, adding phpcs:ignore
gudmdharalds Mar 8, 2022
d1ec535
Adding test for vipgoci_github_prs_urls_get()
gudmdharalds Mar 8, 2022
2d83484
Adding linebreak, previously removed by the function, but not any more.
gudmdharalds Mar 8, 2022
cd254d4
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
1be6fd1
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
d359c95
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
7f8a16b
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
0ca09da
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
8d2a924
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
1fc798d
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
68067f8
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
d826164
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
1bd0063
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
93df850
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
1092a43
Changing "Pull-Request" to "pull request".
gudmdharalds Mar 8, 2022
7298d05
Retry mechanism for vipgoci_github_fetch_url(), replace "Pull-Request…
gudmdharalds Mar 8, 2022
9e45315
Log to IRC when HTTP GET requests fail.
gudmdharalds Mar 8, 2022
819151f
Log to IRC when exiting due to HTTP GET failure.
gudmdharalds Mar 8, 2022
8528517
Add comment explaining the purpose.
gudmdharalds Mar 8, 2022
0e4754c
WP CS fix
gudmdharalds Mar 8, 2022
2d05a0a
WP CS fix
gudmdharalds Mar 8, 2022
533b927
Removing whitespace.
gudmdharalds Mar 8, 2022
b90b152
Exit if temporary file name not found in PHPCS output.
gudmdharalds Mar 8, 2022
4821278
Providing exit code for vipgoci_sysexit()
gudmdharalds Mar 8, 2022
6f4e34b
Providing exit code for vipgoci_sysexit()
gudmdharalds Mar 8, 2022
0713dbf
Providing exit code for vipgoci_sysexit()
gudmdharalds Mar 8, 2022
8119721
Log correct string for PHPCS version.
gudmdharalds Mar 8, 2022
f4cadfe
Define out of memory error.
gudmdharalds Mar 8, 2022
8449953
Handle out of memory situations.
gudmdharalds Mar 8, 2022
d2e5340
Set maximum execution time for PHPCS.
gudmdharalds Mar 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,13 +109,13 @@ While running, `vip-go-ci` will output log of its actions. Here is an example --
"dry-run": false
}
}
[ 2018-04-16T14:10:05+00:00 -- 0 ] Fetching all open Pull-Requests from GitHub; {
[ 2018-04-16T14:10:05+00:00 -- 0 ] Fetching all open pull requests from GitHub; {
"repo_owner": "mygithubuser",
"repo_name": "testing123",
"commit_id": "f978c17f8f648e5ce10aa16694c74a5544b1670e",
"branches_ignore": []
}
[ 2018-04-16T14:10:14+00:00 -- 0 ] Fetching information about all commits made to Pull-Request #17 from GitHub; {
[ 2018-04-16T14:10:14+00:00 -- 0 ] Fetching information about all commits made to pull request #17 from GitHub; {
"repo_owner": "mygithubuser",
"repo_name": "testing123",
"pr_number": 17
Expand Down Expand Up @@ -146,7 +146,7 @@ While running, `vip-go-ci` will output log of its actions. Here is an example --
"filename": "bla-2.php",
"local_git_repo": "\/tmp\/git-testing123"
}
[ 2018-04-16T14:10:51+00:00 -- 0 ] About to PHPCS-scan all files affected by any of the Pull-Requests; {
[ 2018-04-16T14:10:51+00:00 -- 0 ] About to PHPCS-scan all files affected by any of the pull requests; {
"repo_owner": "mygithubuser",
"repo_name": "testing123",
"commit_id": "f978c17f8f648e5ce10aa16694c74a5544b1670e"
Expand Down
11 changes: 0 additions & 11 deletions ap-file-types.php
Original file line number Diff line number Diff line change
Expand Up @@ -101,17 +101,6 @@ function vipgoci_ap_file_types(
}
}

/*
* Reduce memory-usage as possible.
*/
unset( $prs_implicated );
unset( $pr_diff );
unset( $pr_item );
unset( $pr_diff_file_extension );
unset( $pr_diff_file_name );

gc_collect_cycles();

vipgoci_runtime_measure( VIPGOCI_RUNTIME_STOP, 'ap_file_types' );
}

17 changes: 3 additions & 14 deletions ap-hashes-api.php
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ function vipgoci_ap_hashes_api_file_approved(

/**
* Scan a particular commit, look for altered
* files in the Pull-Request we are associated with
* files in the pull request we are associated with
* and for each of these files, check if they
* are approved in the hashes-to-hashes API.
*
Expand All @@ -267,7 +267,7 @@ function vipgoci_ap_hashes_api_scan_commit(
vipgoci_runtime_measure( VIPGOCI_RUNTIME_START, 'hashes_api_scan' );

vipgoci_log(
'Scanning altered or new files affected by Pull-Request(s) ' .
'Scanning altered or new files affected by pull request(s) ' .
'using hashes-to-hashes API',
array(
'repo_owner' => $options['repo-owner'],
Expand Down Expand Up @@ -333,7 +333,7 @@ function vipgoci_ap_hashes_api_scan_commit(

/*
* Add the file to a list of approved files
* of these affected by the Pull-Request.
* of these affected by the pull request.
*/
if ( true === $approval_status ) {
vipgoci_log(
Expand Down Expand Up @@ -366,17 +366,6 @@ function vipgoci_ap_hashes_api_scan_commit(
}
}

/*
* Reduce memory-usage as possible
*/
unset( $prs_implicated );
unset( $pr_item );
unset( $pr_diff );
unset( $pr_diff_contents );
unset( $approval_status );

gc_collect_cycles();

vipgoci_runtime_measure( VIPGOCI_RUNTIME_STOP, 'hashes_api_scan' );
}

21 changes: 4 additions & 17 deletions ap-nonfunctional-changes.php
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ function vipgoci_ap_nonfunctional_changes(

/*
* Save contents of version of
* file at the base of the Pull-Request
* file at the base of the pull request
* to a temporary file ("old version").
*/

Expand Down Expand Up @@ -141,7 +141,7 @@ function vipgoci_ap_nonfunctional_changes(

/*
* Save contents of version of
* file at the head of the Pull-Request
* file at the head of the pull request
* to a temporary file ("new version").
*/

Expand Down Expand Up @@ -182,9 +182,9 @@ function vipgoci_ap_nonfunctional_changes(

/*
* Check if the version at the base of
* of the Pull-Request ("old version")
* of the pull request ("old version")
* is the same as the latest version at
* the head of the Pull-Request ("new version")
* the head of the pull request ("new version")
* are exactly the same, given that we remove
* all whitespacing changes.
*/
Expand Down Expand Up @@ -225,19 +225,6 @@ function vipgoci_ap_nonfunctional_changes(
}
}

/*
* Reduce memory-usage as possible
*/
unset( $prs_implicated );
unset( $pr_diff );
unset( $pr_item );
unset( $pr_diff_file_extension );
unset( $pr_diff_file_name );
unset( $tmp_file_old );
unset( $tmp_file_new );

gc_collect_cycles();

vipgoci_runtime_measure( VIPGOCI_RUNTIME_STOP, 'ap_nonfunctional_changes' );
}

13 changes: 0 additions & 13 deletions ap-svg-files.php
Original file line number Diff line number Diff line change
Expand Up @@ -186,19 +186,6 @@ function vipgoci_ap_svg_files(
}
}

/*
* Reduce memory-usage as possible
*/
unset( $tmp_scan_results );
unset( $prs_implicated );
unset( $pr_diff );
unset( $pr_item );
unset( $pr_diff_file_extension );
unset( $pr_diff_file_name );
unset( $file_issues_arr_master );

gc_collect_cycles();

vipgoci_runtime_measure( VIPGOCI_RUNTIME_STOP, 'ap_svg_files' );
}

68 changes: 26 additions & 42 deletions auto-approval.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
declare(strict_types=1);

/**
* Pull-Request is not approved,
* Pull request is not approved,
* remove label if needed, leave messages
* on files that are approved, dismiss
* any previously approving PRs.
Expand Down Expand Up @@ -52,7 +52,7 @@ function vipgoci_auto_approval_non_approval(
'pull/' . (int) $pr_item->number;

vipgoci_log(
'Will not auto-approve Pull-Request #' .
'Will not auto-approve pull request #' .
(int) $pr_item->number . ' ' .
'as it contains ' .
'files which are not ' .
Expand All @@ -67,8 +67,7 @@ function vipgoci_auto_approval_non_approval(
'files_seen' => $files_seen,
'pr_files_changed' => $pr_files_changed,
),
0,
true // Send to IRC.
0
);

if ( false === $pr_label ) {
Expand Down Expand Up @@ -107,7 +106,7 @@ function vipgoci_auto_approval_non_approval(
) {
/*
* Make sure that the file was
* really altered in the Pull-Request;
* really altered in the pull request;
* this is to avoid any errors when
* submitting inline comments.
*/
Expand All @@ -119,7 +118,7 @@ function vipgoci_auto_approval_non_approval(
vipgoci_log(
'Not adding auto-approved in hashes-api ' .
'database to results for file, as it ' .
'was not altered by the Pull-Request',
'was not altered by the pull request',
array(
'pr_number' => $pr_item->number,
'file_name' => $approved_file,
Expand Down Expand Up @@ -207,12 +206,12 @@ function vipgoci_auto_approval_non_approval(
}

/*
* Get any approving reviews for the Pull-Request
* Get any approving reviews for the pull request
* submitted by us. Then dismiss them.
*/
vipgoci_log(
'Dismissing any approving reviews for ' .
'the Pull-Request, as it is not ' .
'the pull request, as it is not ' .
'approved anymore',
array(
'pr_number' => $pr_item->number,
Expand Down Expand Up @@ -249,7 +248,7 @@ function vipgoci_auto_approval_non_approval(
}

/**
* Approve a particular Pull-Request,
* Approve a particular pull request,
* alter label for the PR if needed,
* remove old comments, and log everything
* we do.
Expand Down Expand Up @@ -303,7 +302,7 @@ function vipgoci_autoapproval_do_approve(
'pull/' . (int) $pr_item->number;

vipgoci_log(
'Will auto-approve Pull-Request #' .
'Will auto-approve pull request #' .
(int) $pr_item->number . ' ' .
'as it alters or creates ' .
'only files that can be ' .
Expand All @@ -318,13 +317,12 @@ function vipgoci_autoapproval_do_approve(
'auto_approved_files_arr' => $auto_approved_files_arr,
'files_seen' => $files_seen,
),
0,
true // Send to IRC.
0
);

/*
* Actually approve, if not in dry-mode.
* Also add a label to the Pull-Request
* Also add a label to the pull request
* if applicable.
*/
vipgoci_github_approve_pr(
Expand All @@ -333,7 +331,7 @@ function vipgoci_autoapproval_do_approve(
$options['token'],
$pr_item->number,
$options['commit'],
'Auto-approved Pull-Request #' .
'Auto-approved pull request #' .
(int) $pr_item->number . ' as it ' .
'contains only auto-approvable files' .
' -- either pre-approved files' .
Expand Down Expand Up @@ -364,7 +362,7 @@ function( $type ) {
);
} else {
vipgoci_log(
'Will not actually approve Pull-Request #' .
'Will not actually approve pull request #' .
(int) $pr_item->number .
', as it is already approved by us',
array(
Expand All @@ -376,13 +374,12 @@ function( $type ) {
'auto_approved_files_arr' => $auto_approved_files_arr,
'files_seen' => $files_seen,
),
0,
true
0
);
}

/*
* Add label to Pull-Request, but
* Add label to pull request, but
* only if it is not associated already.
* If it is already associated, just log
* that fact.
Expand Down Expand Up @@ -418,7 +415,7 @@ function( $type ) {
'Removing any previously submitted comments ' .
'indicating that a particular file ' .
'is approved as the whole ' .
'Pull-Request is approved',
'pull request is approved',
array(
'pr_number' => $pr_item->number,
)
Expand Down Expand Up @@ -452,7 +449,7 @@ function( $type ) {
}

/**
* Process auto-approval(s) of the Pull-Request(s)
* Process auto-approval(s) of the pull request(s)
* involved with the commit specified.
*
* @param array $options Array of options.
Expand All @@ -469,7 +466,7 @@ function vipgoci_auto_approval_scan_commit(
vipgoci_runtime_measure( VIPGOCI_RUNTIME_START, 'auto_approve_commit' );

vipgoci_log(
'Doing auto-approval',
'Performing auto-approval',
array(
'repo_owner' => $options['repo-owner'],
'repo_name' => $options['repo-name'],
Expand Down Expand Up @@ -508,7 +505,7 @@ function vipgoci_auto_approval_scan_commit(

/*
* Loop through all files that are
* altered by the Pull-Request, look for
* altered by the pull request, look for
* files that can be auto-approved.
*/
foreach ( $pr_diff['files'] as
Expand Down Expand Up @@ -551,7 +548,7 @@ function vipgoci_auto_approval_scan_commit(
'pull/' . (int) $pr_item->number;

vipgoci_log(
'No action taken with Pull-Request #' .
'No action taken with pull request #' .
(int) $pr_item->number . ' ' .
'since no files were found' .
' -- PR URL: ' . $tmp_github_url,
Expand All @@ -561,8 +558,7 @@ function vipgoci_auto_approval_scan_commit(
'pr_number' => (int) $pr_item->number,
'pr_diff' => $pr_diff,
),
0,
true
0
);
} elseif (
( true === $did_foreach ) &&
Expand Down Expand Up @@ -593,23 +589,6 @@ function vipgoci_auto_approval_scan_commit(
unset( $files_seen );
}

/*
* Reduce memory-usage as possible
*/

unset( $pr_diff );
unset( $pr_diff_file_name );
unset( $pr_diff_contents );
unset( $pr_item );
unset( $pr_label );
unset( $prs_implicated );
unset( $files_seen );
unset( $did_foreach );
unset( $can_auto_approve );
unset( $tmp_github_url );

gc_collect_cycles();

vipgoci_runtime_measure( VIPGOCI_RUNTIME_STOP, 'auto_approve_commit' );
}

Expand Down Expand Up @@ -710,5 +689,10 @@ function vipgoci_auto_approval_process(
$results,
$auto_approved_files_arr
);

vipgoci_log(
'Auto-approval process complete',
array()
);
}

7 changes: 6 additions & 1 deletion defines.php
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
define(
'VIPGOCI_REVIEW_COMMENTS_TOTAL_MAX',
'Total number of active review comments per ' .
'Pull-Request has been reached and some ' .
'pull request has been reached and some ' .
'comments might not appear as a result. ' .
'Please resolve some issues to see more'
);
Expand Down Expand Up @@ -84,6 +84,11 @@
'<br />'
);

define(
'VIPGOCI_OUT_OF_MEMORY_ERROR',
'Unable to analyze the pull request due to resource constraints. The pull request may be too large to process. Please try submitting a smaller pull request'
);

define(
'VIPGOCI_NO_ISSUES_FOUND_MSG_AND_NO_REVIEWS',
'No issues were found to report when scanning latest commit'
Expand Down
Loading