Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Make scrub_preprocessed_java_class local only
Summary: Noticed several continuous release builds failing with errors similar to: ``` [2023-08-11T14:29:12.035-07:00] Waiting on fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) -- action (preprocess_java_classes) [re_worker_upload], and 0 other actions [2023-08-11T14:29:19.036-07:00] Waiting on fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) -- action (scrub_preprocessed_java_class 29_statics_internal.jar) [re_action_cache], and 405 other actions [2023-08-11T14:29:26.136-07:00] Waiting on fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) -- action (scrub_preprocessed_java_class 6_preconditions.jar) [re_upload], and 32214 other actions [2023-08-11T14:29:33.235-07:00] Waiting on fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) -- action (scrub_preprocessed_java_class 6_preconditions.jar) [re_upload], and 23739 other actions [2023-08-11T14:29:40.335-07:00] Waiting on fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) -- action (scrub_preprocessed_java_class 344_canaries.jar) [re_upload], and 15291 other actions [2023-08-11T14:29:47.336-07:00] Waiting on fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) -- action (scrub_preprocessed_java_class 34295_multiverse.jar) [re_upload], and 7010 other actions [2023-08-11T14:29:48.875-07:00] Action failed: fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) (scrub_preprocessed_java_class 29938_FBReactNativeFXAccessLibrarySpecAndroid.jar) [2023-08-11T14:29:48.877-07:00] Stderr: dotslash error: problem with `/re_cwd/third-party/toolchains/jdk/dotslash_java` [2023-08-11T14:29:48.877-07:00] caused by: failed to download artifact [2023-08-11T14:29:48.877-07:00] caused by: backend fetch failed [2023-08-11T14:29:48.877-07:00] caused by: CASDaemonService::materializeDotslashAndCache failed with ApplicationException [2023-08-11T14:29:48.877-07:00] caused by: Loadshedding: load shedding due to qps limit [2023-08-11T14:29:48.987-07:00] Action failed: fbsource//fbandroid/apps/fb4a:fbandroid_no_face_e2e_staticdi_dextr_splitarsc_dlstr_xzs_hdpi_x86_release (fbsource//tools/build_defs/config/platform/android:fb4a#e9f2c405f2ad03ef) (scrub_preprocessed_java_class 28507_bkactioncaaretrieveblockstore.jar) ... followed by several instances of the same error on different 'scrub_preprocessed_java_class' actions) ``` https://www.internalfb.com/skycastle/run/3012908150711616126 It seems likely that as soon as `preprocess_java_classes` finishes more than 30k `scrub_preprocessed_java_class` actions are scheduled at the same time, all of which run on RE, which all try to fetch the same dotslash artifact at the same time and hit qps limits. We should be able to work around this by just running it locally. Reviewed By: IanChilds Differential Revision: D48283036 fbshipit-source-id: 19b2602b60c535bdc1337d86b3744c1bbf53baf9
- Loading branch information