Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Bazel itself build under an output base with Unicode characters #24457

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Nov 22, 2024

  • All decompression functions used in repository rules are modified to go through Path#getInputStream(), thus avoiding string encoding issues when constructing a FileInputStream manually.
  • A new Path#createTempDirectory function is introduced to replace error-prone usages of Files.createTempDirectory throughout the code base, which requires reencoding of both the arguments and the return value.
  • Fix encoding of argv0 in SubprocessBuilder and WorkerMultiplexer.
  • Fix encoding of the path of and the content in the server log file.
  • Fix encoding of info output.

Work towards #374
Fixes #24444

@fmeum fmeum changed the title Fix even more Unicode bugs Make Bazel itself buildable under an output base with Unicode characters Nov 22, 2024
@fmeum fmeum changed the title Make Bazel itself buildable under an output base with Unicode characters Make Bazel itself build under an output base with Unicode characters Nov 22, 2024
@fmeum
Copy link
Collaborator Author

fmeum commented Nov 22, 2024

I suspect that the RBE failure is due to the remote execution environment not having the C.UTF-8 locale installed which is forced for Java compilation actions. Would it be possible to add it?

@tjgq
Copy link
Contributor

tjgq commented Nov 22, 2024

I suspect that the RBE failure is due to the remote execution environment not having the C.UTF-8 locale installed which is forced for Java compilation actions. Would it be possible to add it?

I think that entails creating a new docker image for RBE (or finding a suitable existing one). @coeuvre, do you know how to do this?

If this is blocking for 8.0.0, I suggest finding a way to either detect the existence of the locale in the test, or disable it when on RBE.

@fmeum
Copy link
Collaborator Author

fmeum commented Nov 22, 2024

The test is failing on other platforms, so I'll look into that. It's not a hard blocker for 8.0.0, it can equally well go into 8.1.0.

@fmeum
Copy link
Collaborator Author

fmeum commented Nov 25, 2024

This needs a fix in rules_java to properly set a UTF-8 environment for bootstrap actions: bazelbuild/rules_java#243

copybara-service bot pushed a commit to bazelbuild/rules_java that referenced this pull request Nov 27, 2024
Copybara Import from #243

BEGIN_PUBLIC
Build bootclasspath in a UTF-8 environment (#243)

`java` and `javac` convert file and classpaths to absolute paths and thus require a UTF-8 locale to work under a path that contains non-ASCII characters.

Unblocks bazelbuild/bazel#24457

Closes #243
END_PUBLIC

COPYBARA_INTEGRATE_REVIEW=#243 from fmeum:utf8-environment 05813e4
PiperOrigin-RevId: 700695134
Change-Id: I2f5753720ec3c838a4dd8b6aabf1050c6935ef3d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants