Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a new tag, Tag_RISCV_x3_reg_usage to record how gp/x3 begin … #387

Merged
merged 1 commit into from
Aug 29, 2023

Conversation

kito-cheng
Copy link
Collaborator

…used.

This tag is designed to prevent the linker from merging objects that use gp for different purposes. Additionally, it can also serve to help users identify the usage within the executable.

Copy link
Collaborator

@luismarques luismarques left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some editing suggestions.

riscv-elf.adoc Outdated Show resolved Hide resolved
riscv-elf.adoc Outdated Show resolved Hide resolved
@kito-cheng
Copy link
Collaborator Author

@luismarques applied, thanks :)

Copy link
Contributor

@ilovepi ilovepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left some minor suggestions for phrasing, and a question regarding merging.

This is the solution we were hoping for in Fuchsia, so I'm glad to see this PR. Thanks for taking the time to do this @kito-cheng.

riscv-elf.adoc Outdated Show resolved Hide resolved
riscv-elf.adoc Outdated Show resolved Hide resolved
riscv-elf.adoc Outdated Show resolved Hide resolved
riscv-elf.adoc Outdated Show resolved Hide resolved
riscv-elf.adoc Outdated Show resolved Hide resolved
riscv-elf.adoc Outdated Show resolved Hide resolved
@kito-cheng kito-cheng changed the title Introduce a new tag, Tag_RISCV_gp_reg_usage to record how gp begin … Introduce a new tag, Tag_RISCV_x3_reg_usage to record how gp/x3 begin … Jun 12, 2023
@kito-cheng
Copy link
Collaborator Author

Changes:

  • Rename gp into x3 to prevent misleading, e.g. using gp as shadow stack pointer seems weird.
  • Update merge rule: explicitly allow merge 0 with 1 and 2 only.

@MaskRay
Copy link
Collaborator

MaskRay commented Jun 12, 2023

Thank you for making GP more useful, especially when many folks don't consider re-purposing GP as an "ABI break" on their platforms.

I still hold the opinion from #371 (comment) that a tag in the attribute section may not be very useful, more so if we emit it by default for -mrelax.
It will increase a few bytes in every relocatable object file, and if we make SHT_RISCV_ATTRIBUTE allocable, waste some bytes in the linked image.

At the moment, I don't have a strong opinion on whether a software shadow stack based on GP should emit a tag by default. If a user chooses to do so and enables --relax-gp (which is the current default in GNU ld), they will encounter an error. However, for ld.lld, which doesn't enable global pointer relaxation by default, this would be more like unnecessary work. (There are many combinations of linker options that can produce broken output, and it is simply not feasible to diagnose every invalid case.)

@sorear
Copy link
Collaborator

sorear commented Jun 13, 2023

Why would -mrelax generate the attribute tag when -mrelax doesn't generate explicit uses of gp? The gp references are not, with the exception of the single point where gp is initialized, generated until link time so the attribute tag shouldn't be created until then either.

@asb
Copy link
Collaborator

asb commented Jun 14, 2023

To ask a slightly higher level question - are we sure we want to go down the route of picking out every potential ABI option and specifying it in attributes (which seems to be the route this and the atomics PR seems to be going down). I can believe it's better this way, but it would be reassuring to know if people spent time considering an alternative that e.g. just specified the platform ABI targeted by a given ELF (potentially with an ABI version).

@palmer-dabbelt
Copy link
Contributor

To ask a slightly higher level question - are we sure we want to go down the route of picking out every potential ABI option and specifying it in attributes (which seems to be the route this and the atomics PR seems to be going down). I can believe it's better this way, but it would be reassuring to know if people spent time considering an alternative that e.g. just specified the platform ABI targeted by a given ELF (potentially with an ABI version).

That's kind of a broader issue, though: coupling together features into a set requires getting consensus on what those sets should be and there's just no mechanism for doing that in RISC-V land. It's essentially the same problem that's lead to an explosion of extensions, just more visible because it's ABI and we're all used to ABI proliferation being a mess.

So I think we just need to suck it up and live with the mess, it's already defacto there from the extensions. We can just punt to the distros to group features into sets and then go deprecate everything else in a few years when it's sufficiently broken.

@kito-cheng
Copy link
Collaborator Author

This issue could break into two separate parts IMO: bare-metal and UNIX-like platforms (Linux, *BSD, Android...)

For bare-metal: The answer is yes; we are going to identify potential ABI options and add attributes to let users configure what they want; that's is the scope of EABI; this part will drive by IAR folks, and I believe this is not the issue we concern here, so I would not spend too much word here :)

For UNIX-like platforms: I would like to minimalize this IF possible; however, as you mentioned, we already have atomic and this PR:

Do we have more ABI options for UNIX-like platforms? It's hard to guarantee to say no more, but this will be carefully considered when introducing new ABI options.

But could we prevent those EABI options from being applied on UNIX-like platforms? It's hard to stop people from doing so; from the psABI aspect, we can add NOTE to mention this is EABI, and not recommended applied on UNIX-like platforms.

but it would be reassuring to know if people spent time considering an alternative that e.g. just specified the platform ABI targeted by a given ELF (potentially with an ABI version).

That would be useful; we could create a separated issue to track that

@kito-cheng
Copy link
Collaborator Author

@asb @MaskRay did you mind give explicitly LGTM if it's OK from the LLVM land. we are OK with that in GNU land, and then I'll gonna prepare PoC for this :)

…used.

This tag is designed to prevent the linker from merging objects that use `gp`
for different purposes. Additionally, it can also serve to help users identify
the usage within the executable.
@kito-cheng
Copy link
Collaborator Author

Changes:

-Rebase and squash patches

@kito-cheng
Copy link
Collaborator Author

binutils PoC: kito-cheng/riscv-binutils-gdb@d13849c

@kito-cheng
Copy link
Collaborator Author

ping @asb @MaskRay, this will be merged if no objection this week.

@kito-cheng kito-cheng merged commit 30746d4 into master Aug 29, 2023
@kito-cheng kito-cheng deleted the tag_riscv_gp-usage branch August 29, 2023 14:55
@MaskRay
Copy link
Collaborator

MaskRay commented Aug 30, 2023

That's kind of a broader issue, though: coupling together features into a set requires getting consensus on what those sets should be and there's just no mechanism for doing that in RISC-V land. It's essentially the same problem that's lead to an explosion of extensions, just more visible becaus

This is exactly my concern. In another thread (I don't recall which) I mentioned that compilers cannot determine whether inline assembly uses x3 and in what way.

I am sorry that I haven't replied earlier. I think I am ok if psABI adds Tag_RISCV_x3_reg_usage, as long as if compilers don't emit Tag_RISCV_x3_reg_usage, as that just doesn't seem useful... and purely unneeded code size for many systems.

If folks loving gp wants to an opt-in option to emit Tag_RISCV_x3_reg_usage, I think I'm fine as well, as a compromise...

@palmer-dabbelt
Copy link
Contributor

That's kind of a broader issue, though: coupling together features into a set requires getting consensus on what those sets should be and there's just no mechanism for doing that in RISC-V land. It's essentially the same problem that's lead to an explosion of extensions, just more visible becaus

This is exactly my concern. In another thread (I don't recall which) I mentioned that compilers cannot determine whether inline assembly uses x3 and in what way.

We can't really expect compilers to be able to tell if inline assembly violates any ABI constraint. It's inline assembly anything goes.

I am sorry that I haven't replied earlier. I think I am ok if psABI adds Tag_RISCV_x3_reg_usage, as long as if compilers don't emit Tag_RISCV_x3_reg_usage, as that just doesn't seem useful... and purely unneeded code size for many systems.

If folks loving gp wants to an opt-in option to emit Tag_RISCV_x3_reg_usage, I think I'm fine as well, as a compromise...

@kito-cheng
Copy link
Collaborator Author

This is exactly my concern. In another thread (I don't recall which) I mentioned that compilers cannot determine whether inline assembly uses x3 and in what way.

We can't really expect compilers to be able to tell if inline assembly violates any ABI constraint. It's inline assembly anything goes.

Yeah, I think user could write any ABI violation code as they want, so I don't really concern about that - user should handle those inline asm very carefully since most compiler didn't really know what inline asm write - we all rely on the info what user give.

And my thought on that tag other than the binutils patch, the binutils patch is just basic tag parsing and merge rule, which not cover all thing we should do:

  • Linker should NOT perform gp relaxation if any input object file contain Tag_RISCV_x3_reg_usage other than 0 (unknown or unset) and 1 (gp relax).
  • Linker should set Tag_RISCV_x3_reg_usage to 1 if gp relaxation has perform.
  • If -mno-gp-relax is given (assembler or compiler), then Tag_RISCV_x3_reg_usage should explicitly set to 0 for the final output.
  • Compiler should emit Tag_RISCV_x3_reg_usage as 2 if it use as shadow stack.

@MaskRay
Copy link
Collaborator

MaskRay commented Aug 30, 2023

This is exactly my concern. In another thread (I don't recall which) I mentioned that compilers cannot determine whether inline assembly uses x3 and in what way.

We can't really expect compilers to be able to tell if inline assembly violates any ABI constraint. It's inline assembly anything goes.

Yeah, I think user could write any ABI violation code as they want, so I don't really concern about that - user should handle those inline asm very carefully since most compiler didn't really know what inline asm write - we all rely on the info what user give.

This is actually my argument against Tag_RISCV_x3_reg_usage.
If users twiddle with x3 in inline asm to be incompatible with global pointer relaxation, they unlikely annotate the translation unit as Tag_RISCV_x3_reg_usage.
On the other hand, if they add an .attribute directive in inline asm, the compiler cannot know it and suppress its default .attribute.
When the two .attribute conflict, should the assembler report an error?

Let's visit the current defined 4 values. 2 (shadow stack) can be determined by a driver option, while the other 3 are really indistinguishable from the compiler.

0:: This object uses x3 as a fixed register with unknown purpose.
1:: This object uses x3 as the global pointer, for relaxation purposes.
3:: This object uses X3 as a temporary register.

And my thought on that tag other than the binutils patch, the binutils patch is just basic tag parsing and merge rule, which not cover all thing we should do:

  • Linker should NOT perform gp relaxation if any input object file contain Tag_RISCV_x3_reg_usage other than 0 (unknown or unset) and 1 (gp relax).
  • Linker should set Tag_RISCV_x3_reg_usage to 1 if gp relaxation has perform.
  • If -mno-gp-relax is given (assembler or compiler), then Tag_RISCV_x3_reg_usage should explicitly set to 0 for the final output.
  • Compiler should emit Tag_RISCV_x3_reg_usage as 2 if it use as shadow stack.

OK, if -mno-gp-relax is introduced, value 1 will now be distinguishable with others.
However, I am still questioning whether this option is really useful.
It presumably doesn't do anything except emit an .attribute. If .attribute is present in inline asm (due to x3 twidding), do we override the 0 value with the user-requested value?

IMHO, an option that does nothing, but just mark something, is rarely a good design. It just restricts choices.

If the compiler emits an .attribute by default, it's small overhead to every operating system that doesn't use global pointer relaxation.
Sure, it's negligible overhead, but this reminds me of https://www.airs.com/blog/archives/518

"These days we could probably change the default: we could probably say that if an object file does not have a .note.GNU-stack section, then it does not require an executable stack."

GNU ld didn't do this, and we are still getting churn in binutils for --warn-execstack, -z noexecstack, and related options.

Let me conclude with an end note from https://www.openwall.com/lists/musl/2023/05/03/1 : how musl rejected glibc-style symbol twidding idea for C23 binary integer binary compatibility.

"""There are not going to be different versions of scanf/strto* because
there's just no way to do that in a conforming way (the standard
allows declaring these functions yourself without including the
header, which would not get the remapping). As above, strict
conformance to outdated versions of the standard is just not a
priority. musl's claim/target is conformance to current versions only
and sometimes, on a case-by-case basis, partial support for older
ones. (Looking at it again, I don't understand how the code in your
repo was actually intended to provide different versions. __intscan,
etc. are not public interface boundaries and references to them never
appear in applications, only internally within libc. Your code seems
to be conditioning which gets used based on the STDC version in use
for building musl, which is completely decoupled from the version of
the standard that a given application is being built/linked for.)
"""

Our GP case does is not the same as that scenario, but we can a lot of insights from the reasoning above.

For global pointer relaxation users who may use x3 for other purposes, they get crash as they desire.
aaelf64 reserves .ARM.attributes but the toolchain never uses it. I think they have found drawbacks in aaelf32's uses.

@aswaterman
Copy link
Contributor

If folks loving gp

You mean folks who are using the ABI as specified? We all know you dislike the ABI, but you should characterize your position a bit more forthrightly.

@asb
Copy link
Collaborator

asb commented Sep 4, 2023

ping @asb @MaskRay, this will be merged if no objection this week.

Now I'm back from vacation, just noting for the record I've got no objection to this flag (and sorry for missing the earlier ping!).

@MaskRay
Copy link
Collaborator

MaskRay commented Sep 4, 2023

You mean folks who are using the ABI as specified? We all know you dislike the ABI, but you should characterize your position a bit more forthrightly.

I am forthright that I don't think global pointer relaxation provides sufficient valuable.
I am forthright that I don't get why marking object files with Tag_RISCV_x3_reg_usage is useful, and my arguments have been provided above.

I do worry that Clang produced relocatable object files will be annotated as Tag_RISCV_x3_reg_usage and lld will somehow need to parse it. I think this is busy work. If the toolchain will not be patched to add Tag_RISCV_x3_reg_usage in the default mode, I am happy to take a compromise and not object to this tag in the ABI.

@kito-cheng
Copy link
Collaborator Author

@MaskRay I feel there's two things mixed in your comment, one concern for Tag_RISCV_x3_reg_usage and one concern for ELF attribute, so let us break it down into two parts to discuss.

Let me describe the ELF attribute first. I think we have discussed this in different PRs/issues, and I know you are not really convinced that we need that.

There are many different containers in ELF format which can carry information like e_flags, properties, ELF attributes, and many different .note.* sections.

And we pick ELF attributes at beginning due to following reason:

  1. It’s implemented in existing toolchain
  2. It’s allow store string and arbitrary length of integer (by ULEB128 format)

However I know it’s come with some drawback like:

  1. It is not easy to do random access, it must parse from the beginning.
  2. NO existing parser implementation within dynamic linker (like glibc)

So we may consider introducing ELF properties to make dynamic linker easier to read, however I think we could keep using ELF attributes, and define some guidelines to clarify how to decide what information/tag should add to attribute or property.

The guideline in my mind is:

  • The tag should go property if it must read the information at dynamic linking time, otherwise use ELF attribute.
  • If the information is only used by static linker and other static time tools like disassembler - then use ELF attribute.

Back to Tag_RISCV_x3_reg_usage, that info mostly used by the static linker to determine the GP relaxation should be performed or not, does it introduce lot of complexity of the linker implementation, I expect that won’t be terrible high cost, we already have RVC flag to determine if the compressed instruction can be used within linker relaxation.

The first use case I expect is (soft) shadow stack, compiler will emit Tag_RISCV_x3_reg_usage when they begin use x3 as shadows stack pointer, then it will suppress linker to perform GP relaxation even -mno-gp-relax is not given.

Without that tag, users must be careful to make sure -mno-gp-relax is given when (soft) shadow stack enabled - otherwise the program might cause random bugs due to gp relaxation, but that could be prevented at the beginning if we have the tag.

For inline asm and hand craft assembly files…I tend to ignore those cases since I believe no matter what mechanism is used, users could play tricks to violate or ignore that.

@MaskRay
Copy link
Collaborator

MaskRay commented Sep 8, 2023

@MaskRay I feel there's two things mixed in your comment, one concern for Tag_RISCV_x3_reg_usage and one concern for ELF attribute, so let us break it down into two parts to discuss.

My two concerns are:

  • whether Tag_RISCV_x3_reg_usage is useful
  • in what cases users will bear the (small) overhead of Tag_RISCV_x3_reg_usage

I do agree that if we ever need an ABI marker, build attributes is the best choice. Actually, no other alternative is ever close to be competitive.

I am more concerned with the second point.
What relocatable object files are going to have Tag_RISCV_x3_reg_usage?

I re-read the comments and think my
commenting "OK, if -mno-gp-relax is introduced, value 1 will now be distinguishable with others."
was mistaken.

Linker should NOT perform gp relaxation if any input object file contain Tag_RISCV_x3_reg_usage other than 0 (unknown or unset) and 1 (gp relax).

OK, a bit of busy work in the linker to me, but I think this is acceptable.

Linker should set Tag_RISCV_x3_reg_usage to 1 if gp relaxation has perform.

I hope that in the absence of build attributes, the linker doesn't need to set Tag_RISCV_x3_reg_usage.

If -mno-gp-relax is given (assembler or compiler), then Tag_RISCV_x3_reg_usage should explicitly set to 0 for the final output.

0 is compatible with global pointer relaxation, then what does the 0 value do?

In the default mode (whether -mrelax or -mno-relax, but there is no -m[no-]gp-relax), is there a Tag_RISCV_x3_reg_usage attribute?

Compiler should emit Tag_RISCV_x3_reg_usage as 2 if it use as shadow stack.

This is small overhead, but I think it's fine.

wangliu-iscas pushed a commit to plctlab/patchwork-binutils-gdb that referenced this pull request Sep 17, 2023
The original discussion,
riscv-non-isa/riscv-elf-psabi-doc#387

This patch is just the first edition that I implemented after discussing it
with Kito Cheng in another thread.  So it will definitly have later editions
in the future.  I start by writing down the details that I remember. Maybe
there are some riscv-psabi that haven't been upadted or mentioned yet.

* Defined values of Tag_RISCV_x3_reg_usage.
For compiler/assembler,
0: default, don't set anything, so allow x3 for gp relaxations.
1: relaxation, allow x3 for gp relaxations.
4: reserved, don't allow x3 for gp relaxations.
others: unknown, report warning.

For linker,
1: relaxation, if doing any gp relaxations, then update/add elf x3_reg_usage
attribute to output files.
4: reserved, if we don't do any gp relaxations, then update/add elf
x3_reg_usage to output files.

If input files have unknown Tag_RISCV_x3_reg_usage values, then regard them
as default in linker, which means we allow to do gp relaxations.  Otherwise,
report error when input files have different Tag_RISCV_x3_reg_usage values.

* The new linker will always generate Tag_RISCV_x3_reg_usage for the output files

* The priority of elf x3_reg_usage attribute and ld option --[no-]gp-relax.
Same as elf arch attribute, elf x3_reg_usage attribute > ld option --[no-]gp-relax.

* ...

Co-authored-by: Kito Cheng <[email protected]>

bfd/
	* cpu-riscv.c (riscv_elf_is_unknown_x3_reg_usage): New function.
	Check if the value of Tag_RISCV_x3_reg_usage is defined or not.
	* cpu-riscv.h (enum riscv_x3_reg_usage): Defined to record the
	usage of x3.
	* elfnn-riscv.c (riscv_elf_link_hash_table): New int x3_reg_usage,
	set to X3_RELAX if doing any gp relaxation.  Otherwise set it to
	X3_RESERVED.
	(riscv_merge_attributes): Handle Tag_RISCV_x3_reg_usage.  Regared
	the undefined value as 0, which is the default value.  Report error
	if two objects have different Tag_RISCV_x3_reg_usage values, except
	default 0.  Disable gp relaxations if Tag_RISCV_x3_reg_usage value
	is X3_RESERVED.
	(_bfd_riscv_relax_lui): Set x3_reg_usage of riscv_elf_link_hash_table
	to X3_RELAX if doing any gp relaxation.
	(_bfd_riscv_relax_pc): Likewise.
	(bfd_elfNN_riscv_update_x3_reg_usage): New function.  Called by
	after_allocation, after all the relaxations (ldelf_map_segments)
	finishing, to update the value of output Tag_RISCV_x3_reg_usage.
	* elfxx-riscv.h:  Defined extern bfd_elf[32|64]_riscv_update_x3_reg_usage.
binutils/
	* readelf.c (riscv_attr_tag_t): Added T(x3_reg_usage).
	(display_riscv_attribute): Print correct Tag_RISCV_x3_reg_usage
	information.
gas/
	* config/tc-riscv.c (riscv_convert_symbolic_attribute): Added
	T(x3_reg_usage).
	(s_riscv_attribute): Report warning if value of Tag_RISCV_x3_reg_usage
	is unknown.
	* testsuite/gas/riscv/attribute-x3-reg-usage-*: New testcases.
include/
	* elf/riscv.h (Tag_RISCV_x3_reg_usage): Defined to 16.
ld/
	* emultempl/riscvelf.em: Called bfd_elf[32|64]_riscv_update_x3_reg_usage
	in after_allocation, after ldelf_map_segments.
	* testsuite/ld-riscv-elf/attr-merge-arch-*: Updated since we always
	generate Tag_RISCV_x3_reg_usage now for the output in ld.
	* testsuite/ld-riscv-elf/attr-merge-priv-spec-*: Likewise.
	* testsuite/ld-riscv-elf/attr-merge-strict-align-*: Likewise.
	* testsuite/ld-riscv-elf/attr-merge-user-ext-01.d: Likewise.
	* testsuite/ld-riscv-elf/attr-merge-x3-reg-usage-*: New testcases.
	* testsuite/ld-riscv-elf/ld-riscv-elf.exp: Updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants