Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create object files internally to avoid invoking GAS #757

Merged
merged 33 commits into from
Aug 3, 2022

Conversation

matthew4850
Copy link
Contributor

This PR uses the existing x86_binary_emitter to construct object files directly, allowing us to avoid calling out to GAS.
As can be seen in the table below this results in a substantial performance speed up.

File GAS Internal Assembler
Small file (2 lines) 7ms 1ms
Typecore.ml 208ms 76ms
Translmod.ml 48ms 8ms
Translcoreml 46ms 8ms

In the case of Typecore.ml the total build time reduced from 2.737 to 2.624 resulting overall in a ~4% speed up in build time for this file.

@mshinwell mshinwell marked this pull request as draft August 3, 2022 07:53
@mshinwell
Copy link
Collaborator

One detail is that this vendors owee (including in the boot compiler) and as a consequence needs #751.

I'll have to discuss with @let-def but some work probably needs doing to somehow unify owee and the existing DWARF libraries in the compiler, in due course. Two follow-on parts to this PR are going to involve DWARF work: dealing with the CFI directives and emitting the .debug_line information.

external/owee/owee_stubs.c Outdated Show resolved Hide resolved
external/owee/owee_elf.ml Outdated Show resolved Hide resolved
external/owee/owee_buf.ml Show resolved Hide resolved
external/owee/owee_buf.ml Outdated Show resolved Hide resolved
external/owee/owee_buf.ml Outdated Show resolved Hide resolved
backend/asmgen.ml Outdated Show resolved Hide resolved
backend/x86_binary_emitter.ml Outdated Show resolved Hide resolved
backend/x86_binary_emitter.ml Outdated Show resolved Hide resolved
backend/x86_binary_emitter.ml Outdated Show resolved Hide resolved
backend/amd64/emit.mlp Outdated Show resolved Hide resolved
Copy link
Contributor

@gretay-js gretay-js left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the files outside of internal_assembler and owee folders.
What is the diff between upstream owee and the vendored version?

driver/flambda_backend_args.ml Outdated Show resolved Hide resolved
backend/x86_proc.mli Outdated Show resolved Hide resolved
backend/x86_proc.mli Outdated Show resolved Hide resolved
dune Show resolved Hide resolved
backend/x86_dsl.mli Show resolved Hide resolved
backend/asmgen.ml Outdated Show resolved Hide resolved
backend/x86_proc.ml Outdated Show resolved Hide resolved
backend/x86_proc.ml Outdated Show resolved Hide resolved
@gretay-js
Copy link
Contributor

Are the speed-ups comparable, if the internal assembler doesn't generate debug_line sections? What portion of does gas spend in handling dwarf?

@mshinwell
Copy link
Collaborator

Are the speed-ups comparable, if the internal assembler doesn't generate debug_line sections? What portion of does gas spend in handling dwarf?

We don't know yet for sure, but I think there will still be a significant speed-up.

backend/internal_assembler/symbol_table.mli Show resolved Hide resolved
backend/internal_assembler/symbol_table.mli Show resolved Hide resolved
backend/internal_assembler/symbol_table.mli Outdated Show resolved Hide resolved
backend/internal_assembler/relocation_entry.mli Outdated Show resolved Hide resolved
external/owee/owee_buf.ml Outdated Show resolved Hide resolved
external/owee/owee_buf.ml Outdated Show resolved Hide resolved
backend/x86_dsl.mli Show resolved Hide resolved
driver/flambda_backend_args.ml Outdated Show resolved Hide resolved
backend/asmgen.ml Outdated Show resolved Hide resolved
backend/internal_assembler/section_table.ml Outdated Show resolved Hide resolved
backend/internal_assembler/relocation_entry.ml Outdated Show resolved Hide resolved
backend/internal_assembler/internal_assembler.ml Outdated Show resolved Hide resolved
backend/internal_assembler/internal_assembler.ml Outdated Show resolved Hide resolved
backend/internal_assembler/internal_assembler.ml Outdated Show resolved Hide resolved
@mshinwell mshinwell marked this pull request as ready for review August 3, 2022 16:34
@mshinwell mshinwell merged commit 9b7ebe7 into ocaml-flambda:main Aug 3, 2022
ccasin added a commit to ccasin/flambda-backend that referenced this pull request Aug 31, 2022
ccc356d flambda-backend: Prevent dynamic loading of the same .cmxs twice in private mode, etc. (ocaml-flambda#784)
14eb572 flambda-backend: Make local extension point equivalent to local_ expression (ocaml-flambda#790)
487d11b flambda-backend: Fix tast_iterator and tast_mapper for include functor. (ocaml-flambda#795)
a50a818 flambda-backend: Reduce closure allocation in List (ocaml-flambda#792)
96c9c60 flambda-backend: Merge ocaml-jst
a775b88 flambda-backend: Fix ocaml/otherlibs/unix 32-bit build (ocaml-flambda#767)
f7c2679 flambda-backend: Create object files internally to avoid invoking GAS (ocaml-flambda#757)
c7a46bb flambda-backend: Bugfix for Cmmgen.expr_size with locals (ocaml-flambda#756)
b337cb6 flambda-backend: Fix build_upstream for PR749 (ocaml-flambda#750)
8e7e81c flambda-backend: Differentiate is_int primitive between generic and variant-only versions (ocaml-flambda#749)

git-subtree-dir: ocaml
git-subtree-split: ccc356d
mshinwell added a commit to mshinwell/flambda-backend that referenced this pull request Oct 24, 2022
25188da flambda-backend: Missed comment from PR802 (ocaml-flambda#887)
9469765 flambda-backend: Improve the semantics of asynchronous exceptions (new simpler version) (ocaml-flambda#802)
d9e4dd0 flambda-backend: Fix `make runtest` on NixOS (ocaml-flambda#874)
4bbde7a flambda-backend: Simpler symbols (ocaml-flambda#753)
ef37262 flambda-backend: Add opaqueness to Obj.magic under Flambda 2 (ocaml-flambda#862)
a9616e9 flambda-backend: Add build system hooks for ocaml-jst (ocaml-flambda#869)
045ef67 flambda-backend: Allow the compiler to build with stock Dune (ocaml-flambda#868)
3cac5be flambda-backend: Simplify Makefile logic for natdynlinkops (ocaml-flambda#866)
c5b12bf flambda-backend: Remove unnecessary install lines (ocaml-flambda#860)
ff12bbe flambda-backend: Fix unused variable warning in st_stubs.c (ocaml-flambda#861)
c84976c flambda-backend: Static check for noalloc: attributes (ocaml-flambda#825)
ca56052 flambda-backend: Build system refactoring for ocaml-jst (ocaml-flambda#857)
39eb7f9 flambda-backend: Remove integer comparison involving nonconstant polymorphic variants (ocaml-flambda#854)
c102688 flambda-backend: Fix soundness bug by using currying information from typing (ocaml-flambda#850)
6a96b61 flambda-backend: Add a primitive to enable/disable the tick thread (ocaml-flambda#852)
f64370b flambda-backend: Make Obj.dup use a new primitive, %obj_dup (ocaml-flambda#843)
9b78eb2 flambda-backend: Add test for ocaml-flambda#820 (include functor soundness bug) (ocaml-flambda#841)
8f24346 flambda-backend: Add `-dtimings-precision` flag (ocaml-flambda#833)
65c2f22 flambda-backend: Add test for ocaml-flambda#829 (ocaml-flambda#831)
7b27a49 flambda-backend: Follow-up PR#829 (comballoc fixes for locals) (ocaml-flambda#830)
ad7ec10 flambda-backend: Use a custom condition variable implementation (ocaml-flambda#787)
3ee650c flambda-backend: Fix soundness bug in include functor (ocaml-flambda#820)
2f57378 flambda-backend: Static check noalloc (ocaml-flambda#778)
aaad625 flambda-backend: Emit begin/end region only when stack allocation is enabled (ocaml-flambda#812)
17c7173 flambda-backend: Fix .cmt for included signatures (ocaml-flambda#803)
e119669 flambda-backend: Increase delays in tests/lib-threads/beat.ml (ocaml-flambda#800)
ccc356d flambda-backend: Prevent dynamic loading of the same .cmxs twice in private mode, etc. (ocaml-flambda#784)
14eb572 flambda-backend: Make local extension point equivalent to local_ expression (ocaml-flambda#790)
487d11b flambda-backend: Fix tast_iterator and tast_mapper for include functor. (ocaml-flambda#795)
a50a818 flambda-backend: Reduce closure allocation in List (ocaml-flambda#792)
96c9c60 flambda-backend: Merge ocaml-jst
a775b88 flambda-backend: Fix ocaml/otherlibs/unix 32-bit build (ocaml-flambda#767)
f7c2679 flambda-backend: Create object files internally to avoid invoking GAS (ocaml-flambda#757)
c7a46bb flambda-backend: Bugfix for Cmmgen.expr_size with locals (ocaml-flambda#756)
b337cb6 flambda-backend: Fix build_upstream for PR749 (ocaml-flambda#750)
8e7e81c flambda-backend: Differentiate is_int primitive between generic and variant-only versions (ocaml-flambda#749)

git-subtree-dir: ocaml
git-subtree-split: 25188da
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants