-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
.eh_frame: Initial DWARF expression support
This commits adds partial support for DWARF expressions, in particular,
for unwind ranges that cover PLTs for the CFA calculation
(`DW_CFA_def_cfa_expression`).
Arbitrary expressions require a VM to evaluate them, which is not easy
to do in BPF, so after doing some quick data analysis (see below) it was
clear that by adding some hardcoded expressions to increase the success
ratio of the stack walker without bloating it too much. I don't think we
should add every "common" expression, but the PLT ones seemed like
good bang for the buck.
On my Fedora machine, these were the most common expressions for
binaries, and libraries, as well as for processes running:
```
[javierhonduco@fedora parca-agent]$ sudo readelf -wf /bin/* 2>/dev/null | grep DW_CFA_def_cfa_expression | sort | uniq -c | sort -k1h
1 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 160; DW_OP_deref)
5 DW_CFA_def_cfa_expression (DW_OP_breg4 (esp): 4; DW_OP_breg8 (eip): 0; DW_OP_lit15; DW_OP_and; DW_OP_lit9; DW_OP_ge; DW_OP_lit2; DW_OP_shl; DW_OP_plus)
5 DW_CFA_def_cfa_expression (DW_OP_breg5 (ebp): -16; DW_OP_deref)
12 DW_CFA_def_cfa_expression (DW_OP_breg6 (rbp): -40; DW_OP_deref)
62 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 8; DW_OP_breg16 (rip): 0; DW_OP_lit15; DW_OP_and; DW_OP_lit11; DW_OP_ge; DW_OP_lit3; DW_OP_shl; DW_OP_plus)
1673 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 8; DW_OP_breg16 (rip): 0; DW_OP_lit15; DW_OP_and; DW_OP_lit10; DW_OP_ge; DW_OP_lit3; DW_OP_shl; DW_OP_plus)
```
```
[javierhonduco@fedora parca-agent]$ sudo readelf -wf /lib64/* 2>/dev/null | grep DW_CFA_def_cfa_expression | sort | uniq -c | sort -k1h
[...]
47 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 8; DW_OP_breg16 (rip): 0; DW_OP_lit15; DW_OP_and; DW_OP_lit11; DW_OP_ge; DW_OP_lit3; DW_OP_shl; DW_OP_plus)
237 DW_CFA_def_cfa_expression (DW_OP_breg6 (rbp): -24; DW_OP_deref)
910 DW_CFA_def_cfa_expression (DW_OP_breg6 (rbp): -8; DW_OP_deref)
1006 DW_CFA_def_cfa_expression (DW_OP_breg6 (rbp): -40; DW_OP_deref)
2563 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 8; DW_OP_breg16 (rip): 0; DW_OP_lit15; DW_OP_and; DW_OP_lit10; DW_OP_ge; DW_OP_lit3; DW_OP_shl; DW_OP_plus)
```
```
[javierhonduco@fedora parca-agent]$ sudo readelf -wf /proc/*/exe 2>/dev/null | grep DW_CFA_def_cfa_expression | sort | uniq -c | sort -k1h
[...]
25 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 120; DW_OP_deref; DW_OP_plus_uconst: 8)
25 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 152; DW_OP_deref; DW_OP_plus_uconst: 8)
25 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 88; DW_OP_deref; DW_OP_plus_uconst: 8)
30 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 0; DW_OP_deref; DW_OP_plus_uconst: 8)
35 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 40; DW_OP_deref; DW_OP_plus_uconst: 8)
45 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): -8; DW_OP_deref; DW_OP_plus_uconst: 8)
130 DW_CFA_def_cfa_expression (DW_OP_breg7 (rsp): 8; DW_OP_breg16 (rip): 0; DW_OP_lit15; DW_OP_and; DW_OP_lit10; DW_OP_ge; DW_OP_lit3; DW_OP_shl; DW_OP_plus)
```
As it can be seen above, by adding PLT expression support, we can get ~50% more unwind sections working.
PLT sections are particularly important as libc is typically dynamically
linked. In this case, most libc calls go through the Procedure Linkage
Table, and before this commit, we will stop walking the stack.
See test plan in the PR.
Signed-off-by: Francisco Javier Honduvilla Coto <[email protected]>- Loading branch information
1 parent
3f0d04f
commit b7d873d
Showing
8 changed files
with
179 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| package frame | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| // Spot check some of the constants. | ||
| func TestDWARFExpressionConstants(t *testing.T) { | ||
| require.Equal(t, 0x0a, DW_OP_const2u) | ||
| require.Equal(t, 0x2f, DW_OP_skip) | ||
| require.Equal(t, 0x30, DW_OP_lit0) | ||
| require.Equal(t, 0x9d, DW_OP_bit_piece) | ||
| require.Equal(t, 0xe0, DW_OP_lo_user) | ||
| require.Equal(t, 0xff, DW_OP_hi_user) | ||
| require.Equal(t, 0x22, DW_OP_plus) | ||
| require.Equal(t, 0x08, DW_OP_const1u) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| // Copyright 2022 The Parca Authors | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // | ||
|
|
||
| package unwind | ||
|
|
||
| import ( | ||
| "github.com/parca-dev/parca-agent/internal/dwarf/frame" | ||
| ) | ||
|
|
||
| type DwarfExpressionID int16 | ||
|
|
||
| const ( | ||
| ExpressionUnknown DwarfExpressionID = iota | ||
| ExpressionPlt1 | ||
| ExpressionPlt2 | ||
| ) | ||
|
|
||
| // DWARF expressions that we recognize. | ||
|
|
||
| // plt1 is equivalent to: sp + 8 + ((((ip & 15) >= 11)) << 3. | ||
| var plt1 = [...]byte{ | ||
| frame.DW_OP_breg7, | ||
| frame.DW_OP_const1u, | ||
| frame.DW_OP_breg16, | ||
| frame.DW_OP_lit15, | ||
| frame.DW_OP_and, | ||
| frame.DW_OP_lit11, | ||
| frame.DW_OP_ge, | ||
| frame.DW_OP_lit3, | ||
| frame.DW_OP_shl, | ||
| frame.DW_OP_plus, | ||
| } | ||
|
|
||
| // plt2 is quivalent to: sp + 8 + ((((ip & 15) >= 10)) << 3. | ||
| var plt2 = [...]byte{ | ||
| frame.DW_OP_breg7, | ||
| frame.DW_OP_const1u, | ||
| frame.DW_OP_breg16, | ||
| frame.DW_OP_lit15, | ||
| frame.DW_OP_and, | ||
| frame.DW_OP_lit10, | ||
| frame.DW_OP_ge, | ||
| frame.DW_OP_lit3, | ||
| frame.DW_OP_shl, | ||
| frame.DW_OP_plus, | ||
| } | ||
|
|
||
| // equalBytes checks whether two byte slices are equal. | ||
| func equalBytes(a, b []byte) bool { | ||
| if len(a) != len(b) { | ||
| return false | ||
| } | ||
| for i := range a { | ||
| if a[i] != b[i] { | ||
| return false | ||
| } | ||
| } | ||
| return true | ||
| } | ||
|
|
||
| // ExpressionIdentifier returns the identifier for recognized | ||
| // DWARF expressions. | ||
| func ExpressionIdentifier(expression []byte) DwarfExpressionID { | ||
| cleanedExpression := make([]byte, 0, len(expression)) | ||
| for _, opcode := range expression { | ||
| if opcode == 0x0 { | ||
| continue | ||
| } | ||
| cleanedExpression = append(cleanedExpression, opcode) | ||
| } | ||
|
|
||
| if equalBytes(plt1[:], cleanedExpression) { | ||
| return ExpressionPlt1 | ||
| } | ||
|
|
||
| if equalBytes(plt2[:], cleanedExpression) { | ||
| return ExpressionPlt2 | ||
| } | ||
|
|
||
| return ExpressionUnknown | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters