Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(types/parser): parse by fragment and process in pipeline #855

Conversation

xcoulon
Copy link
Member

@xcoulon xcoulon commented Nov 14, 2021

Fragments are blocks of contiguous lines separated by blank lines.
Blank lines within delimited blocks are preserved (hence they do not
split the fragments)
Once a fragment is read, it is sent on a channel to be processed
by the next stages of the pipeline until the aggregator which combines
all the fragments into the resulting types.Document, which
can then be rendered

BREAKING CHANGES:

  • using pointers on all structs in pkg/types
  • removing types.Document.Attributes, holding attributes in context
    as parsing/rendering progresses.
  • types.Paragraph.Lines ([][]interface{}) is replaced by
    types.Paragraph.Elements ([]interface{})
  • all delimited blocks types are merged into types.DelimitedBlock
  • all lists are merged into types.List
  • removed the Level field in types.ListElement struct
  • refactor document structure with header (level 0) and
    sections (level 1 to 5)

Fixes #843

Signed-off-by: Xavier Coulon [email protected]

Fragments are blocks of contiguous lines separated by blank lines.
Blank lines within delimited blocks are preserved (hence they do not
split the fragments)
Once a fragment is read, it is sent on a channel to be processed
by the next stages of the pipeline until the aggregator which combines
all the fragments into the resulting `types.Document`, which
can thn be rendered

Also:
- removed tests in "raw documents" for quoted text, since it has little
 to no value.

BREAKING CHANGE:
- using pointers on all structs in `pkg/types`
- removing `types.Document.Attributes`, holding attributes in context
  as parsing/rendering progresses.
- `types.Paragraph.Lines` (`[][]interface{}`) is replaced by
  `types.Paragraph.Elements` (`[]interface{}`)
- all delimited blocks types are merged into `types.DelimitedBlock`
- all lists are merged into `types.List`
- removed the `Level` field in `types.ListElement` struct
- refactor document structure with header (level 0) and
  sections (level 1 to 5)

Fixes bytesparadise#843

Signed-off-by: Xavier Coulon <[email protected]>
@xcoulon xcoulon force-pushed the Issue843_refactor_parser_raw_fragment_blocks_parse_fragments branch from 4194681 to 78a9120 Compare November 14, 2021 08:35
@codecov
Copy link

codecov bot commented Nov 14, 2021

Codecov Report

Merging #855 (8037812) into master (82bfd34) will increase coverage by 1.38%.
The diff coverage is 85.09%.

@@            Coverage Diff             @@
##           master     #855      +/-   ##
==========================================
+ Coverage   78.78%   80.16%   +1.38%     
==========================================
  Files          88       85       -3     
  Lines        5289     6499    +1210     
==========================================
+ Hits         4167     5210    +1043     
- Misses        694      806     +112     
- Partials      428      483      +55     

Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
Signed-off-by: Xavier Coulon <[email protected]>
@xcoulon
Copy link
Member Author

xcoulon commented Nov 20, 2021

FWIW, running the benchs in libasciidoc_bench_test.go compared to the code tagged v0.6.0 gives the following improvements (via benchstat):

name                   old time/op    new time/op    delta
ParseBasicDocument-12    6.69ms ± 3%    0.68ms ± 3%  -89.79%  (p=0.000 n=9+9)
ParseLongDocument-12      295ms ± 2%      25ms ± 2%  -91.54%  (p=0.000 n=9+10)

name                   old alloc/op   new alloc/op   delta
ParseBasicDocument-12    2.01MB ± 0%    0.24MB ± 0%  -87.83%  (p=0.000 n=10+9)
ParseLongDocument-12     99.5MB ± 0%    11.5MB ± 0%  -88.44%  (p=0.000 n=9+10)

name                   old allocs/op  new allocs/op  delta
ParseBasicDocument-12     41.2k ± 0%      4.1k ± 0%  -89.97%  (p=0.001 n=8+9)
ParseLongDocument-12      2.06M ± 0%     0.20M ± 0%  -90.08%  (p=0.000 n=9+10)

@xcoulon xcoulon merged commit dbec12b into bytesparadise:master Nov 20, 2021
@xcoulon xcoulon deleted the Issue843_refactor_parser_raw_fragment_blocks_parse_fragments branch November 20, 2021 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor the parser
1 participant