Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor reader 1 #17569

Closed
wants to merge 213 commits into from
Closed

Refactor reader 1 #17569

wants to merge 213 commits into from

Conversation

triump2020
Copy link
Contributor

@triump2020 triump2020 commented Jul 16, 2024 •
edited
Loading

What type of PR is this?

  • API-change
  • BUG
  • [x ] Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #11805

What this PR does / why we need it:

  1. Refactor reader
  • Optimize select count(*), Min, Max.
  • Read orderBy
  • Handle partition table.

@matrix-meow matrix-meow added the size/L Denotes a PR that changes [500,999] lines label Jul 16, 2024
@matrix-meow matrix-meow added size/XL Denotes a PR that changes [1000, 1999] lines and removed size/L Denotes a PR that changes [500,999] lines labels Jul 17, 2024
@matrix-meow matrix-meow added size/XXL Denotes a PR that changes 2000+ lines and removed size/XL Denotes a PR that changes [1000, 1999] lines labels Jul 17, 2024
triump2020 and others added 13 commits July 30, 2024 21:37
@triump2020
Merge pull request #74 from LeftHandCold/add_BlockDataReadNoCopy
bf98cf7 
Add block data read no copy
@XuPeng-SH
Small reader update (#75)
f39edde 
* update

* update 2

* update 3

* update 4

* update 5
@LeftHandCold
Change read new rowid (#76)
d33af41 
* change readNewRowid

* Update code

* Update code

* Update code

* Add log

* Update code

* Update code

* Update code

* Update code

* Update code

* Update code

* Update code

* Update code

* Update code

* remove debug info

* Update code

* Remove BlockRead
@LeftHandCold
Fix GetDeltaLoc panic (#77)
bdf5669
@triump2020
Refactor reader 1 remove batch (#80)
2a1337e 
* Revert "Small reader update (#75)"

This reverts commit f39edde.

* Revert "Revert "Small reader update (#75)""

This reverts commit a414863.

* update

* update
@LeftHandCold
Update read new rowid (#81)
9c95a10 
* update readNewRowid

* update code

* Update code

* Update code

* Update code
@LeftHandCold
Fix bug (#82)
3717f94 
* Fix bug

* Updat code

* Update code
@gouhongshen
Reader data source ut (#79)
0446b63 
* ranges to ranges slice, order by

* fix order by

* prefetch, db_test.BlockRead

* update db_test delta loc datasource

* fix prefetch

* fix engine_mock

* clean unused code

* fix local datasource close

* remove delta loc in filter, ranges
@LeftHandCold
remove_BlockRead (#83)
4f44ff2 
* remove_BlockRead

* remove ForeachBlkInObjStatsList
@XuPeng-SH
update for marshal and unmarshal (#84)
b3ae789 
* update 1

* update 2

* update 3

* update 4
@XuPeng-SH
update 1 (#85)
a44eb3f
@XuPeng-SH
update 1 (#86)
4911cd5
@gouhongshen
Reader data source ut (#88)
0e234af 
* ranges to ranges slice, order by

* fix order by

* prefetch, db_test.BlockRead

* update db_test delta loc datasource

* fix prefetch

* fix engine_mock

* clean unused code

* fix local datasource close

* remove delta loc in filter, ranges

* adding cmd to force shuffle reader, build remote ds

* adding local datasource string
triump2020 and others added 7 commits August 1, 2024 12:21
@triump2020
Refactor reader 1 add remote ds switch (#89)
a9f4661 
* Revert "Small reader update (#75)"

This reverts commit f39edde.

* Revert "Revert "Small reader update (#75)""

This reverts commit a414863.

* add switch for ctl remote ds
@XuPeng-SH
update (#90)
0738faf
@LeftHandCold
Remove BlockInfo (#87)
aeffda1
@triump2020
Refactor reader 1 fix bug for distribute (#91)
5ff0532 
* Revert "Small reader update (#75)"

This reverts commit f39edde.

* Revert "Revert "Small reader update (#75)""

This reverts commit a414863.

* fix bug for remote distribute
@XuPeng-SH
Update marshal and remove some pointer (#92)
8a730ab 
* update datasource marshal

* rm some pointer
@XuPeng-SH
rm some rel data fields (#93)
a986445
@XuPeng-SH
rm typ in tombstone data and rel data (#95)
8a390d0 
* rm typ in tombstone data

* rm typ in rel data
@LeftHandCold
Add LocationSlice (#96)
93d8edd 
* Add LocationSlice

* Fix bug
@triump2020 triump2020 closed this Aug 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daviszhen daviszhen daviszhen requested changes

@XuPeng-SH XuPeng-SH Awaiting requested review from XuPeng-SH XuPeng-SH is a code owner

@LeftHandCold LeftHandCold Awaiting requested review from LeftHandCold LeftHandCold is a code owner

@reusee reusee Awaiting requested review from reusee reusee is a code owner

@iamlinjunhong iamlinjunhong Awaiting requested review from iamlinjunhong iamlinjunhong is a code owner

@m-schen m-schen Awaiting requested review from m-schen

@ouyuanning ouyuanning Awaiting requested review from ouyuanning ouyuanning is a code owner

@aunjgr aunjgr Awaiting requested review from aunjgr aunjgr is a code owner

@badboynt1 badboynt1 Awaiting requested review from badboynt1 badboynt1 is a code owner

@qingxinhome qingxinhome Awaiting requested review from qingxinhome qingxinhome is a code owner

@zhangxu19830126 zhangxu19830126 Awaiting requested review from zhangxu19830126 zhangxu19830126 is a code owner

@volgariver6 volgariver6 Awaiting requested review from volgariver6

@aptend aptend Awaiting requested review from aptend

@xzxiong xzxiong Awaiting requested review from xzxiong

@fengttt fengttt Awaiting requested review from fengttt

@aressu1985 aressu1985 Awaiting requested review from aressu1985

@heni02 heni02 Awaiting requested review from heni02

Assignees
No one assigned
Labels
do-not-merge/wip size/XXL Denotes a PR that changes 2000+ lines
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@triump2020 @daviszhen @matrix-meow @gouhongshen @XuPeng-SH @LeftHandCold