Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Protocol] binary row-format protocol #331

Closed
chaokunyang opened this issue May 21, 2023 · 0 comments · Fixed by #330
Closed

[Protocol] binary row-format protocol #331

chaokunyang opened this issue May 21, 2023 · 0 comments · Fixed by #330
Labels
enhancement New feature or request java

Comments

@chaokunyang
Copy link
Collaborator

chaokunyang commented May 21, 2023

Is your feature request related to a problem? Please describe.

Implement binary row-format for fury.

Following graph are the detailed protocol, which is havily inspired by spark tungsten:

image

The main differences are:

  • using fury primitives for memory operations
  • using arrow schema
  • using arrow decimal format
  • using arrow data type visitor pattern for debug field access.

Describe the solution you'd like

The first version implementation doesn't count for :

  • String ascii/unicode16 support
  • variable-size data inline
  • schema forward-backward compatibility

Additional context

#314

@chaokunyang chaokunyang added enhancement New feature or request java labels May 21, 2023
@chaokunyang chaokunyang changed the title [Java] binary row-format java support [Java] binary row-format protocol Jun 5, 2023
@chaokunyang chaokunyang changed the title [Java] binary row-format protocol [Protocol] binary row-format protocol Jun 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request java
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant