Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encrypt demo all #84

Closed
wants to merge 83 commits into from
Closed

Encrypt demo all #84

wants to merge 83 commits into from

Conversation

acelyc111
Copy link
Owner

@acelyc111 acelyc111 commented Aug 27, 2023

Not encrypt files

local_file_object file_metadata | |
backup metadata | block service:/<backup_id>/<appname_appid>/meta/app_metadata | json encoded
config file
bulk_load_metadata

TODO: The nfs module now checks all files' sizes as sensitive, check if it is OK.

Need tests

manual test on HDFS backup
pegasus_geo_test

手动测试

老版本备份的数据,是用新版本(开启/不开启加密)恢复是否能成功

onebox/block_service/local_service
|-- onebox
|   `-- 1693401530311
|       |-- backup_info
|       `-- test_app_3
|           |-- 0
|           |   |-- chkpt_127.0.0.1_34803
|           |   |   |-- 000010.sst
|           |   |   |-- 000013.sst
|           |   |   |-- 000014.sst
|           |   |   |-- CURRENT
|           |   |   |-- MANIFEST-000005
|           |   |   |-- OPTIONS-000012
|           |   |   `-- backup_metadata
|           |   `-- current_checkpoint
|           |-- 1
|           |   |-- chkpt_127.0.0.1_34802
|           |   |   |-- 000010.sst
|           |   |   |-- 000013.sst
|           |   |   |-- 000014.sst
|           |   |   |-- CURRENT
|           |   |   |-- MANIFEST-000005
|           |   |   |-- OPTIONS-000012
|           |   |   `-- backup_metadata
|           |   `-- current_checkpoint
|           |-- 2
|           |   |-- chkpt_127.0.0.1_34801
|           |   |   |-- 000010.sst
|           |   |   |-- 000013.sst
|           |   |   |-- 000014.sst
|           |   |   |-- CURRENT
|           |   |   |-- MANIFEST-000005
|           |   |   |-- OPTIONS-000012
|           |   |   `-- backup_metadata
|           |   `-- current_checkpoint
|           |-- 3
|           |   |-- chkpt_127.0.0.1_34803
|           |   |   |-- 000010.sst
|           |   |   |-- 000013.sst
|           |   |   |-- 000014.sst
|           |   |   |-- CURRENT
|           |   |   |-- MANIFEST-000005
|           |   |   |-- OPTIONS-000012
|           |   |   `-- backup_metadata
|           |   `-- current_checkpoint
|           `-- meta
|               `-- app_metadata




/home/laiyingchun/dev/pegasus/onebox/block_service/local_service/bulk_load_root/cluster/temp/
|-- 0
|   |-- bulk_load_metadata
|   |-- file0.sst
|   `-- filename0.sst
|-- 1
|   |-- bulk_load_metadata
|   |-- file1.sst
|   `-- filename1.sst
|-- 2
|   |-- bulk_load_metadata
|   |-- file2.sst
|   `-- filename2.sst
|-- 3
|   |-- bulk_load_metadata
|   |-- file3.sst
|   `-- filename3.sst
|-- 4
|   |-- bulk_load_metadata
|   |-- file4.sst
|   `-- filename4.sst
|-- 5
|   |-- bulk_load_metadata
|   |-- file5.sst
|   `-- filename5.sst
|-- 6
|   |-- bulk_load_metadata
|   |-- file6.sst
|   `-- filename6.sst
|-- 7
|   |-- bulk_load_metadata
|   |-- file7.sst
|   `-- filename7.sst
`-- bulk_load_info

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1684 files.

Valid Invalid Ignored Fixed
1057 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1682 files.

Valid Invalid Ignored Fixed
1055 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1682 files.

Valid Invalid Ignored Fixed
1055 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1682 files.

Valid Invalid Ignored Fixed
1055 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1682 files.

Valid Invalid Ignored Fixed
1055 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

license-eye has totally checked 1682 files.

Valid Invalid Ignored Fixed
1055 1 626 0
Click to see the invalid file list
  • src/nfs/nfs_node_simple.cpp

@acelyc111 acelyc111 closed this Sep 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant