HBASE-23176 delete_all_snapshot does not work with regex #725

karthikhw · 2019-10-15T05:15:31Z

Delete_all_snapshot.rb is using deprecated method SnapshotDescription#getTable but this method is already removed in 3.0.x.

karthikhw · 2019-10-15T05:31:37Z

I am really not sure if we need testcase. I added general test case for this issue. Just realized, similar testcase exists already.

Testcase will pass even without this fix. The reason I could not add testcase because delete_all_snapshot.rb#command(regex) accept only one argument and second argument will get it from user ('y/n') at runtime, for confirming delete.

But here I don't think I can pass argument at runtime.

karthikhw · 2019-10-15T05:36:20Z

I tried passing args at runtime but test case hungs ever. I don't think we need test case for init script.

Apache-HBase · 2019-10-15T06:00:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	6m 29s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	6m 37s	master passed
💚	javadoc	0m 15s	master passed
		_ Patch Compile Tests _
💚	mvninstall	6m 6s	the patch passed
💚	rubocop	0m 9s	There were no new rubocop issues.
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	javadoc	0m 15s	the patch passed
		_ Other Tests _
💔	unit	13m 44s	hbase-shell in the patch failed.
💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		35m 6s

Reason	Tests
Failed junit tests	hadoop.hbase.client.TestAdminShell

Subsystem	Report/Notes
Docker	Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/1/artifact/out/Dockerfile
GITHUB PR	#725
Optional Tests	dupname asflicense javac javadoc unit rubocop
uname	Linux bf49f0f4e578 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-725/out/precommit/personality/provided.sh
git revision	master / `7924ba3`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/1/artifact/out/patch-unit-hbase-shell.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/1/testReport/
Max. process+thread count	2680 (vs. ulimit of 10000)
modules	C: hbase-shell U: hbase-shell
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/1/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.75.1
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

karthikhw · 2019-10-15T17:49:44Z

Removed this test-case, as similar functionality test exist already.

Apache-HBase · 2019-10-15T17:55:48Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	0m 0s	Docker mode activated.
💔	docker	0m 1s	Docker failed to build yetus/hbase:1cb4f687c1.

Subsystem	Report/Notes
GITHUB PR	#725
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/2/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2019-10-15T21:00:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	1m 8s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	6m 11s	master passed
💚	javadoc	0m 14s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 34s	the patch passed
💚	rubocop	0m 9s	There were no new rubocop issues.
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	javadoc	0m 12s	the patch passed
		_ Other Tests _
💚	unit	10m 23s	hbase-shell in the patch passed.
💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		25m 0s

Subsystem	Report/Notes
Docker	Client=19.03.3 Server=19.03.3 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/3/artifact/out/Dockerfile
GITHUB PR	#725
Optional Tests	dupname asflicense javac javadoc unit rubocop
uname	Linux b4b1bb25dfbe 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-725/out/precommit/personality/provided.sh
git revision	master / `7924ba3`
Default Java	1.8.0_181
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/3/testReport/
Max. process+thread count	2630 (vs. ulimit of 10000)
modules	C: hbase-shell U: hbase-shell
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/3/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.75.1
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

guangxuCheng · 2019-10-16T11:23:39Z

hbase/hbase-shell/src/main/ruby/shell/commands/delete_all_snapshot.rb

Line 57 in fa05907

    
           formatter.row([snapshot.getName, snapshot.getTable + ' (' + creation_time + ')'])

Also need to be modified here

karthikhw · 2019-10-16T18:17:52Z

Thank you very much @guangxuCheng for checking this. There are 3 commits here, the first commit changes for delete_all_snapshot.rb.

guangxuCheng · 2019-10-17T02:09:10Z

hbase/hbase-shell/src/main/ruby/shell/commands/delete_all_snapshot.rb

Lines 31 to 58 in 0043dfe

    
           def command(regex) 
        
             formatter.header(['SNAPSHOT', 'TABLE + CREATION TIME']) 
        
             list = admin.list_snapshot(regex) 
        
             count = list.size 
        
             list.each do |snapshot| 
        
               creation_time = Time.at(snapshot.getCreationTime / 1000).to_s 
        
               formatter.row([snapshot.getName, snapshot.getTable + ' (' + creation_time + ')']) 
        
             end 
        
             puts "\nDelete the above #{count} snapshots (y/n)?" unless count == 0 
        
             answer = 'n' 
        
             answer = gets.chomp unless count == 0 
        
             puts "No snapshots matched the regex #{regex}" if count == 0 
        
             return unless answer =~ /y.*/i 
        
             @start_time = Time.now 
        
             admin.delete_all_snapshot(regex) 
        
             @end_time = Time.now 
        
             list = admin.list_snapshot(regex) 
        
             leftOverSnapshotCount = list.size 
        
             successfullyDeleted = count - leftOverSnapshotCount 
        
             puts "#{successfullyDeleted} snapshots successfully deleted." unless successfullyDeleted == 0 
        
             return if leftOverSnapshotCount == 0 
        
             puts "\nFailed to delete the below #{leftOverSnapshotCount} snapshots." 
        
             formatter.header(['SNAPSHOT', 'TABLE + CREATION TIME']) 
        
             list.each do |snapshot| 
        
               creation_time = Time.at(snapshot.getCreationTime / 1000).to_s 
        
               formatter.row([snapshot.getName, snapshot.getTable + ' (' + creation_time + ')']) 
        
             end 
        
           end

@karthikhw There are two places in delete_all_snapshot.rb (L37 and L56) that need to be modified, but you only modified one of them.

karthikhw · 2019-10-17T07:34:47Z

Thank you @guangxuCheng highlighting missing one. Sorry, I somehow missed it.

Apache-HBase · 2019-10-17T08:05:40Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
💙	reexec	1m 11s	Docker mode activated.
		_ Prechecks _
💚	dupname	0m 0s	No case conflicting files found.
💚	@author	0m 0s	The patch does not contain any @author tags.
💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
💚	mvninstall	6m 52s	master passed
💚	javadoc	0m 14s	master passed
		_ Patch Compile Tests _
💚	mvninstall	5m 29s	the patch passed
💚	rubocop	0m 10s	There were no new rubocop issues.
💚	whitespace	0m 0s	The patch has no whitespace issues.
💚	javadoc	0m 13s	the patch passed
		_ Other Tests _
💚	unit	10m 17s	hbase-shell in the patch passed.
💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		25m 35s

Subsystem	Report/Notes
Docker	Client=19.03.3 Server=19.03.3 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/4/artifact/out/Dockerfile
GITHUB PR	#725
Optional Tests	dupname asflicense javac javadoc unit rubocop
uname	Linux f34471f58eee 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-725/out/precommit/personality/provided.sh
git revision	master / `0043dfe`
Default Java	1.8.0_181
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/4/testReport/
Max. process+thread count	2621 (vs. ulimit of 10000)
modules	C: hbase-shell U: hbase-shell
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-725/4/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.75.1
Powered by	Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

guangxuCheng

+1

HBASE-23176 delete_all_snapshot does not work with regex

3c3de83

HBASE-23176 Removed test-case

960c2fc

HBASE-23176 Fix for delete_table_snapshots.rb

fa05907

HBASE-23176 remove depricated method

d41384b

guangxuCheng approved these changes Oct 17, 2019

View reviewed changes

guangxuCheng merged commit 0f910f0 into apache:master Oct 17, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-23176 delete_all_snapshot does not work with regex #725

HBASE-23176 delete_all_snapshot does not work with regex #725

karthikhw commented Oct 15, 2019

karthikhw commented Oct 15, 2019 •

edited

Loading

karthikhw commented Oct 15, 2019

Apache-HBase commented Oct 15, 2019

karthikhw commented Oct 15, 2019

Apache-HBase commented Oct 15, 2019

Apache-HBase commented Oct 15, 2019

guangxuCheng commented Oct 16, 2019

karthikhw commented Oct 16, 2019

guangxuCheng commented Oct 17, 2019 •

edited

Loading

karthikhw commented Oct 17, 2019

Apache-HBase commented Oct 17, 2019

guangxuCheng left a comment

HBASE-23176 delete_all_snapshot does not work with regex #725

HBASE-23176 delete_all_snapshot does not work with regex #725

Conversation

karthikhw commented Oct 15, 2019

karthikhw commented Oct 15, 2019 • edited Loading

karthikhw commented Oct 15, 2019

Apache-HBase commented Oct 15, 2019

karthikhw commented Oct 15, 2019

Apache-HBase commented Oct 15, 2019

Apache-HBase commented Oct 15, 2019

guangxuCheng commented Oct 16, 2019

karthikhw commented Oct 16, 2019

guangxuCheng commented Oct 17, 2019 • edited Loading

karthikhw commented Oct 17, 2019

Apache-HBase commented Oct 17, 2019

guangxuCheng left a comment

Choose a reason for hiding this comment

karthikhw commented Oct 15, 2019 •

edited

Loading

guangxuCheng commented Oct 17, 2019 •

edited

Loading