HBASE-25161 [hbase-meta-repair] Add tool for repairing metadata. by VictorZeng · Pull Request #77 · apache/hbase-operator-tools

VictorZeng · 2020-10-07T09:17:23Z

Apache-HBase · 2020-10-07T09:51:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 15s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 45s	master passed
+1 💚	compile	0m 9s	master passed
+1 💚	checkstyle	0m 6s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 24s	the patch passed
+1 💚	compile	0m 17s	the patch passed
+1 💚	javac	0m 17s	the patch passed
-1 ❌	checkstyle	0m 5s	hbase-meta-repair: The patch generated 193 new + 0 unchanged - 0 fixed = 193 total (was 0)
-1 ❌	checkstyle	0m 6s	root: The patch generated 193 new + 0 unchanged - 0 fixed = 193 total (was 0)
-1 ❌	whitespace	0m 0s	The patch has 10 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	javadoc	0m 10s	the patch passed
		_ Other Tests _
-1 ❌	unit	10m 55s	hbase-meta-repair in the patch failed.
-1 ❌	unit	18m 30s	root in the patch failed.
-1 ❌	asflicense	0m 9s	The patch generated 1 ASF License warnings.
		33m 19s

Reason	Tests
Failed junit tests	hbase.repair.TestMetaRepair
	hbase.TestRegionsMerger
	hbase.repair.TestMetaRepair

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 2926be98c042 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `5cdc0e2`
Default Java	Oracle Corporation-1.8.0_265-b01
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-meta-repair.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/diff-checkstyle-root.txt
whitespace	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/whitespace-eol.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-unit-hbase-meta-repair.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/testReport/
asflicense	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-asflicense-problems.txt
Max. process+thread count	941 (vs. ulimit of 1000)
modules	C: hbase-meta-repair . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-10-07T10:35:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 41s	master passed
+1 💚	compile	0m 9s	master passed
+1 💚	checkstyle	0m 6s	master passed
+1 💚	javadoc	0m 8s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 25s	the patch passed
+1 💚	compile	0m 17s	the patch passed
+1 💚	javac	0m 17s	the patch passed
-1 ❌	checkstyle	0m 4s	hbase-meta-repair: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
-1 ❌	checkstyle	0m 4s	root: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 1s	The patch has no ill-formed XML file.
+1 💚	javadoc	0m 10s	the patch passed
		_ Other Tests _
-1 ❌	unit	0m 31s	hbase-meta-repair in the patch failed.
-1 ❌	unit	6m 44s	root in the patch failed.
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		10m 21s

Reason	Tests
Failed junit tests	hbase.repair.TestMetaRepair
	hbase.TestRegionsMerger
	hbase.repair.TestMetaRepair

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 02832fc2b867 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `5cdc0e2`
Default Java	Oracle Corporation-1.8.0_265-b01
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-meta-repair.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/artifact/yetus-precommit-check/output/diff-checkstyle-root.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/artifact/yetus-precommit-check/output/patch-unit-hbase-meta-repair.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/artifact/yetus-precommit-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/testReport/
Max. process+thread count	939 (vs. ulimit of 1000)
modules	C: hbase-meta-repair . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/2/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-10-07T11:07:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 48s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 9s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 7s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 42s	the patch passed
+1 💚	compile	0m 17s	the patch passed
+1 💚	javac	0m 17s	the patch passed
-1 ❌	checkstyle	0m 3s	hbase-meta-repair: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
-1 ❌	checkstyle	0m 4s	root: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	javadoc	0m 9s	the patch passed
		_ Other Tests _
-1 ❌	unit	0m 31s	hbase-meta-repair in the patch failed.
-1 ❌	unit	26m 49s	root in the patch failed.
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		31m 0s

Reason	Tests
Failed junit tests	hbase.repair.TestMetaRepair
	hbase.repair.TestMetaRepair

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 90555a1d353a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `5cdc0e2`
Default Java	Oracle Corporation-1.8.0_265-b01
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-meta-repair.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/artifact/yetus-precommit-check/output/diff-checkstyle-root.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/artifact/yetus-precommit-check/output/patch-unit-hbase-meta-repair.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/artifact/yetus-precommit-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/testReport/
Max. process+thread count	935 (vs. ulimit of 1000)
modules	C: hbase-meta-repair . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/3/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-10-07T11:43:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 42s	master passed
+1 💚	compile	0m 9s	master passed
+1 💚	checkstyle	0m 6s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 25s	the patch passed
+1 💚	compile	0m 17s	the patch passed
+1 💚	javac	0m 17s	the patch passed
-1 ❌	checkstyle	0m 3s	hbase-meta-repair: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
-1 ❌	checkstyle	0m 5s	root: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	javadoc	0m 10s	the patch passed
		_ Other Tests _
-1 ❌	unit	3m 19s	hbase-meta-repair in the patch passed.
-1 ❌	unit	17m 40s	root in the patch failed.
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		24m 6s

Reason	Tests
Failed junit tests	hbase.repair.TestMetaRepair
	hbase.TestRegionsMerger
	hbase.repair.TestMetaRepair

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux a0c3cca40594 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `5cdc0e2`
Default Java	Oracle Corporation-1.8.0_265-b01
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-meta-repair.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/artifact/yetus-precommit-check/output/diff-checkstyle-root.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/artifact/yetus-precommit-check/output/patch-unit-hbase-meta-repair.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/artifact/yetus-precommit-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/testReport/
Max. process+thread count	942 (vs. ulimit of 1000)
modules	C: hbase-meta-repair . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/4/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2020-10-07T12:05:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 36s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 44s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 6s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 25s	the patch passed
+1 💚	compile	0m 17s	the patch passed
+1 💚	javac	0m 17s	the patch passed
+1 💚	checkstyle	0m 8s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	javadoc	0m 10s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 46s	hbase-meta-repair in the patch passed.
-1 ❌	unit	8m 19s	root in the patch failed.
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		12m 16s

Reason	Tests
Failed junit tests	hbase.TestRegionsMerger

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/5/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 2fdc34118b31 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `5cdc0e2`
Default Java	Oracle Corporation-1.8.0_265-b01
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/5/artifact/yetus-precommit-check/output/patch-unit-root.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/5/testReport/
Max. process+thread count	942 (vs. ulimit of 1000)
modules	C: hbase-meta-repair . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/5/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil

Have you tried HBCK2 addFsRegionsMissingInMeta and extraRegionsInMeta functions? I believe those do the same as what you are proposing here, in other words, use hdfs info as the source of truth to rebuild meta.

VictorZeng · 2020-10-07T12:16:07Z

This testcase for RegionsMerger run failed:

[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hbase.TestRegionsMerger
[ERROR] Tests run: 9, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 102.515 s <<< FAILURE! - in org.apache.hbase.TestRegionsMerger
[ERROR] testMergeRegionsInvalidParams(org.apache.hbase.TestRegionsMerger) Time elapsed: 0.422 s <<< ERROR!
java.io.IOException: java.lang.Exception: java.lang.OutOfMemoryError: unable to create new native thread
......

VictorZeng · 2020-10-07T12:19:32Z

@wchevreuil There is no HbckService in versions of hbase before 2.0.3 and 2.1.1. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata.

Apache9 · 2020-10-07T12:53:44Z

Have you tried HBCK2 addFsRegionsMissingInMeta and extraRegionsInMeta functions? I believe those do the same as what you are proposing here, in other words, use hdfs info as the source of truth to rebuild meta.

This is a fallback tool to reconstruct the meta region when users are deploying a 2.x version without HBCK2 support. I think it is useful for our users?

wchevreuil · 2020-10-08T09:31:57Z

There is no HbckService in versions of hbase before 2.0.3 and 2.1.1. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata.

Have you tried use these functions and got some error? If so, we should fix that, as neither addFsRegionsMissingInMeta nor extraRegionsInMeta rely on HbckService, they perform fixes on their own, thus, there should be no hbase version validation when using these two functions, and they should be compatible with the above versions.

VictorZeng · 2020-10-08T10:46:10Z

There is no HbckService in versions of hbase before 2.0.3 and 2.1.1. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata.

Have you tried use these functions and got some error? If so, we should fix that, as neither addFsRegionsMissingInMeta nor extraRegionsInMeta rely on HbckService, they perform fixes on their own, thus, there should be no hbase version validation when using these two functions, and they should be compatible with the above versions.

Hi wchevreuil, although both addFsRegionsMissingInMeta and extraRegionsInMeta do not rely on HbckService, but assigns command rely on HbckService... So I think this tool is very useful.

wchevreuil · 2020-10-08T15:59:48Z

There is no HbckService in versions of hbase before 2.0.3 and 2.1.1. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata.

Have you tried use these functions and got some error? If so, we should fix that, as neither addFsRegionsMissingInMeta nor extraRegionsInMeta rely on HbckService, they perform fixes on their own, thus, there should be no hbase version validation when using these two functions, and they should be compatible with the above versions.

Hi wchevreuil, although both addFsRegionsMissingInMeta and extraRegionsInMeta do not rely on HbckService, but assigns command rely on HbckService... So I think this tool is very useful.

Hi wchevreuil, although both addFsRegionsMissingInMeta and extraRegionsInMeta do not rely on HbckService, but assigns command rely on HbckService... So I think this tool is very useful.

Then the only extra bit is a bulk assignment? Could it reuse addFsRegionsMissingInMeta and extraRegionsInMeta, instead of providing extra lines of code that do same as the already existing methods? I do agree the command proposed here is very convenient, but we should re-use already existing code as much as possible.

VictorZeng · 2020-10-09T02:12:55Z

There is no HbckService in versions of hbase before 2.0.3 and 2.1.1. HBCK2 will not work against these versions. If users use Hbase 2.1.0 version, you can use this tool to repair metadata.

Have you tried use these functions and got some error? If so, we should fix that, as neither addFsRegionsMissingInMeta nor extraRegionsInMeta rely on HbckService, they perform fixes on their own, thus, there should be no hbase version validation when using these two functions, and they should be compatible with the above versions.

Hi wchevreuil, although both addFsRegionsMissingInMeta and extraRegionsInMeta do not rely on HbckService, but assigns command rely on HbckService... So I think this tool is very useful.

Hi wchevreuil, although both addFsRegionsMissingInMeta and extraRegionsInMeta do not rely on HbckService, but assigns command rely on HbckService... So I think this tool is very useful.

Then the only extra bit is a bulk assignment? Could it reuse addFsRegionsMissingInMeta and extraRegionsInMeta, instead of providing extra lines of code that do same as the already existing methods? I do agree the command proposed here is very convenient, but we should re-use already existing code as much as possible.

This tool function is more similar to the hbck1 fixMeta command, it can repair the metadata when users are deploying a 2.x version without HBCK2 support more quickly and conveniently. So it's necessary to provide extra codes. I think it's better to be independent.

wchevreuil · 2020-10-19T10:58:31Z

it can repair the metadata when users are deploying a 2.x version without HBCK2 support more quickly and conveniently.

I agree with that, and that we can do it without the server side support for hbck2

So it's necessary to provide extra codes. I think it's better to be independent.

We already have functions/methods on hbck2 that perform fixes without relying on server side support for hbck2, so I see no reason to add an entirely extra module whose scope could/should be contained within hbck2. And for this specific solution, I already mentioned some methods that could be reused by some sort of coordination logic that could give a less interactive experience, without the need to duplicate code to do essentially the same thing.

wchevreuil

See my latest comment above.

busbey · 2021-01-15T18:57:41Z

@VictorZeng still working on this?

… into HBASE-25161

VictorZeng · 2021-02-07T02:33:29Z

@busbey What do you think of megre the Meta Repair Tool into hbase-tools module ?

Apache-HBase · 2021-02-07T02:36:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 46s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 8s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for patch
+1 💚	mvninstall	0m 26s	the patch passed
+1 💚	compile	0m 19s	the patch passed
+1 💚	javac	0m 19s	the patch passed
+1 💚	checkstyle	0m 8s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	javadoc	0m 11s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 48s	hbase-meta-repair in the patch passed.
+1 💚	unit	6m 14s	root in the patch passed.
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		10m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/6/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 8e13243b309e 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `2958eeb`
Default Java	Oracle Corporation-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/6/testReport/
Max. process+thread count	1285 (vs. ulimit of 5000)
modules	C: hbase-meta-repair . U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/6/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-02-15T02:41:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 57s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 53s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 2s	Maven dependency ordering for patch
-1 ❌	mvninstall	0m 3s	root in the patch failed.
-1 ❌	mvninstall	0m 4s	hbase-meta-repair in the patch failed.
-1 ❌	compile	0m 3s	root in the patch failed.
-1 ❌	compile	0m 3s	hbase-meta-repair in the patch failed.
-1 ❌	javac	0m 3s	root in the patch failed.
-1 ❌	javac	0m 3s	hbase-meta-repair in the patch failed.
-1 ❌	checkstyle	0m 1s	The patch fails to run checkstyle in root
-1 ❌	checkstyle	0m 1s	The patch fails to run checkstyle in hbase-meta-repair
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
-1 ❌	javadoc	0m 3s	root in the patch failed.
-1 ❌	javadoc	0m 3s	hbase-meta-repair in the patch failed.
		_ Other Tests _
-1 ❌	unit	0m 4s	root in the patch failed.
-1 ❌	unit	0m 3s	hbase-meta-repair in the patch failed.
+0 🆗	asflicense	0m 6s	ASF License check generated no output?
		3m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 46b49e44696a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `8d667b1`
Default Java	Oracle Corporation-1.8.0_282-b08
mvninstall	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-mvninstall-root.txt
mvninstall	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-mvninstall-hbase-meta-repair.txt
compile	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-root.txt
compile	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-hbase-meta-repair.txt
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-root.txt
javac	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-hbase-meta-repair.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/buildtool-patch-checkstyle-root.txt
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/buildtool-patch-checkstyle-hbase-meta-repair.txt
javadoc	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-javadoc-root.txt
javadoc	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-javadoc-hbase-meta-repair.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-unit-root.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-unit-hbase-meta-repair.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/testReport/
Max. process+thread count	38 (vs. ulimit of 5000)
modules	C: . hbase-meta-repair U: .
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-02-07T04:58:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 53s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	markdownlint	0m 0s	markdownlint was not available.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 54s	master passed
+1 💚	compile	0m 11s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 3s	Maven dependency ordering for patch
-1 ❌	mvninstall	0m 3s	root in the patch failed.
-1 ❌	mvninstall	0m 3s	hbase-meta-repair in the patch failed.
-1 ❌	compile	0m 4s	root in the patch failed.
-1 ❌	compile	0m 3s	hbase-meta-repair in the patch failed.
-1 ❌	javac	0m 4s	root in the patch failed.
-1 ❌	javac	0m 3s	hbase-meta-repair in the patch failed.
-1 ❌	checkstyle	0m 1s	The patch fails to run checkstyle in root
-1 ❌	checkstyle	0m 1s	The patch fails to run checkstyle in hbase-meta-repair
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
-1 ❌	javadoc	0m 3s	root in the patch failed.
-1 ❌	javadoc	0m 3s	hbase-meta-repair in the patch failed.
		_ Other Tests _
-1 ❌	unit	0m 4s	root in the patch failed.
-1 ❌	unit	0m 3s	hbase-meta-repair in the patch failed.
+0 🆗	asflicense	0m 6s	ASF License check generated no output?
		3m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#77
Optional Tests	dupname asflicense markdownlint javac javadoc unit xml compile spotbugs findbugs checkstyle
uname	Linux 6cf55e0ac371 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 GNU/Linux
Build tool	maven
git revision	master / `b7d9fc8`
Default Java	Oracle Corporation-1.8.0_282-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-mvninstall-root.txt
mvninstall	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-mvninstall-hbase-meta-repair.txt
compile	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-root.txt
compile	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-hbase-meta-repair.txt
javac	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-root.txt
javac	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-compile-hbase-meta-repair.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/buildtool-patch-checkstyle-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/buildtool-patch-checkstyle-hbase-meta-repair.txt
javadoc	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-javadoc-root.txt
javadoc	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-javadoc-hbase-meta-repair.txt
unit	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-unit-root.txt
unit	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/artifact/yetus-precommit-check/output/patch-unit-hbase-meta-repair.txt
Test Results	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/testReport/
Max. process+thread count	38 (vs. ulimit of 5000)
modules	C: . hbase-meta-repair U: .
Console output	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-77/1/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

HBASE-25161 [hbase-meta-repair] Add tool for repairing metadata.

50995ab

VictorZeng added 2 commits October 7, 2020 18:23

HBASE-25161 [hbase-meta-repair] fix code style.

7e9dd0b

HBASE-25161 [hbase-meta-repair] fix code style.

9dfcb16

VictorZeng added 3 commits October 7, 2020 18:36

HBASE-25161 [hbase-meta-repair] fix testcase issue.

e582d26

HBASE-25161 [hbase-meta-repair] fix code style.

8d9358a

HBASE-25161 [hbase-meta-repair] fix testcase issue.

ffe3aa5

VictorZeng added 2 commits October 7, 2020 19:47

HBASE-25161 [hbase-meta-repair] fix code style.

ac7b00e

HBASE-25161 [hbase-meta-repair] fix testcase issue.

afe272d

wchevreuil requested changes Oct 7, 2020

View reviewed changes

VictorZeng mentioned this pull request Oct 7, 2020

关于捐赠代码到hbase-operator-tools DarkPhoenixs/hbase-meta-repair#3

Open

VictorZeng requested a review from wchevreuil October 8, 2020 04:57

wchevreuil requested changes Oct 21, 2020

View reviewed changes

Merge branch 'master' of https://github.com/apache/hbase-operator-tools…

7771028

… into HBASE-25161

Conversation

VictorZeng commented Oct 7, 2020

Uh oh!

Apache-HBase commented Oct 7, 2020

Uh oh!

Apache-HBase commented Oct 7, 2020

Uh oh!

Apache-HBase commented Oct 7, 2020

Uh oh!

Apache-HBase commented Oct 7, 2020

Uh oh!

Apache-HBase commented Oct 7, 2020

Uh oh!

wchevreuil left a comment

Choose a reason for hiding this comment

Uh oh!

VictorZeng commented Oct 7, 2020

Uh oh!

VictorZeng commented Oct 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Apache9 commented Oct 7, 2020

Uh oh!

wchevreuil commented Oct 8, 2020

Uh oh!

VictorZeng commented Oct 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wchevreuil commented Oct 8, 2020

Uh oh!

VictorZeng commented Oct 9, 2020

Uh oh!

wchevreuil commented Oct 19, 2020

Uh oh!

wchevreuil left a comment

Choose a reason for hiding this comment

Uh oh!

busbey commented Jan 15, 2021

Uh oh!

VictorZeng commented Feb 7, 2021

Uh oh!

Apache-HBase commented Feb 7, 2021

Uh oh!

Apache-HBase commented Feb 15, 2021

Uh oh!

Apache-HBase commented Feb 7, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

VictorZeng commented Oct 7, 2020 •

edited

Loading

VictorZeng commented Oct 8, 2020 •

edited

Loading