HBASE-30058 Eliminate unnecessary connection creation in snapshot operations#8030
HBASE-30058 Eliminate unnecessary connection creation in snapshot operations#8030mini666 wants to merge 1 commit intoapache:masterfrom
Conversation
3e6e68c to
c0497ca
Compare
junegunn
left a comment
There was a problem hiding this comment.
Two suggestions:
Keep the old method signatures and overload
Could we add overloads that take Connection rather than changing existing signatures? It would minimize code changes and keep the existing API working. For example:
@Deprecated
public static SnapshotDescription validate(SnapshotDescription snapshot, Configuration conf)
throws IOException {
try (Connection conn = ConnectionFactory.createConnection(conf)) {
return validate(conn, snapshot, conf);
}
}
public static SnapshotDescription validate(Connection conn, SnapshotDescription snapshot,
Configuration conf) throws IOException {
// ... new implementation
}Same idea for isSecurityAvailable.
Commit message
Please drop the Signed-off-by: JeongMin Ju <mini666@apache.org> line from the commit message. In the HBase project, Signed-off-by is used to attribute reviewers of the PR, not the author.
|
@junegunn Thanks for the review! Updated:
Diff is now down to 5 production files (no test changes). Please take another look when you get a chance. |
|
Do we need to deprecate the original ones? We're still using them in multiple places. I think it's enough to mention that the new ones are for avoiding connection overhead. |
|
@junegunn Good point — agreed. Dropped |
| public static ListMultimap<String, UserPermission> getTablePermissions(Configuration conf, | ||
| TableName tableName, Table t) throws IOException { | ||
| return getPermissions(conf, tableName != null ? tableName.getName() : null, t, null, null, null, | ||
| false); | ||
| } |
There was a problem hiding this comment.
The existing one without t can call this:
public static ListMultimap<String, UserPermission> getTablePermissions(Configuration conf,
TableName tableName) throws IOException {
return getTablePermissions(conf, tableName, null);
}There was a problem hiding this comment.
Good catch — done. The 2-arg overload now delegates to the 3-arg one with t=null to avoid duplicating the getPermissions(...) call.
…rations Each snapshot operation created two short-lived connections in SnapshotDescriptionUtils.validate() — one for isSecurityAvailable() to check hbase:acl table existence, and another in writeAclToSnapshotDescription() to read ACL data. In Kerberos environments with ZKConnectionRegistry, each connection triggered a new ZK session with GSSAPI authentication and a TGS request to the KDC, causing excessive KDC load during batch snapshot operations. This patch reuses the caller's existing connection instead of creating new ones: - isSecurityAvailable() now accepts a Connection parameter - writeAclToSnapshotDescription() passes the shared connection's Table to PermissionStorage.getTablePermissions() - All callers (MasterRpcServices, RestoreSnapshotProcedure, CloneSnapshotProcedure) pass through their available connection The original validate(SnapshotDescription, Configuration) and isSecurityAvailable(Configuration) signatures are preserved as overloads that delegate to the new Connection-based methods, so existing callers (including tests) keep working unchanged. The new Connection-based overloads are documented as preferred for callers that already hold a Connection, to avoid the per-call connection setup cost.
Summary
Each snapshot operation created two short-lived connections in
SnapshotDescriptionUtils.validate():isSecurityAvailable(conf)— created aConnection+Adminjust to checkhbase:acltable existencewriteAclToSnapshotDescription()— created anotherConnectionto read ACL data fromhbase:aclIn Kerberos environments with the default
ZKConnectionRegistry, each connection triggered a new ZK session with GSSAPI authentication and a TGS request to the KDC. During batch snapshot operations, this caused excessive KDC load that could lead to IP blocking.This patch reuses the caller's existing connection instead of creating new ones:
isSecurityAvailable()now accepts aConnectionparameterwriteAclToSnapshotDescription()passes the shared connection'sTabletoPermissionStorage.getTablePermissions()MasterRpcServices,RestoreSnapshotProcedure,CloneSnapshotProcedure) pass through their available connectionZero behavioral change — the same checks are performed, the same data is read, the same ACL is written to snapshots.
See HBASE-30058 for detailed root cause analysis including design ambiguity discussion.