Skip to content

Removes running cache from coordinator#6244

Merged
keith-turner merged 4 commits intoapache:mainfrom
keith-turner:remove-running-cache
Mar 26, 2026
Merged

Removes running cache from coordinator#6244
keith-turner merged 4 commits intoapache:mainfrom
keith-turner:remove-running-cache

Conversation

@keith-turner
Copy link
Copy Markdown
Contributor

@keith-turner keith-turner commented Mar 25, 2026

This contains uncommitted changes from many other PRs related to the running cache, so opening as a draft for now. This is the last PR in the series that actually removes the running cache, now that most of its usages are removed.

@keith-turner keith-turner force-pushed the remove-running-cache branch from 7d71352 to 3ccccad Compare March 25, 2026 17:53
@keith-turner keith-turner marked this pull request as ready for review March 25, 2026 18:06
LOG.warn(
"External compaction {} running in group {} on compactor {},"
+ " but group not found in current configuration. Failing compaction...",
tec.getJob().getExternalCompactionId(), tec.getGroupName(), tec.getCompactor());
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this functionality and not sure it should be carried forward in anyway. If a user has a compactor group A that is running long running compactions. They start a new compactor group B up and configure new long running compactions to go there. They want the compactions running in A to finish and will kill all processes in A when that happens. This code would interfere w/ that intent. Seems like if this done it should be user directed.

* @throws ThriftSecurityException when permission error
*/
@Override
public void updateCompactionStatus(TInfo tinfo, TCredentials credentials,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with todo below. This can be removed. It was to update the status of the entry in the running cache, and to centralize logging of status updates in the coordinator.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will do that as a follow on, would be good to remove the RPC and the compactor calls to it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #6252

@keith-turner keith-turner merged commit 491c0d6 into apache:main Mar 26, 2026
9 checks passed
@keith-turner keith-turner deleted the remove-running-cache branch March 26, 2026 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants