HBASE-30102 Add metric to account for region data classified as cold by the Time Based Priority logic#8128
HBASE-30102 Add metric to account for region data classified as cold by the Time Based Priority logic#8128wchevreuil wants to merge 3 commits intoapache:masterfrom
Conversation
…by the Time Based Priority logic Change-Id: I5601a37300a3f5d10fe4886ba988f2d25e66b546
taklwu
left a comment
taklwu
left a comment
There was a problem hiding this comment.
when Time Based Priority is disabled, what would be the % Cold Data? is it always showing?
| getAllCacheKeysForFile(hFileInfo.getHFileContext().getHFileName(), 0, Long.MAX_VALUE); | ||
| int evictedBlocks = evictBlockSet(keySet); | ||
| if (evictedBlocks > 0) { | ||
| LOG.info("Evicted {} blocks for file {} as it is now considered cold by DataTieringManager", |
There was a problem hiding this comment.
nit: should we have it as debug level? I'm wondered if we see a lot of these message.
There was a problem hiding this comment.
Maybe, yeah. Although it would be triggered only upon enabling of the time based priority on the individual store, and once for each affected file, it can still flood the logs. Let me switch it to DEBUG.
| if (key.getCfName() != null) { | ||
| builder.setFamilyName(key.getCfName()); | ||
| } | ||
| if (key.getRegionName() != null) { | ||
| builder.setRegionName(key.getRegionName()); | ||
| } |
There was a problem hiding this comment.
question: why weren't the cf and regionname filled before ? is it because the cold data needs for log message or other compute usage?
There was a problem hiding this comment.
Yes. This is required not only by the new "coldDataRatio" metric we are adding, but also the existing "regionCachedRatio" that is critical for the CacheAwareLoadBalancer. Without this change here, we cannot calculate these metrics when recovering the persistent cache. IMO, it's a bug in the current CacheAwareLoadBalancer implementation.
Change-Id: I392517f882e7c5a8c6063b16f525f6467956a3bb
taklwu
left a comment
taklwu
left a comment
There was a problem hiding this comment.
LGTM.
something is wrong with the github action, can you give a try to trigger them ?
2 participants
No description provided.