Fix TLS variable access issue under PGO optimization#3284
Open
zcfh wants to merge 1 commit intoapache:masterfrom
Open
Fix TLS variable access issue under PGO optimization#3284zcfh wants to merge 1 commit intoapache:masterfrom
zcfh wants to merge 1 commit intoapache:masterfrom
Conversation
In task_group, there is the following call chain where ready_to_run_in_worker is indirectly called: `task_runner -> ending_sched -> sched_to -*-> ready_to_run_in_worker` With PGO optimization is enabled, these functions may be inlined into `task_runner`, which will cause TLS access error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number:
Problem Summary:
What is changed and the side effects?
在编译器,开启PGO优化后,task_group.cpp 中,
tls_unique_user_ptr/tls_task_group的访问,还会有异常。具体原因
在task_group 中,包含下面的调用链,其中 ready_to_run_in_worker 是间接调用
task_runner -> ending_sched -> sched_to -*-> ready_to_run_in_worker在开启pgo优化的场景下,存在这样的问题:
ending_sched/sched_to,是个热函数时,会调高这两个函数的inline阈值,此时 sched_to 和 ending_sched 会被inline,此时void* saved_unique_user_ptr = tls_unique_user_ptr;这个关于 tls_unique_user_ptr 的访问就会有异常。复现途径
由于PGO优化,需要profile,我们的复现场景profile较为复杂无法直接提供。一个简单的复现方式是,使用
-mllvm --inline-threshold=10000的编译选项,手动调高inline的阈值。环境信息:
总结
对于tls变量的访问,原先在函数的"开头"部分,在函数不被inline的情况下,是不会发生tls的访问异常的。但是类似PGO这类的 profile-guided-optimization,会挖掘一些潜在的inline机会。所以,这些发生在函数"开头"部分的tls变量访问,仍需通过宏来禁用优化。
补充
此外,还发现了一处调用链, 这里也会出现把tls基地址提前缓存起来的问题,不过没对我们的服务产生影响,可能需要你们评估一下,是否需要禁用优化。
Changed:
ready_to_run_in_worker\ready_to_run_in_worker_ignoresignal\sched_to中对tls变量的访问,都改为宏BAIDU_GET_VOLATILE_THREAD_LOCALCheck List: