-
Notifications
You must be signed in to change notification settings - Fork 5.3k
[CoreCLR][Signal] Bump shutdown notif and crashdump before prev handler #123735
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mdh1418
wants to merge
1
commit into
dotnet:main
Choose a base branch
from
mdh1418:reorder_previous_signal_handler
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+8
−5
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not think this works well when there are multiple runtimes loaded in the process that all want to handle segfaults. It can be multiple .NET runtimes (e.g. CoreCLR + NAOT, or multiple NAOT), or it can be .NET and some other runtime (e.g. Java runtime).
The expected behavior in these situations is that the given runtime will check whether the signal happened in the code that it cares about. If yes, it will handle the signal. If no, it will forward the signal to the next runtime, and so on.
With this change, I think we will shutdown our runtime instance and generate crashdump if there is segfault gracefully handled by some other runtime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gracefully handled by another runtime, as in sa_sigaction/sa_handler? Wouldn't those still hit PROCNotifyProcessShutdown/PROCCreateCrashDumpIfEnabled in the original implementation? Or do they somehow return from
invoke_previous_actionThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the signal is handled in some other runtime, the handler registered by that runtime would not return.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, so given that we cannot tell how the other runtime will handle the signal, would it make sense then to pivot to an opt-in config switch that allows triggering a shutdown/create dump, even if the other runtime handled it gracefully?
I am not sure yet if there is a way for Android CoreCLR to not have a previously registered signal handler, so in those cases we wouldn't ever create crash dumps for signals.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we know what the handlers that are registered on Android before us do?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Mono we have this as opt-in feature and it will create the native crash report before chaining any signal handlers, but its off by default and apps opt in to enable the feature through the embedding API's (as described in the PR description). dotnet Android SDK currently opt-in to this feature, meaning that it will dump the native crash report before chaining signal handlers.
The "crash dump" that we collect on Android is mainly crash details going into logcat in text format, since that is the only way to get crash data out of retail devices in cases when using default error reporting, logcat output will be one of few artifacts uploaded from installed app on retail devices, unless app is running 3'rd party crash services that could upload additional data into their own services.
The Android signal handler is most likely invoking the Android crash daemon generating the Android crash report and tombstone and terminate the app, so it won't return, meaning there is no way for us to report additional information in a SIGSEGV scenario unless we do it before that handler gets executed.
There is AFAIK no reliable way to detect that its the Andorid crash report handler that has been registered, maybe we could identify the handler as being part of Android libraries and act based on that information, but that will probably break sooner or later, so probably not a good path forward.
Let say we have this as an opt-in as we do on Mono, meaning we can generate a crash report before chaining signals, should we still call PROCNotifyProcessShutdown or is it enough to just call PROCCreateCrashDumpIfEnabled triggering the Android specific implementation of that function?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we could do something along these lines instead (default config value is false):
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm seeing that
PROCNotifyProcessShutdowndoesn't actually handle/signal shutdown. It currently only applies to Unix and just cleans up the debugger transport and diagnostic server.For non-Android,
PROCCreateCrashDumpIfEnabledwill triggerPROCCreateCrashDumpwhich will likely terminate the process withexit().On Android CoreCLR, there is no CreateDump (hopefully we will have one in the future), so its PROCCreateCrashDumpIfEnabled is just
runtime/src/coreclr/pal/src/thread/process.cpp
Lines 2743 to 2751 in cf05e72
Do other CoreCLR platforms not do so if crash dumps are enabled?
I think having parity with other platforms makes sense, but has any customer hit a crash on desktop CoreCLR and not produced a dump with
DOTNET_DbgEnableMiniDump=1because of signal handler registered out of their control?Unless we coordinate with Android's crash reporter resolve our symbols, it feels like we should atleast allow an opt-in to generate a crash report without terminating the process, and then pass the signal along to the previously registered handler.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, CoreCLR post-mortem diagnostics tooling is oriented around crash dumps. It would be about figuring out the whole flow - what would people need to do to opt-in on the given retail device (drop a config file to some known user writeable location?) and what would they need to do to exfiltrate the crashdump from the device (find the crash dump in some known location?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we can start with that. I do not think we even need to pass the signal along to the previously registered handler if it is an opt-in.