Skip to content

Remove atos for backtrace generation. #4930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
+32 −141
Open

Remove atos for backtrace generation. #4930

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
010f7a2
Add dladdr symbolication
LunaTheFoxgirl May 15, 2025
da0f49b
Completely remove atos
LunaTheFoxgirl May 15, 2025
642941a
Added cleanup hooks
LunaTheFoxgirl May 16, 2025
0fe3c91
Add atos removal note
LunaTheFoxgirl May 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 32 additions & 141 deletions runtime/druntime/src/core/internal/backtrace/dwarf.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
* Reference: http://www.dwarfstd.org/
* Copyright: Copyright Digital Mars 2015 - 2015.
* License: $(HTTP www.boost.org/LICENSE_1_0.txt, Boost License 1.0).
* Authors: Yazan Dabain, Sean Kelly
* Authors: Yazan Dabain, Sean Kelly, Luna Nielsen
* Source: $(DRUNTIMESRC core/internal/backtrace/dwarf.d)
*/

Expand All @@ -69,6 +69,7 @@ else

import core.internal.container.array;
import core.stdc.string : strlen, memcpy;
import core.attribute : weak;

//debug = DwarfDebugMachine;
debug(DwarfDebugMachine) import core.stdc.stdio : printf;
Expand Down Expand Up @@ -184,13 +185,18 @@ int traceHandlerOpApplyImpl(size_t numFrames,
startIdx = idx + 1;
}

// Symbolicate locations
rt_dwarfSymbolicate(locations[startIdx .. $]);

if (!image.isValid())
return locations[startIdx .. $].processCallstack(null, 0, dg);

// find address -> file, line mapping using dwarf debug_line
return image.processDebugLineSectionData(
(line) => locations[startIdx .. $].processCallstack(line, image.baseAddress, dg));

// Allow cleaning up after ourselves.
rt_dwarfSymbolicateCleanup(locations[startIdx .. $]);
}

struct TraceInfoBuffer
Expand Down Expand Up @@ -237,18 +243,38 @@ struct TraceInfoBuffer
}
}

/**
* A weakly linked hook which can be implemented by external libraries
* to extend the symbolication capabilites when debug info is missing.
*
* NOTE:
* There used to be an atos based symbolication implementation built in
* here, but atos is not a portable solution on darwin derived OSes.
* atos conflicts with things such as the hardened runtime, iOS releases,
* App Store certification and the like. I've removed that implementation
* to ensure that D can easily be used to publish to the App Store.
* Please avoid adding other private APIs in its place directly in to
* druntime. If it resides in PrivateFrameworks or is a dev tool,
* don't use it. - Luna
*/
@weak
extern(C) void rt_dwarfSymbolicate(Location[] locations)
{
}

/// ditto
@weak
extern(C) void rt_dwarfSymbolicateCleanup(Location[] locations)
{
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more or less already handled earlier:

ldc/runtime/druntime/src/core/runtime.d

Lines 925 to 948 in 53ffdf1

// https://code.woboq.org/userspace/glibc/debug/backtracesyms.c.html
// The logic that glibc's backtrace use is to check for for `dli_fname`,
// the file name, and error if not present, then check for `dli_sname`.
// In case `dli_fname` is present but not `dli_sname`, the address is
// printed related to the file. We just print the file.
static const(char)[] getFrameName (const(void)* ptr)
{
import core.sys.posix.dlfcn;
Dl_info info = void;
// Note: See the module documentation about `-L--export-dynamic`
if (dladdr(ptr, &info))
{
// Return symbol name if possible
if (info.dli_sname !is null && info.dli_sname[0] != '\0')
return info.dli_sname[0 .. strlen(info.dli_sname)];
// Fall back to file name
if (info.dli_fname !is null && info.dli_fname[0] != '\0')
return info.dli_fname[0 .. strlen(info.dli_fname)];
}
// `dladdr` failed
return "<ERROR: Unable to retrieve function name>";
}

Copy link
Contributor Author

@LunaTheFoxgirl LunaTheFoxgirl May 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good to know, then this can just be a no-op stub. I am calling it for the day (spent 6 hours getting ldc2 compiling due to fighting with llvm and cmake for some time. Did get it working eventually)

Copy link
Member

@kinke kinke May 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. The hook's job is to populate the file and line fields of the Locations, if possible. So the name could reflect that (resolveSourceLocs() or so).

Edit: Well, that's the primary job at least. There's nothing standing in the way of improving/adding the symbol names too. E.g., to make it work without -L--export-dynamic too.

Copy link
Contributor Author

@LunaTheFoxgirl LunaTheFoxgirl May 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've called it symbolicate since it may, on macOS, also end up handling symbolication of Objective-C which requires another kind of roundtrip not related to DWARF. So in general it's supposed to be a hook that pre-symbolicates stuff (including the name of the procedure if relevant). Then letting the default implementation handle anything left over.


private:

int processCallstack(Location[] locations, const(ubyte)[] debugLineSectionData,
size_t baseAddress, scope int delegate(ref size_t, ref const(char[])) dg)
{
if (debugLineSectionData)
resolveAddresses(debugLineSectionData, locations, baseAddress);
version (Darwin)
{
if (!debugLineSectionData)
resolveAddressesWithAtos(locations);
}

TraceInfoBuffer buffer;
foreach (idx, const ref loc; locations)
Expand All @@ -267,141 +293,6 @@ int processCallstack(Location[] locations, const(ubyte)[] debugLineSectionData,
return 0;
}

version (Darwin) {
/**
* Resolve the addresses of `locations` using `atos` (executable that ships with XCode)
*
* Spawns a child process that calls `atos`. Communication is through stdin/stdout pipes.
*
* After this function successfully completes, `locations` will contain
* file / lines informations.
*
* The lifetime of the `Location` data surpases function return (strndup is used).
*
* Params:
* locations = The locations to resolve
*/
private void resolveAddressesWithAtos(Location[] locations) @nogc nothrow
{
import core.stdc.stdio : fclose, fflush, fgets, fprintf, printf, snprintf;
import core.stdc.stdlib : exit;
import core.sys.posix.stdio : fdopen;
import core.sys.posix.unistd : close, dup2, execlp, fork, getpid, pipe;
// Create in/out pipes to communicate with the forked exec
int[2] dummy_pipes; // these dummy pipes are there to prevent funny issues when stdin/stdout is closed and pipe returns id 0 or 1
int[2] pipes_to_atos;
int[2] pipes_from_atos;
if ( pipe(dummy_pipes) < 0 || pipe(pipes_to_atos) < 0 || pipe(pipes_from_atos) < 0 ) {
printf("some pipe creation error!\n");
return;
}
close(dummy_pipes[0]);
close(dummy_pipes[1]);
auto write_to_atos = pipes_to_atos[1];
auto read_from_atos = pipes_from_atos[0];
auto atos_stdin = pipes_to_atos[0];
auto atos_stdout = pipes_from_atos[1];
auto self_pid = cast(int) getpid();
// Spawn a child process that calls atos, reads/writes from the pipes, and then exits.
auto child_id = fork();
if (child_id == -1)
{
printf("some fork error!\n");
return;
}
else if (child_id == 0)
{
// We are in the child process, spawn atos and link pipes
// Close unused read/write ends of pipes
close(write_to_atos);
close(read_from_atos);
// Link pipes to stdin/stdout
dup2(atos_stdin, 0);
close(atos_stdin);
dup2(atos_stdout, 1);
close(atos_stdout);
char[10] pid_str;
snprintf(pid_str.ptr, pid_str.sizeof, "%d", cast(int) self_pid);
const(char)* atos_executable = "atos";
const(char)* atos_p_arg = "-p";
const(char)* atos_fullpath_arg = "-fullPath";
execlp(atos_executable, atos_executable, atos_fullpath_arg, atos_p_arg, pid_str.ptr, null);
// If exec returns, an error occurred, need to exit the forked process here.
printf("some exec error!\n");
exit(0);
}
// Parent process just continues from here.
// Close unused pipes
close(atos_stdin);
close(atos_stdout);
auto to_atos = fdopen(write_to_atos, "w");
auto from_atos = fdopen(read_from_atos, "r");
// buffer for atos reading. Note that symbol names can be super large...
static char[16 * 1024] read_buffer = void;
char* status_ptr = null;
foreach (ref loc; locations)
{
fprintf(to_atos, "%p\n", loc.address);
fflush(to_atos);
read_buffer[0] = '\0';
status_ptr = fgets(read_buffer.ptr, read_buffer.sizeof, from_atos);
if (!status_ptr)
break;
Location parsed_loc = parseAtosLine(read_buffer.ptr);
if (parsed_loc.line != -1)
{
// Only update the file:line info, keep the procedure name as found before (preserving the standard truncation).
loc.file = parsed_loc.file;
loc.line = parsed_loc.line;
}
}
if (!status_ptr)
printf("\nDid not succeed in using 'atos' for extra debug information.\n");
fclose(to_atos);
fclose(from_atos);
close(write_to_atos);
close(read_from_atos);
}
private Location parseAtosLine(char* buffer) @nogc nothrow
{
// The line from `atos` is in one of these formats:
// myfunction (in library.dylib) (sourcefile.c:17)
// myfunction (in library.dylib) + 0x1fe
// myfunction (in library.dylib) + 15
// 0xdeadbeef (in library.dylib) + 0x1fe
// 0xdeadbeef (in library.dylib) + 15
// 0xdeadbeef (in library.dylib)
// 0xdeadbeef
import core.stdc.stdlib : atoi;
import core.stdc.string : strchr, strstr;
import core.sys.posix.string : strndup;
Location loc;
if (!buffer)
return loc;
if (buffer[0] == '0' && buffer[1] == 'x')
// no named symbol found
return loc;
const symbolname_end = strstr(buffer, " (in ");
if (!symbolname_end)
return loc;
const symbolname_size = symbolname_end - buffer;
loc.procedure = strndup(buffer, symbolname_size)[0..symbolname_size];
const filename_start = strstr(symbolname_end, ") (") + 3;
if (cast(size_t)filename_start < 4)
return loc;
const colon_location = strchr(filename_start, ':');
if (!colon_location)
return loc;
const filename_size = colon_location - filename_start;
loc.file = strndup(filename_start, filename_size)[0..filename_size];
const final_paren = strchr(colon_location+1, ')');
if (!final_paren)
return loc;
loc.line = atoi(colon_location+1);
return loc;
}
}

/**
* Resolve the addresses of `locations` using `debugLineSectionData`
*
Expand Down
Loading