[SR-1251] Can't import libbsd from modulemap

[drewcrawford]

Previous ID	SR-1251
Radar	rdar://problem/66398034
Original Reporter	@drewcrawford
Type	Bug

Attachment: Download

Environment

swift-DEVELOPMENT-SNAPSHOT-2020-07-22-a
Linux x64

Additional Detail from JIRA

Votes	1
Component/s
Labels	Bug, ClangImporter, Linux, ModuleMaps
Assignee	None
Priority	Medium

md5: 990948b78eef73ab0863e3462d1ceb9d

Issue Description:

I'm attempting to use libBSD.

Modulemap:

module BSD [system] {
    header "/usr/include/bsd/stdlib.h"
    export *
}

import BSD

Producing error

/usr/include/bsd/libutil.h:63:59: error: declaration of 'pid_t' must be imported from module 'SwiftGlibc.C.signal' before it is required
struct pidfh *pidfile_open(const char *path, mode_t mode, pid_t *pidptr);

See also SR-685

[belkadan]

This sounds like <bsd/stdlib.h> doesn't include all its dependencies. Do you know if that's the case?

[drewcrawford]

I'm not sure I understand the question... what I know is that this used to work, but stopped working around a month or so ago.

My understanding of the problem is that libbsd tries to import unistd.h, which defines pid_t. Swift seems to dislike the importing of unistd.h for some reason, although I have not worked out why or why this changed.

[belkadan]

Hm. The actual definer of pid_t in this case is sys/types.h, but Clang thinks it's in signal.h. @benlangmuir, any ideas?

I imagine you'd be able to reproduce this issue with just Clang. I tried this on our Linux machine:

echo '@import BSD;' | ~/public/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/bin/clang -fmodule-map-file=$HOME/test.modulemap -fsyntax-only -x objective-c -fmodules -

but it compiled successfully there.

[benlangmuir]

The actual definer of pid_t in this case is sys/types.h, but Clang thinks it's in signal.h.

Sounds like a non-modular include. Could you try removing the module cache and then building with -Xcc -Wnon-modular-include-in-module -Xcc -Wsystem-headers to see if we get a useful warning here when building the SwiftGlibc module.

If that's too noisy you can prefix those warning flags with -Xcc -Wno-everything so that only that one warning is emitted. Note that the no-everything flag needs to come first so that the later warning flags will override it.

[drewcrawford]

I can't reproduce with clang either:

# cat module.modulemap
module BSD [system] {
    header "/usr/include/bsd/stdlib.h"
    export *
}
# echo '@import BSD;' | clang -I . -fsyntax-only -x objective-c -fmodules -
# echo $?
0

But I can with swift:

# cat test.swift
import BSD
print("hello world")
# swiftc -I . test.swift
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/usr/include/bsd/stdlib.h"
         ^
/usr/include/bsd/stdlib.h:39:10: note: in file included from /usr/include/bsd/stdlib.h:39:
#include <bsd/libutil.h>
         ^
/usr/include/bsd/libutil.h:63:59: error: declaration of 'pid_t' must be imported from module 'SwiftGlibc.C.signal' before it is required
struct pidfh *pidfile_open(const char *path, mode_t mode, pid_t *pidptr);
                                                          ^
/usr/include/signal.h:61:17: note: previous declaration is here
typedef __pid_t pid_t;
                ^
test.swift:1:8: error: could not build Objective-C module 'BSD'
import BSD
       ^

Could you try removing the module cache and then building with -Xcc -Wnon-modular-include-in-module -Xcc -Wsystem-headers to see if we get a useful warning here when building the SwiftGlibc module.

Sorry to be pedantic, I suspect I have not understood something important here. AFAIK, SwiftGlibc is built with swift, so I believe you are asking if I can rebuild Swift with these compile flags. If so, I don't know how to get those compile flags inside of build-script.

Or maybe you are asking to use those flags when building my hello world program? In that case:

# rm -rf /tmp/org.llvm.clang.0/
# swiftc -I . -Xcc -Wnon-modular-include-in-module -Xcc -Wsystem-headers test.swift 2> log

That produces the following log: log Maybe that is useful.

[swift-ci]

Comment by Honza Dvorsky (JIRA)

FWIW I was having a similar issue and fixed it by adding the missing include: ZewoGraveyard/CZlib@c102e18?diff=unified

OS X seems to be adding that automatically, it seems.

[swift-ci]

Comment by Tomáš Linhart (JIRA)

@benlangmuir@belkadan I have same issue when trying to use GTK+3 on Linux. I have created a simple package that you can use to reproduce this issue https://github.com/TomasLinhart/CTest Just use this on Ubuntu with latest stable Swift (3.0.2) and you should see the problem as well.

[swift-ci]

Comment by Helge Heß (JIRA)

Seeing the same thing when including httpd.h on Linux. Is there a workaround? :-)

[swift-ci]

Comment by Bryan Dela Cruz (JIRA)

TomasLinhart (JIRA User) I also encountered the same issue while trying to use SwiftGtk 🙂. I am not an expert in C++ but I think what I did to resolve the issue is very wrong. D: I modified "/usr/include/glib-2.0/gio/gcredentials.h" and added a line to include the header "termios.h" right after the line that says "#ifndef G_OS_UNIX". After that, I was able to run your sample SwiftGtk code.

[swift-ci]

Comment by Bryan Dela Cruz (JIRA)

Creating a seperate header file in the system module and including termios.h in it resolves the problem.

module BSD [system] {
  header "./includes.h"
  export *
}

#include <termios.h>
#include <bsd/stdlib.h>

[swift-ci]

Comment by Helge Heß (JIRA)

I can confirm that, that is what I have discovered by accident too :-) This is what I do for APR now:

  cat /usr/local/lib/swift/shims/APRShim.h 
  #ifndef __APR_SWIFT_SHIM_H__
  #define __APR_SWIFT_SHIM_H__
  
  #ifdef __linux__
  #  include <termios.h>
  #endif
  
  #include "apr.h"

[swift-ci]

Comment by Vincent Woo (JIRA)

I'm having this problem on Swift 4.0, whereas I did not in the latest version of Swift 3.x. My reproduction case is extremely easy and can be done on any Ubuntu (or deb-like) machine.

Previously:

apt-get install libbsd-dev

swift -lbsd -import-objc-header /usr/include/bsd/bsd.h HelloWorld.swift

would work. In Swift 4, we see the termios.h error as show in this bug. Is there a recommendation? Is this intentional?

[gwynne]

Incredibly, this is still happening with Swift 5.2.4, and with the currently nightly snapshots!

$ docker run -ti --rm --privileged swiftlang/swift:nightly-master-focal
 ################################################################
 #                              #
 # Swift Nightly Docker Image                   #
 # Tag: swift-DEVELOPMENT-SNAPSHOT-2020-07-22-a         #
 #                              #
 ################################################################

root@c0f369fa8b0d:/# swift --version
Swift version 5.3-dev (LLVM e966de21c9ff9ed, Swift 493b4a8ef836af0)
Target: x86_64-unknown-linux-gnu
root@c0f369fa8b0d:/# apt-get -q -q update && apt-get -q -q install libbsd-dev
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libbsd-dev:amd64.
(Reading database ... 16756 files and directories currently installed.)
Preparing to unpack .../libbsd-dev_0.10.0-1_amd64.deb ...
Unpacking libbsd-dev:amd64 (0.10.0-1) ...
Setting up libbsd-dev:amd64 (0.10.0-1) ...
root@c0f369fa8b0d:/# echo 'import BSD' > test.swift
root@c0f369fa8b0d:/# swift -lbsd -import-objc-header /usr/include/bsd/bsd.h test.swift
/usr/include/bsd/bsd.h:40:10: note: in file included from /usr/include/bsd/bsd.h:40:
#include <bsd/stdlib.h>
         ^
/usr/include/bsd/stdlib.h:39:10: note: in file included from /usr/include/bsd/stdlib.h:39:
#include <bsd/libutil.h>
         ^
/usr/include/bsd/libutil.h:61:59: error: missing '#include <termios.h>'; declaration of 'pid_t' must be imported from module 'SwiftGlibc.POSIX.termios' before it is required
struct pidfh *pidfile_open(const char *path, mode_t mode, pid_t *pidptr);
                                                          ^
/usr/include/termios.h:30:17: note: previous declaration is here
typedef __pid_t pid_t;
                ^
<unknown>:0: error: failed to import bridging header '/usr/include/bsd/bsd.h'
root@c0f369fa8b0d:/# head -n 50 /usr/include/bsd/libutil.h | tail -n 10

#ifdef LIBBSD_OVERLAY
#include <sys/cdefs.h>
#else
#include <bsd/sys/cdefs.h>
#endif
#include <sys/types.h>
#include <stdint.h>
#include <stdio.h>

root@c0f369fa8b0d:/# grep pid_t /usr/include/x86_64-linux-gnu/sys/types.h
#ifndef __pid_t_defined
typedef __pid_t pid_t;
# define __pid_t_defined

The appropriate system header is included by libbsd's header, and that header does define pid_t. termios.h also contains a definition of pid_t, for good measure. I get the idea this is a case of the preprocessor making a mess of the module system again; on Focal there appear to be no less than eleven system headers which will typedef pid_t if it hasn't been already. There is no such ambiguity in the macOS SDK; pid_t is defined by sys/_types/_pid_t.h and nowhere else.

Unfortunately, I don't have much idea what a real fix for this would look like; there's a lot of other such witchcraft going on in the Linux "toolchain"... 🙁

[typesanitizer]

@swift-ci create