defer statement fails to increment refcount of `consuming` binding, causing segmentation fault

[tayloraswift]

Description

the following test program crashes with a segfault when compiled with a 5.9.1 toolchain.

Reproduction

class Class
{
    init()
    {
    }
}

struct ClassList
{
    var array:[Class]

    init(_ x:consuming Class)
    {
        self.array = []

        defer
        {
            self.array.append(x)
        }

        _ = copy x
    }
}

func main()
{
    let _:ClassList = .init(Class.init())
}

main()

$ swiftc Snippets/_Crashes/Refcounting.swift 
$ ./Refcounting

Stack dump

💣 Program crashed: Bad pointer dereference at 0x00007fd15e861868

Thread 0 "Refcounting" crashed:

0 0x00007fd15e861868
1 0x00007fd15e60429b bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1> >::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 122 in libswiftCore.so
2 0x00007fd15e5f789b swift_arrayDestroy + 58 in libswiftCore.so
3 0x00007fd15e3a53b7 _ContiguousArrayStorage.__deallocating_deinit + 38 in libswiftCore.so
4 0x00007fd15e6038e0 _swift_release_dealloc + 15 in libswiftCore.so
5 0x00007fd15e60429b bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1> >::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 122 in libswiftCore.so
6 0x000056073eea4f68 main() + 39 in Refcounting
7 0x000056073eea4d29 main + 8 in Refcounting

Segmentation fault (core dumped)

Expected behavior

the defer statement should correctly increment the ref count of the captured binding.

Environment

$ swift --version
Swift version 5.9.1 (swift-5.9.1-RELEASE)
Target: x86_64-unknown-linux-gnu

Additional information

No response

[MaxDesiatov]

Is this still reproducible with Swift 5.9.2 for you? If so, can you also clarify which exact Linux distribution you're using and its exact version?

[tayloraswift]

this is still happening for me on 5.9.2.

$ ./crash3

💣 Program crashed: Bad pointer dereference at 0x00007f5fed982868

Thread 0 "crash3" crashed:

0 0x00007f5fed982868
1 0x00007f5fed72532b bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1> >::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 122 in libswiftCore.so
2 0x00007f5fed71892b swift_arrayDestroy + 58 in libswiftCore.so
3 0x00007f5fed4c6447 _ContiguousArrayStorage.__deallocating_deinit + 38 in libswiftCore.so
4 0x00007f5fed724970 _swift_release_dealloc + 15 in libswiftCore.so
5 0x00007f5fed72532b bool swift::RefCounts<swift::RefCountBitsT<(swift::RefCountInlinedness)1> >::doDecrementSlow<(swift::PerformDeinit)1>(swift::RefCountBitsT<(swift::RefCountInlinedness)1>, unsigned int) + 122 in libswiftCore.so
6 0x000055c23758cf68 main() + 39 in crash3
7 0x000055c23758cd29 main + 8 in crash3

Backtrace took 0.04s

Segmentation fault (core dumped)

i’m using Amazon Linux 2 in a docker container.

dockerfile

FROM amazonlinux:2

RUN yum -y update 
# install sysadmin basics 
RUN yum -y install sudo passwd 
# install swift dependencies
RUN yum -y install \
    binutils \
    gcc \
    git \
    unzip \
    glibc-static \
    gzip \
    libbsd \
    libcurl-devel \
    libedit \
    libicu \
    libstdc++-static \
    libuuid \
    libxml2-devel \
    tar \
    tzdata \
    zlib-devel

# create the `ec2-user`, and switch to her
RUN useradd -ms /bin/bash ec2-user
RUN passwd -d ec2-user
RUN usermod -aG wheel ec2-user
USER ec2-user

WORKDIR /home/ec2-user/

# optional, but python, and iptables are very useful in a container
RUN sudo yum -y install python3 python3-devel iptables nc 

# jemalloc
RUN sudo yum -y install bzip2 make
RUN curl https://github.com/jemalloc/jemalloc/releases/download/5.3.0/jemalloc-5.3.0.tar.bz2 -L -o jemalloc-5.3.0.tar.bz2
RUN tar -xf jemalloc-5.3.0.tar.bz2
RUN cd jemalloc-5.3.0 && ./configure && make && sudo make install

# install swift

RUN sudo yum -y install rsync

RUN curl https://download.swift.org/swift-5.9.2-release/amazonlinux2/swift-5.9.2-RELEASE/swift-5.9.2-RELEASE-amazonlinux2.tar.gz -o toolchain.tar.gz

RUN mkdir toolchain
RUN tar -xvz -f toolchain.tar.gz -C toolchain --strip-components=1
RUN rm toolchain.tar.gz
RUN sudo rsync -av toolchain/usr /
RUN sudo rm -r toolchain

# generate script that will run on terminal creation, 
# enables showing the PWD prompt
RUN echo "PS1='\w\$ '" >> .bashrc
RUN echo "force_color_prompt=yes" >> .bashrc
ENV TERM xterm-256color

CMD sleep infinity

[MaxDesiatov]

@gottesmm this looks like something specific to new ownership features?

[tayloraswift]

this miscompilation still occurs on the 5.10 nightly branch, however it became slightly harder to reproduce due to what i assume are 5.10-only ARC optimizations.

updated reproducer:

class Class
{
    init()
    {
    }
}

struct ClassList
{
    var array:[Class]

    @inline(never)
    init(_ x:consuming Class)
    {
        self.array = []

        defer
        {
            self.array.append(x)
        }

        _ = copy x
    }
}

@inline(never)
func f(x:Class)
{
    let _:ClassList = .init(x)
}

func main()
{
    f(x: Class.init())
}

main()

note the two @inline(never)s; without f(x:) i believe the compiler is allocating the class on the stack which prevents the segfault.