Possible memory leak

[PHLAK]

• Laravel Version: 9.30.1
• PHP Version: 8.1.10
• Database Driver & Version: N/A

Description:

We've been having memory related issues in CI recently where the memory consumption continuously grows. This is a particular problem when enabling coverage checks (i.e. artisan test --coverage) as this eventually exhausts the memory in the CI environment. A few weeks ago we thought we had this traced back to this issue in PHP core which was fixed as of v8.1.10, however, after updating to 8.1.10 we're still having memory issues in CI.

I recently found this article which recommends creating a test to repeatedly create, boot and flush the (Laravel) application multiple times to see the memory consumption over time. I did this on a fresh Laravel application and, as you can see below, the memory usage constantly rises over time.

I tried to reproduce this on a personal project that doesn't use Laravel to determine if it was an issue with phpunit or PHP core but was unable to reproduce the leak so I'm leaning towards Laravel at the moment (however, I'm not convinced of this yet).

I haven't dug any deeper at this point as I wanted to bring this to your attention but am happy to keep investigating if necessary.

Steps To Reproduce:

Create a new Laravel application.

$ laravel new test-app

Add the following test case to the newly created application.

public function test_leaks_memory_on_1000_iterations()
{
    $this->app->flush();
    $this->app = null;

    for ($i = 1; $i < 1000; ++$i) {
        $this->createApplication()->flush();

        dump('Using ' . ((int) (memory_get_usage(true) / (1024 * 1024))) . 'MB in ' . $i . ' iterations.');
    }

    $this->app = $this->createApplication();
}

Run the test and observe the ever increasing memory consumption.

$ phpunit tests/Feature/ExampleTest.php
PHPUnit 9.5.24 #StandWithUkraine

^ "Using 20MB in 1 iterations."
^ "Using 20MB in 2 iterations."
^ "Using 20MB in 3 iterations."
[TRUNCATED FOR BREVITY]
^ "Using 26MB in 100 iterations."
[TRUNCATED FOR BREVITY]
^ "Using 28MB in 200 iterations."
[TRUNCATED FOR BREVITY]
^ "Using 38MB in 500 iterations."
[TRUNCATED FOR BREVITY]
^ "Using 48MB in 800 iterations."
[TRUNCATED FOR BREVITY]
^ "Using 54MB in 997 iterations."
^ "Using 54MB in 998 iterations."
^ "Using 54MB in 999 iterations."

The more iterations, the more memory used.

[rodrigopedra]

Adding this line:

\Illuminate\Console\Application::forgetBootstrappers();

Right after $this->createApplication()->flush(); reduced the memory increase dramatically (from 54MB to 28MB)

And commenting the last 2 bootstrapers in:

framework/src/Illuminate/Foundation/Console/Kernel.php

Lines 82 to 90 in f40fe7d

	protected $bootstrappers = [
	\Illuminate\Foundation\Bootstrap\LoadEnvironmentVariables::class,
	\Illuminate\Foundation\Bootstrap\LoadConfiguration::class,
	\Illuminate\Foundation\Bootstrap\HandleExceptions::class,
	\Illuminate\Foundation\Bootstrap\RegisterFacades::class,
	\Illuminate\Foundation\Bootstrap\SetRequestForConsole::class,
	\Illuminate\Foundation\Bootstrap\RegisterProviders::class,
	\Illuminate\Foundation\Bootstrap\BootProviders::class,
	];

Further reduced from 28MB to 18MB. (in my machine it starts with 12MB)

This last one is less objective (could be any loaded service provider), and impacts the application. But at least it gives a hint on where to look further.

[PHLAK]

I've found a slightly better way to illustrate that there is no code outside of Laravel causing the memory usage to increase.

Add the following test class as tests/Feature/ExampleTest.php to a brand new Laravel installation.

class ExampleTest extends TestCase
{
    protected function tearDown(): void
    {
        parent::tearDown();

        dump(sprintf('%d MB', (int) (memory_get_usage(true) / (1024 * 1024))));
    }

    public function test_true()
    {
        $this->assertTrue(true);
    }
}

Run the test repeatedly using the --repeat flag and watch the memory usage grow.

$ phpunit --repeat 1000 tests/Feature/ExampleTest.php
^ "20 MB"
.^ "20 MB"
.^ "20 MB"
[TRUNCATED]
.^ "22 MB"
[TRUNCATED]
.^ "24 MB"
[TRUNCATED]
.^ "30 MB"

[DarkGhostHunter]

Lol that's my article.

I assume these test are done in vanilla Laravel. Did anyone tried the Opcache trick?

BTW, seems that RegisterProviders deals also with compiling providers, retrieving a manifest, and requiring them manually. I could bet is the getRequire() line in Illuminate\Foundation\ProviderRepository::loadManifest().

[driesvints]

Seems this is indeed still a culprit even after a fix in PHP core was merged here: php/php-src#9265. Also see #30736

@olsavmic do you might have knowledge of any other memory leaks in PHP core?

BTW, seems that RegisterProviders deals also with compiling providers, retrieving a manifest, and requiring them manually. I could bet is the getRequire() line in Illuminate\Foundation\ProviderRepository::loadManifest().

@DarkGhostHunter I disabled that call and the memory leak still happens.

Also, I think for this to be investigated we shouldn't focus on Opcache? Not sure what that has to do with anything.

[PHLAK]

I'd like to continue digging into this but not sure how to proceed (aside from a lot of code commenting). Is there an easy way to profile the code while running tests?

[DarkGhostHunter]

@driesvints I know but is just a workaround while we pinpoint the culprit.

[driesvints]

@PHLAK not that I know. Just use time <cmd> maybe?

[KristianI]

I'd like to continue digging into this but not sure how to proceed (aside from a lot of code commenting). Is there an easy way to profile the code while running tests?

Maybe xdebug profiler could be helpful in this connection.

[PHLAK]

I've profiled my application code while running tests. This generated several cachegrind.out.*.gz files that I am able to extract and open in qcachegrind. I read this Stack Overflow post to familiarize myself with the tool but there's still so much data to comb through so any recommendations as to what I should be looking for would be helpful.

[driesvints]

Just pinging you @brendt as well (since you originally opened the other issue) to see if you had any thoughts here?

[nunomaduro]

Going to take a look at this next Monday. ✅

[taylorotwell]

Has anyone tried to recreate the issue on initial release of Laravel 8.0? or Laravel 7.0? So we can see possibly when the memory leak was introduced if it is indeed something new. Or, if it something that has been around for some time.

[PHLAK]

I ran the example test against Laravel v8.83.24 with similar results.

I'm getting errors when trying to install Laravel 7 (i.e. composer create-project laravel/laravel=7.* laratest) so couldn't evaluate it there. Can dig into it more later.