Analyzer reported to be very slow in 1.13

[mit-mit]

Several users are reporting very poor performance in the Analyzer in the 1.13 release candidate builds:
https://groups.google.com/a/dartlang.org/forum/#!topic/misc/pQCl_cGA_q0

[mit-mit]

@scheglov is this related to #24812 ?

[alexander-doroshko]

Might be also related to #24611 as people in that email thread are saying that they have a lot of pubspec-based Dart packages configured as a single IntelliJ IDEA/WebStorm project.

[bwilkerson]

As stated, this issue is not actionable. Are there problems beyond those reported in other issues? Is this intended to be a tracking issue?

[mit-mit]

This came in through the misc group, if you need more info, just reply to the thread there.

[clayberg]

See the thread https://groups.google.com/a/dartlang.org/forum/?fromgroups=#!topic/misc/pQCl_cGA_q0

[clayberg]

Are the users using .package files?

[clayberg]

We need to see their logs right away. While I doubt we are seeing a regression in actual Analyzer performance, we have seen a couple of scenarios that can result in a severe performance problem in IntelliJ using the Analyzer. For example, are the users using .package files? We had reports from Alex last month about issues running Pub too many times when the projects did not contain .package files (or had a huge number of pubspecs in a single project). We've also seen severe slowness in situations where Analyzer fails and restarts repeatedly (which the logs might shed some light on).

[zoechi]

I just run pub upgrade. This currently creates the .packages file and the packages directory with the symlinks. I don't know if the analyzer uses the file or the directory if both exist.

[bwilkerson]

It uses the file if it exists and the directory only if the file doesn't exist. The plan is for pub to stop producing the directory.

[dgrove]

Any updates on this issue?

[bwilkerson]

We have not been able to reproduce the same level of performance problems that our users have seen, but we did find one problem that might account for part of the slowness being reported. It would effect performance for anyone that has a large numbers of HTML files in the doc directory (such as are produced by running dartdoc).

[dgrove]

@zoechi and anyone else who's on this thread - can you please try out our new build of the SDK, which has a number of fixes which may help?

It's available at https://storage.googleapis.com/dart-archive/channels/dev/raw/1.13.0-dev.7.11/sdk/dartsdk-{macos,linux,windows}-x64-release.zip

Thanks for any info that you can provide after trying it!

[zoechi]

I assume these fixes are in bleeding_edge as well

[scheglov]

Correct.
We first make changes in bleeding_edge and then cherry-pick some of them into dev.

[zoechi]

Hard to say, I don't know myself how to reliably reproduce the issue.
Today I have trouble getting anything useful from the analyzer at all.
For example #24910

Currently I'm working on a small console application (with a few dependencies like shelf, shelf_static, args, unscripted, grinder, test) I started today.
Even this project consisting of 3 small Dart files seems to completely overtax dartanalyzer. I have not yet tried a project with a lot of packages.
When I edit it it gets out of sync and not even syntax highlighting works (different colors in single words), "reanalyzer sources" sometimes fixes syntax highlighting but only until I start editing again.

Dart VM version: 1.13.0-edge.50aa43569d892b3e395a812cdb2a42e66069c1be (Fri Nov 13 00:34:34 2015) on "linux_x64

[dgrove]

We're working hard on this - in the meantime, if you are having issues with the analyzer, here's a document to help you help us by gathering as much info as possible:

https://docs.google.com/document/d/1s4PTof1x7RyxXYX1yO31ByvZsQoDMM7edFIW1SKP25U/edit?pli=1

[bwilkerson]

As Dan mentioned, we’re working hard on resolving these issues. One of the things we’ve discovered is that there can be a significant performance improvement if packages folders are removed (these folders contain a lot of symlinks which can sometimes cause performance problems). Before you remove them, though, we’d like to gather some information to get a better understanding of how big this issue is. We would appreciate it if you could send us the results of running these two commands (from the root of your repository):

ls -lR | wc -l
ls -LlR | wc -l

The first counts the number of files reachable from your root directory without following symlinks, the second counts the number of files reachable from your root directory by following symlinks.

Then, you can remove all of the packages folders by running the following script:

find . -name pubspec.yaml -execdir pwd \; -execdir pub get --no-package-symlinks \;

Please let us know whether running the second script improves server’s performance for you.

[bwilkerson]

The 7.11 build that we posted a link to last Thursday had some problems that are fixed by the 7.12 build (http://gsdview.appspot.com/dart-archive/channels/dev/raw/1.13.0-dev.7.12/sdk/). We would be grateful to hear whether this fixes the problems you've been running into.

[mit-mit]

Closing this bug as there is no new feedback. If new issues are found, please open new issues.