Language server leaks/eats 30+GB of RAM

Question

Language server leaks/eats 30+GB of RAM

Ark-kun opened this issue 7 years ago · 25 comments

I'm not installing/updating any new packages. It just leaks lots of RAM over the time.

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                                                                                                             
139520 avolkov   20   0   30.8g  24.1g   1452 S   1.0  38.4 254:31.27 /home/avolkov/.vscode/extensions/ms-python.python-2018.10.1/languageServer.0.1.60/Microsoft.Python.LanguageServer                                                                                  
203203 avolkov   20   0   17.6g   9.4g   1540 S   1.0  14.9  99:36.68 /avolkov/.vscode/extensions/ms-python.python-2018.10.1/languageServer.0.1.60/Microsoft.Python.LanguageServer

MikhailArkhipov commented 7 years ago

#546

Answer 1 · 2018-12-03T20:49:00.000Z

0.1.60.0 is an older build. The current stable release is 0.1.65.0 and should be getting downloaded by the extension. (Your extension is also last month's build, but the language server download should be independent.) I don't think we had any fixes specifically for memory usage between those releases, but you may want to try the newer build.

It would be helpful to know what project you're working on and if it's public such that we could test it. Your example in particular is interesting because it's high memory usage without the CPU usage being high.

Answer 2 · 2018-12-04T19:00:55.000Z

I had this at one point too today with 0.1.65.0.

We use pipenv with the PIPENV_VENV_IN_PROJECT=1 which means that we have a .venv folder in our workspace (though also in .gitignore). No idea if that might be the cause.

I've also noticed that ctrl+click on relative package references sometimes seems to spin for ever so perhaps that is causing an infinite loop that increases memory or something.

Unfortunately I can't share our code and I haven't got time at the moment to recreate it

Answer 3 · 2018-12-10T16:12:47.000Z

I can confirm the memory leak too. After few hours the memory usage is huge (something about 5GB), after reload it comes back to the normal usage. But using vscode actively (but I sometimes noticed that the memory usage incresased without doing anything) slowly increases the memory usage.

Don't know if this information will be helpful but I've noticed that in the output log was printed:

Reloading modules ....
done.
done.
done.

I was wondering why the LS reloads module so many times. The upper lines were interleaved with the linting output.

I am using venv, mypy.

If I notice anything else, I will edit this comment or add a new one.

Answer 4 · 2018-12-15T17:41:06.000Z

It seems to me that this issue was fixed with new version of python language server (actual version 0.1.72.0, previous version 0.1.65.0).
Running vs code 2 days straight and the memory usage didn't exceed 600MB of RAM.

It still needs further testing, but I just wanted to give this feedback, as I was happy seeing this issue (probably) gone in my case! Thank you very much!

Answer 5 · 2018-12-16T19:22:30.000Z

While I do not know how to reproduce it I just peaked at 12GB again.
I have the 0.1.72 Version of the language server and python extension "2018.12.1"

The "Process Explorer" in VS Code only showed a memory usage of ~ 3.8GB for the language server, as compared to Windows 10 Task Manager that showed ~12GB

Answer 6 · 2018-12-23T16:34:14.000Z

Unfortunately the leaks are still here. But still dont know how to reproduce it.

Answer 7 · 2018-12-31T09:09:35.000Z

I am using the latest version of languageServer 0.1.73. This python language server eats all my ram. For that, I can`t use vs code. Please tell me how to fix it

Answer 8 · 2019-01-08T03:09:46.000Z

same... .vscode/extensions/ms-python.python 2018.12.1/languageServer.0.1.72/Microsoft.Python.LanguageServer

eat too much RAM, i just open two simple python file..

Answer 9 · 2019-01-11T12:40:13.000Z

Got the same thing, 25+ GB after a couple of days, not even using VS Code, just leaving it open.

Answer 10 · 2019-01-16T18:05:12.000Z

Strange thing is, that the python language server causes those memory leaks even if I haven't opened any .py file. It took 75% memory of my 16GB total memory.

Answer 11 · 2019-01-16T18:15:26.000Z

When the language server starts, it begins analyzing every file within the workspace, which means that there won't be any difference between opening a file and not opening a file.

Without an example codebase, this is very hard to pin down. If you're also seeing CPU usage during that time, then it may be the case that the analysis isn't stopping, continuing to use more memory as it grows type information (which is #495 and #479).

Know that like #495 and #479, the memory usage issues should be handled with our new analysis approach. See Mikhail's comment here for more info: #479 (comment)

The analysis approach as a whole will be different, and the files you open will impact what gets analyzed (versus analyzing unopened files that may not be referenced).

Answer 12 · 2019-01-22T17:00:52.000Z

Uhhh... Explains why everything was slower then normal I guess... Best part is that after a certain point intellisense stops working in python altogether.

Microsoft Python Language Server version 0.1.75.0
Python Extension 2018.12.1
VSCode 1.30.2
Mac OSX High Sierra 10.13.6

Answer 13 · 2019-02-12T13:04:56.000Z

I still observe regular memory leaks with current development build (2019.2.5135-alpha, Microsoft Python Language Server version 0.1.78.0). Is it supposed to be fixed in this version, @MikhailArkhipov ?

Answer 14 · 2019-02-12T16:13:35.000Z

No, it is not. That's the release branch. Our master contains an updated analyzer/language server but is not yet released for any users. When we make it generally available, we'll probably be bumping the version to v0.2.x.

Answer 15 · 2019-02-12T16:35:12.000Z

Thanks a lot for your prompt answer and all your effort! Will be waiting for the 0.2.x release.

Answer 16 · 2019-02-20T11:57:46.000Z

I have the same issue, leaking many GBs of memory per hour that VS Code is open. Looking forward to testing the 0.2.x release!

Answer 17 · 2019-03-03T14:38:13.000Z

I have the same issue cause I need restart vs frequently.
Waiting for 0.2.x to resolve it.

Answer 18 · 2019-03-06T22:48:05.000Z

0.2.16 is now available on beta channel. You can set this in your VS Code settings.json to get it:

"python.analysis.downloadChannel": "beta"

Answer 19 · 2019-03-07T12:19:46.000Z

Language Server version < 0.2.16 still has the memory leak for me. Installing version 0.2.16 following @jakebailey comment throws an error:

[Info  - 1:05:59 PM] Microsoft Python Language Server version 0.2.16.0
[Info  - 1:05:59 PM] Initializing for /home/user/dev/virtualenv/pytorch/bin/python
[Error - 1:11:34 PM] Request textDocument/completion failed.
  Message: Specified argument was out of the range of valid values.
Parameter name: index
  Code: -32000 
   at System.Char.IsLetter(String s, Int32 index)
   at Microsoft.Python.LanguageServer.Completion.CompletionItemSource.CreateCompletionItem(String text, CompletionItemKind kind)
   at System.Linq.Enumerable.SelectEnumerableIterator`2.ToArray()
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Microsoft.Python.LanguageServer.Completion.ImportCompletion.TryGetCompletions(ImportStatement import, CompletionContext context)
   at Microsoft.Python.LanguageServer.Completion.CompletionSource.GetCompletions(IDocumentAnalysis analysis, SourceLocation location)
   at Microsoft.Python.LanguageServer.Implementation.Server.Completion(CompletionParams params, CancellationToken cancellationToken)
   at Microsoft.Python.LanguageServer.Implementation.LanguageServer.Completion(JToken token, CancellationToken cancellationToken)

Answer 20 · 2019-03-08T08:51:08.000Z

0.2.16 is now available on beta channel. You can set this in your VS Code settings.json to get it:
"python.analysis.downloadChannel": "beta"

Thanks a lot for letting us know.

Answer 21 · 2019-03-26T14:46:16.000Z

The beta version solved the problem.
Thanks

Answer 22 · 2019-08-12T06:39:23.000Z

Today I faced this issue! it took more than 10GB+ of my system RAM. ~~I cant reproduce it now~~ (see below), it seems when this happens, the intellisense also stops working. and analysis goes on forever!
I'm using the latest version

Python Language Server:
Microsoft Python Language Server version 0.3.46.0
Python Extension :
2019.8.29288 (6 August 2019)
VSCode info

Version: 1.37.0 (user setup)
Commit: 036a6b1d3ac84e5ca96a17a44e63a87971f8fcc8
Date: 2019-08-08T02:33:50.993Z
Electron: 4.2.7
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Windows_NT x64 10.0.17763

UPDATE:

OK this happened again and I could save the log and record it before the vscode crashed!
Here is the console log :

Answer 23 · 2019-08-12T16:12:54.000Z

@Coderx7 Please open a new issue; this one is old and unrelated. Providing the code that causes the issue (what you have in that file) would be helpful.

Those logs are for VS Code, not the language server, and can't help us. Check this section of our troubleshooting guide for how to enable more logging for the language server and the info we'd be interested in: https://github.com/microsoft/python-language-server/blob/master/TROUBLESHOOTING.md#filing-an-issue

Answer 24 · 2019-08-12T19:33:42.000Z

@jakebailey : Thanks for the heads up. I created a new issue here and provided the logs you suggested in addition to the test script I used to replicate this.