Skip to content
This repository has been archived by the owner on Apr 14, 2022. It is now read-only.

Analysis of peewee with the new LS never finishes #193

Closed
MikhailArkhipov opened this issue Oct 4, 2018 · 10 comments
Closed

Analysis of peewee with the new LS never finishes #193

MikhailArkhipov opened this issue Oct 4, 2018 · 10 comments
Assignees
@MikhailArkhipov
Labels
bug Something isn't working feature: symbols
Milestone
Oct 2018.1

Comments

@MikhailArkhipov
Copy link

MikhailArkhipov commented Oct 4, 2018 •
edited
Loading

microsoft/vscode-python#2682

"Check out peewee. When the new LS tries to analyse the main file, peewee.py, it uses >100% of CPU and the RAM usage increases continuously (over 40GB at some point). This doesn't seem to complete, even after leaving it open and running for an hour. Things like the Go to symbol list never get rendered."
@MikhailArkhipov MikhailArkhipov added the bug Something isn't working label Oct 4, 2018
@MikhailArkhipov MikhailArkhipov self-assigned this Oct 4, 2018
@MikhailArkhipov
Copy link
Author

Looks like #173

Assertion Failed
Hash code changed when merging AnalysisValues

   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddOne(BucketSet& buckets, AnalysisValue key, Int32 hc, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 425
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddOne(BucketSet& buckets, AnalysisValue key, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 329
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddFromEnumerator(BucketSet& buckets, IEnumerator`1 items, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 240
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet.AddFromEnumerator(IEnumerator`1 items) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 232
   at Microsoft.PythonTools.Analysis.AnalysisSetDetails.AnalysisHashSet..ctor(IEnumerable`1 enumerable, IEqualityComparer`1 comparer) in F:\PLS\src\Analysis\Engine\Impl\AnalysisHashSet.cs:line 85
   at Microsoft.PythonTools.Analysis.AnalysisSet.AsUnion(IAnalysisSet set, UnionComparer comparer, Boolean& wasChanged) in F:\PLS\src\Analysis\Engine\Impl\AnalysisSet.cs:line 288
   at Microsoft.PythonTools.Analysis.AnalysisSet.AsUnion(IAnalysisSet set, Int32 strength, Boolean& wasChanged) in F:\PLS\src\Analysis\Engine\Impl\AnalysisSet.cs:line 247
   at Microsoft.PythonTools.Analysis.Analyzer.TypedDependencyInfo.MakeUnion(Int32 strength) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DependencyInfo.cs:line 130
   at Microsoft.PythonTools.Analysis.TypedDef`1.MakeUnion(Int32 strength) in F:\PLS\src\Analysis\Engine\Impl\VariableDef.cs:line 415
   at Microsoft.PythonTools.Analysis.TypedDef`1.MakeUnionStronger() in F:\PLS\src\Analysis\Engine\Impl\VariableDef.cs:line 406
   at Microsoft.PythonTools.Analysis.TypedDef`1.MakeUnionStrongerIfMoreThan(Int32 typeCount, IAnalysisSet extraTypes) in F:\PLS\src\Analysis\Engine\Impl\VariableDef.cs:line 398
   at Microsoft.PythonTools.Analysis.Analyzer.FunctionScope.UpdateParameters(FunctionAnalysisUnit unit, ArgumentSet others, Boolean enqueue, FunctionScope scopeWithDefaultParameters, Boolean usePlaceholders) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\FunctionScope.cs:line 198
   at Microsoft.PythonTools.Analysis.Analyzer.FunctionAnalysisUnit.UpdateParameters(ArgumentSet callArgs, Boolean enqueue) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\FunctionAnalysisUnit.cs:line 64
   at Microsoft.PythonTools.Analysis.Values.FunctionInfo.DoCall(Node node, AnalysisUnit callingUnit, FunctionAnalysisUnit calledUnit, ArgumentSet callArgs) in F:\PLS\src\Analysis\Engine\Impl\Values\FunctionInfo.cs:line 140
   at Microsoft.PythonTools.Analysis.Values.FunctionInfo.Call(Node node, AnalysisUnit unit, IAnalysisSet[] args, NameExpression[] keywordArgNames) in F:\PLS\src\Analysis\Engine\Impl\Values\FunctionInfo.cs:line 96
   at Microsoft.PythonTools.Analysis.Values.ClassInfo.AddCall(Node node, NameExpression[] keywordArgNames, AnalysisUnit unit, IAnalysisSet[] args) in F:\PLS\src\Analysis\Engine\Impl\Values\ClassInfo.cs:line 66
   at Microsoft.PythonTools.Analysis.Values.ClassInfo.Call(Node node, AnalysisUnit unit, IAnalysisSet[] args, NameExpression[] keywordArgNames) in F:\PLS\src\Analysis\Engine\Impl\Values\ClassInfo.cs:line 56
   at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.EvaluateCall(ExpressionEvaluator ee, Node node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\ExpressionEvaluator.cs:line 382
   at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.EvaluateWorker(Node node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\ExpressionEvaluator.cs:line 214
   at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.Evaluate(Expression node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\ExpressionEvaluator.cs:line 54
   at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(ReturnStatement node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DDG.cs:line 578
   at Microsoft.PythonTools.Parsing.Ast.ReturnStatement.Walk(PythonWalker walker) in F:\PLS\src\Analysis\Engine\Impl\Parsing\Ast\ReturnStatement.cs:line 32
   at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(SuiteStatement node) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DDG.cs:line 662
   at Microsoft.PythonTools.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker) in F:\PLS\src\Analysis\Engine\Impl\Parsing\Ast\SuiteStatement.cs:line 34
   at Microsoft.PythonTools.Analysis.Analyzer.FunctionAnalysisUnit.AnalyzeWorker(DDG ddg, CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\FunctionAnalysisUnit.cs:line 89
   at Microsoft.PythonTools.Analysis.AnalysisUnit.Analyze(DDG ddg, CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\AnalysisUnit.cs:line 139
   at Microsoft.PythonTools.Analysis.Analyzer.DDG.Analyze(Deque`1 queue, CancellationToken cancel, Action`1 reportQueueSize, Int32 reportQueueInterval) in F:\PLS\src\Analysis\Engine\Impl\Analyzer\DDG.cs:line 75
   at Microsoft.PythonTools.Analysis.PythonAnalyzer.AnalyzeQueuedEntries(CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\PythonAnalyzer.cs:line 925
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.GroupAnalysis.Analyze(CancellationToken cancel) in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 224
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.HandleAnalyzable(IAnalyzable item, AnalysisPriority priority, CancellationToken cancellationToken) in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 148
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.HandleAnalyzable(IAnalyzable item, AnalysisPriority priority, CancellationToken cancellationToken)
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.<>c__DisplayClass28_0.<Enqueue>b__0(CancellationToken ct) in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 132
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.ConsumerLoop() in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 60
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
   at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult result)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetExistingTaskResult(TResult result)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult result)
   at Microsoft.PythonTools.Intellisense.AnalysisQueue.ConsumeAsync() in F:\PLS\src\Analysis\Engine\Impl\Intellisense\AnalysisQueue.cs:line 85
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

@MikhailArkhipov MikhailArkhipov mentioned this issue Oct 4, 2018
Closed
@markw-t markw-t mentioned this issue Oct 5, 2018
Closed
@MikhailArkhipov MikhailArkhipov mentioned this issue Oct 5, 2018
Closed
@MikhailArkhipov MikhailArkhipov added this to the Oct 2018.1 milestone Oct 5, 2018
@MikhailArkhipov
Copy link
Author

#201

This was referenced Oct 5, 2018
Closed
Closed
@stessaris
Copy link

I still have the RAM problem on OSX with Version 2018.9.0

@MikhailArkhipov
Copy link
Author

MikhailArkhipov commented Oct 17, 2018 •
edited
Loading

@stessaris - how big is the RAM consumption? Runaway or large but stable? What is the project size? Is it better with Jedi?

@stessaris
Copy link

@MikhailArkhipov - over 5 GB (enough to bring my 8GB iMac to its knees with the rest of applications running), I got the feeling that is not growing a lot after that but I'll do some tests. The project is 20 files for a total of 6500 lines of code. No issues with Jedi

@MikhailArkhipov
Copy link
Author

@stessaris - This bug is specifically about peewee, could you open another specific to your project please.

@GinRyan
Copy link

GinRyan commented Nov 10, 2018 •
edited
Loading

It's still happening in language-server.0.1.60 (Platform: on Windows 10 1803, VSCode 1.28.2, Python 3.6.4)

default

There are just 5 python scripts in my project.

After I killed this process, it launched again and auto increment to 4GB even 10GB .Auto increment step by 10MB per sec.

I just import numpy and matplotlib in this project.

@jakebailey
Copy link
Member

@GinRyan As previously mentioned, this issue is specifically about peewee. If this is occurring for another project, I'd appreciate it if you opened a new issue (especially if you can share the project that causes the high memory usage so we can profile it).

@MikhailArkhipov
Copy link
Author

0.2.16 is now available on beta channel.

"python.analysis.downloadChannel": "beta"

@maxn0d3x
Copy link

@MikhailArkhipov
What is this ? ""python.analysis.downloadChannel": "beta""
Why we should put this odds line in setting.json without explanation ?
My Mac is always +300% whenever turning of vscode with Python. This is not good !

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@MikhailArkhipov MikhailArkhipov

Labels
bug Something isn't working feature: symbols
Projects
None yet
Milestone
Oct 2018.1
Development

No branches or pull requests
5 participants
@GinRyan @jakebailey @MikhailArkhipov @maxn0d3x @stessaris