Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix/perf: don't call language detection command in enabled() #285

Merged
merged 1 commit into from
Dec 31, 2020

Conversation

lnu
Copy link
Contributor

@lnu lnu commented Dec 29, 2020

Prerequisites

  • I have read and understand the CONTRIBUTING guide
  • The commit message follows the conventional commits guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

Description

Currently, language version detection is done in the enabled() function, which is called in any directory. This call can be avoided by moving the call to get the version inside the string() method. All tests are green and only two had to be modified but not in a way that changes the current behaviour.

@lnu lnu changed the title fix/segment_language fix/perf: don't call language detection command in enabled() Dec 29, 2020
@lnu lnu marked this pull request as draft December 29, 2020 17:04
@lnu
Copy link
Contributor Author

lnu commented Dec 29, 2020

I'm also aligning the dotnet segment like other languages segment

@JanDeDobbeleer
Copy link
Owner

@lnu I suppose you're still working on this, right?

src/segment_language_test.go Outdated Show resolved Hide resolved
@lnu
Copy link
Contributor Author

lnu commented Dec 29, 2020

@lnu I suppose you're still working on this, right?

yes, I put it in draft. It's almost done but there's one thing about the dotnet segment with exit code 145. I'm not able to test it, the tests are ok but I'd like to test it a once manually.

@lnu
Copy link
Contributor Author

lnu commented Dec 29, 2020

I think I found something related to the perf issues when calling executables on windows.
Before git segment was taking 350ms, now it takes 228ms. Not blazing fast but 35% is still nice.
It also applies to other segments calling windows executables.
We always call hasCommand, then call runCommand. Since the command are not fully qualified, a call to LookPath is done, checking all paths to check if the executable exists. By caching the executable full path on the first hasCommand call, we can avoid calling this method multiple times.

@lnu lnu force-pushed the fix/segment_language branch from e200eb5 to e9e7ffd Compare December 29, 2020 20:52
@JanDeDobbeleer
Copy link
Owner

JanDeDobbeleer commented Dec 29, 2020

I think I found something related to the perf issues when calling executables on windows.

Before git segment was taking 350ms, now it takes 228ms. Not blazing fast but 35% is still nice.

It also applies to other segments calling windows executables.

We always call hasCommand, then call runCommand. Since the command are not fully qualified, a call to LookPath is done, checking all paths to check if the executable exists. By caching the executable full path on the first hasCommand call, we can avoid calling this method multiple times.

Awesome. That's something I didn't check indeed. I have a working rust lib which is faster on Windows when it comes to executing commands. So that combination could be even faster.

@JanDeDobbeleer
Copy link
Owner

@lnu let me know when this is ready to go.

@lnu lnu force-pushed the fix/segment_language branch from 7a72fcb to 006f6c5 Compare December 30, 2020 07:28
@lnu
Copy link
Contributor Author

lnu commented Dec 30, 2020

@lnu let me know when this is ready to go.

For me it looks fine. But it needs some testing about the languages segment(especially the dotnet one) and a second pair of eyes would be useful.
For the dotnet segment,I have no idea how to reproduce the exit code 145 scenario. Has someone seen it working before? it seems this code is returned when you do a dotnet restore but I saw nothing related to that code and dotnet --version.

The only difference with the previous behaviour is that hasCommand is called instead of getVersion to check if the segment should be enabled or not.

@lnu lnu force-pushed the fix/segment_language branch from 006f6c5 to 9176a9a Compare December 30, 2020 07:54
@lnu lnu marked this pull request as ready for review December 30, 2020 08:50
@lnu lnu force-pushed the fix/segment_language branch 3 times, most recently from 4b51add to 9f7c1f9 Compare December 30, 2020 09:28
@lnu
Copy link
Contributor Author

lnu commented Dec 30, 2020

found info here: golang/go#38736

@JanDeDobbeleer
Copy link
Owner

I have no idea how to reproduce the exit code 145 scenario. Has someone seen it working before?

The original author did. So we'd have to check with @tillig :-)

@lnu
Copy link
Contributor Author

lnu commented Dec 30, 2020

I have no idea how to reproduce the exit code 145 scenario. Has someone seen it working before?

The original author did. So we'd have to check with @tillig :-)

I tried to test it by using a global.json config file with an sdk not installed on my machine:

{
    "sdk": {
      "version": "3.1.100",
      "rollForward": "disable"
    }
}

Here are the results:
Main
image
Main with runCommand fix
image
fix/segment_language(current branch)
image

You can see there's already an issue with the current main branch. A change has been done when trying to optimize runCommand but we loste the exitCode in the meantime. I put the old code back and it looks ok now with acceptable performance.

I also changed to the dotnet segment to return the unsupported whenever an exit code != 0 is returned by the command. It seems ok to me, maybe @tillig can confirm?

@lnu lnu force-pushed the fix/segment_language branch from 765af8e to 4c157be Compare December 30, 2020 13:02
@tillig
Copy link
Contributor

tillig commented Dec 30, 2020

145 is actually special for dotnet. There are other exit codes you can get, like if you call the command incorrectly or if something else is messed up in the environment. I'd recommend keeping the dotnet segment checking specifically for 145 to indicate "unsupported version" and considering any other non-zero code to be a more severe unrecoverable error and return "" instead.

@lnu lnu force-pushed the fix/segment_language branch from 4c157be to ece6694 Compare December 30, 2020 21:07
@lnu
Copy link
Contributor Author

lnu commented Dec 30, 2020

145 is actually special for dotnet. There are other exit codes you can get, like if you call the command incorrectly or if something else is messed up in the environment. I'd recommend keeping the dotnet segment checking specifically for 145 to indicate "unsupported version" and considering any other non-zero code to be a more severe unrecoverable error and return "" instead.

Thank you @tillig. Could you tell me how I could test this? As said above, I put a global.json with a not existing sdk and it returns this:
image

I have these sdks installed:
image

And this global.json:

{
    "sdk": {
      "version": "3.1.100",
      "rollForward": "disable"
    }
}

@lnu lnu force-pushed the fix/segment_language branch from ece6694 to c9944dc Compare December 30, 2020 21:37
@JanDeDobbeleer
Copy link
Owner

@lnu you might want to rebase too, while testing the new experimental Rust lib, I found a bug in the current implementation of runCommand where the error did not contain the actual error output.

@lnu
Copy link
Contributor Author

lnu commented Dec 30, 2020

@lnu you might want to rebase too, while testing the new experimental Rust lib, I found a bug in the current implementation of runCommand where the error did not contain the actual error output.

Yep. I saw the change but I think we still don't get the right exit code from the command(error output is ok though).
The actual code(in main) of the dotnet segment does this:

// Exit code 145 is a special indicator that dotnet
// ran, but the current project config settings specify
// use of an SDK that isn't installed.
var exerr *commandError
if errors.As(err, &exerr) && exerr.exitCode == 145 {
    d.unsupportedVersion = true
    return true
}

But runCommand does not return the exit code but only the error output. I wonder if the original code(withoutput) was slower than the current change?

@tillig
Copy link
Contributor

tillig commented Dec 30, 2020

@lnu The bad global.json is how you test it. That should yield both the info you saw and a 145 exit code.

@JanDeDobbeleer
Copy link
Owner

JanDeDobbeleer commented Dec 31, 2020

But runCommand does not return the exit code but only the error output

True. I'll align it with command error.

JanDeDobbeleer added a commit that referenced this pull request Dec 31, 2020
JanDeDobbeleer added a commit that referenced this pull request Dec 31, 2020
@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

@lnu The bad global.json is how you test it. That should yield both the info you saw and a 145 exit code.

Ok then it seems something has changed, it returns this:
image
image

@tillig
Copy link
Contributor

tillig commented Dec 31, 2020

Unfortunately, I'm on vacation without any real access to a dev environment for another week. Right now I'm on my phone. This isn't something I'll be able to check or help troubleshoot before then.

@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

But runCommand does not return the exit code but only the error output

True. I'll align it with command error.

Now the exit code is there but processtate ==nil and it always return -1 and not the real exit code of the external command.

image
image

@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

Unfortunately, I'm on vacation without any real access to a dev environment for another week. Right now I'm on my phone. This isn't something I'll be able to check or help troubleshoot before then.

You've been already very helpful. Since I kept the exitCode check and all the tests are green, everything should be ok.
Happy new year and all the best!

@JanDeDobbeleer
Copy link
Owner

not the real exit code of the external command

I'll check what does contain the value, I did this rather swift in between taking care of the kids. I can have a look somewhere around lunch.

@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

not the real exit code of the external command

I'll check what does contain the value, I did this rather swift in between taking care of the kids. I can have a look somewhere around lunch.

I think it's not so urgent :), it can wait until 2021.

Happy new year and all the best for you and your family.

@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

Ok, I tried in ubuntu and yes the exit code is correct(you can see the issued with exit code on the first screenshot):
image
with the fixed zsh script:
image

So I confirm it works(with the old runCommand version calling output) well in Ubuntu, same as before. We only have to fix the runCommand and the dotnet segment will be ok.
We're getting closer 👍
image

@lnu lnu force-pushed the fix/segment_language branch from c9944dc to 0551900 Compare December 31, 2020 11:01
JanDeDobbeleer added a commit that referenced this pull request Dec 31, 2020
without wait(), the *ProcessState is nil, meaning we can't access
the ExitCode(). On Windows, calling wait() introduces a timeout
which makes things run slower, which is why we only call wait()
in case of an error. That should not be the main use-case.

relates to #285
@JanDeDobbeleer
Copy link
Owner

@lnu found a fix, validated it too. This should hopefully make this ready to merge. Would be cool if we can do this before OEY 😓

JanDeDobbeleer added a commit that referenced this pull request Dec 31, 2020
without wait(), the *ProcessState is nil, meaning we can't access
the ExitCode(). On Windows, calling wait() introduces a timeout
which makes things run slower, which is why we only call wait()
in case of an error. That should not be the main use-case.

relates to #285
@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

@lnu found a fix, validated it too. This should hopefully make this ready to merge. Would be cool if we can do this before OEY 😓

Cool! Sounds like a deadline? 😅

@lnu lnu force-pushed the fix/segment_language branch from 0551900 to 3d2e61b Compare December 31, 2020 13:37
@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

@lnu found a fix, validated it too. This should hopefully make this ready to merge. Would be cool if we can do this before OEY 😓

Ok so I rebased everything and tested. The exit code is ok and the perf are better. Still, the dotnet segment works correctly under ubuntu but not in windows. But it was already the case before, so we still have the same behaviour.

@lnu lnu force-pushed the fix/segment_language branch from 3d2e61b to ef579e6 Compare December 31, 2020 13:48
@JanDeDobbeleer
Copy link
Owner

Ill do some testing on this layer today too, that way we can merge with confidence.

fix: align dotnet segment with other languages
feat: missing command text + json schema updated
chore: doc updated
perf: cache executable path
chore: not supported version icon updated(previus one was unreadable)
@lnu lnu force-pushed the fix/segment_language branch from ef579e6 to edf1e8d Compare December 31, 2020 14:15
@JanDeDobbeleer
Copy link
Owner

@lnu This works fine, I would just remove DisplayModeNever as that's a bit pointless. It's the same as not adding the segment. But I'll remove that rather than ask you to once again rebase :-)

@lnu
Copy link
Contributor Author

lnu commented Dec 31, 2020

Ill do some testing on this layer today too, that way we can merge with confidence.

@lnu This works fine, I would just remove DisplayModeNever as that's a bit pointless. It's the same as not adding the segment. But I'll remove that rather than ask you to once again rebase :-)

True, no idea what's the point of never displaying the segment. It's also harmless to let it now and clean it later.

@JanDeDobbeleer JanDeDobbeleer merged commit abfbb27 into JanDeDobbeleer:main Dec 31, 2020
@lnu lnu deleted the fix/segment_language branch January 1, 2021 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants