Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

upgrade fail under windows #6161

Closed
iconben opened this issue Jul 23, 2017 · 27 comments
Closed

upgrade fail under windows #6161

iconben opened this issue Jul 23, 2017 · 27 comments
Labels
theme: upgrade
Milestone
4.7.0

Comments

@iconben
Copy link
Contributor

iconben commented Jul 23, 2017

Overview of the issue

I tried to upgrade my code to 4.6.2 but encountered error:

$ jhipster upgrade
Executing jhipster:upgrade
Options:
Welcome to the JHipster Upgrade Sub-Generator
This will upgrade your current application codebase to the latest JHipster version
Looking for latest generator-jhipster version...
New generator-jhipster version found: 4.6.2
Git repository detected
Checked out branch "jhipster_upgrade"
Updating generator-jhipster to 4.6.2 . This might take some time...
Updated generator-jhipster to version 4.6.2
Cleaned up directory
Regenerating application with JHipster 4.6.2...
events.js:160
throw er; // Unhandled 'error' event
^

Error: ERROR! Something went wrong while generating project! '$' is not recognized as an internal or external command,
operable program or batch file.

at Environment.error (C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\yeoman-environment\lib\environment.js:140:40)
at constructor.error (C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\generator-base.js:1610:18)
at shelljs.exec (C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\upgrade\index.js:86:23)
at C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\shelljs\src\exec.js:204:9
at ChildProcess.exithandler (child_process.js:211:5)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:886:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
Motivation for or Use Case

To help who encounter same issue under windows when upgrading a jhipster generated project

Reproduce the error

Step 1:
yarn global upgrade generator-jhipster

Step 2:
jhipster upgrade

This error occurs in power shell AND git bash.

Related issues

None

Suggest a Fix
JHipster Version(s)

Upgraded jhipster generator to 4.6.2 then try to upgrade my code base from 4.5.6 to 4.6.2

JHipster configuration

$ jhipster info
Executing jhipster:info
Options:
Welcome to the JHipster Information Sub-Generator

JHipster Version(s)
C:\code\sibodell-hse\hse-backend
`-- [email protected]
JHipster configuration, a .yo-rc.json file generated in the root folder
.yo-rc.json file 
{
  "generator-jhipster": {
    "promptValues": {
      "packageName": "com.bigcompany.hse",
      "nativeLanguage": "zh-cn"
    },
    "jhipsterVersion": "4.5.6",
    "baseName": "hse",
    "packageName": "com.bigcompany.hse",
    "packageFolder": "com/bigcompany/hse",
    "serverPort": "8080",
    "authenticationType": "jwt",
    "hibernateCache": "ehcache",
    "clusteredHttpSession": false,
    "websocket": false,
    "databaseType": "sql",
    "devDatabaseType": "mysql",
    "prodDatabaseType": "mysql",
    "searchEngine": false,
    "messageBroker": false,
    "serviceDiscoveryType": false,
    "buildTool": "maven",
    "enableSocialSignIn": false,
    "jwtSecretKey": "replaced-by-jhipster-info",
    "clientFramework": "angular1",
    "useSass": false,
    "clientPackageManager": "yarn",
    "applicationType": "monolith",
    "testFrameworks": [],
    "jhiPrefix": "jhi",
    "enableTranslation": true,
    "nativeLanguage": "zh-cn",
    "languages": [
      "zh-cn",
      "en"
    ]
  }
}
JDL for the Entity configuration(s) entityName.json files generated in the .jhipster directory

ls: no such file or directory: .jhipster/*.json

JDL entity definitions
Environment and Tools

java version "1.8.0_141"
Java(TM) SE Runtime Environment (build 1.8.0_141-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode)

git version 2.12.2.windows.2

node: v6.10.2

npm: 5.0.4

bower: 1.8.0

gulp:
[12:11:04] CLI version 1.3.0
[12:11:04] Local version 3.9.1

yeoman: 2.0.0

yarn: 0.27.5

Docker version 1.12.5, build 7392c3b

docker-compose version 1.9.0, build 2585387

Execution complete

Entity configuration(s) entityName.json files generated in the .jhipster directory

NONE (We generated some entities then moved the .json files to other directory, this bug should have nothing to do with entities)

Browsers and Operating System

Windows 10 Pro

  • [* ] Checking this box is mandatory (yes, I've read everything)
@deepu105
Copy link
Member

deepu105 commented Jul 23, 2017 via email

@gmarziou
Copy link
Contributor

gmarziou commented Jul 23, 2017 •
edited
Loading

This error occurs in power shell AND git bash.

Could you try with cmd.exe? The simplest shell could help.

Few hints to help you debug this issue in your environment:

The error about $ makes me think this about PATH variable with folders where you can find same script with 2 names: jhipster (bash) and jhipster.cmd (cmd.exe or PS) or maybe another command than jhipster. At some point of the process, I suspect you're trying to execute a script with wrong shell so getting syntax errors.

Sometimes I worked around this kind of issues by either:

  • deleting one of the 2 variants of the failing script
  • using yo jhipsterrather than jhipster as this skips a level of indirection

Some users may have also a combination of npm and yarn folders in their PATH which makes it more tedious to debug. This is why sometimes npm could work better than yarn in such cases. yarn global bin shows you where yarn looks for global binaries.

@gmarziou
Copy link
Contributor

gmarziou commented Jul 23, 2017 •
edited
Loading

Nodejs doc has specific instructions about executing Windows commands ,

upgrade.js uses shelljs.exec() which runs everything through cmd.exe on Windows by default (see doc). So we should be isolated from user's own shell.

@gmarziou
Copy link
Contributor

gmarziou commented Jul 23, 2017 •
edited
Loading

@iconben could you try to reproduce with shelljs in verbose mode? Just edit generator-jhipster\generators\upgrade\index.js and set verbose mode:

const shelljs = require('shelljs');
shelljs.config.verbose = true;

I tried to reproduce but I have a more basic issue about tabtab that is unrelated but that I need to fix first.

@deepu105 I think we should always set shelljs in verbose mode at least for the upgrade sub generator, just compare the 2 outputs below.
Or maybe have a verbose option on jhipster command.

shelljs in normal mode 
jhipster upgrade
Executing jhipster:upgrade
Welcome to the JHipster Upgrade Sub-Generator
This will upgrade your current application codebase to the latest JHipster version
Looking for latest generator-jhipster version...
New generator-jhipster version found: 4.6.2
Git repository detected
WARNING!  local changes found.
        Please commit/stash them before upgrading
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: �[31m�[1mERROR!�[22m�[39m Exiting process

at Environment.error (C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\node_modules\yeoman-environment\lib\environment.js:140:40)

at constructor.error (C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\generator-base.js:1645:18)

at gitExec (C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\upgrade\index.js:179:26)

at C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\node_modules\shelljs\src\exec.js:199:9

at ChildProcess.exithandler (child_process.js:195:7)

at emitTwo (events.js:106:13)

at ChildProcess.emit (events.js:191:7)

at maybeClose (internal/child_process.js:886:16)

at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)

shelljs in verbose mode 
jhipster upgrade
Executing jhipster:upgrade
Welcome to the JHipster Upgrade Sub-Generator
This will upgrade your current application codebase to the latest JHipster version
Looking for latest generator-jhipster version...
New generator-jhipster version found: 4.6.2
Git repository detected
Created branch jhipster_upgrade
Cleaned up directory
Installing JHipster 4.6.1 locally
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: �[31m�[1mERROR!�[22m�[39m Something went wrong while installing the JHipster generator! yarn add v0.27.5

info No lockfile found.

[1/4] Resolving packages...

[2/4] Fetching packages...

[3/4] Linking dependencies...

[4/4] Building fresh packages...

info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

error E:\projets\issues\upgrade\node_modules\generator-jhipster: Command failed.

Exit code: 1

Command: C:\WINDOWS\system32\cmd.exe

Arguments: /d /s /c tabtab install --name jhipster --auto

Directory: E:\projets\issues\upgrade\node_modules\generator-jhipster

Output:

module.js:471

throw err;

^


Error: Cannot find module 'E:\projets\issues\upgrade\node_modules\issues\upgrade\node_modules\tabtab\bin\tabtab'

at Function.Module._resolveFilename (module.js:469:15)

at Function.Module._load (module.js:417:25)

at Module.runMain (module.js:604:10)

at run (bootstrap_node.js:393:7)

at startup (bootstrap_node.js:150:9)

at bootstrap_node.js:508:3


at Environment.error (C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\node_modules\yeoman-environment\lib\environment.js:140:40)
at constructor.error (C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\generator-base.js:1645:18)
at shelljs.exec (C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\upgrade\index.js:226:31)
at C:\Users\Gaël\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\node_modules\shelljs\src\exec.js:204:9
at ChildProcess.exithandler (child_process.js:211:5)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:886:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)

@gmarziou gmarziou mentioned this issue Jul 23, 2017
Closed
1 task
@pascalgrimaud
Copy link
Member

Related to the discussion #5883

I think we should reconsider to remove the upgrade sub generator, or at least, move to a specific project for people who want to maintain it.
Every new release of JHipster has new issues with this upgrade sub generator.

@gmarziou
Copy link
Contributor

gmarziou commented Jul 23, 2017 •
edited
Loading

This sub generator is very important, personally I would prefer to keep it in core project.
It's just that it's too silent for all it does so when it fails, user has no idea what went wrong.
This is why I proposed a verbose mode for shelljs, this way we know exactly what fails and user can reproduce by running failing command manually to debug.

Also I don't know if it's feasible to add it to our CI but it could help.

@mraible
Copy link
Contributor

mraible commented Jul 23, 2017 via email

@jdubois
Copy link
Member

jdubois commented Jul 23, 2017

Just as I wanted to have one of those Surface Pro laptop :-)
(Btw, Matt, I just did a photo for you, I'm tweeting this right now)

@iconben
Copy link
Contributor Author

iconben commented Jul 23, 2017

Hi @deepu105, I had both yarn and npm installed and both have generator-jhipster at the begininng. As I didn't set up yarn global bin path environment then(I didn't complete yarn setup maybe because npm runs so far so good), I actually was using npm to upgrade my project, the error is the same (of course path of shelljs is different).

The above error message I pasted was after I removed generator-jhipster under npm and enabled yarn global bin environment.

My yarn global bin path:
C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules.bin

@iconben
Copy link
Contributor Author

iconben commented Jul 23, 2017

Hello @gmarziou , I just tried 'jhipster upgrade' with cmd.exe, same error:

Regenerating application with JHipster 4.6.2...
events.js:160
throw er; // Unhandled 'error' event
^

Error: �[31m�[1mERROR!�[22m�[39m Something went wrong while generating project! '$' is not recognized as an internal or external command,
operable program or batch file.

@iconben
Copy link
Contributor Author

iconben commented Jul 23, 2017 •
edited by gmarziou
Loading

@gmarziou I tried shelljs in verbose mode, here is what I got:

$ jhipster upgrade
Executing jhipster:upgrade
Options:
Welcome to the JHipster Upgrade Sub-Generator
This will upgrade your current application codebase to the latest JHipster version
exec git --version { async: true, silent: true } (code) => {
            if (code !== 0) {
                this.warning('git is not found on your computer.\n',
                    ` Install git: ${chalk.yellow('https://git-scm.com/')}`
                );
            }
            if (callback) callback(code);
        }
Looking for latest generator-jhipster version...
exec yarn info generator-jhipster version { silent: true } (code, msg, err) => {
                if (err) {
                    this.warning(`Something went wrong fetching the latest JHipster version number...\n${err}`);
                    this.error('Exiting process');
                }
                this.latestVersion = this.clientPackageManager === 'yarn' ? msg.split('\n')[1] : msg.replace('\n', '');
                if (semver.lt(this.currentVersion, this.latestVersion)) {
                    this.log(chalk.green(`New ${GENERATOR_JHIPSTER} version found: ${this.latestVersion}`));
                } else if (this.force) {
                    this.log(chalk.yellow('Forced re-generation'));
                } else {
                    this.error(`${chalk.green('No update available.')} Application has already been generated with latest version.`);
                }
                done();
            }
New generator-jhipster version found: 4.6.2
exec git rev-parse -q --is-inside-work-tree { async: true, silent: true } (code, msg, err) => {
                if (code !== 0) gitInit();
                else {
                    this.log('Git repository detected');
                    done();
                }
            }
Git repository detected
exec git status --porcelain { async: true, silent: true } (code, msg, err) => {
                if (code !== 0) this.error(`Unable to check for local changes:\n${msg} ${err}`);
                if (msg) {
                    this.warning(' local changes found.\n' +
                        '\tPlease commit/stash them before upgrading');
                    this.error('Exiting process');
                }
                done();
            }
exec git rev-parse -q --abbrev-ref HEAD { async: true, silent: true } (code, msg, err) => {
                if (code !== 0) this.error(`Unable to detect current Git branch:\n${msg} ${err}`);
                this.sourceBranch = msg.replace('\n', '');
                done();
            }
exec git rev-parse -q --verify jhipster_upgrade { async: true, silent: true } (code, msg, err) => {
                if (code !== 0) createUpgradeBranch();
                else done();
            }
exec git checkout -q jhipster_upgrade { async: true, silent: true } (code, msg, err) => {
            if (code !== 0) this.error(`Unable to checkout branch ${branch}:\n${err}`);
            this.log(`Checked out branch "${branch}"`);
            callback();
        }
Checked out branch "jhipster_upgrade"
Updating generator-jhipster to 4.6.2 . This might take some time...
exec yarn add [email protected] --dev --no-lockfile { silent: true } (code, msg, err) => {
                if (code === 0) this.log(chalk.green(`Updated ${GENERATOR_JHIPSTER} to version ${this.latestVersion}`));
                else this.error(`Something went wrong while updating JHipster! ${msg} ${err}`);
                done();
            }
Updated generator-jhipster to version 4.6.2
ls -A
rm -rf .bowerrc
rm -rf .editorconfig
rm -rf .eslintignore
rm -rf .eslintrc.json
rm -rf .gitattributes
rm -rf .gitignore
rm -rf .mvn
rm -rf bower.json
rm -rf gulp
rm -rf gulpfile.js
rm -rf mvnw
rm -rf mvnw.cmd
rm -rf package.json
rm -rf pom.xml
rm -rf README.md
rm -rf src
Cleaned up directory
Regenerating application with JHipster 4.6.2...
exec $(yarn bin)/jhipster --with-entities --force --skip-install { silent: true } (code, msg, err) => {
            if (code === 0) this.log(chalk.green(`Successfully regenerated application with JHipster ${version}`));
            else this.error(`Something went wrong while generating project! ${err}`);
            callback();
        }
events.js:160
      throw er; // Unhandled 'error' event
      ^

Error: ERROR! Something went wrong while generating project! '$' is not recognized as an internal or external command,
operable program or batch file.

    at Environment.error (C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\yeoman-environment\lib\environment.js:140:40)
    at constructor.error (C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\generator-base.js:1610:18)
    at shelljs.exec (C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\generator-jhipster\generators\upgrade\index.js:87:23)
    at C:\Users\Powerful User\AppData\Local\Yarn\config\global\node_modules\shelljs\src\exec.js:204:9
    at ChildProcess.exithandler (child_process.js:211:5)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:886:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)

@iconben
Copy link
Contributor Author

iconben commented Jul 23, 2017

Last time I had this issue was when I tried to upgrade from 4.5.4 to 4.5.6. I then manually executed the command that shelljs was trying to do (jhipster --with-entities --force --skip-install) and committed the jhipster-upgrade branch, then solved merge issues and merged to master. This time I think it should be something we have to fix.

Not sure if this help: when I type this in cmd/shell:
echo $(yarn bin)
the output is:
C:\code\sibodell-hse\hse-backend\node_modules.bin

@gmarziou
Copy link
Contributor

gmarziou commented Jul 23, 2017 •
edited
Loading

Interesting, this line looks suspicious to me :

generatorCommand = this.clientPackageManager === 'yarn' ? '$(yarn bin)/jhipster' : '$(npm bin)/jhipster';

It can be expanded only in a unix shell and as shelljs.exec uses cmd.exe, it's normal that it fails
$(yarn bin)/jhipster --with-entities --force --skip-install ....

So this bug affects all Windows users whatever shell they use.

The fix should be to use shelljs to execute yarn bin or npm bin and capture output to set generatorCommand's value.

I'll work on it in few hours unless you want to do it in meantime @iconben

@iconben
Copy link
Contributor Author

iconben commented Jul 23, 2017

Please go ahead @gmarziou and thanks a lot 👍

@gmarziou
Copy link
Contributor

Unfortunately, I am unable to run the upgrade process, it always fails because of tabtab even after I completely wiped out yarn's node_modules dir and re-installed node. I'm completely puzzled.

@iconben
Copy link
Contributor Author

iconben commented Jul 24, 2017

I changed line 82 from:

generatorCommand = this.clientPackageManager === 'yarn' ? '$(yarn bin)/jhipster' : '$(npm bin)/jhipster';

to these two lines:

let generatorDir = this.clientPackageManager === 'yarn' ? shelljs.exec('yarn bin') : shelljs.exec('npm bin');
generatorCommand = generatorDir.replace('\n', '') + '/jhipster';

Now it works. Out put around these lines are:

......
Regenerating application with JHipster 4.6.2...
C:\Code\sibodell-hse\hse-backend\node_modules.bin
Successfully regenerated application with JHipster 4.6.2
......

Notice there's output when shelljs.exec('yarn bin') run, should I mute it by adding a parameter { silent: this.silent }? Not familiar with shelljs.

The code might be not robust and only tested on my win10 pro box so I'll leave it for your comments before starting a pull request. By the way this is done by directly modifying upgrade/index.js in yarn global bin directory. I guess there's no test case around upgrade feature in the generator project?

@deepu105
Copy link
Member

@iconben @gmarziou yes we need to add some tests for this. Its actually bit hard to test :(
@iconben could you do a PR since you are using shellJs its should work in all OS anyway we can test your PR

@gmarziou
Copy link
Contributor

Great job @iconben

this is done by directly modifying upgrade/index.js in yarn global bin directory

Does it mean that yarn link did not work for you?

@deepu105 what about having a -v flag for jhipster command to enable verbose mode?

iconben pushed a commit to iconben/generator-jhipster that referenced this issue Jul 24, 2017
fix command path in upgrade/index.js for windows users
5bcf789 
Fix jhipster#6161
@iconben iconben mentioned this issue Jul 24, 2017
Merged
@iconben
Copy link
Contributor Author

iconben commented Jul 24, 2017

Code was slightly changed to make eslint happy, please test/review, thanks.

@gmarziou I mean I didn't setup a generator development environment yet. Yarn link works for me.

@deepu105
Copy link
Member

@gmarziou we already use -v for version. we also have --debug which prints more debug info and may be we could use that, or we can also add --verbose flag if you like.
In the generator I also added --debug flag support to do this, so you can add debug logs when this.debugEnabled is true

@gmarziou
Copy link
Contributor

gmarziou commented Jul 25, 2017 •
edited
Loading

The more I learn, the more I realize how much I don't know ;)

Hopefully your book will surely help.

@deepu105
Copy link
Member

@gmarziou I'm bad at documenting stuff 😸

@deepu105
Copy link
Member

@gmarziou here is a sample usage https://github.com/jhipster/generator-jhipster/blob/master/generators/generator-base.js#L1904
I wanted to add such debug statements in other places as well but didn't get around it

@gmarziou
Copy link
Contributor

Thanks, I will use it on the upgrade sub generator.

@ruddell
Copy link
Member

ruddell commented Jul 25, 2017

The upgrade sugbenerator already has a --verbose flag (docs) which outputs all of the shelljs commands, maybe we should merge that with the --debug flag

@gmarziou
Copy link
Contributor

yes good idea, I also think we should give an advice to user when an error is thrown during upgrade, something like "please run the same command with --verbose flag"

@iconben
Copy link
Contributor Author

iconben commented Jul 26, 2017

Agree with this idea. It will definitely help when someone needs to troubleshoot (or at least to locate) errors.

@gmarziou gmarziou mentioned this issue Jul 28, 2017
Closed
Tcharl pushed a commit to Tcharl/generator-jhipster that referenced this issue Jul 30, 2017
@Tcharl
fix command path in upgrade/index.js for windows users
03a667c 
Fix jhipster#6161
@jdubois jdubois modified the milestone: 4.7.0 Aug 22, 2017
@pvlastaridis pvlastaridis mentioned this issue Dec 19, 2017
Closed
@ghost ghost mentioned this issue Dec 14, 2018
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
theme: upgrade
Projects
None yet
Milestone
4.7.0
Development

No branches or pull requests
7 participants
@mraible @jdubois @gmarziou @deepu105 @ruddell @pascalgrimaud @iconben