v4: Move to npm scripts #20332
v4: Move to npm scripts #20332
Conversation
|
First time here 😄 and i think this would be a great move @mdo 👍 |
| @@ -0,0 +1 @@ | |||
| node-sass --output-style expanded --source-map true --precision 6 docs/assets/scss/docs.scss docs/assets/css/docs.css | |||
There was a problem hiding this comment.
probably better if you reference the local npm installs of these (in .bin ) so that people don't need global installs 👍
There was a problem hiding this comment.
Well this whole approach will have issues on Windows. But I'll try it later
or tomorrow to verify this.
On Jul 20, 2016 22:44, "fat" [email protected] wrote:
In tasks/dist/css
#20332 (comment):@@ -0,0 +1 @@
+node-sass --output-style expanded --source-map true --precision 6 docs/assets/scss/docs.scss docs/assets/css/docs.cssprobably better if you reference the local npm installs of these (in .bin
) so that people don't need global installs 👍—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/twbs/bootstrap/pull/20332/files/795f6d924b1c1eb9b637917a41f2c83655408c97#r71592896,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVVtVO5bBIj0o15D8lDel5ya1-05Smrks5qXnqfgaJpZM4JRHjn
.
There was a problem hiding this comment.
In https://docs.npmjs.com/cli/run-script, it sounded like if the devDependency is there, npm run will do that for us.
In addition to the shell's pre-existing PATH, npm run adds node_modules/.bin to the PATH provided to scripts. Any binaries provided by locally-installed dependencies can be used without the node_modules/.bin prefix. For example, if there is a devDependency on tap in your package, you should write:
Reading that, maybe I just need to add node-sass to devDependency? Again, I've no idea so I'm likely wrong 😁.
There was a problem hiding this comment.
This is true for the case where you call the bin via package.json. Not sure
if it will work here since you call it in a js file. I'll try it when im
home and let you know.
On Jul 20, 2016 23:19, "Mark Otto" [email protected] wrote:
In tasks/dist/css
#20332 (comment):@@ -0,0 +1 @@
+node-sass --output-style expanded --source-map true --precision 6 docs/assets/scss/docs.scss docs/assets/css/docs.cssIn https://docs.npmjs.com/cli/run-script, it sounded like if the
devDependency is there, npm run will do that for us.In addition to the shell's pre-existing PATH, npm run adds
node_modules/.bin to the PATH provided to scripts. Any binaries provided by
locally-installed dependencies can be used without the node_modules/.bin
prefix. For example, if there is a devDependency on tap in your package,
you should write:Reading that, maybe I just need to add node-sass to devDependency? Again,
I've no idea so I'm likely wrong 😁.—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/twbs/bootstrap/pull/20332/files/795f6d924b1c1eb9b637917a41f2c83655408c97#r71598457,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVVtdykpzNATYk8LkXMda5OCPTJJQIlks5qXoLrgaJpZM4JRHjn
.
There was a problem hiding this comment.
Running these task files using npm run * will run them in the npm environment. as you mentioned above @mdo
https://docs.npmjs.com/misc/scripts#environment
Dependency versus DevDependency just has to do with what gets installed when someone using the npm package run npm install you-package. Only dependencies needed at run-time will be installed on the user's system. DevDependencies will only be installed locally when npm install is called within the repository. It's useful / helpful to only include run-time dependencies in dependencies so devs know what's needed to be installed.
FWIW, bootstrap's run-time dependencies are all bundled in the project's files right? So it shouldn't have really have any dependencies.
More info on that here: https://docs.npmjs.com/files/package.json#devdependencies
|
As much as I love Gulp, this does seem like the right way to go about things. 👍 |
|
Hey, Allow me to share my thoughts on this. At Edenspiekermann, we had projects running on Grunt, and we were more and more unhappy about it every day. Then we started a couple of projects with nothing but npm scripts. It seemed fine at first, and it made things much simpler to actually run. And then, we needed more commands. More options. More scripts. To do more things. We kept adding more and more scripts to the package.json, without the ability to properly document them because y’know, JSON. Clever developers did clever things with clever commands, and while it worked fine, it got harder and harder to figure out what was actually going on. Let alone update it. From there, we started moving our projects to Gulp because this is just what we needed. Simpler than Grunt, but more explanatory, obvious and flexible than npm scripts. Since then, most of our projects run on Gulp and this is a decision we did not regret. We usually have a Gulpfile.js acting as some task manifest, and do have our individual tasks in separate files. That’s it for the story time. Now regarding Bootstrap, I can think of a few reasons why moving to npm scripts might not be the best idea of all. First, some commands are not cross-platform, like Secondly, command line skills are not a given. I do like CLI much, but I know a lot of developers — especially beginners — who are quite afraid of it, or simply can’t really make their way around it. I wonder whether moving to npm scripts could be some kind of deal breaker for some potential contributors. In a way, Gulp or Grunt appear simpler (even if they come with their share of issues). Third, this tweet is just full of truth. Just my 2 cents. :) |
|
I agree too with the gist of this post. Npm scripts will be just as hard to Not sure Gulp is the solution either. It's nice to follow the trends and I for one don't like Gulp at all for my personal reasons. To summarize this pull request does not bring any improvement in my On Jul 20, 2016 23:12, "Hugo Giraudel" [email protected] wrote:
|
|
Great idea, Also, since alpha 3 is getting late, can i ask you to move this to alpha 4 phase please. its been 7 months you released an alpha and this is a breaking change which requires lot of time to fix everything...which makes alpha3 delay again. 2days back, no. of open issues in alpha 3 are 25 and i am happy that its getting closer but today again its 27 and this issue (big change) is one of those new ones. so please consider moving it to alpha 4, so you will have lot of time to work on it. thanks. |
|
What are the reasons not to use Gulp? As now Gulp4 hasn't been released yet but it's already in 4.0.0-alpha.2 stable version which let you run tasks asynchronously using |
|
Has there been webpack on the table? It's been a great replacement for our projects replacing gulp& grunt for CSS and JavaScript pipeline. If the decision is already set for pure npm scripts then of course my note is just a noise and let's move on with the current plan :) |
|
When I started working on the Salesforce Design System, we had a lot of bespoke scripts that were sort of using gulp in places, but not in a standard way. We were mostly using npm for our task orchestration. It was very challenging to troubleshoot performance issues and other bugs because of the lack of documentation, testing, and the inability to find community help. We started porting as much of our development environment to gulp as we could.
I hope this helps making a decision. I am not advocating Gulp in particular, but definitely advising against npm scripts for everything as I don't think it's a maintainable solution. Now we have a mix where we use gulp for what it's good at (running a front-end development environment) and npm for our testing, installation and production build process orchestration. We're pretty happy with it! If you're curious about the way we do things, here's our package.json: https://github.com/salesforce-ux/design-system/blob/master/package.json#L115 |
|
So, having worked a lot on the build process, I'm heavily in favor of npm scripts, though perhaps not this exact implementation. (I favor doing it piece-by-piece in several small PRs.) Firstly, there are currently N different JS build tools (Grunt, Gulp, Brunch, Broccoli, etc.) battling it out for dominance. It's not obvious which is going to win, or how long the winner will last until a hip new rival appears, or even how long the "winner" will stay maintained. (Remember how Grunt got really popular but then its maintenance stagnated for a long while?) So, I'd rather stay out of the brawl and not pick any of them. Secondly, these build tools typically involve relying on a layer of build-tool-specific plugins that are often not as actively maintained as their upstream projects, which can delay updates and fixes. By using the upstream CLIs directly, we avoid this problem. With regard to Windows compatibility, I'm optimistic about https://github.com/shelljs/shx (or similar) as a solution to that problem.
Newbies at that level are unlikely to be modifying the build in the first place, and I would feel nervous about accepting build-related PRs from such folks.
At the least, we'd have one less layer of dependencies (no "plugins") to deal with, and we'd depend on shx (or similar) instead of the build tool. And the build tools implement complicated things, whereas shx implements simple things like |
|
I also share the same opinions like @hugogiraudel & @kaelig. npm scripts tends to get really complex at some point and hard to maintain. especially when you need add more flags, configs, etc... I just finished migrating our app to use gulp/webpack & in fact our What I think is pretty good setup is this:
This tweet can show you also how things can get out of hand at some point https://twitter.com/sindresorhus/status/724259780676575232 |
|
First of all, I'm all for refactoring and simplifying the tooling and their dependencies but I'm with @hugogiraudel on this. You'll be placing all your hope for twbs4 on a single build dependency which currently has 88% coverage and 6 contributors by enforcing sfx for the entire community. and if you don't want to enforce but expect it to work magically across all OS'es You'll be putting all your hopes on IDE vendors that they'll update their integrated tooling to support this or expect developers to manually work around commands if we run into command-line issues like @hugogiraudel pointed out... We'll all follow you where ever you take us with v4, but by not picking one out of the brawl, you're also making a choice and its not necesarrily one that makes our live easier because for us, Bootstrap's always part of a bigger solution and not the final end-result... |
|
Totally agree with @ahmedelgabri. npm should be the interface to go and abstract the underlying tooling. That way no one will get confused if the tooling change. ...
"scripts": {
"clean": "rimraf ./dist",
"start": "NODE_ENV=development webpack-dev-server --hot --host 0.0.0.0",
"build": "NODE_ENV=production webpack -p",
"build:debug": "NODE_ENV=debug webpack",
"lint": "eslint ./",
"test": "karma start",
"test:watch": "npm test -- --auto-watch --no-single-run",
"publish": "npm run clean && node scripts/publish.js"
}
...I myself never executed these scripts on Windows but others on the team use Windows on a daily basis and no one ever complained. Running tasks sequentially with I know Bootstrap is an exponentially larger project and there will be many more dificulties but the upsides seems to outweight the downsides. @jbokkers |
|
Well why is there still alpha 5 after alpha 4? I thought alpha 4 is the last Bootstrap 4 alpha release, and then it will go Beta. |
That was more of a hopeful prediction as opposed to some sort of hard-and-fast promise. |
|
so would you consider add them back? thanks for great and tremendous work. |
|
uh-oh, did not check deep enough :) |
|
Would it be possible to have a sans-docs package.json with the documentation scripts and dependencies removed? That way a beginner can see the bare minimum they need to start working with the source files. Web designers that just want a custom build of Bootstrap could get overwhelmed easily. |
|
I've spent some time trying to continue the effort to migrate the build process to npm scripts only or at least to improve it. Problems encountered with a full npm script solutionSome tasks would require a lot of custom code to be migrated
Simplicity for the user / developerEven if we externalize the npm scripts, each one of them has to be referenced in the package.json and are exposed to the user/developer. That would be several tasks and subtask and it would be difficult to know which one is a subtask only and which one is supposed to be run. ConfigurationIdeally we would want to have a config file that define all the paths in the projects (dist, dist/js, dist/css scss, js/src etc... and global parameters like sass precision, babel plugins etc...). It's possible in package.json to define a DependenciesOne of the objective of the full npm solution was to limit the number of dependencies. In practice we would just replace a grunt plugin dependency by a command line one. I found that in many cases the cli package is less maintained and offer less configuration than the Grunt or Gulp plugin. In several cases, as mentioned above, there is no cli tool while there is a good Grunt or Gulp plugin. Other than sass, cleancss, eslint and a few others the npm cli tools is quite limited. ConclusionMany tasks would require to write custom node scripts (in javascript) and add more complexity and more difficulty to maintain, therefore defeating the purpose. ObjectivesDespite the issues mentioned above I think the build process can be greatly improved and simplified. Here is what I think we can achieve. Simple to useWe should expose a limited number of command:
All these command should be started from the same tool and should be the only one mentioned in the documentation. A lot fasterWe should run the tasks we can in parallel to make the process as fast as possible.
With a reorganization of the tasks and parallelisation I managed to achieve ~15s for a full build, a ~2s to build the js and the css. More modularIdeally we should avoid big files and try to have smaller, more manageable tasks in their own files. MaintainableThe less custom JS scripts the better. The less config and path duplicated the better (i.e. --precision 6 for node should be set only once). Propositions1/ Grunt as the engine, NPM as the user interface
2/ Gulp as the engine, NPM as the user interfaceSame as 1/ but with Gulp as the engine. 3/ Gulp + NPM as the engine, NPM as the user interfaceMigrate to npm for the task that are simple simple enough, but keep Grunt for the others.
Next stepsAfter all that, I think /1 is the best solution to achieve the objective stated in the first comment by @mdo (even though the solution is the opposite of the one proposed). I understand the desire to move to npm in order to simplify the build process. I think npm scripts are the most simple solution as long as you build is limited to sass, concat, minify and lint. Everything beyond that (babel, qunit, file headers, linting other than stylestat and eslint, saucelab, conditional build for Travis) would require so much custom JS scripts that it defeat the purpose. @twbs and community, I'd love to have your feedback. Hopefully, if a decision is made I think I can implement it in the following couple weeks. |
|
Migrating to Webpack should also be considered in my humble opinion. |
|
@vanduynslagerp you present some valid points.
We could use
I still need to work out how we are going to do this, but as you'll notice, shelljs/shelljs#637 is my issue, and I'm working on a PR for it.
As per #21003, the inject file can be removed if needed.
I need to review our choice of HTML linter anyway. We are using two, one of which requires Java, and it all seems a bit weird.
We might be dropping SauceLabs soon and moving to BrowserStack (#18754), but we can switch to this in the meantime.
Normal git commands will work in this instance. In conclusion, there are going to be one or two rough edges until a few upstream issues get fixed, but I think that we should be able to drop grunt without too many issues. You might be interested in looking at bardiharborow/bootstrap/build, my heavily work-in-progress build branch. |
|
@bardiharborow I see we reached more or less the same situation :-) I think the initial objective was to make the build tool more simple, more maintainable and more pleasant to work with.
So I wondered "what if we keep what we have that works and rearrange the part that are not that great?" I understand there is an effort to move to npm, and frankly my opinion is "why not?". This is why I spend time at the beginning to help in that direction. But now that I have something that works pretty well and I think achieved at least part of the objectives mentioned in the very first post, I thought it might worth to share :-) |
|
I'm trying to convert the Grunt build process to Gulp - but I'm coming unstuck at the JS processes as I'm not able to figure out where some of the JS comes from. I see that the Official Bootstrap Theme's state they use Gulp for their build tasks though? |
|
Closing for #22045. |
This PR is heavily WIP, but addresses a growing concern I've had with developing Bootstrap—the build process is way to convoluted. Currently we make use of Grunt, npm, Bundler, Jekyll, etc all just to run our docs and compile source assets. More personally, I find the current dev process incredibly fatiguing—as a maintainer, it's hard to be motivated to jump into this codebase right now.
My goal here is to overhaul the build process and make Bootstrap development more approachable to everyone (myself included). Originally I was thinking Gulp would be our answer, but it's still a task runner build on npm rather than npm itself.
So far I've started to pull apart a few bits of our current tooling and replacing them with npm scripts. Right now they all live under
tasks/and are split bydist/anddocs/. I'm probably doing a ton wrong, but maybe you can help with that :D.Here's the list of tasks we need for developing and distributing Bootstrap:
stylelintoverscss-lint)_sitedirectory to remote to update docs siteI'm probably missing some tasks/scripts we need, but I do believe we need the above. These tasks should be runnable by us as maintainers, as well as casual users who wish to extend Bootstrap's code and docs, so I want them to be clear, concise, and documented.
/cc @twbs/team