Port glossary and Java tutorial to scala-lang, ...

[soc]

... update README, add Gemfile/Bundler

[soc]

@SethTisue A bit of a prototype, language support needs to be integrated, but the move to incremental compilation with Jekyll 3.0.1 looks promising. The idea is to unify the documentation into one site. I'm determined to find a solution for the slow indexing issue @heathermiller mentioned, too. Until now, everything is blazingly fast.

[soc]

I'll do a smaller change against the spec site to move that closer to the main style, too.

[heathermiller]

Just to repeat what I've said in a few different locations (another earlier PR scala/docs.scala-lang#467 (comment), and on gitter):

While I very much appreciate the move to improve the appearance of the website, I don't think we should be attempting to merge two repositories into one.

Instead, I think we should be keeping two separate repositories (one for documentation, and one for the website front page/blog/announcements/news).

It is, however, ok if we merge the appearance. I think we can do that a lot easier than by migrating content between repositories. Instead, it's likely easier if we keep the content where it is, and move over the stylesheets/templates from scala-lang to docs.scala-lang. If that's clear.

[soc]

I just checked and until now everything builds in a sub-second range. I keep watching the build-time issue, and act accordingly. I rewrote some code which "looked" slow (unnecessarily traversing all pages for minor stuff), and it hasn't been an issue so far. I propose that I try to migrate more and more, and if the build time issue never crops up, we could go forward with the migration. @heathermiller what do you think? My idea is that keeping stuff up-to-date is much easier with one repo instead of three.

[heathermiller]

As I said several times now, I would like to keep things in separate repositories. There are not three respositories.

[soc]

Why? I don't understand it. There is the spec in scala/scala, the website (with some documentation) in scala-lang and a lot of stuff in scala.github.com.

[heathermiller]

I provided rationale several times in several locations, even days prior to this pull request. I refer you to the many other locations where we discussed this (see links above).

For convenience, I will repeat some of the above, and I will provide additional rationale.

1. docs.scala-lang is the documentation website. We publish and maintain documentation there. Documentation is categorically different than releases/downloads/announcements. I prefer keeping concerns separate. Both projects have typically separate committers that do not need to be blocked by one or the other when problems do arise.
2. This pull request complicates the build. You've added several more dependencies, and require new versions of lots of things, including Ruby, that people who have been contributing for years likely do not have. This includes the machine at EPFL that we build and serve scala-lang on. For these people who want to just make a quick update to the site (e.g., adding an event), you're introducing a 30 minute long headache– updating Ruby etc is a pain. All for no appreciable benefit – two separate repositories is working just fine, and it's preferable for us.
3. The documentation site is slow to build. We don't want both in one repository in the first place, but even if you were to migrate, I would still wager that you are going to increase build times significantly by lumping them into one megasiterepo.

If you would like to help, the most constructive way (other than brute forcing your own changes, regardless of our views) would be to consider suggestions that we have made. Originally this discussion was about providing the same look and feel across both sites. We'd still like that. And I think it's an easy PR to make if you're so inclined (move stylesheets/templates over from scala-lang to docs.scala-lang, that + light tweaking is about all it takes). But we do not want to merge our 2 websites into one mega repo, and to drastically change all of our builds.

The cost outweighs the benefit.

[soc]

Hi @heathermiller!

I appreciate your clarifications, but I substantially disagree with the following claims:

Regarding 1: I think that users don't care about the distinction between releases/downloads/announcements and documentation, and keeping different software stacks, styles and repos is unnecessary work for contributors.

Regarding 2: I think the change simplifies the build overall:

• It doesn't add several more dependencies, the dependencies you see in the .lock file are those which are required by jekyll, nothing more. The only change is that the versions are mentioned explicitly now. In fact, there are now less dependencies than before: Jekyll 3 dropped dependencies on jekyll-paginate, jekyll-coffeescript, jekyll-gist, pygments.rb, redcarpet, toml, classifier-reborn. These are all required by our current build.
• It eliminates 3 slightly different and incompatible Ruby-Gem-Bundler-Jekyll-Markdown stacks in favor of one.
• One of that is the GitHub Jekyll build which disallows plugins, further complicating things if the need for plugins ever arises: Add Algolia search on header bar docs.scala-lang#469
• Ruby 2.0 was released in February 2013. The new version requirements will start to apply to the scala.github.com documentation whether we want it or not, because GitHub is already moving to Jekyll 3: Jekyll 3.0.x github/pages-gem#175. That 30 minute long (one-time) headache will be coming. This way or via gh-pages.

Regarding 3: No page on the documentation site, with my changes, takes more than 1.5 seconds, most of them are less than 0.9 seconds (usually 0.7 seconds). With incremental compilation this matters even less, because if you want to update one single file, you just do it. No need to recompile all the other things in the repo.

If you would like to help, the most constructive way (other than brute forcing your own changes, regardless of our views) would be to consider suggestions that we have made. Originally this discussion was about providing the same look and feel across both sites.

That's were I started from, and my conclusion very early on is that it's just not worth the work of keeping things between different repos in sync forever. If I'm touching this stuff anyway, I will go the whole way: the effort is only insufficiently larger from my side, while the mid- and large-term benefits are much higher.

I hope the current stance can be revisited in the future, given the information above.

[heathermiller]

RE point 1:
As I said at least 4 times in several conversations spanning several days, we would like the visual style to carry over to the documentation. We agree on that point. However, that does not require it all to be in one repository.

RE point 2:
Requiring people to update Ruby and to suddenly use bundler to build everything is a big change. No one wants to update Ruby or its tangled web of dependencies in order to update a website. You have not had to deal with angry people storming your office about having to update tons of Ruby libraries just because they want to quickly fix a typo on a website. The cost of your proposed change is high (it's annoying to lots of people), and there is no benefit to having everything in one repo.

RE point 3:
I will repeat my previous point with the important part in bold: "The documentation site is slow to build. We don't want both in one repository in the first place, but even if you were to migrate, I would still wager that you are going to increase build times significantly by lumping them into one megasiterepo. That is to say you have not put it all in one repo yet, so you have no idea how long it all takes to build. You don't have an argument here. In any case, for reasons I've stated above, we don't want everything in one repo. We'd like to separate concerns.

That 30 minute long (one-time) headache will be coming. This way or via gh-pages.

Maybe so, but for the doc-site only. We'll cross that bridge when using Jekyll 3 is a requirement for gh-pages. In any case, bundler etc (that you've added to the build in this PR) wouldn't be required if gh-pages forces us to update Jekyll. My point still stands that your PR complicates the build, and that everyone will have to update several things (Ruby, Jekyll) and they will have to install new packages (bundler) for no benefit that I see. Just to put everything in one repo, which is unnecessary in the first place.

it's just not worth the work of keeping things between different repos in sync forever.

We never sync between repos. There's never been a need to. Totally unfounded and false statement. Baseless claim.

This is a completely unnecessary battle. Let's make the doc-site look like scala-lang, that's established. Let's work towards that goal – I agree with you on the point that unifying it all would be nice. However, for you to ignore our views, and to brute force your way around with large PRs doing the exactly the opposite of what we earlier suggested is not constructive. We don't need to force dozens of people to update Ruby etc and deal with a megasite repo for both sites to have the same visual appearance.

Anyway – this argument has consumed quite a bit of my time on my Sunday. Getting back to enjoying the weekend :) I suggest you do too! :)

[soc]

Hi Heather,

RE point 2:

• Ruby < 2.0 is equivalent to Debian oldstable or oldoldstable. For 99% of the users the existing Ruby version will be perfectly fine, they won't need to do anything.
• Bundler is completely optional. I added it because it was already used in two of the three repos (scala.github.com is one of them), and because it simplifies the process for new users (gem install by default tries insane things like install stuff globally into /var).

You have not had to deal with angry people storming your office about having to update tons of Ruby libraries just because they want to quickly fix a typo on a website.

Has this been an issue in the last few years? Looking at content changes to the docs, it kind of looks like the state of documentation is somewhere between "on life-support" and "abandoned".

RE point 3:

I would still wager that you are going to increase build times significantly by lumping them into one megasiterepo. That is to say you have not put it all in one repo yet, so you have no idea how long it all takes to build. You don't have an argument here.

The PR is quite out-of-date. I ported a substantial part, including the linking of translations, and the issue just isn't there. Additionally, the diff between the current scala-lang and a "megasiterepo" is basically 15 *.md files for the spec, and maybe 100 *.md files depending on how much can be salvaged from scala.github.com.

bundler etc (that you've added to the build in this PR) wouldn't be required if gh-pages forces us to update Jekyll

There is zero change for the doc site compared to the state today, it only moves scala-lang in line with docs.scala and scala/spec.
This is what the README of the doc site says:

cd into the directory where you cloned this repository,
then install the required gems with bundle install.
https://github.com/scala/scala.github.com#building--viewing

.

We never sync between repos. There's never been a need to. Totally unfounded and false statement. Baseless claim.

Are you proposing to let one Jekyll instance in one repo run against the Jekyll templates and includes in a different repo? That sounds pretty complicated. It's either that, or copying the templates and includes from one repo to another and keeping the files in sync.

Anyway, I don't really see a way forward here. I'll nuke my branch and move on. If not updating a Debian oldstable box is more important than not losing contributors ... well, then I have plenty of other work to pick from.

Enjoy your weekend,

Simon

[SethTisue]

No one wants to update [Ruby's] tangled web of dependencies in order to update a website

Doesn't Bundler take care of that?

(Or is this a naïve question? It's definitely possible, I'm pretty ignorant about Ruby stuff.)

EDIT: I had said some stuff about how I'd found it painful to work on the sites, but it's entirely possible I just haven't educated myself sufficiently about Bundler.

[SethTisue]

(OK, so, after I fixed a problem in my Ruby config:)

Now in the docs repo, I'm able to do bundle exec jekyll serve and know the right versions are being used, because (if I understand this stuff correctly) there's a Gemfile.lock in that repo that enforces the versions.

But here in this repo, there is no Gemfile.lock. Would a more targeted PR be accepted, that simply added Gemfile.lock and anything else needed to make bundle exec jekyll serve do the right thing in this repo?

[Stargator]

@SethTisue Gemfile.lock doesn't enforce the versions. It is generated as a snapshot for what gems it installed at that time. It's more for humans to read than anything else.

[SethTisue]

@Stargator I'm reading at http://bundler.io/rationale.html that:

When your co-developers (or you on another machine) check out your code, it will come with the exact versions of all the third-party code your application used on the machine that you last developed on (in the Gemfile.lock). When they run bundle install, bundler will find the Gemfile.lock and skip the dependency resolution step. Instead, it will install all of the same gems that you used on the original machine.

In other words, you don't have to guess which versions of the dependencies you should install

what you're telling me seems utterly different, and contradictory. can you explain?

[Stargator]

I got that impression from reading:

After installing any needed gems to your system, bundler writes a snapshot of all of the gems and versions that it installed to Gemfile.lock.

I admit I didn't read further than this. There may be programs that need to maintain specific gem versions, but I haven't experienced any cases where that was needed. You can already set specific versions in the gemspec and/or Gemfile. So I didn't realize what other use Gemfile.lock may have.

Thanks for pointing out my error.

[SethTisue]

np. still figuring this stuff out myself