Reorganize content structure

[straight-shoota]

While working on porting this site to Jekyll (#261 (comment)) it became evident that the content in this book needs to be restructured.
The current navigation structure (SUMMARY.md) is very confusing. The individual pages and sections should be better arranged.

Sections

Looking at the current content, I would roughly categorize it into three section. The contents in each section build a collection and can have different properties / visuals from others. For a clearly arranged structure, the navigation menu should only provide links to other pages in the same section.

• Manuals: Using the compiler and The shards command are manuals for the CLI programs coming with Crystal
• Guides: Besides the obvious Guides section, Database and Overview and Examples are essentially also guides and should be merged into this section.
• Language: Syntax and semantics contains what I would consider the actual language reference. That's obviously the biggest chunk.

I'm not sure what to do with what's currently labeled Conventions. Documenting code is a mixture of language reference, compiler manual and style guide. I suppose this should be more streamlined, the respective aspects moved to Coding style and the compiler manual. The heart of what's left clearly belongs to the language reference unless we consider the documentation syntax not part of the core language itself.
Coding style is primarily an internal resource for contributors to the compiler and stdlib. As such it could even be referenced more prominently. I think most of it is pretty much undisputed standard anyway and some even enforced by the formatter (indentation for example).
It could fit into the Guides section, but maybe it could also be placed in a Contribute/Developer section, together with some pages ported from the GitHub Wiki which would help centralize documentation in one place (see Overhauling the GitHubWiki).

Navigating Syntax and Semantics

Even when reduced to only a single section, the navigation menu for Syntax and Semantics is still pretty huge and hard to grasp. It currently lists 108 pages, stacked up to 4 layers deep with very different sizes.

General outline

The structure has probably grown from more humble beginnings, but now it's at a place where certain things don't make much sense. The top-level items:

• Comments
• Literals
• Assignment
• Local variables
• Control expressions
• Requiring files
• Types and methods
• Exception handling
• Type grammar
• Type reflection
• Macros
• Annotations
• Low-level primitives
• Compile-time flags
• C bindings
• Unsafe code

Types and methods is the biggest subsection and pretty convoluted. While there are also other subsections related to types not included in it, one of its children is Classes and methods and it's not really clear what's the difference. That needs some reorganizing. I'm not yet sure about the best approach, though. A first step would probably be to separate the concepts types and methods.

Low level primitives, C bindings and Unsafe code could probably be merged into one subsection for advanced topics.
Compile time flags should probably be in the Macros subsection. And its sub-page Cross-compilation is actually part of the compiler manual.

Hierarchical nesting

It's hard to follow the structure when nested deeply into a hierarchy of long sibling pages. As an example, the default navigation menu provided by the Jekyll theme minimal-mistakes I've used for my test, only supports two nesting levels in the navigation menu. That's pretty limiting, and we'd probably have to increase that. But in general, it's not a bad idea to impose such strict limitations because it forces to keep the hierarchy flat and in clearly arranged chunks.

Page boundaries

While some pages are really huge with 1000+ words, there are also very tiny ones. About a third is less than 100 words. So it begs to question whether there should be so many small pages or rather integrate them into bigger ones. Of course, the circumstances are different in each case. In general, I'd prefer to combine related content into a single page. Especially with a in-page table of contents, this is often better to combine everything about a topic into a single page, rather than having it distributed among several ones. In general the topics should be language concepts, not the keywords used for them.

• I would prefer to combine all the variations of if semantics into a single page instead of eight sub pages. Most of the content on these pages is code examples.
• Some of the pages about individual literals are quite small as well, but others are pretty huge. Combining them into a single page doesn't seem feasible.
• The entire C bindings section could be merged into a single page. The combined word count would just be 2500 (also with lots of code examples).
• The entire Type reflection section could be merged into a single page. The combined word count would just be 1000.

[bcardiff]

I agree that the content needs to be restructured.

I've been iterating on some alternatives. My current take is that the top level could be

• Getting Started: with a quick check that the environment was installed correctly and can create a hello world kind of app.
• Examples: to show intro for custom domain apps like web, cli, data science, game, etc.
• Fundamentals: to explain core concepts used in the language without falling into a full description. Strings, Collections, Struct and Classes (without macros)
• Techniques: to offer a first-time explanation of more advanced concepts like Channels, Concurrency, using macros; and presenting top used modules like http, serializacion, logging, db, specs.
• Recipes: explain how to solve some cross-cutting concerns idiomatically like user-input, validation, configuration, pluggable designs. Perform non pure coding tasks like profiling, CI, publishing shards, documentation.
• Language Specification: more extensive description of the language. Most current Syntax and semantics content.
• Deep understanding: guides of how things work or how it can be used and tweaked. This could be content used as a reference and not needed from the beginning. I imagine from explanation of how the scheduler work, static linking, and full description of cli compiler/shards/tools options.

The list of contents on each top level is not extensive. But in order to make the book a good resource for learning the organization needs to allow people to use crystal before knowing the whole language and features. I believe that splitting fundaments, techniques and recipes is a good way to present content that will get bigger, complex and uses concepts of previous top-level section.

I don't believe that a single concept needs to appear only once in the whole book. There might be a simple instruction of collectiopns in fundamentals and there might be more stuff to cover in techniques like transformation, serializing them etc. Linking content to simplify discovery is key.

I agree that the language specification current navigation is huge. We can either merge some pages or allow some pages to not be reached from the top navigation. Also collapsing, and sending it more towards the end will help. I expect the language specification to be more a catalog of features than an explanation guide.

Some other changes I notice I would do are: split the database connection and pool in the deep understanding section, while leaving and intro of it in techniques; split the current using the compiler and shards command in a getting started and in a deep understanding of the tooling.

[oren]

I like the top level list! One aspect I would like to mention is the examples. Many people like to learn by looking at a well designed and real-world example. It will be useful to have a bunch of comments throughout the code and have links in the comments to different parts of the language specification docs.

[straight-shoota]

The number of different sections seems quite overwhelming. It might be something for the future, but such a dispersed structure wouldn't be very useful when there would only be a handful of entries in each section. I just can't see how we would possibly fill that with content. And for the start it's certainly better to group related content into combined sections than having too many sections with sparse content. Planning ahead for future development is good, but the current architecture needs to be focused on what we have currently.

From what I understand the distinction between Fundamentals, Techniques, Recipies, Deep understanding and Examples would be very wobbly. They're all some kind of guides with different levels of detail and context. That's why I'd suggest to collect them in a Guides section. Guides can still have different levels and contexts, but they'd show up in one section. When the number of guides grows in the future, we can discuss how to organize it then based on the actual content. For example, if nobody writes Deep understanding guides we won't need a section for that.

A getting started guide shouldn't stop at a hello world app but introduce into the language. Something like https://tour.golang.org/welcome/1 (cf. crystal-lang/crystal#4462). Maybe that's what you mean by Fundamentals.

split the current using the compiler and shards command in a getting started and in a deep understanding of the tooling.

For the tools we should have manual pages and introductory guides. Maybe later some deep understanding topics, but that's low priority.

[bcardiff]

My proposal is that the different sections requires/assumes different current knowledge and understanding. IME holistic learning is better than a vertical one. In that regard is better to have guides grouped by complexity rather than topic. But linking can help the discovery, of course.

I don't think a guide regarding db connection pool belongs to the same section where a guide for basic collection manipulation. Neither the usage of std-lib macros for avoiding boilerplate, or the creation of such macros, or dealing with the transformation of AST nodes.

Where each guide fits it will depends on where the requirements to understand it are. Ideally techniques would require fundamentals. And recipes would assume techniques are clear. Not all of them probably, but is a way to layer the learning.

How we can go from the current state to the proposed one iteratively might be a challenge by itself, but I don't fear a bit of time with some less polluted section if that is the path. Yet, I don't want empty sections. Building a long book iteratively where every increment is a deliverable itis definitely a hard task.

A guide that has a similar structure to what I am proposing is https://docs.nestjs.com/ , yet been a framework offers some clear path to follow.

[straight-shoota]

When looking at the nestjs documentation as someone completely unfamiliar with it, I feel immediately lost in the navigation tree. It's at least pretty clear to start with the Overview section, but after that I have no idea what everything's supposed to mean.
The same applies to your proposal which essentially results in a seven-tier top navigation.

I don't think a guide regarding db connection pool belongs to the same section where a guide for basic collection manipulation. Neither the usage of std-lib macros for avoiding boilerplate, or the creation of such macros, or dealing with the transformation of AST nodes.

All these examples certainly vary by complexity, but most importantly by topic. And that's the main focus of a guide's readers: They typically want to learn about a topic they're interested. Looking for guides available at a specific level of detail seems of much lesser concern.

[bcardiff]

And that's the main focus of a guide's readers

It depends. If we have vertical topics only experienced readers (or the ones with a specific topic goal) can jump to them.
To learn the language, from zero, there is no clear path.

If it is separated by complexity and then topic both kind of readers can navigate.

[straight-shoota]

Since this discussion started, we have moved to mkdocsmaterial as site generator. That has improved many things, but the navigation is still very crowded and badly designed.
#470 suggests adding another top-level section, and I'm preparing a Tutorials section which would be another one.

mkdocs-material has a navigation tabs feature (that link also shows them in action) which allows us to effectively structure the content into separate sections with individual navigation trees (technically, it's still one tree but the sidebar shows only the items from the current top-level section).
Activating that would be a great improvement to help structuring our content. The biggest section Syntax and Semantics should definitely get their own tab (as Language Specification), and it's still gonna have a growded sub-tree (see OP).

I've re-read #384 (comment) and comparing it with my ideas, it seems we pretty much mean the same, just using different words. I'll try to combine both approaches into a pracitcal proposal for which top-level nav sections we could introduce. Don't nail me on the exact section titles, they're up for debate.

• Home: This section includes the landing page (see Landing Page #475), quick setup instructions (Getting started) and examples (Examples). It would also serve as home to general content that don't fit anywhere else like Platform Support.
• Language Specification: The current Syntax and Semantics section.
• Tutorials: New section with different courses, targeting beginners and advanced users. This aligns with the Fundamentals and Techniques sections in Brian's proposal. They're just tutorial courses on different knowledge levels.
• Guides: Current Guides section. I think this aligns very well with the description of Recipes in Brian's proposal. But the label "guide" might be easier to understand. I don't mind calling it Recipes, though.
  The Deep understanding category probably fits here as well, it's just about very advanced topics.

The current Conventions section, doesn't really fit in either concept, and it should be dispersed. Documenting code belongs to the language specification. Coding style might just go into the default section.

This would leave us with four nav tabs, which is really ideal. We could perhaps also add a tab for the API docs (as external link), because it's a major documentation feature.