feat(tsdocs): add integration with api-extractor/documenter #2834

raymondfeng · 2019-05-03T22:17:05Z

See #2507

Checklist

👉 Read and sign the CLA (Contributor License Agreement) 👈

npm test passes on your machine
New tests added or existing tests modified to cover all changes
Code conforms with the style guide
API Documentation in code was updated
Documentation in /docs/site was updated
Affected artifact templates in packages/cli were updated
Affected example projects in examples/* were updated

👉 Check out how to submit a PR 👈

bajtos

Thank you for taking a stab at replacing strong-tools with api-extractor!

What's the best way for reviewing the new version of API docs that will be generated with this new tooling?

Are there breaking changes to be aware of? For example, what is the format of anchors and URLs in general? Will we need to update our hand-written docs in docs/site to use the new URL format?

packages/tsdocs/README.md

bajtos · 2019-05-06T07:41:34Z

packages/tsdocs/README.md

+### Generate api docs as markdown files
+
+```sh
+npm run generate-apidocs


While this may work inside loopback-next monorepo, I don't think it will work for people installing @loopback/tsdocs into their own projects. I see that the packages is marked as private: true, which means it's not intended for usage outside of loopback-next monorepo. Can you please mention this in package's README too?

If this package is private, how do you envision to generate API docs for experimental packages in loopback-labs? Wouldn't it better if @loopback/tsdocs was a generic tool that can be used in any monorepo? Maybe not, I am just asking.

raymondfeng · 2019-05-06T15:09:16Z

The purpose to have reports and reports-temp is to detect API changes (api reports are tracked by git). See https://api-extractor.com/pages/setup/configure_api_report/.
I'm debating between making @loopback/tsdocs a public module or a just an internal utility. The @loopback/tsdocs can be applied to loopback-labs too as loopback-labs mirrors loopback-next. Let's keep it private for now.
How to preview newly generated apidocs?

Check out ms-apidocs branch. Run npm run build:apidocs from inside tsdocs. Use vscode to open index.md in docs/site/apidocs and preview it.

raymondfeng · 2019-05-06T22:26:04Z

Please follow the following instructions to try out new apidocs:

cd loopback-next
git checkout ms-apidocs
npm i
npm run build
npm run tsdocs
cd docs
node ./bin/copy-readmes.js
npm pack

Preview with loopback.io ([email protected]:strongloop/loopback.io.git):

cd loopback.io
npm i
npm i <docs/loopback/loopback-docs-1.16.0.tgz>
npm start

raymondfeng · 2019-05-07T04:45:35Z

Home page:

Module:

bajtos · 2019-05-07T07:22:31Z

@raymondfeng how about the format of links - are there any breaking changes?

Are there breaking changes to be aware of? For example, what is the format of anchors and URLs in general? Will we need to update our hand-written docs in docs/site to use the new URL format?

bajtos · 2019-05-07T14:07:53Z

IIUC, this pull request is entirely reworking the way how API documentation works in LB4 and moves it from http://apidocs.loopback.io to https://loopback.io/doc/en/lb4. I love that ❤️

Having said that, moving API docs is not only about adding markdown files and a new sidebar. Obviously, we need to fix all places in our current documentation that are pointing to http://apidocs.loopback.io and change them to point to the new API doc location. Another place to change is the top header/navbar shown by https://loopbac.io/doc:

I am not asking to make those changes right now, but we should have a good understanding of what else needs to be done and a list of follow-up tasks to make those change happen.

The purpose to have reports and reports-temp is to detect API changes (api reports are tracked by git). See https://api-extractor.com/pages/setup/configure_api_report/.

+1 to detect API changes.

However, I don't understand why we need to directories for that? Isn't it enough to use git diff? The doc page you provided does not explain change detection, or at least I did not find any explanation.

I'm debating between making @loopback/tsdocs a public module or a just an internal utility. The @loopback/tsdocs can be applied to loopback-labs too as loopback-labs mirrors loopback-next. Let's keep it private for now.

Fair enough.

How to preview newly generated apidocs?
Check out ms-apidocs branch. Run npm run build:apidocs from inside tsdocs. Use vscode to open index.md in docs/site/apidocs and preview it.
Please follow the following instructions to try out new apidocs:
(...)

Thank you for the detailed instruction! I'll try to find some time later this week to play with this. In general and at high level, I am strongly in favor of moving to api-extractor, even if it will require few follow-up pull requests to clean things up.

@strongloop/loopback-maintainers I would like to get few more opinions on the proposed change please. Follow the instructions provided by Raymond above, look around the new API docs, look for unusual or suspicious things.

chore(docs): add apidocs markdown files for loopback.io

Is it really necessary to keep these generated markdown files in git? I am concerned about the noise it will add to our pull request, many of which are already way too large.

So far, we are generating API docs files as part of the release process. I would like to preserve that behavior.

I understand that we need the generated markdown files for Jekyll to render the docs. I believe this can be solved by adding a new build step to ./bin/build-docs-site.sh.

bajtos · 2019-05-07T14:08:58Z

chore(docs): add reports from api-extractor to track api changes

I think this pull request would be easier to review if this commit was left out. Since those reports are auto-generated, they will be easy to commit in a follow-up pull request.

raymondfeng · 2019-05-07T14:11:54Z

However, I don't understand why we need to directories for that? Isn't it enough to use git diff? The doc page you provided does not explain change detection, or at least I did not find any explanation.

api-extractor uses reports and reports-temp for the comparison. Detected changes are reported as part of the extraction process.

raymondfeng · 2019-05-07T14:14:06Z

how about the format of links - are there any breaking changes?

The links will be changed to doc/en/lb4/apidocs.*.html.

raymondfeng · 2019-05-07T14:17:30Z

I think this pull request would be easier to review if this commit was left out. Since those reports are auto-generated, they will be easy to commit in a follow-up pull request.

I removed commits for generated docs.

raymondfeng · 2019-05-07T14:23:31Z

@bajtos I'm thinking about the following steps to roll out this feature:

Leave current apidocs (strong-docs, apidocs.loopback.io) as is for now.
Land this PR with updated sidebar and generated md files for apidocs so that loopback.io starts to render new style of apidocs.
Review new pages for apidocs, fix issues and improve the look and feel
Switch to new style of apidocs when we are happy
- Fix links to apidocs.loopback.io
- ...

raymondfeng · 2019-05-07T15:14:26Z

I understand that we need the generated markdown files for Jekyll to render the docs. I believe this can be solved by adding a new build step to ./bin/build-docs-site.sh.

IIUC, ./bin/build-docs-site.sh. checks out loopback.io repo into sandbox for validation of docs site building. It is not a script used by loopback.io to serve our docs.

Please note that index.md needs to be generated within loopback-next to leverage lerna. Patching md files with Jekyll metadata can be done independently.

raymondfeng · 2019-05-07T15:16:53Z

IMO, seeing diffs in generated apidocs md files for a PR is probably a good side effect. It helps reviewers easily find out if there are api changes.

raymondfeng · 2019-05-21T17:58:19Z

Parameter documentation is not rendered either.

I tried on https://microsoft.github.io/tsdoc/ and it turned out the @param has to include -:

@param x - The first input number

Without x, the parameter name becomes '' and there is no matching description.

See microsoft/tsdoc#128

raymondfeng · 2019-05-21T18:25:25Z

See #2933

bajtos · 2019-05-24T13:27:50Z

I just adjusted build:site to generate apidocs before provisioning loopback.io. It should work out of box now

👍

Parameter documentation is not rendered either.
See #2933

Parameter documentation seems ok now.

There is still not documentation for class constructors and their parameters though.

docs/site/sidebars/lb4_sidebar.yml

raymondfeng · 2019-05-24T18:06:12Z

There is still not documentation for class constructors and their parameters though.

It's probably an issue with api-extractor/documenter. I'll investigate. But it should not hold up this PR.

raymondfeng · 2019-05-25T00:17:25Z

Constructor support - microsoft/rushstack#1295

Another PR to make it easier to troubleshoot unknown tags: hhttps://github.com/microsoft/rushstack/pull/1296

The project maintainers don't seem to be very responsive. Based on their disclaimer at https://github.com/microsoft/web-build-tools/tree/master/apps/api-documenter, we might to fork it to make enhancements.

raymondfeng · 2019-05-27T16:04:12Z

As microsoft/rushstack#1295 is merged and released, constructor tsdoc is now supported.

raymondfeng · 2019-05-27T16:09:49Z

@bajtos PTAL.

bajtos

LGTM as the initial implementation to be improved as we learn more about its limitations.

Please get few more approvals from other team members in @strongloop/loopback-maintainers before landing.

As I commented elsewhere, we should find a way how to programmatically check tsdoc comments for conformity with api-extractor format, so that npm test fails if any of the tsdoc comments are not valid. I think that's best left out of this pull request though.

docs/site/sidebars/lb4_sidebar.yml

bajtos · 2019-05-30T08:33:09Z

packages/tsdocs/fixtures/monorepo/packages/pkg1/dist/index.js

+// This file is licensed under the MIT License.
+// License text available at https://opensource.org/licenses/MIT
+Object.defineProperty(exports, '__esModule', {value: true});
+const pkg2_1 = require('pkg2');


Fair enough.

Please capture this information in developer documentation. For example, you can create packages/tsdocs/fixtures/monorepo/README.md to explain what is in this fixture. Ideally, this documentation should also explain the process for updating the fixture files, should we need to do so in the future.

bajtos · 2019-05-30T08:39:10Z

The PR description says Fixes #2507. I disagree with that - the spike is done only after we have a list of follow-up tasks. Let's keep #2507 open even after this PR is landed - I am going to edit the PR description.

bajtos · 2019-05-30T08:40:33Z

@raymondfeng BTW, did you follow all steps listed in Adding a new package?

raymondfeng · 2019-05-30T17:21:19Z

Please capture this information in developer documentation. For example, you can create packages/tsdocs/fixtures/monorepo/README.md to explain what is in this fixture. Ideally, this documentation should also explain the process for updating the fixture files, should we need to do so in the future.

I added README and made it possible to rebuild the monorepo.

emonddr · 2019-05-30T20:18:52Z

Should files with .../dist/... be included in commit?
e.g.
packages/tsdocs/fixtures/monorepo/packages/pkg1/dist/index.d.ts

raymondfeng · 2019-05-30T20:21:19Z

Should files with .../dist/... be included in commit?
e.g.
packages/tsdocs/fixtures/monorepo/packages/pkg1/dist/index.d.ts

This is part of the fixture for a mock-up monorepo to test out apidocs generation. We commit such files to avoid build steps/costs.

jannyHou · 2019-05-30T20:35:40Z

package.json

@@ -41,6 +41,7 @@
    "version": "npm run update-packages && git add greenkeeper.json \"**/package-lock.json\" && npm run update-template-deps && npm run apidocs",
    "outdated": "npm outdated --depth 0 && lerna exec --no-bail \"npm outdated --depth 0\"",
    "apidocs": "node bin/run-lerna run build:apidocs",
+    "tsdocs": "lerna run --scope @loopback/tsdocs build:tsdocs",


do we still need command apidocs at the above line?

We keep existing strong-docs based apidocs for now. The PR will allow new styles to be seen on loopback.io. Once we are happy with the new approach, we'll remove strong-docs.

jannyHou

Cool 🚢 LGTM

I run the preview of website and it looks great. One thing to improve(we can do it in following PRs):

Types like Registry doesn't point to their definitions, which our current doc does, e.g. clicking ApplicationConfig it scrolls to its definition.

raymondfeng requested a review from bajtos as a code owner May 3, 2019 22:17

raymondfeng force-pushed the ms-apidocs branch 2 times, most recently from 2f3f2ad to cfeaf26 Compare May 4, 2019 17:18

raymondfeng mentioned this pull request May 5, 2019

chore: fix invalid tsdoc tags in comments #2836

Merged

7 tasks

raymondfeng force-pushed the ms-apidocs branch 5 times, most recently from 81f2171 to 89e18cc Compare May 6, 2019 03:44

bajtos reviewed May 6, 2019

View reviewed changes

bajtos added feature Internal Tooling Issues related to our tooling and monorepo infrastructore labels May 6, 2019

raymondfeng force-pushed the ms-apidocs branch 3 times, most recently from b61c686 to a91888b Compare May 6, 2019 22:06

raymondfeng force-pushed the ms-apidocs branch 2 times, most recently from 397b119 to 500f76f Compare May 7, 2019 04:34

raymondfeng force-pushed the ms-apidocs branch from 500f76f to 35d1490 Compare May 7, 2019 14:16

raymondfeng force-pushed the ms-apidocs branch from acbc239 to 4e4a400 Compare May 21, 2019 17:10

raymondfeng force-pushed the ms-apidocs branch 2 times, most recently from 5304d47 to 1c1d976 Compare May 22, 2019 18:20

bajtos reviewed May 24, 2019

View reviewed changes

docs/site/sidebars/lb4_sidebar.yml Show resolved Hide resolved

raymondfeng requested a review from bajtos May 25, 2019 04:10

raymondfeng force-pushed the ms-apidocs branch 2 times, most recently from cf643d1 to 0af199b Compare May 27, 2019 06:39

raymondfeng force-pushed the ms-apidocs branch from 0af199b to 8205629 Compare May 28, 2019 16:59

bajtos approved these changes May 30, 2019

View reviewed changes

raymondfeng force-pushed the ms-apidocs branch 2 times, most recently from 9afabaa to 36f4d15 Compare May 30, 2019 16:40

feat(tsdocs): add integration with api-extractor/documenter

431f048

raymondfeng force-pushed the ms-apidocs branch from 36f4d15 to 431f048 Compare May 30, 2019 17:33

emonddr self-requested a review May 30, 2019 19:59

jannyHou reviewed May 30, 2019

View reviewed changes

jannyHou approved these changes May 30, 2019

View reviewed changes

raymondfeng merged commit c8d9572 into master May 30, 2019

raymondfeng deleted the ms-apidocs branch May 30, 2019 22:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tsdocs): add integration with api-extractor/documenter #2834

feat(tsdocs): add integration with api-extractor/documenter #2834

raymondfeng commented May 3, 2019 •

edited

Loading

bajtos left a comment

bajtos May 6, 2019

raymondfeng commented May 6, 2019

raymondfeng commented May 6, 2019 •

edited

Loading

raymondfeng commented May 7, 2019

bajtos commented May 7, 2019

bajtos commented May 7, 2019

bajtos commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 21, 2019 •

edited

Loading

raymondfeng commented May 21, 2019

bajtos commented May 24, 2019

raymondfeng commented May 24, 2019

raymondfeng commented May 25, 2019 •

edited

Loading

raymondfeng commented May 27, 2019

raymondfeng commented May 27, 2019

bajtos left a comment

bajtos May 30, 2019

bajtos commented May 30, 2019 •

edited

Loading

bajtos commented May 30, 2019

raymondfeng commented May 30, 2019 •

edited

Loading

emonddr commented May 30, 2019

raymondfeng commented May 30, 2019

jannyHou May 30, 2019

raymondfeng May 30, 2019

jannyHou left a comment

feat(tsdocs): add integration with api-extractor/documenter #2834

feat(tsdocs): add integration with api-extractor/documenter #2834

Conversation

raymondfeng commented May 3, 2019 • edited Loading

Checklist

bajtos left a comment

Choose a reason for hiding this comment

bajtos May 6, 2019

Choose a reason for hiding this comment

raymondfeng commented May 6, 2019

raymondfeng commented May 6, 2019 • edited Loading

raymondfeng commented May 7, 2019

bajtos commented May 7, 2019

bajtos commented May 7, 2019

bajtos commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 7, 2019

raymondfeng commented May 21, 2019 • edited Loading

raymondfeng commented May 21, 2019

bajtos commented May 24, 2019

raymondfeng commented May 24, 2019

raymondfeng commented May 25, 2019 • edited Loading

raymondfeng commented May 27, 2019

raymondfeng commented May 27, 2019

bajtos left a comment

Choose a reason for hiding this comment

bajtos May 30, 2019

Choose a reason for hiding this comment

bajtos commented May 30, 2019 • edited Loading

bajtos commented May 30, 2019

raymondfeng commented May 30, 2019 • edited Loading

emonddr commented May 30, 2019

raymondfeng commented May 30, 2019

jannyHou May 30, 2019

Choose a reason for hiding this comment

raymondfeng May 30, 2019

Choose a reason for hiding this comment

jannyHou left a comment

Choose a reason for hiding this comment

raymondfeng commented May 3, 2019 •

edited

Loading

raymondfeng commented May 6, 2019 •

edited

Loading

raymondfeng commented May 21, 2019 •

edited

Loading

raymondfeng commented May 25, 2019 •

edited

Loading

bajtos commented May 30, 2019 •

edited

Loading

raymondfeng commented May 30, 2019 •

edited

Loading