Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add changeset for content layer experimental release #11644

Merged
merged 4 commits into from
Aug 13, 2024

Conversation

ascorbic
Copy link
Contributor

@ascorbic ascorbic commented Aug 7, 2024

Changes

Adds the changeset docs for the experimental content layer release

Testing

Docs

Copy link

changeset-bot bot commented Aug 7, 2024

⚠️ No Changeset found

Latest commit: 5e4b703

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions bot added the semver: minor Change triggers a `minor` release label Aug 7, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is blocked because it contains a minor changeset. A reviewer will merge this at the next release if approved.

@sarah11918
Copy link
Member

Hi @ascorbic ! Great to see this take shape!

This is a bit of a deviation from our usual process, which is fine, but just want to mention that usually we'd get the JSDoc that ends up in docs all figured out first, and then we pull from that for the changeset. The reason for this is that the changeset is often a subset of the docs we'll post (because it doesn't have to list every single thing), and to make sure everything is consistent between both changeset and docs, it helps to have the larger content written first. (Use the same examples, same phrases etc.)

Can we maybe consider this what you'd be proposing for the docs, and we can negotiate "full docs" in here first, then use that to add to the implementation PR and whittle down the changeset from that?

@ascorbic
Copy link
Contributor Author

ascorbic commented Aug 7, 2024

@sarah11918 I did it like this because there isn't a flag, so there's no JSDoc for it. Is there somewhere else it could go?

@sarah11918
Copy link
Member

@ascorbic Then this is really a departure from how we usually release experimental features! 😅

There's no flag to enable? Is there a particular reason we're doing it this way? (Might help me figure out where we're going to put docs, because usually they'd be with the documentation to enable the flag.)

@ascorbic
Copy link
Contributor Author

ascorbic commented Aug 7, 2024

@sarah11918 the initial reason was because the content config file doesn't have access to the astro config, so we used a type instead. However off the back of this we've had a discussion and worked out a way to implement it with a flag so I'll do that instead.

@sarah11918
Copy link
Member

@ascorbic Not gonna lie, that makes it simpler for docs since we already have an established procedure! (Not that I wouldn't have been able to come up with something, of course, but it would have meant some potentially conflicting info on the content collections page itself -- do this, but if you're using experimental, do this.)

And from our stats we know that only around 3% of people have ANY of experimental flags enabled, so if we can avoid adding complexity in the main docs when we don't have to, it just makes for cleaner docs.

As for this PR, maybe we can add the @docs content in the types file here too, then we can hash out all the docs (config for the flag, changeset) together in this one PR?

Typically they'll be pretty close to each other, but the docs should be minimally complete (for an experimental feature), then we refer people to the RFC link for discussion, input, and to keep up to the minute with it. The changeset can be taken from that, but can also be a bit more hype to introduce the feature, and you might choose to not be as doc-y there.

@ascorbic
Copy link
Contributor Author

ascorbic commented Aug 7, 2024

The changeset here is very much focussed on how to migrate and try it out. Would that same format be appropriate for the flag docs, or should it be approached more as a standalone thing ("here's how to use it" vs "here's how to try it with your current content")

@sarah11918
Copy link
Member

I think both are appropriate! Ideally, it should be enough for someone to (understand how to) get started trying it out.

We're pretty flexible on whatever feels helpful to someone wanting to experiment with a feature. You can see the others here: https://docs.astro.build/en/reference/configuration-reference/#experimental-flags
(actions, server islands, and env were all similarly hyped features, so they'd be the best comparisons)

I think showing the migration makes perfect sense in this case, since a LOT of people will already be using collections. I do also see this feature opening up the feature to those who use e.g. remote content, and so for whom using existing content collections was a non-starter. But now, they want to try this version of collections. (I see this sentiment a lot, so we can't just assume everyone will already be using collections!)

Maybe a structure like:

  • intro/description/hype
  • how to enable it
  • how to USE this new iteration of collections (pointing out WHICH of the existing content collections docs may be appropriate? e.g. "You get all the benefits of the existing content collections feature, with the same helpful schema/APIs with a few key differences. Additionally, you have the option to specify a source loader... "
  • how to upgrade an existing collection (which focuses on the differences)
  • See the RFC for discussion/more info standard line.

I think that could all be fair game here!

Copy link
Member

@sarah11918 sarah11918 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this! Just two thoughts that I think might be nice tiny additions that give a good bang for your buck! Do with them what you will, but otherwise, this looks perfect!

})
```

You can then create collections in your `src/content/config.ts` using the Content Layer API.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can then create collections in your `src/content/config.ts` using the Content Layer API.
You can then create collections in your `src/content/config.ts` the Content Layer API in addition to continuing to use the existing content collections API. You can configure on a per-collection basis, and your schema may contain both kinds of collections at once.

Because this is abridged, we don't show the "migrate to / from" nor do we mention that you can keep your existing collections if you don't want to move them all over immediately. So I think frontloading this idea makes sense here for more confident adoption!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included a similar sentence in the first paragraph, so I'm not sure this is needed too.

.changeset/smooth-chicken-wash.md Outdated Show resolved Hide resolved
@ascorbic ascorbic merged commit f1d8538 into content-layer Aug 13, 2024
2 checks passed
@ascorbic ascorbic deleted the cl-changeset branch August 13, 2024 17:00
ascorbic added a commit that referenced this pull request Aug 14, 2024
* Empty commit

* Changeset

* feat: add Content Layer loader (#11334)

* wip

* wip

* wip

* Update demo

* Add meta

* wip

* Add file loader

* Add schema validation

* Remove log

* Changeset

* Format

* Lockfile

* Fix type

* Handle loading for data store JSON

* Use rollup util to import JSON

* Fix types

* Format

* Add tests

* Changes from review

* fix: sync content layer in dev (#11365)

* wip

* wip

* wip

* Update demo

* Add meta

* wip

* Add file loader

* Add schema validation

* Remove log

* Changeset

* Format

* Lockfile

* Fix type

* Handle loading for data store JSON

* Use rollup util to import JSON

* Fix types

* Format

* Add tests

* Changes from review

* Sync content layer in dev

* feat: add typegen for loaders (#11358)

* fix: watch for content layer changes (#11371)

* fix: watch for content layer changes

* Add test

* feat: adds simple loader (#11386)

* wip

* Add simple loader

* Fix type guard

* Tighten loader schema

* Add loader function to type

* Reinstall vitest

* feat: add glob loader (#11398)

* feat: add glob loader

* Enable watching and fix paths

* Store the full entry object, not just data

* Add generateId support

* Fix test

* Rename loaders to sync

* Refacctor imports

* Use getEntry

* Format

* Fix import

* Remove type from output

* Windows path

* Add test for absolute path

* Update lockfile

* Debugging windows

* Allow file URL for base dir

* Reset time limit

* feat: add markdown rendering to content layer (#11440)

* feat: add glob loader

* Enable watching and fix paths

* Store the full entry object, not just data

* Add generateId support

* Fix test

* Rename loaders to sync

* Refacctor imports

* Use getEntry

* Format

* Fix import

* Remove type from output

* Windows path

* Add test for absolute path

* Update lockfile

* Debugging windows

* Allow file URL for base dir

* Reset time limit

* wip: add markdown rendering to content layer

* use cached entries

* CLean up types

* Instrument more of the build

* Add digest helper

* Add comments

* Make image extraction work

* feat: image support for content layer (#11469)

* wip

* wip

* Add image to benchmark

* Stub assets if missing

* Resolve assets in data

* Ignore virtual module

* Format

* rm log

* Handle images when using cached data

* Fix CCC

* Add a comment

* Changes from review

* Format

* Use relative paths for asset files

* Pass all md props to getImage

* Ensure dotastro dir exists

* Fix tests

* Changes from review

* Don't use temp array in getcollection

* Add error handling

* Format

* Handle paths that are already relative

* Dedupe sync runs

* Fix syncing in dev

* Changes from review

* Windows paths ftw

* feat(content-layer): support references in content layer (#11494)

* Support references in content layer

* Fix utf8 rendering

* Warn for invalid entries

* Fix test

* lol windows paths

* Remove assertion

* chore: fix content layer types (#11527)

* Add experimental_content type

* Fix import

* Make data store methods generic

* fix loader types

* Lockfile

* Clean content layer with `--force` (#11541)

* Clearn content layer with `--force`

* Add tests

* Document --force flag

* Fixes to content layer render types (#11558)

* Lockfile

* feat: use devalue to serialize content layer data (#11562)

* feat: use devalue to serialize content layer data

* Fix import

* Use devalue stringify

* Unused import

* Propagate error messages correctly

* Support --force flag in sync and dev (#11581)

* Support --force flag in sync and dev

* Fix test

* Separate render function and merge content layer types (#11579)

* Separate render function and merge content layer types

* Changes from review

* fix: clear content layer cache if config has changed (#11591)

* fix: clear content layer cache if config has changed

* Add test

* Watch config

* Change from review

* fix: skip glob files in content dir (#11622)

* fix: skip glob files in content dir

* Changes from review

* Log pattern

* Refactor content layer into shared instance (#11625)

* Refactor content layer into shared instance

* Clean up when testing

* Handle cleanup

* fix: support filters in content layer getCollection (#11631)

* Throw when using deprecated getEntryByX functions with content layer (#11637)

* Updates to content layer types and jsdocs (#11643)

* Add hot key to reload content layer (#11626)

* Add hot key to reload content layer

* Fix filename

* Remove cli message

* Update example

* Change key to "s"

* feat: handle simple mdx rendering (#11633)

* feat: handle simple mdx rendering

* cleanup

* feedback

* fix regression

* remove log

* flip condition

* update tests

* log collections to understand the error

* let's try this alternative

* try parallel test to understand the issue

* chore: use a new fixture to fix tests

* rebase and docs

* fix regressions

* remove old code

* address feedback

* rename param

* log error

* rebase

* chore: try a different cache dir to solve the error test

* fix invalidation of the module when there's no store available

* address suggestion

* run formatter

* update lock file

* Lint

* Add experimental content layer flag (#11652)

* Add experimental content layer flag

* Syntax and format

* Aside

* Format

* Reset content config between runs

* Update fixture

* Update terminology

* Lint

* wut

* Normalize render function return value (#11663)

* Add markdoc support to content layer (#11664)

* Add markdoc support to content layer

* Switch test to cheerio

* Update benchmarks

* update lock file

* Update content layer flag docs (#11682)

* Update content layer flag docs

* Apply suggestions from code review

Co-authored-by: Sarah Rainsberger <[email protected]>

* More markdoc

* Typo

* Apply suggestions from code review

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update

---------

Co-authored-by: Sarah Rainsberger <[email protected]>

* Add changeset for content layer experimental release (#11644)

* Add changeset for content layer experimental release

* Update changeset

* Update .changeset/smooth-chicken-wash.md

Co-authored-by: Sarah Rainsberger <[email protected]>

---------

Co-authored-by: Sarah Rainsberger <[email protected]>

* feat: injectTypes (#11551)

* feat: make inline config 1st arg

* fix: run config done in sync

* feat: start working on injectTypes

* feat: write files

* feat: adapt core features

* feat: migrate db to injectTypes

* feat: special db handling

* feat: update settings instead of workarounds

* fix: create dotAstroDir

* feat: refactor sync tests

* fix: path

* fix: paths

* chore: add comments

* feat: overwrite content file if exists

* chore: remove unused db env related code

* feat: use dotAstroDir for settings

* chore: simplify astro env sync

* feat: use dotAstroDir for preferences

* feat: handle db in integration api

* chore: reorganize

* feat: format

* feat: add test

* Discard changes to examples/basics/astro.config.mjs

* Discard changes to examples/basics/package.json

* Discard changes to pnpm-lock.yaml

* chore: remove test files

* feat: update examples dts

* fix: dts

* chore: changesets

* fix: indentation

* Apply suggestions from code review

Co-authored-by: Chris Swithinbank <[email protected]>

* Apply suggestions from code review

Co-authored-by: Chris Swithinbank <[email protected]>

* chore: format

* Update packages/astro/src/integrations/hooks.ts

* Update .changeset/mean-horses-kiss.md

* feat: remove formatting

* feat: handle fs errors

* feat: remove astro:db special path handling

* feat: add fs error

* Update packages/astro/src/content/types-generator.ts

* Update .changeset/mean-horses-kiss.md

* Update errors-data.ts

* Update .changeset/mean-horses-kiss.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update .changeset/mean-horses-kiss.md

Co-authored-by: Sarah Rainsberger <[email protected]>

---------

Co-authored-by: Chris Swithinbank <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Erika <[email protected]>

* Add file generation and flag for content intellisense (#11639)

* feat: add type to infer input type of collection

* refactor:

* feat: generate json schema for content too

* feat: generate a manifest of all the collections

* refactor: unnecessary type

* fix: only add content collections to manifest

* chore: changeset

* fix: generate file URLs

* fix: flag it properly

* fix: save in lower case

* docs: add jsdoc to experimental option

* nit: move function out

* fix: match vscode flag name

* Update packages/astro/src/@types/astro.ts

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update packages/astro/src/@types/astro.ts

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update serious-pumas-run.md

* test: add tests

* Add content layer support

* Apply suggestions from code review

* fix: test

* Update .changeset/serious-pumas-run.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Apply suggestions from code review

* Remove check for json

---------

Co-authored-by: Matt Kane <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>

* nit: use same filesystem error as injectTypes

* fix: code component was missing support for meta string (#11605)

* fix: code component was missing support for meta string

Fixed #11604

* Create odd-buttons-pay.md

* <Code>: add reference link for meta prop

* Apply suggestions from code review

Co-authored-by: Sarah Rainsberger <[email protected]>

* Update .changeset/odd-buttons-pay.md

* Update .changeset/odd-buttons-pay.md

Co-authored-by: Sarah Rainsberger <[email protected]>

---------

Co-authored-by: Erika <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Emanuele Stoppa <[email protected]>

* Deprecates exporting prerender with dynamic values (#11657)

* wip

* done i think

* Add changeset

* Use hook instead

* Reorder hooks [skip ci]

* Update .changeset/eleven-pens-glow.md

Co-authored-by: Sarah Rainsberger <[email protected]>

* Fix run

* Fix link

* Add link

Co-authored-by: Sarah Rainsberger <[email protected]>

* More accurate migration [skip ci]

---------

Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>

* Use node parseArgs instead of yargs-parser and arg (#11645)

* wip

* done

* Add changeset

* Format

* Update

* Fix houston

* Fix test

* Fix test

* [ci] format

* resolve conflict

---------

Co-authored-by: Emanuele Stoppa <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Florian Lefebvre <[email protected]>
Co-authored-by: Chris Swithinbank <[email protected]>
Co-authored-by: Erika <[email protected]>
Co-authored-by: Julien Cayzac <[email protected]>
Co-authored-by: Bjorn Lu <[email protected]>
Co-authored-by: Sarah Rainsberger <[email protected]>
Co-authored-by: Bjorn Lu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: minor Change triggers a `minor` release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants