Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

publish snapshot releases on every merge to main #438

Merged
merged 5 commits into from
Mar 15, 2024
Merged

Conversation

finn-block
Copy link
Contributor

@finn-block finn-block commented Mar 7, 2024

This is part of a project being undertaken by OSP to help new code make it's way to TBD's downstream projects faster. It creates a snapshot release for each merge to main, which is published to our Artifactory* registry. Once this is done, downstream projects (ie. tbdex-js and developer.tbd.website) can pull in new web5-js features before a real release has been produced by depending on the snapshot version.

It creates snapshot release for any packages that have changed since the last snapshot version. It does this by counting the number of commits in each package since the last time the package.json version was changed, and producing a snapshot release name in the format <version>-SNAPSHOT.<commit count> (for example, there have been 55 commits that touched packages/api since it was bumped to 0.8.4, so it will produce snapshot version 0.8.4-SNAPSHOT.55). EDIT: this format has changed slightly to include the sha of the last commit as well, see thread. It checks if a snapshot with this version has already been published, and if not it builds and publishes the snapshot release.

Once that has been published, someone writing documentation for developer.tbd.website can set the example to depend on @web5/api version 0.8.4-SNAPSHOT.55 and write sample code that can be validated against that snapshot.

*Artifactory: behaves like npmjs.com, but isn't real npmjs.com so the high number of releases this creates will not spam up our npmjs.com page.

Copy link
Contributor

github-actions bot commented Mar 7, 2024

TBDocs Report

🛑 Errors: 0
⚠️ Warnings: 130

@web5/api

  • Project entry file: packages/api/src/index.ts
📄 File: ./packages/api/src/protocol.ts
⚠️ extractor:typedoc:missing-reference: ProtocolMetadata is referenced by Protocol._metadata but not included in the documentation.
⚠️ extractor:typedoc:missing-docs: Protocol._agent (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Protocol._metadata (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Protocol._protocolsConfigureMessage (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Protocol.send.__type.status (Property) does not have any documentation.
📄 File: ./packages/api/src/send-cache.ts
⚠️ extractor:typedoc:missing-reference: SendCache is referenced by Record._sendCache but not included in the documentation.
📄 File: ./packages/api/src/web5.ts
⚠️ extractor:typedoc:missing-reference: Web5Options is referenced by Web5.constructor.new Web5.options but not included in the documentation.
⚠️ extractor:typedoc:missing-docs: TechPreviewOptions.__type.dwnEndpoints (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Web5ConnectOptions.__type.connectedDid (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Web5.connect.__type.web5 (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Web5.connect.__type.did (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Web5.connectedDid (Property) does not have any documentation.
📄 File: ./packages/api/src/utils.ts
⚠️ extractor:typedoc:missing-docs: utils.dataToBlob.__type.dataBlob (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: utils.dataToBlob.__type.dataFormat (Property) does not have any documentation.
📄 File: ./packages/api/src/did-api.ts
⚠️ extractor:typedoc:missing-docs: DidApi.agent (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: DidApi.connectedDid (Property) does not have any documentation.
📄 File: ./packages/api/src/dwn-api.ts
⚠️ extractor:typedoc:missing-docs: ResponseStatus.__type.status (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ResponseStatus.__type.status.__type.code (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ResponseStatus.__type.status.__type.detail (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ProtocolsConfigureRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ProtocolsConfigureResponse.__type.protocol (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ProtocolsQueryReplyEntry.__type.descriptor (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ProtocolsQueryRequest.__type.from (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ProtocolsQueryRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ProtocolsQueryResponse.__type.protocols (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsCreateFromRequest.__type.author (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsCreateFromRequest.__type.data (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsCreateFromRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsCreateFromRequest.__type.record (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsDeleteRequest.__type.from (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsDeleteRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsQueryRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsQueryResponse.__type.records (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsReadRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsReadResponse.__type.record (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsWriteRequest.__type.data (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsWriteRequest.__type.message (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsWriteRequest.__type.store (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordsWriteResponse.__type.record (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: DwnApi.agent (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: DwnApi.connectedDid (Property) does not have any documentation.
📄 File: ./packages/api/src/record.ts
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.author (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.connectedDid (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.encodedData (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.data (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.initialWrite (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.protocolRole (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordOptions.__type.remoteOrigin (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordModel.__type.author (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordModel.__type.protocolRole (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordModel.__type.recordId (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.data (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.dataCid (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.dataSize (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.dateModified (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.datePublished (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.published (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: RecordUpdateOptions.__type.protocolRole (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._sendCache (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._agent (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._connectedDid (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._encodedData (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._readableStream (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._remoteOrigin (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._author (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._attestation (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._authorization (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._contextId (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._descriptor (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._encryption (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._initialWrite (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._initialWriteStored (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._initialWriteSigned (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._recordId (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record._protocolRole (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record.data.__type.then (CallSignature) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record.data.__type.catch (CallSignature) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Record.processRecord (CallSignature) does not have any documentation.
📄 File: ./packages/api/src/vc-api.ts
⚠️ extractor:typedoc:missing-docs: VcApi.agent (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VcApi.connectedDid (Property) does not have any documentation.
🔀 Misc.
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/agent#Web5Agent" in comment for Web5ConnectOptions.__type.agent. You may have wanted "@web5/agent!Web5Agent"
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/user-agent#Web5UserAgent" in comment for Web5ConnectOptions.__type.agent. You may have wanted "@web5/user-agent!Web5UserAgent"
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/agent#AppDataStore" in comment for Web5ConnectOptions.__type.appData. You may have wanted "@web5/agent!AppDataStore"
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/agent#AppDataStore" in comment for Web5ConnectOptions.__type.appData. You may have wanted "@web5/agent!AppDataStore"
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/agent#Web5Agent" in comment for Web5.connect. You may have wanted "@web5/agent!Web5Agent"
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/user-agent#Web5UserAgent" in comment for Web5.connect. You may have wanted "@web5/user-agent!Web5UserAgent"

@web5/crypto

  • Project entry file: packages/crypto/src/index.ts

@web5/crypto-aws-kms

  • Project entry file: packages/crypto-aws-kms/src/index.ts
📄 File: ./packages/crypto-aws-kms/src/ecdsa.ts
⚠️ extractor:typedoc:missing-docs: EcdsaAlgorithm (Class) does not have any documentation.
📄 File: ./packages/crypto-aws-kms/src/key-manager.ts
⚠️ extractor:typedoc:missing-docs: AwsKeyManager (Class) does not have any documentation.
📄 File: ./packages/crypto-aws-kms/src/utils.ts
⚠️ extractor:typedoc:missing-docs: getKeySpec (CallSignature) does not have any documentation.
🔀 Misc.
⚠️ extractor:typedoc:invalid-link: Failed to resolve link to "@web5/crypto#Secp256k1.adjustSignatureToLowS | here" in comment for EcdsaAlgorithm.sign. You may have wanted "@web5/crypto!Secp256k1.adjustSignatureToLowS | here"

@web5/dids

  • Project entry file: packages/dids/src/index.ts

@web5/credentials

  • Project entry file: packages/credentials/src/index.ts
📄 File: ./packages/credentials/src/jwt.ts
⚠️ extractor:typedoc:missing-docs: JwtParseResult.__type.decoded (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: JwtParseResult.__type.encoded (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: JwtParseResult.__type.encoded.__type.header (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: JwtParseResult.__type.encoded.__type.payload (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: JwtParseResult.__type.encoded.__type.signature (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: ParseJwtOptions.__type.jwt (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: SignJwtOptions.__type.signerDid (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: SignJwtOptions.__type.payload (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifyJwtOptions.__type.jwt (Property) does not have any documentation.
📄 File: ./packages/credentials/src/presentation-exchange.ts
⚠️ extractor:typedoc:missing-docs: PresentationDefinitionV2 (Interface) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: Validated (TypeAlias) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: PresentationExchange (Class) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: PresentationExchange.resetPex (CallSignature) does not have any documentation.
📄 File: ./packages/credentials/src/verifiable-credential.ts
⚠️ extractor:typedoc:missing-docs: DEFAULT_VC_CONTEXT (Variable) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: DEFAULT_VC_TYPE (Variable) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialCreateOptions.__type.type (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialCreateOptions.__type.issuer (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialCreateOptions.__type.subject (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialCreateOptions.__type.data (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialCreateOptions.__type.issuanceDate (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialCreateOptions.__type.expirationDate (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredentialSignOptions.__type.did (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredential.verify.__type.issuer (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredential.verify.__type.subject (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredential.verify.__type.vc (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredential.type (GetSignature) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredential.issuer (GetSignature) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiableCredential.subject (GetSignature) does not have any documentation.
📄 File: ./packages/credentials/src/verifiable-presentation.ts
⚠️ extractor:typedoc:missing-docs: DEFAULT_VP_TYPE (Variable) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentationCreateOptions.__type.holder (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentationCreateOptions.__type.vcJwts (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentationCreateOptions.__type.type (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentationCreateOptions.__type.additionalData (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentationSignOptions.__type.did (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentation.verify.__type.issuer (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentation.verify.__type.subject (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentation.verify.__type.vc (Property) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentation.type (GetSignature) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentation.holder (GetSignature) does not have any documentation.
⚠️ extractor:typedoc:missing-docs: VerifiablePresentation.verifiableCredential (GetSignature) does not have any documentation.

TBDocs Report Updated at 2024-03-15T17:09:50Z 515842a

Copy link

codecov bot commented Mar 7, 2024

Codecov Report

Merging #438 (515842a) into main (7442465) will not change coverage.
Report is 1 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #438   +/-   ##
=======================================
  Coverage   93.35%   93.35%           
=======================================
  Files          80       80           
  Lines       23839    23839           
  Branches     1891     1891           
=======================================
  Hits        22255    22255           
  Misses       1544     1544           
  Partials       40       40           
Components Coverage Δ
agent 88.72% <ø> (ø)
api 97.30% <ø> (ø)
common 98.68% <ø> (ø)
credentials 94.77% <ø> (ø)
crypto 93.81% <ø> (ø)
dids 97.51% <ø> (ø)
identity-agent 56.81% <ø> (ø)
crypto-aws-kms 100.00% <ø> (ø)
proxy-agent 58.43% <ø> (ø)
user-agent 55.22% <ø> (ø)

@finn-block finn-block changed the title [WIP] publish snapshot releases on every merge to main publish snapshot releases on every merge to main Mar 8, 2024
@finn-block finn-block marked this pull request as ready for review March 8, 2024 22:43
@finn-block finn-block requested a review from leordev March 8, 2024 22:44
Copy link
Contributor

@leordev leordev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! I think I might be able to simplify things after merging #416 (the next publication could publish to jfrog too! and changesets is smart enough to do the diff logic)

@frankhinek
Copy link
Contributor

@finn-tbd Is it possible to publish the short SHA in the version <version>-SNAPSHOT.<short-SHA> rather than commit count?

If no, how does a consumer of the published release know which commit it is associated with? I'm imagining a scenario where a downstream consumer reports they ran into an issue/bug with 0.8.4-SNAPSHOT.55. During troubleshooting, how would we determine which commit in the main branch this release was cut from? Is the only way to manually look up the commit for 0.8.4 and then count 55 commits later?

@finn-block
Copy link
Contributor Author

It's possible, but that would break npm's versioning scheme. in particular, it would make it unable to determine which is the latest snapshot. If that's something we're super concerned about, we could combine the two approaches (ie. <version>-SNAPSHOT.<count>-<sha>) and probably have comparable versions, but i'm 100% sure how npm's version comparing works. Should I implement the combined approach?

@frankhinek
Copy link
Contributor

It's possible, but that would break npm's versioning scheme. in particular, it would make it unable to determine which is the latest snapshot.

Can you elaborate on the concern?

Based on npm publish command in this PR it appears the desired outcome is that if someone runs npm install @web5/<package-name> --registry https://blockxyz.jfrog.io/artifactory/api/npm/tbd-oss-snapshots-npm/ without specifying a specific version that latest will be the most recent commit to main?

Do we really want latest to be the latest commit to main or should latest be the most recent version that went through the normal release process?

@finn-block
Copy link
Contributor Author

Based on npm publish command in this PR it appears the desired outcome is that if someone runs npm install @web5/ --registry https://blockxyz.jfrog.io/artifactory/api/npm/tbd-oss-snapshots-npm/ without specifying a specific version that latest will be the most recent commit to main?

correct

Do we really want latest to be the latest commit to main or should latest be the most recent version that went through the normal release process?

I would expect the latest version in the snapshot repository to be the latest snapshot. If we just used the SHA and had no count, it would still install a snapshot release, it just wouldn't be the latest one. I'm not 100% sure how it would pick but i think it would be the (alphabetically) highest SHA, so closest to fffffff.

i've updated it to include the SHA in the snapshot version, so version numbers are computed like this:

snapshot_version="${base_version}-SNAPSHOT.${commits_since_version_bump}-${last_commit_to_package:0:7}"

for example, @web5/crypto's latest snapshot number is 0.4.0-SNAPSHOT.97-7442465

with:
cache: "true"

- uses: jfrog/setup-jfrog-cli@v4
Copy link
Contributor

@frankhinek frankhinek Mar 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the security tools added by OSP raise security issues as a result of unpinned actions, please use the hash and include a comment with which version it is.

Otherwise looks good to ✅ and merge

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops, thanks for catching that. Also pinned cashapp/activate-hermit above

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries. thanks! nice work @finn-tbd

@leordev leordev requested a review from frankhinek March 15, 2024 19:15
@finn-block finn-block merged commit c2359ee into main Mar 15, 2024
31 checks passed
@finn-block finn-block deleted the snapshot-releases branch March 15, 2024 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants