Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@actions/artifact package #304

Merged
merged 48 commits into from
Feb 11, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
3a20113
Initial commit for @actions/artifact
Jan 13, 2020
776270f
Alphabetize imports
Jan 13, 2020
f3ac746
Fix majority of lint errors
Jan 13, 2020
bd38a3f
Fix all lint errors
Jan 14, 2020
b2d67bb
Various fixes due to PR feedback
Jan 14, 2020
029f2a4
Add tests for search
Jan 15, 2020
4fda766
Rename test file
Jan 15, 2020
e025ef0
Test improvements
Jan 15, 2020
11c006c
Misc formatting fixes
Jan 15, 2020
4431cd8
Tests for artifactName
konradpabjan Jan 15, 2020
7cf5770
Misc improvements
konradpabjan Jan 28, 2020
b56cdb0
Hopefully fix failing tests
konradpabjan Jan 28, 2020
6b04ca5
🤞 this works
konradpabjan Jan 28, 2020
e12553e
Hmmm
konradpabjan Jan 28, 2020
83c8848
Really weird...
konradpabjan Jan 28, 2020
b6f59a0
Update tests for util.ts
konradpabjan Jan 28, 2020
c6d23c0
Succesfull http mocking along with more tests
konradpabjan Jan 30, 2020
da72c73
Spelling fixes and misc updates
Jan 30, 2020
206233a
Logic for fast failing in the event a chunk fails
Jan 30, 2020
e6f5591
Tests for artifact uploads
Jan 31, 2020
7f48237
Minor fixes
Jan 31, 2020
ddfdb57
PR Feedback
Jan 31, 2020
0768705
Format fix
Jan 31, 2020
31ff9e6
Fixes error with unbound methods being used in tests
konradpabjan Feb 1, 2020
c5216d2
.
konradpabjan Feb 1, 2020
a375c57
Remove directory deletion after tests finish
konradpabjan Feb 3, 2020
c71237b
Remove project GUID from API call
Feb 3, 2020
0cc46af
Misc Improvements
Feb 3, 2020
6f6aac1
Code cleanup
Feb 3, 2020
cd5d31a
Extra debug
Feb 3, 2020
09b1c39
Move glob search out of the package
Feb 4, 2020
049110f
Code cleanup
Feb 4, 2020
abd642c
Rename some thing and cleanup
Feb 4, 2020
813bcf1
Rename uploadInfo to uploadResponse
Feb 6, 2020
c8d2dd1
Misc Improvements
Feb 7, 2020
1dd4152
Format Fix
Feb 7, 2020
1fa883d
Test Improvements
Feb 7, 2020
7c6e6ac
Updates to continueOnError
Feb 7, 2020
6b6a5e0
Extra test
Feb 7, 2020
4ac6ef6
Normalize and Resolve all paths
Feb 7, 2020
d01f0f3
Change console.log to warning
Feb 7, 2020
7917754
Update upload-response.ts
konradpabjan Feb 10, 2020
21ce5cb
PR feedback
konradpabjan Feb 10, 2020
78605ae
Merge branch 'konradpabjan/actions/artifact' of https://github.com/ac…
konradpabjan Feb 10, 2020
fcec6a7
Update upload-http-client.ts
konradpabjan Feb 10, 2020
5288533
Silence output when running tests and rename files with internal
konradpabjan Feb 10, 2020
8a1220b
Use Factory to create ArtifactClient
konradpabjan Feb 10, 2020
332dfce
Merge branch 'konradpabjan/actions/artifact' of https://github.com/ac…
konradpabjan Feb 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added packages/artifact/README.md
Empty file.
54 changes: 54 additions & 0 deletions packages/artifact/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

43 changes: 43 additions & 0 deletions packages/artifact/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"name": "@actions/artifact",
"version": "0.1.0",
"preview": true,
"description": "Actions artifact lib",
"keywords": [
"github",
"actions",
"artifact"
],
"homepage": "https://github.com/actions/toolkit/tree/master/packages/artifact",
"license": "MIT",
"main": "lib/artifact.js",
"types": "lib/artifact.d.ts",
"directories": {
"lib": "lib",
"test": "__tests__"
},
"files": [
"lib"
],
"publishConfig": {
"access": "public"
},
"repository": {
"type": "git",
"url": "git+https://github.com/actions/toolkit.git",
"directory": "packages/artifact"
},
"scripts": {
"audit-moderate": "npm install && npm audit --audit-level=moderate",
"test": "echo \"Error: no test specified\" && exit 1",
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
"tsc": "tsc"
},
"bugs": {
"url": "https://github.com/actions/toolkit/issues"
},
"dependencies": {
"@actions/core": "^1.2.1",
"@actions/glob": "^0.1.0",
"@actions/http-client": "^1.0.1"
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
}
}
108 changes: 108 additions & 0 deletions packages/artifact/src/artifact.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import * as core from '@actions/core'
import {CreateArtifactResponse} from './contracts'
import {SearchResult, findFilesToUpload} from './search'
import {
createArtifactInFileContainer,
uploadArtifactToFileContainer,
patchArtifactSize
} from './upload-artifact-http-client'
import {UploadInfo} from './upload-info'
import {checkArtifactName} from './utils'

/**
* Uploads an artifact
*
* @param name the name of the artifact, required
* @param path the directory, file, or glob pattern to denote what will be uploaded, required
* @returns single UploadInfo object
*/
export async function uploadArtifact(
name: string,
path: string
): Promise<UploadInfo> {
// Check that the required inputs are valid
if (!name) {
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
throw new Error('Artifact name must be provided')
}

checkArtifactName(name)

if (!path) {
throw new Error('Upload path must be provided')
}

// Search for the items that will be uploaded
const filesToUpload: SearchResult[] = await findFilesToUpload(name, path)
let reportedSize = -1

if (filesToUpload === undefined) {
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
core.setFailed(
`Unable to succesfully search fo files to upload with the provided path: ${path}`
)
} else if (filesToUpload.length === 0) {
core.warning(
`No files were found for the provided path: ${path}. No artifacts will be uploaded.`
)
} else {
/**
* Step 1 of 3
* Create an entry for the artifact in the file container
*/
const response: CreateArtifactResponse = await createArtifactInFileContainer(
name
)
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
if (!response.fileContainerResourceUrl) {
console.log(response)
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
throw new Error('Unable to get fileContainerResourceUrl')
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
}
core.debug(`We will be uploading to: ${response.fileContainerResourceUrl}`)
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved

/**
* Step 2 of 3
* Upload each of the files that were found concurrently
*/
const uploadingArtifact: Promise<number> = Promise.resolve(
uploadArtifactToFileContainer(
response.fileContainerResourceUrl,
filesToUpload
)
)
uploadingArtifact.then(async size => {
console.log(`Size of what we just uploaded is ${size}`)
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
/**
* Step 3 of 3
* Update the size of the artifact to indicate we are done uploading
*/
await patchArtifactSize(size, name)
reportedSize = size
})
}

return {
artifactName: name,
artifactItems: filesToUpload.map(item => item.absoluteFilePath),
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
size: reportedSize
}
}

/*
Downloads a single artifact associated with a run

export async function downloadArtifact(
name: string,
path?: string,
createArtifactFolder?:boolean
): Promise<DownloadInfo> {

TODO
}

Downloads all artifacts associated with a run. Because there are multiple artifacts being downloaded, a folder will be created for each one in the specified or default directory

export async function downloadAllArtifacts(
path?: string
): Promise<DownloadInfo[]>{

TODO
}
*/
28 changes: 28 additions & 0 deletions packages/artifact/src/contracts.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
export interface CreateArtifactResponse {
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
containerId?: string
size?: number
signedContent?: string
fileContainerResourceUrl?: string
type?: string
name?: string
url?: string
}

export interface CreateArtifactParameters {
Type: string
Name: string
}

export interface PatchArtifactSize {
Size: number
}

export interface PatchArtifactSizeSuccessResponse {
containerId: number
size: number
signedContent: string
type: string
name: string
url: string
uploadUrl: string
}
11 changes: 11 additions & 0 deletions packages/artifact/src/download-info.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export interface DownloadInfo {
/**
* The name of the artifact that was downloaded
*/
artifactName: string

/**
* The full Path to where the artifact was downloaded
*/
downloadPath: string
}
96 changes: 96 additions & 0 deletions packages/artifact/src/search.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
import {debug} from '@actions/core'
import * as glob from '@actions/glob'
import {lstatSync} from 'fs'
import {join, basename} from 'path'

export interface SearchResult {
absoluteFilePath: string
uploadFilePath: string
}

/**
* Searches the provided path and returns the files that will be uploaded as part of the artifact
* @param {string} searchPath Wildcard pattern, directory or individual file that is provided by the user to specify what should be uploaded
* @param {string} artifactName The name of the artifact, used as the root directory when uploading artifacts to glob storage
* @return A list of files that should be uploaded along with the paths they will be uploaded with
*/
export async function findFilesToUpload(
artifactName: string,
searchPath: string
): Promise<SearchResult[]> {
const searchResults: SearchResult[] = []
const itemsToUpload: string[] = []
const options: glob.GlobOptions = {
konradpabjan marked this conversation as resolved.
Show resolved Hide resolved
followSymbolicLinks: true,
implicitDescendants: true,
omitBrokenSymbolicLinks: true
}
const globber = await glob.create(searchPath, options)

const rawSearchResults: string[] = await globber.glob()
/**
* Directories will be rejected if attempted to be uploaded. This includes just empty
* directories so filter any directories out from the raw search results
*/
for (const searchResult of rawSearchResults) {
if (!lstatSync(searchResult).isDirectory()) {
itemsToUpload.push(searchResult)
} else {
debug(
`Removing ${searchResult} from rawSearchResults because it is a directory`
)
}
}

if (itemsToUpload.length === 0) {
return searchResults
}
console.log(
`Found the following ${itemsToUpload.length} items that will be uploaded as part of the artifact:`
)
console.log(itemsToUpload)

/**
* Only a single search pattern is being included so only 1 searchResult is expected. In the future if multiple search patterns are
* simultaneously supported this will change
*/
const searchPaths: string[] = globber.getSearchPaths()
if (searchResults.length > 1) {
console.log(searchResults)
throw new Error('Only 1 search path should be returned')
}

/**
* Creates the path that the artifact will be uploaded with. The artifact name will always be the root directory so that
* it can be distinguished from other artifacts that are uploaded to the same file Container/glob storage during a run
*/
if (itemsToUpload.length === 1) {
// A single artifact will be uploaded, the upload path will always be in the form ${artifactName}\${singleArtifactName}
searchResults.push({
absoluteFilePath: itemsToUpload[0],
uploadFilePath: join(artifactName, basename(searchPaths[0]))
})
} else {
/**
* multiple files will be uploaded as part of the artifact
* The search path denotes the base path that was used to find the file. It will be removed from the absolute path and
* the artifact name will be prepended to create the path used during upload
*/
for (const uploadItem of itemsToUpload) {
const uploadPath: string = uploadItem.replace(searchPaths[0], '')
searchResults.push({
absoluteFilePath: uploadItem,
uploadFilePath: artifactName.concat(uploadPath)
})
}
}

debug('SearchResult includes the following information:')
for (const searchResult of searchResults) {
debug(
`Absolute File Path: ${searchResult.absoluteFilePath}\nUpload file path: ${searchResult.uploadFilePath}`
)
}

return searchResults
}
Loading