-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny This PR adds a Debian package registry. You can follow [this tutorial](https://www.baeldung.com/linux/create-debian-package) to build a *.deb package for testing. Source packages are not supported at the moment and I did not find documentation of the architecture "all" and how these packages should be treated.  Part of #20751. Revised copy of #22854. --------- Co-authored-by: Brian Hong <[email protected]> Co-authored-by: techknowlogick <[email protected]> Co-authored-by: Giteabot <[email protected]>
- Loading branch information
4 people
authored
May 2, 2023
1 parent
1f52560
commit bf999e4
Showing
57 changed files
with
2,008 additions
and
96 deletions.
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| --- | ||
| date: "2023-01-07T00:00:00+00:00" | ||
| title: "Debian Packages Repository" | ||
| slug: "packages/debian" | ||
| draft: false | ||
| toc: false | ||
| menu: | ||
| sidebar: | ||
| parent: "packages" | ||
| name: "Debian" | ||
| weight: 35 | ||
| identifier: "debian" | ||
| --- | ||
|
|
||
| # Debian Packages Repository | ||
|
|
||
| Publish [Debian](https://www.debian.org/distrib/packages) packages for your user or organization. | ||
|
|
||
| **Table of Contents** | ||
|
|
||
| {{< toc >}} | ||
|
|
||
| ## Requirements | ||
|
|
||
| To work with the Debian registry, you need to use a HTTP client like `curl` to upload and a package manager like `apt` to consume packages. | ||
|
|
||
| The following examples use `apt`. | ||
|
|
||
| ## Configuring the package registry | ||
|
|
||
| To register the Debian registry add the url to the list of known apt sources: | ||
|
|
||
| ```shell | ||
| echo "deb https://gitea.example.com/api/packages/{owner}/debian {distribution} {component}" | sudo tee -a /etc/apt/sources.list.d/gitea.list | ||
| ``` | ||
|
|
||
| | Placeholder | Description | | ||
| | -------------- | ----------- | | ||
| | `owner` | The owner of the package. | | ||
| | `distribution` | The distribution to use. | | ||
| | `component` | The component to use. | | ||
|
|
||
| If the registry is private, provide credentials in the url. You can use a password or a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}): | ||
|
|
||
| ```shell | ||
| echo "deb https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/debian {distribution} {component}" | sudo tee -a /etc/apt/sources.list.d/gitea.list | ||
| ``` | ||
|
|
||
| The Debian registry files are signed with a PGP key which must be known to apt: | ||
|
|
||
| ```shell | ||
| sudo curl https://gitea.example.com/api/packages/{owner}/debian/repository.key -o /etc/apt/trusted.gpg.d/gitea-{owner}.asc | ||
| ``` | ||
|
|
||
| Afterwards update the local package index: | ||
|
|
||
| ```shell | ||
| apt update | ||
| ``` | ||
|
|
||
| ## Publish a package | ||
|
|
||
| To publish a Debian package (`*.deb`), perform a HTTP `PUT` operation with the package content in the request body. | ||
|
|
||
| ``` | ||
| PUT https://gitea.example.com/api/packages/{owner}/debian/pool/{distribution}/{component}/upload | ||
| ``` | ||
|
|
||
| | Parameter | Description | | ||
| | -------------- | ----------- | | ||
| | `owner` | The owner of the package. | | ||
| | `distribution` | The distribution may match the release name of the OS, ex: `bionic`. | | ||
| | `component` | The component can be used to group packages or just `main` or similar. | | ||
|
|
||
| Example request using HTTP Basic authentication: | ||
|
|
||
| ```shell | ||
| curl --user your_username:your_password_or_token \ | ||
| --upload-file path/to/file.deb \ | ||
| https://gitea.example.com/api/packages/testuser/debian/pool/bionic/main/upload | ||
| ``` | ||
|
|
||
| If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password. | ||
| You cannot publish a file with the same name twice to a package. You must delete the existing package version first. | ||
|
|
||
| The server reponds with the following HTTP Status codes. | ||
|
|
||
| | HTTP Status Code | Meaning | | ||
| | ----------------- | ------- | | ||
| | `201 Created` | The package has been published. | | ||
| | `400 Bad Request` | The package name, version, distribution, component or architecture are invalid. | | ||
| | `409 Conflict` | A package file with the same combination of parameters exists already. | | ||
|
|
||
| ## Delete a package | ||
|
|
||
| To delete a Debian package perform a HTTP `DELETE` operation. This will delete the package version too if there is no file left. | ||
|
|
||
| ``` | ||
| DELETE https://gitea.example.com/api/packages/{owner}/debian/pool/{distribution}/{component}/{package_name}/{package_version}/{architecture} | ||
| ``` | ||
|
|
||
| | Parameter | Description | | ||
| | ----------------- | ----------- | | ||
| | `owner` | The owner of the package. | | ||
| | `package_name` | The package name. | | ||
| | `package_version` | The package version. | | ||
| | `distribution` | The package distribution. | | ||
| | `component` | The package component. | | ||
| | `architecture` | The package architecture. | | ||
|
|
||
| Example request using HTTP Basic authentication: | ||
|
|
||
| ```shell | ||
| curl --user your_username:your_token_or_password -X DELETE \ | ||
| https://gitea.example.com/api/packages/testuser/debian/pools/bionic/main/test-package/1.0.0/amd64 | ||
| ``` | ||
|
|
||
| The server reponds with the following HTTP Status codes. | ||
|
|
||
| | HTTP Status Code | Meaning | | ||
| | ----------------- | ------- | | ||
| | `204 No Content` | Success | | ||
| | `404 Not Found` | The package or file was not found. | | ||
|
|
||
| ## Install a package | ||
|
|
||
| To install a package from the Debian registry, execute the following commands: | ||
|
|
||
| ```shell | ||
| # use latest version | ||
| apt install {package_name} | ||
| # use specific version | ||
| apt install {package_name}={package_version} | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| // Copyright 2023 The Gitea Authors. All rights reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| package v1_20 //nolint | ||
|
|
||
| import ( | ||
| "xorm.io/xorm" | ||
| ) | ||
|
|
||
| func AddIsInternalColumnToPackage(x *xorm.Engine) error { | ||
| type Package struct { | ||
| ID int64 `xorm:"pk autoincr"` | ||
| OwnerID int64 `xorm:"UNIQUE(s) INDEX NOT NULL"` | ||
| RepoID int64 `xorm:"INDEX"` | ||
| Type string `xorm:"UNIQUE(s) INDEX NOT NULL"` | ||
| Name string `xorm:"NOT NULL"` | ||
| LowerName string `xorm:"UNIQUE(s) INDEX NOT NULL"` | ||
| SemverCompatible bool `xorm:"NOT NULL DEFAULT false"` | ||
| IsInternal bool `xorm:"NOT NULL DEFAULT false"` | ||
| } | ||
|
|
||
| return x.Sync(new(Package)) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| // Copyright 2023 The Gitea Authors. All rights reserved. | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| package debian | ||
|
|
||
| import ( | ||
| "context" | ||
| "strconv" | ||
|
|
||
| "code.gitea.io/gitea/models/db" | ||
| "code.gitea.io/gitea/models/packages" | ||
| debian_module "code.gitea.io/gitea/modules/packages/debian" | ||
|
|
||
| "xorm.io/builder" | ||
| ) | ||
|
|
||
| type PackageSearchOptions struct { | ||
| OwnerID int64 | ||
| Distribution string | ||
| Component string | ||
| Architecture string | ||
| } | ||
|
|
||
| // SearchLatestPackages gets the latest packages matching the search options | ||
| func SearchLatestPackages(ctx context.Context, opts *PackageSearchOptions) ([]*packages.PackageFileDescriptor, error) { | ||
| var cond builder.Cond = builder.Eq{ | ||
| "package_file.is_lead": true, | ||
| "package.type": packages.TypeDebian, | ||
| "package.owner_id": opts.OwnerID, | ||
| "package.is_internal": false, | ||
| "package_version.is_internal": false, | ||
| } | ||
|
|
||
| props := make(map[string]string) | ||
| if opts.Distribution != "" { | ||
| props[debian_module.PropertyDistribution] = opts.Distribution | ||
| } | ||
| if opts.Component != "" { | ||
| props[debian_module.PropertyComponent] = opts.Component | ||
| } | ||
| if opts.Architecture != "" { | ||
| props[debian_module.PropertyArchitecture] = opts.Architecture | ||
| } | ||
|
|
||
| if len(props) > 0 { | ||
| var propsCond builder.Cond = builder.Eq{ | ||
| "package_property.ref_type": packages.PropertyTypeFile, | ||
| } | ||
| propsCond = propsCond.And(builder.Expr("package_property.ref_id = package_file.id")) | ||
|
|
||
| propsCondBlock := builder.NewCond() | ||
| for name, value := range props { | ||
| propsCondBlock = propsCondBlock.Or(builder.Eq{ | ||
| "package_property.name": name, | ||
| "package_property.value": value, | ||
| }) | ||
| } | ||
| propsCond = propsCond.And(propsCondBlock) | ||
|
|
||
| cond = cond.And(builder.Eq{ | ||
| strconv.Itoa(len(props)): builder.Select("COUNT(*)").Where(propsCond).From("package_property"), | ||
| }) | ||
| } | ||
|
|
||
| cond = cond. | ||
| And(builder.Expr("pv2.id IS NULL")) | ||
|
|
||
| joinCond := builder. | ||
| Expr("package_version.package_id = pv2.package_id AND (package_version.created_unix < pv2.created_unix OR (package_version.created_unix = pv2.created_unix AND package_version.id < pv2.id))"). | ||
| And(builder.Eq{"pv2.is_internal": false}) | ||
|
|
||
| pfs := make([]*packages.PackageFile, 0, 10) | ||
| err := db.GetEngine(ctx). | ||
| Table("package_file"). | ||
| Select("package_file.*"). | ||
| Join("INNER", "package_version", "package_version.id = package_file.version_id"). | ||
| Join("LEFT", "package_version pv2", joinCond). | ||
| Join("INNER", "package", "package.id = package_version.package_id"). | ||
| Where(cond). | ||
| Desc("package_version.created_unix"). | ||
| Find(&pfs) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return packages.GetPackageFileDescriptors(ctx, pfs) | ||
| } | ||
|
|
||
| // GetDistributions gets all available distributions | ||
| func GetDistributions(ctx context.Context, ownerID int64) ([]string, error) { | ||
| return getDistinctPropertyValues(ctx, ownerID, "", debian_module.PropertyDistribution) | ||
| } | ||
|
|
||
| // GetComponents gets all available components for the given distribution | ||
| func GetComponents(ctx context.Context, ownerID int64, distribution string) ([]string, error) { | ||
| return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyComponent) | ||
| } | ||
|
|
||
| // GetArchitectures gets all available architectures for the given distribution | ||
| func GetArchitectures(ctx context.Context, ownerID int64, distribution string) ([]string, error) { | ||
| return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyArchitecture) | ||
| } | ||
|
|
||
| func getDistinctPropertyValues(ctx context.Context, ownerID int64, distribution, propName string) ([]string, error) { | ||
| var cond builder.Cond = builder.Eq{ | ||
| "package_property.ref_type": packages.PropertyTypeFile, | ||
| "package_property.name": propName, | ||
| "package.type": packages.TypeDebian, | ||
| "package.owner_id": ownerID, | ||
| } | ||
| if distribution != "" { | ||
| innerCond := builder. | ||
| Expr("pp.ref_id = package_property.ref_id"). | ||
| And(builder.Eq{ | ||
| "pp.ref_type": packages.PropertyTypeFile, | ||
| "pp.name": debian_module.PropertyDistribution, | ||
| "pp.value": distribution, | ||
| }) | ||
| cond = cond.And(builder.Exists(builder.Select("pp.ref_id").From("package_property pp").Where(innerCond))) | ||
| } | ||
|
|
||
| values := make([]string, 0, 5) | ||
| return values, db.GetEngine(ctx). | ||
| Table("package_property"). | ||
| Distinct("package_property.value"). | ||
| Join("INNER", "package_file", "package_file.id = package_property.ref_id"). | ||
| Join("INNER", "package_version", "package_version.id = package_file.version_id"). | ||
| Join("INNER", "package", "package.id = package_version.package_id"). | ||
| Where(cond). | ||
| Find(&values) | ||
| } |
Oops, something went wrong.