Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build cioos ckan off ckan/ckan base docker image #224

Merged
merged 120 commits into from
Apr 25, 2024
Merged

Conversation

fostermh
Copy link
Member

No description provided.

wardi and others added 30 commits October 4, 2022 14:22
package_update performance backport version
…e-perf-backport

Revert "package_update performance backport version"
In the /dataset/<id>/changes_multiple page, if the user selects
the same date in both select controls a 404 is returned.
The case when the second date was more recent than the first one
 is already handled (selecting the most recent version), we just
need to account for the case when both dates are the same.
Fix 404 when selecting the same date in the changes view [2.9]
Allow only a limited number of formats, that can be expanded with
`ckan.upload.user.mimetypes` and `ckan.upload.user.types` (same for
`group` instead of user). Added new tests to cover common formats used
by spammers.
Set `ckan.auth.create_user_via_web = false` as this is the most common
deployment scenario for CKAN, and leaving open the user registration
leads to Spam problems.
…sion

Add defaults to check_ckan_version helper on 2.9
The manual installation instructions for solr is replaced with master branch instructions
…ocker-installation-step-for-solr

Updated readme to include instructions to install solr using docker
…-backport

package update performance fix with resource purge (backport)
Update typo in search-index command (rebuild-fast instead of rebuild_fast)
When running `ckan user token add` you get this output by default:

```
API Token created:
        eyJ0eXAiOiJKV1QiL...
```

This is user-friendly but is not great for integrating in user scripts
as you need to clean up the output a bit, eg:

```
export CKAN_API_TOKEN=$(ckan user token add ckan_user my_token | tail -n 1 | tr -d '\t')
```

The changes suggested add a `--quiet` / `-q` flag that make the command
just output the token directly so you can do:

```
export CKAN_API_TOKEN=$(ckan user token add ckan_user my_token -q)
```
Copy link

Image has been pushed to cioos/ckan

Testing Quick Start

Pull image

sudo docker pull cioos/ckan:DEV_PR224
or
sudo CKAN_TAG=DEV_PR224 docker-compose pull ckan

Remove Home Volume and Restart

sudo docker-compose down
sudo docker volume rm docker_ckan_home
sudo CKAN_TAG=DEV_PR224 docker-compose up -d

for full documentation see TBD

@fostermh fostermh merged commit 753c4f9 into cioos_dev Apr 25, 2024
2 checks passed
@fostermh fostermh deleted the refactor_repo branch April 25, 2024 18:43
fostermh added a commit that referenced this pull request May 23, 2024
* add projects facet

* change projects type yo stringCollection
remove unused solr config
correctly populate projects in the solr index
update solr8 config

* add more font weights to gui and cleanup repeating subfields display

* fix display of fluent fields in subfields

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* update license and match on legacy ids

* update

* Updated submodule contrib/docker/src/cioos-siooc-schema

* add a few fields to the solr index so we can include them in field list (fl) queries

* Updated submodule contrib/docker/src/ckanext-cioos_harvest

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* add ECV field

* update solr schema and entrypoint files

* change responsible org to groups

* Updated submodule contrib/docker/src/ckanext-cioos_harvest

* test resp. org. group and new harvest source features

* Updated submodule contrib/docker/src/cioos-siooc-schema

* Updated submodule contrib/docker/src/ckanext-spatial

* split french and english index fields so that a french specific text parsing can be used

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* update submodules

* merge in latest changes

* populate groups during ckan harvest

* fix bugs

* hide empty facets on group and organization pages

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* adjust indexing to handle french and english separately

* add update instructions

* add file rotation to default log file

* update dev branches of submodules

* add a build css script

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/cioos-siooc-schema

* update dev github actions docker build

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* build cioos ckan off ckan/ckan base docker image (#224)

* [ckan#7119] package_update performance backport version

* Revert "package_update performance backport version"

* Fix 404 when selecting the same date in the changes view

In the /dataset/<id>/changes_multiple page, if the user selects
the same date in both select controls a 404 is returned.
The case when the second date was more recent than the first one
 is already handled (selecting the most recent version), we just
need to account for the case when both dates are the same.

* [i18n] Update 2.9 translation catalogue (pot)

* Restrict user and group/org image upload formats by default

Allow only a limited number of formats, that can be expanded with
`ckan.upload.user.mimetypes` and `ckan.upload.user.types` (same for
`group` instead of user). Added new tests to cover common formats used
by spammers.

* Disable public registration of users by default

Set `ckan.auth.create_user_via_web = false` as this is the most common
deployment scenario for CKAN, and leaving open the user registration
leads to Spam problems.

* Fix tests and lint

* Defaults were missing in backport to check_ckan_version

* [ckan#7119] package_update performance backport version

* [ckan#7119] filter first, then purge

* Fix "_" override

* Updated readme to include instructions to install solr using docker

* Removed duplicate links

* Replaced instructions from master branch

The manual installation instructions for solr is replaced with master branch instructions

* Removed tomcat installation part from readme

* [ckan#7230] Add dev containers / codespaces config for CKAN 2.9

* Make test_user_list_order_by_default locale-independent

* [ckan#7031] prepare_dataset_blueprint: support dataset type

* [ckan#7031] add changes file

* Fix typo in search-index command

Update typo in search-index command (rebuild-fast instead of rebuild_fast)

* Add option to output token value without extra messages

When running `ckan user token add` you get this output by default:

```
API Token created:
        eyJ0eXAiOiJKV1QiL...
```

This is user-friendly but is not great for integrating in user scripts
as you need to clean up the output a bit, eg:

```
export CKAN_API_TOKEN=$(ckan user token add ckan_user my_token | tail -n 1 | tr -d '\t')
```

The changes suggested add a `--quiet` / `-q` flag that make the command
just output the token directly so you can do:

```
export CKAN_API_TOKEN=$(ckan user token add ckan_user my_token -q)
```

* changelog

* Add pillow as dev requirement

* Add command to clear users (by @pdelboca)

Adds new command `ckan clean users` that will scan the system for users with
invalid images and delete both the image and the user from the database.

Invalid images are all images that doesn't contain allowed mimetypes set
in `ckan.upload.user.mimetypes` config option.

* Add entity type to package get_index

* Fix package_update documentation

* Delete ckan/plugins/toolkit.py.orig

* Fixed color contrast in dashboard

* Fix color contrast for "Markdown" link

* Fix color contrast for Danger button

* Color contrast for warning button

* Add config values to test since they are default starting 2.10

* Install version of Pillow compatible with py2

* Skip clean tests on py2

* lint

* Update templates.rst

* Update docs for jwt secrets'

* Merge pull request ckan#7310 from shashigharti/7193-fix-color-contrast-accessibility-in-dashboard-for-v2_9

Fix color contrast in dashboard

* Merge pull request ckan#7308 from shashigharti/7199-fix-color-contrast-accessibility-in-useredit-for-v2_9

Fix color contrast accessibility in user edit page

* [ckan#7110] Enable DateTime to be returned through Actions by @EricSoroos

* [ckan#7351] Document how to add models to SQLAlchemy metadata by @smotornyuk

* Prefix literal

* Update install-from-source.rst

* Do not auto-register prefixed resource blueprint;

* Added changelog file;

* Rename 7374.fix to 7374.feature

* Load the `resource` blueprint after all other package blueprints;

* Update changelog for 2.9.8

* Rebuild frontend

* Update .tx/config file format

To be used with the new version of the Transifex client:

https://github.com/transifex/cli

Obtained running the ``tx migrate`` command (and manually adding the
existing comments)

* [i18n] Pull po files from Transifex before 2.9.8

* [i18n] Compile mo files

* Update version number for 2.9.8

* Update version number for 2.9.9b

* [doc] Fix versions in banners and package names

The old sorting was string based, so 2.10 broke things. Also the
packages were showing the latest stable version, not the current one

* [doc] Fix versions in banners and package names

The old sorting was string based, so 2.10 broke things. Also the
packages were showing the latest stable version, not the current one

* [doc] We only support 2 versions

* [doc] We only support 2 versions

* Fix version name function

* Remove outdated documentation for upgrading postgres

* Add note about target branches in contributing docs

* [ckan#7508] Escape names in emails

Fix ckan#7508 by quoting names in To/From headers

* Fix test

* Use `email.utils.formataddr` from stdlib to format name/address pairs

* Add changelog

* Tighten checks around incoming resource ids

* Remove Python 2 tests, use Debian buster image

* Update version for 2.9.9

* Fix openjdk package version

* Use py 3.9

* Use dockerized solr in tests

* Don't use Resource factories when creating new Datasets

The Resource factory creates an actual Resource model linked to an
automatically created dataset. If trying to create a Dataset factory
passing a resource one, it will fail as the id already exists

* Use proper solr url

* pep8

* Fix another resource factory

* [ckan#7602] Mention extra requirement in migration notes for 2.10.1

* Update solr.rst

* Add RTD conf file

* Use py3 in docs builds

* Upgrade pyyaml to avoid AttributeError: cython_sources error

* remove setuptools req on docs

* Build on py 3.9

* remove core ckan code

* fix package version

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* switch to using core ckan docker image as base

* remove unneeded lines from dockerfile

* move config settings into .env file

* add  some local backup files to git ignore

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* update submodules and add longer delays on health checks

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* Updated submodule contrib/docker/src/ckanext-spatial

* update to using apline ckan base image

* update environment template file

---------

Co-authored-by: Ian Ward <[email protected]>
Co-authored-by: Sergey <[email protected]>
Co-authored-by: amercader <[email protected]>
Co-authored-by: Brett Jones <[email protected]>
Co-authored-by: Jari Voutilainen <[email protected]>
Co-authored-by: avdata99 <[email protected]>
Co-authored-by: shashi gharti <[email protected]>
Co-authored-by: Francesco Frassinelli <[email protected]>
Co-authored-by: Patricio Del Boca <[email protected]>
Co-authored-by: Evert Ramos <[email protected]>
Co-authored-by: Jari Voutilainen <[email protected]>
Co-authored-by: Rukshan Ranatunge <[email protected]>
Co-authored-by: Sergey Motornyuk <[email protected]>
Co-authored-by: Konstantin Sivakov <[email protected]>
Co-authored-by: JVickery-TBS <[email protected]>
Co-authored-by: Jesse Vickery <[email protected]>
Co-authored-by: Ben Soroos <[email protected]>
Co-authored-by: Brett <[email protected]>

* Updated submodule contrib/docker/src/ckanext-spatial

* update config, maybe this will make the github build work?

* limit flask version to 2.3.3 or less 3.0.3 breaks the import of _request_ctx_stack as it is removed in v3

* fix flask debug toolbar version to fix the rest

* fix default config missing key

* Updated submodule contrib/docker/src/ckanext-spatial

* use ckan base startup script.

* remove unneeded config settings as they are set in the .env now

* Updated submodule contrib/docker/src/ckanext-spatial

* update the docs. WIP

* reformat plugins list so it is replaced on ckan start correctly

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* remove hardcoded site url from ckan build process

* Updated submodule contrib/docker/src/ckanext-spatial

* Updated submodule contrib/docker/src/ckanext-spatial

* fix related works and add lineage

* Updated submodule contrib/docker/src/ckanext-cioos_theme

* add to update instructions

* update instructions again

* clean up instructions and config variables a little. remove secrets from production.ini so that they are read from .env file

* fix sitemap generation and bump docker-compose version

* update to production versions of submodules

---------

Co-authored-by: Ian Ward <[email protected]>
Co-authored-by: Sergey <[email protected]>
Co-authored-by: amercader <[email protected]>
Co-authored-by: Brett Jones <[email protected]>
Co-authored-by: Jari Voutilainen <[email protected]>
Co-authored-by: avdata99 <[email protected]>
Co-authored-by: shashi gharti <[email protected]>
Co-authored-by: Francesco Frassinelli <[email protected]>
Co-authored-by: Patricio Del Boca <[email protected]>
Co-authored-by: Evert Ramos <[email protected]>
Co-authored-by: Jari Voutilainen <[email protected]>
Co-authored-by: Rukshan Ranatunge <[email protected]>
Co-authored-by: Sergey Motornyuk <[email protected]>
Co-authored-by: Konstantin Sivakov <[email protected]>
Co-authored-by: JVickery-TBS <[email protected]>
Co-authored-by: Jesse Vickery <[email protected]>
Co-authored-by: Ben Soroos <[email protected]>
Co-authored-by: Brett <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.