diff --git a/CHANGES.md b/CHANGES.md index 392b58d4d..e61675a72 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,3 +1,9 @@ +# Unreleased + +## New Features + +- Added HTTPS support in Docker Compose + Enabled bind-mount volume for Nginx config + add documentation in README.md (PR #1653) + # 6.5.0 ## New Features diff --git a/pytest.ini b/pytest.ini index 202452a91..f648e49e9 100644 --- a/pytest.ini +++ b/pytest.ini @@ -2,3 +2,4 @@ log_cli_level = DEBUG log_level = DEBUG asyncio_mode=strict +norecursedirs = src/bandersnatch_docker_compose diff --git a/src/bandersnatch_docker_compose/Dockerfile b/src/bandersnatch_docker_compose/Dockerfile index 103e71366..d37809b1f 120000 --- a/src/bandersnatch_docker_compose/Dockerfile +++ b/src/bandersnatch_docker_compose/Dockerfile @@ -1 +1 @@ -../banderx/Dockerfile \ No newline at end of file +../banderx/Dockerfile diff --git a/src/bandersnatch_docker_compose/README.md b/src/bandersnatch_docker_compose/README.md index 6c88f5952..83f9de0b1 100644 --- a/src/bandersnatch_docker_compose/README.md +++ b/src/bandersnatch_docker_compose/README.md @@ -1,5 +1,25 @@ # Bandersnatch with docker compose +# Table of Contents + +[Introduction](#introduction) + +[Preparation](#preparation) + +[Pull the Docker Image](#pull-the-docker-image) + +[Run with Docker Compose](#run-with-docker-compose-v2) + +[Watching Live Logs](#watching-live-logs) + +[(Optional) - Enabling HTTPS Support](#optional---enabling-https-support) + +[Removing the Repository](#removing-the-repository) + +[Caveats](#caveats) + +## Introduction + Similar to [banderx](https://github.com/pypa/bandersnatch/tree/main/src/banderx) but with docker compose. ## Preparation @@ -33,24 +53,105 @@ docker compose logs -f bandersnatch docker compose logs -f bandersnatch_nginx ``` +### (Optional) - Enabling HTTPS Support + +The `bandersnatch_docker_compose` setup now includes optional HTTPS support for the Nginx server. + +To enable HTTPS: + +- **Uncomment the HTTPS sections**: + +In `docker-compose.yml` related to SSL certificate and key volumes, as well as the exposed HTTPS port. + +- **Provide SSL Certificates**: + +Place your SSL certificate and key files in the `src/banderx` directory and name them `certificate.crt` and `private.key`, respectively. + +- Ensure that these files are not publicly accessible. + +- In case you need just a Self-Signed certificate just use the guide [here](https://github.com/pypa/bandersnatch/blob/main/src/banderx/README.md) + +- **Uncomment the HTTPS sections in `nginx.conf`**: + +In the `nginx.conf` file located in the `banderx` directory, uncomment the server block for HTTPS and the server block for redirecting HTTP to HTTPS. + +- **Rebuild and Restart the Containers**: + +After making these changes, rebuild and restart your Docker containers. + +```bash +docker-compose down +docker-compose up --build -d +``` + +#### Integrating with Docker Compose + +```yaml +services: + bandersnatch_nginx: + # ... other configurations ... + volumes: + # ... Other volunes ... + - "../banderx/certificate.crt:/etc/ssl/certs/nginx.crt:ro" # SSL certificate + - "../banderx/private.key:/etc/ssl/private/nginx.key:ro" # SSL key +``` + +#### Test HTTPS Connection + +Ensure that your Nginx server is correctly serving content over HTTPS and access it using https. + +##### Using a Web Browser + +- Open your web browser. + +- Navigate to `https://pypi-repo-domain>:44300`. + +- You should see your site served over HTTPS. + +> Note - Your browser may display a security warning if you are using a self-signed certificate. + +##### Using `curl` Command + +- Open a terminal or command prompt. + +- Run the following command: + + ```bash + curl -vk https://pypi-repo-domain:44300 + ``` + + - Replace `` with your server's domain name or IP address. + + - The `-k` option allows `curl` to perform "insecure" SSL connections and transfers, useful if you're using a self-signed certificate. + +- If the Nginx server is correctly serving content over HTTPS, you should see the HTML content of your website in the terminal output. + +> Note +> +> - Ensure that the port `44300` is open and accessible from your network. +> +> - If you are using a self-signed certificate, web browsers and tools like `curl` may show a warning because the certificate is not signed by a recognized Certificate Authority. +> +> - This is expected behavior for self-signed certificates. For a production environment, it's recommended to use a certificate from a recognized Certificate Authority. + ## Removing the Repository To remove the Bandersnatch repository that you've set up using Docker Compose, follow these steps. Please be aware that this process will delete all the packages and configuration files you have downloaded or created. Ensure you have backups if necessary. -1. **Stop the Docker Containers**: Before removing any files, it's important to stop the running Docker containers to prevent any file corruption or data loss. Use the command: +- **Stop the Docker Containers**: Before removing any files, it's important to stop the running Docker containers to prevent any file corruption or data loss. Use the command: ```bash docker compose down ``` -2. **Remove the Packages and Configuration Files**: To delete all the downloaded packages and configuration files, run the following command. This will remove the `packages` and entire `config` directory directory inside your `config` folder, which contains all the mirrored Python packages. +- **Remove the Packages and Configuration Files**: To delete all the downloaded packages and configuration files, run the following command. This will remove the `packages` and entire `config` directory directory inside your `config` folder, which contains all the mirrored Python packages. ```bash rm -rf ./data/ rm -rf ./config/ ``` -3. **Clean up Docker Artifacts**: Finally, to remove any Docker volumes, networks, or other artifacts that were created with the Docker Compose file, you can use the following command: +- **Clean up Docker Artifacts**: Finally, to remove any Docker volumes, networks, or other artifacts that were created with the Docker Compose file, you can use the following command: ```bash docker system prune --volumes diff --git a/src/bandersnatch_docker_compose/docker-compose.yml b/src/bandersnatch_docker_compose/docker-compose.yml index 7237edf03..977e73dcb 100644 --- a/src/bandersnatch_docker_compose/docker-compose.yml +++ b/src/bandersnatch_docker_compose/docker-compose.yml @@ -7,22 +7,22 @@ services: - "./data:/srv/pypi/web" # Mount the local data directory where the mirror will be stored # To run bandersnatch in mirror mode. # It will mirror all the packages in the mirror list of the config file. - command: bandersnatch mirror --config-file /conf/bandersnatch.conf + command: bandersnatch --config /conf/bandersnatch.conf mirror # To run bandersnatch in --force mode, uncomment the following line. # It will delete all packages that are not in the mirror list. - # command: bandersnatch mirror --config-file /conf/bandersnatch.conf --force + # command: bandersnatch --config /conf/bandersnatch.conf --force mirror # To run bandersnatch in --debug mode, uncomment the following line. # It will print out all the debug messages. This is useful for debugging. - # command: bandersnatch mirror --config-file /conf/bandersnatch.conf --debug + # command: bandersnatch --config /conf/bandersnatch.conf --debug mirror # To run bandersnatch in --verbose mode, uncomment the following line. # It will print out all the verbose messages. This is useful for debugging. - # command: bandersnatch mirror --config-file /conf/bandersnatch.conf --verbose + # command: bandersnatch --config /conf/bandersnatch.conf --verbose mirror # To run bandersnatch in verify mode, uncomment the following line. # It will verify all the packages in the mirror list and delete the ones that are not in the list anymore. - # command: bandersnatch verify --config-file /conf/bandersnatch.conf + # command: bandersnatch --config /conf/bandersnatch.conf verify # To run bandersnatch in once mode, uncomment the following line. # It will run bandersnatch once and exit. This is useful for testing. - # command: bandersnatch once --config-file /conf/bandersnatch.conf + # command: bandersnatch --config ./conf/bandersnatch.conf once networks: - bandersnatch_net # Connects the service to the custom bridge network @@ -30,11 +30,19 @@ services: bandersnatch_nginx: image: bandersnatch_nginx # If the image not built yet, it will be built from the nginx image from the Dockerfile in the current directory - build: . + build: + context: ../banderx + dockerfile: Dockerfile volumes: - - "./data:/data/pypi/web:ro" # Mount the data directory as read-only for nginx + - "./data:/data/pypi/web:ro" # Mount the data directory as read-only for nginx for serving the downloaded repository + - "../banderx/nginx.conf:/config/nginx.conf:ro" # Mount custom nginx.conf as recommended in ../banderx/README.md + # Uncomment for HTTPS support + #- "../banderx/certificate.crt:/etc/ssl/certs/nginx.crt:ro" # Uncomment for HTTPS support + #- "../banderx/private.key:/etc/ssl/private/nginx.key:ro" # Uncomment for HTTPS support ports: - "40080:80" # Map port 40080 on the host to port 80 in the container + # Uncomment for HTTPS support + #- "44300:443" # Uncomment for expose port 44300 on the host and map it to port 443 in the container networks: - bandersnatch_net diff --git a/src/bandersnatch_docker_compose/nginx.conf b/src/bandersnatch_docker_compose/nginx.conf index 5f9fb2403..db61c0bf0 120000 --- a/src/bandersnatch_docker_compose/nginx.conf +++ b/src/bandersnatch_docker_compose/nginx.conf @@ -1 +1 @@ -../banderx/nginx.conf \ No newline at end of file +../banderx/nginx.conf diff --git a/src/banderx/README.md b/src/banderx/README.md index 62b1c59d0..bbd7f34cb 100644 --- a/src/banderx/README.md +++ b/src/banderx/README.md @@ -11,3 +11,37 @@ Simple nginx PEP 501 + 691 mirror filesystem serving example. ## Run - `docker run --detach --name banderx banderx` + +## (Optional) - Generate a Self-Signed Certificate for Bandersnatch + +> Note - Use this only for HTTPS support + +- **Generate a 2048-bit RSA Private Key**: + + This key is used to decrypt traffic. Run the following command in PowerShell or Command Prompt (on Windows): + + ```bash + openssl genrsa -out ./src/banderx/private.key 2048 + ``` + + This will create a private key file `key.key`. + +- **Generate a Certificate S igning Request (CSR)**: + + Using the private key from the previous step, generate a CSR: + + ```bash + openssl req -new -key ./src/banderx/private.key -out ./src/banderx/certificate.csr + ``` + + You will be prompted to enter details for the certificate; you can fill these out as needed or leave them blank. + +- **Self-Sign the Certificate**: + + Sign the CSR with the private key, setting the certificate's validity period. For example, for a validity of 365 days: + + ```bash + openssl x509 -req -days 180 -in ./src/banderx/certificate.csr -signkey ./src/banderx/private.key -out ../banderx/certificate.crt + ``` + + This creates a self-signed certificate `cert.crt` in the `banderx` directory. diff --git a/src/banderx/nginx.conf b/src/banderx/nginx.conf index 71f965326..e5d62d520 100644 --- a/src/banderx/nginx.conf +++ b/src/banderx/nginx.conf @@ -35,6 +35,45 @@ http { "~*text/html" ".html"; } + # Uncomment the following lines to enable HTTPS + #server { + # listen 443 ssl; + # server_name banderx; + # ssl_certificate /etc/ssl/certs/nginx.crt; + # ssl_certificate_key /etc/ssl/private/nginx.key; + # root /data/pypi/web; + # autoindex on; + # charset utf-8; + + # location /simple/ { + # index index$mirror_suffix; + # try_files $uri$mirror_suffix $uri $uri/ =404; + # } + + # location /json/ { + # default_type application/json; + # } + + # location /pypi/ { + # default_type application/json; + # } + + # error_page 404 /404.html; + # location = /404.html { + # } + + # error_page 500 502 503 504 /50x.html; + # location = /50x.html { + # } + #} + + # Uncomment the following lines to redirect HTTP to HTTPS + #server { + # listen 80; + # server_name banderx; + # return 301 https://$host$request_uri; + #} + server { listen 80 default_server; listen [::]:80 default_server;