Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

udx-docker-sftp/master -> icamiami-docker-sftp/master #5

Open
wants to merge 76 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
7ffb51c
Merge pull request #6 from udx/latest
planv Apr 6, 2023
5f5a241
Merge pull request #7 from udx/latest
planv Apr 7, 2023
12540eb
Merge pull request #9 from udx/latest
kavaribes Apr 12, 2023
d94e783
Merge pull request #11 from udx/latest
kavaribes Sep 27, 2023
1d2a00e
Merge pull request #12 from udx/latest
kavaribes Sep 27, 2023
20a0015
Merge pull request #13 from udx/latest
kavaribes Sep 27, 2023
c4e5932
Merge pull request #15 from udx/latest
kavaribes Oct 5, 2023
7b73562
Merge pull request #16 from udx/latest
kavaribes Jan 31, 2024
7f16556
Merge pull request #17 from udx/latest
planv Feb 5, 2024
0cad95d
Merge pull request #18 from udx/latest
planv Feb 5, 2024
6580f51
Merge pull request #24 from udx/latest
planv Feb 7, 2024
85c17b0
Merge pull request #26 from udx/latest
planv Feb 7, 2024
46ee680
fixed JS errors that blocked creating keys
planv Mar 20, 2024
f9662c5
added new GitHub Action to release and create SBOM
planv Mar 20, 2024
0b0040d
removed GitHub Action for release, will be improved and then committed
planv Apr 1, 2024
e719e71
Merge pull request #27 from udx/develop-vova
planv Apr 3, 2024
cecc9b5
UDXI-360 version bump, updated Node modules
planv Apr 3, 2024
07510d0
UDXI-360 version bump, updated Node modules
planv Apr 4, 2024
9ac70ea
UDXI-360 version bump, updated Node modules
planv Apr 4, 2024
f0dfeb7
Merge pull request #28 from udx/develop-vova
planv Apr 8, 2024
2e9512c
Merge pull request #29 from udx/latest
planv Apr 19, 2024
12dc662
increased version of NodeJS Image
planv Apr 19, 2024
5e649b7
increased version of NodeJS Image
planv Apr 19, 2024
7a4f398
UDXI-394 enabled Dependabot for Docker, npm, GitHub Actions
planv Aug 20, 2024
a5ba62c
reverted NodeJS version
planv Aug 20, 2024
211caac
Merge pull request #30 from udx/develop-vova
planv Aug 20, 2024
9bfac4d
UDXI-395
planv Aug 20, 2024
69845f0
UDXI-395
planv Aug 20, 2024
783ee2f
version bump
planv Aug 20, 2024
0e9f2e8
pulling latest version of openssh from openbsd
planv Aug 20, 2024
e08aa9b
pulling latest version of openssh from openbsd
planv Aug 20, 2024
e56daac
clean up
planv Aug 20, 2024
3ba9589
removed openssh instalation using apk
planv Aug 21, 2024
fef640e
debug
planv Aug 21, 2024
2e65eed
debug
planv Aug 21, 2024
d7a89bc
debug
planv Aug 21, 2024
47f823d
change sshd path
planv Aug 21, 2024
5605f3f
change sshd path
planv Aug 21, 2024
0500aa5
UDXI-396
planv Aug 21, 2024
d0778d1
UDXI-396
planv Aug 21, 2024
6e72618
UDXI-396
planv Aug 21, 2024
8f312af
UDXI-396
planv Aug 21, 2024
170f68e
UDXI-396
planv Aug 21, 2024
5f7aee3
small fix
planv Aug 21, 2024
6ff3c78
Merge pull request #32 from udx/develop-vova
planv Aug 21, 2024
c4c8e4c
Merge pull request #33 from udx/latest
planv Aug 22, 2024
7d0ccf0
improved github action workflow
planv Aug 22, 2024
4f3b921
Merge pull request #34 from udx/develop-vova
planv Aug 22, 2024
cb53e8c
improved github action workflow
planv Aug 22, 2024
5a3222e
Merge pull request #37 from udx/develop-vova
planv Aug 22, 2024
95d4029
Merge pull request #36 from udx/latest
planv Aug 22, 2024
0840b83
fix github action workflow
planv Aug 28, 2024
1a7325e
fix github action workflow
planv Aug 28, 2024
9a58910
updated parent Docker Image
planv Aug 28, 2024
988d192
updated node module axios to 1.7.5
planv Aug 28, 2024
c7dc96b
version bump
planv Aug 28, 2024
4c6131e
version bump
planv Aug 28, 2024
b4147b0
Merge pull request #39 from udx/develop-vova
planv Aug 29, 2024
be6aad8
* OS updates * added config for a cronjob to scheduled container rest…
kavaribes Nov 12, 2024
1288338
Merge pull request #52 from udx/UDXI-427
kavaribes Nov 12, 2024
ca3a854
update version
kavaribes Nov 13, 2024
58c56e5
Merge pull request #53 from udx/UDXI-427
kavaribes Nov 13, 2024
8c98c86
update modules
kavaribes Nov 13, 2024
c8ddc46
Merge pull request #54 from udx/UDXI-427
kavaribes Nov 13, 2024
4af6cb9
Updated node modules. Updated Alpine to node:23.4-alpine UDXI-427
kavaribes Dec 11, 2024
ba23b57
Merge pull request #56 from udx/UDXI-427
kavaribes Dec 11, 2024
8066c9a
updated openssh UDXI-427
kavaribes Dec 11, 2024
e690b30
Merge pull request #59 from udx/UDXI-427
kavaribes Dec 11, 2024
e3947ca
fix: Improve SSH/SFTP handling and documentation
andypotanin Jan 8, 2025
54009db
fix: Improve SSH config and add logging documentation
andypotanin Jan 8, 2025
630b4ea
docs: Use generic naming in examples
andypotanin Jan 8, 2025
e5e274d
Merge remote-tracking branch 'origin/master' into develop-andy
kavaribes Jan 8, 2025
000e0e8
updated Dockerfile
kavaribes Jan 8, 2025
403ba33
Merge pull request #65 from udx/develop-andy
kavaribes Jan 10, 2025
3c47eed
updated kubectl
kavaribes Jan 10, 2025
27a4626
Merge pull request #71 from udx/develop-ihor
kavaribes Jan 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
Makefile
Makefile

# Ignore generated credentials from google-github-actions/auth
gha-creds-*.json
26 changes: 26 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file

version: 2
updates:
# Enable Dependabot alerts for `Docker`
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"

# Enable Dependabot alerts for `npm`
- package-ecosystem: "npm"
directory: "/"
schedule:
interval: "weekly"

# Enable Dependabot alerts for `GitHub Actions`
- package-ecosystem: "github-actions"
directory: ".github/workflows"
schedule:
interval: "weekly"


119 changes: 108 additions & 11 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# To get a newer version, you will need to update the SHA.
# You can also reference a tag or branch, but the action may change without warning.

name: Build and Deploy to GKE
name: Build, Release and Deploy to GKE

on:
push:
Expand All @@ -25,6 +25,7 @@ env:
SLACK_NOTIFICACTION_CHANNEL: ${{ secrets.SLACK_NOTIFICACTION_CHANNEL }}
AR_LOCATION: ${{ vars.AR_LOCATION }}
AR_REPOSITORY: ${{ vars.AR_REPOSITORY }}
IMAGE_NAME: ${{vars.AR_LOCATION}}-docker.pkg.dev/${{secrets.GKE_PROJECT}}/${{vars.AR_REPOSITORY}}/${{github.ref_name}}

jobs:
setup-build-publish-deploy:
Expand All @@ -33,16 +34,64 @@ jobs:
if: github.event_name == 'push'
environment:
name: ${{ github.ref_name }}
permissions:
contents: write
checks: write
# required for all workflows
security-events: write

steps:
- uses: actions/checkout@v4

# fetch Tag from package.json version
- name: Get Tag from package.json
id: version
run: echo "TAG=$(node -p "require('./package.json').version")" >> $GITHUB_OUTPUT

# echo Tag
- name: Echo Tag
run: echo ${{ steps.version.outputs.TAG }}

# parse the changelog to get the release description
- name: Parse Changelog Entries
uses: actions/github-script@v7
id: changelog
with:
script: |
const { open } = require('fs/promises');

const version ='${{ steps.version.outputs.TAG }}';
const delimiter = '### ';
const file = await open('./changes.md');

let description = [];
let found = false;

for await (let line of file.readLines()) {
line = line.trim();

if ( line.startsWith(`${delimiter}${version}`) ) {
found = true;
continue;
}

if (!found) continue;
if ( line.startsWith(delimiter) ) break;

description.push(line);
}

if ( !description.length ) core.setFailed(`Release ${version} not found in the changelog!`);

core.setOutput('description', description.join('\n') );

# Setup gcloud CLI
- id: 'auth'
uses: 'google-github-actions/auth@v2'
with:
credentials_json: '${{ secrets.GKE_SA_KEY }}'

# Set up Cloud SDK
- name: 'Set up Cloud SDK'
uses: 'google-github-actions/setup-gcloud@v2'
with:
Expand All @@ -52,25 +101,69 @@ jobs:
# helper for authentication
- run: |-
gcloud --quiet auth configure-docker $AR_LOCATION-docker.pkg.dev
# Get the GKE credentials so we can deploy to the cluster
- uses: google-github-actions/get-gke-credentials@v2
with:
cluster_name: ${{ env.GKE_CLUSTER }}
location: ${{ env.GKE_REGION }}
project_id: ${{ secrets.GKE_PROJECT }}

# Build the Docker image
- name: Build
run: |-
docker build \
--tag "$AR_LOCATION-docker.pkg.dev/$PROJECT_ID/$AR_REPOSITORY/$GITHUB_REF_NAME:$GITHUB_SHA" \
--tag "${{env.IMAGE_NAME}}:${{ steps.version.outputs.TAG }}" \
--build-arg GITHUB_SHA="$GITHUB_SHA" \
--build-arg GITHUB_REF="$GITHUB_REF" \
.

# Push the Docker image to Google Container Registry
- name: Publish
run: |-
docker push "$AR_LOCATION-docker.pkg.dev/$PROJECT_ID/$AR_REPOSITORY/$GITHUB_REF_NAME:$GITHUB_SHA"
docker push "${{env.IMAGE_NAME}}:${{ steps.version.outputs.TAG }}"

# Scan Docker image for vulnerabilities
- name: Scan Docker Image using Trivy
if: github.ref_name == 'master'
uses: aquasecurity/trivy-action@master
with:
image-ref: "${{env.IMAGE_NAME}}:${{ steps.version.outputs.TAG }}"
exit-code: '0'
timeout: '60m0s'
format: 'sarif'
output: 'trivy-results.sarif'

# Upload Trivy scan results to GitHub Security tab
- name: Upload Trivy scan results to GitHub Security tab
if: github.ref_name == 'master'
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'trivy-results.sarif'

# Generate SPDX SBOM
- name: Generate SBOM with Anchore Action
if: github.ref_name == 'master'
id: sbom
uses: anchore/sbom-action@v0
with:
image: "${{env.IMAGE_NAME}}:${{ steps.version.outputs.TAG }}"
output-file: sbom.spdx.json
format: spdx-json

# Create a release with the version changelog as a description and attach the sbom
- name: Create GitHub Release
if: github.ref_name == 'master'
id: create_github_release
uses: softprops/action-gh-release@v2
with:
name: "Release ${{ steps.version.outputs.TAG }}"
body: "${{ steps.changelog.outputs.description }}"
tag_name: ${{ steps.version.outputs.TAG }}
prerelease: false
files: |
./sbom.spdx.json

# Get the GKE credentials so we can deploy to the cluster
- uses: google-github-actions/get-gke-credentials@v2
with:
cluster_name: ${{ env.GKE_CLUSTER }}
location: ${{ env.GKE_REGION }}
project_id: ${{ secrets.GKE_PROJECT }}

# Set up kustomize
- name: Set up Kustomize
run: |-
Expand All @@ -81,6 +174,7 @@ jobs:
sed -i.bak "s|CLUSTER_NAME_VALUE|${{ vars.GKE_CLUSTER }}|g" ci/deployment-v2.yml
sed -i.bak "s|CLUSTER_ENDPOINT_VALUE|${{ secrets.KUBERNETES_CLUSTER_ENDPOINT }}|g" ci/deployment-v2.yml
sed -i.bak "s|CLUSTER_NAMESPACE_VALUE|${{ secrets.KUBERNETES_CLUSTER_NAMESPACE }}|g" ci/deployment-v2.yml
sed -i.bak "s|CLUSTER_NAMESPACE_VALUE|${{ secrets.KUBERNETES_CLUSTER_NAMESPACE }}|g" ci/deployment-restart-cronjob.yml
sed -i.bak "s|CLUSTER_USER_TOKEN_VALUE|${{ secrets.KUBERNETES_CLUSTER_USER_TOKEN }}|g" ci/deployment-v2.yml
sed -i.bak "s|CLUSTER_SERVICEACCOUNT_VALUE|${{ secrets.KUBERNETES_CLUSTER_SERVICEACCOUNT }}|g" ci/deployment-v2.yml
sed -i.bak "s|CLUSTER_CERTIFICATE_VALUE|${{ secrets.KUBERNETES_CLUSTER_CERTIFICATE }}|g" ci/deployment-v2.yml
Expand All @@ -89,14 +183,17 @@ jobs:
sed -i.bak "s|ACCESS_TOKEN_VALUE|${{ secrets.ACCESS_TOKEN }}|g" ci/deployment-v2.yml
sed -i.bak "s|SLACK_NOTIFICACTION_URL_VALUE|${{ secrets.SLACK_NOTIFICACTION_URL }}|g" ci/deployment-v2.yml
sed -i.bak "s|SLACK_NOTIFICACTION_CHANNEL_VALUE|${{ secrets.SLACK_NOTIFICACTION_CHANNEL }}|g" ci/deployment-v2.yml
sed -i.bak "s|IMAGE_VERSION|$GITHUB_SHA|g" ci/deployment-v2.yml
sed -i.bak "s|IMAGE_VERSION|${{ steps.version.outputs.TAG }}|g" ci/deployment-v2.yml
sed -i.bak "s|GITHUB_ORG|$GITHUB_REPOSITORY_OWNER|g" ci/service.yml
sed -i.bak "s|GITHUB_ORG|$GITHUB_REPOSITORY_OWNER|g" ci/deployment-v2.yml
sed -i.bak "s|GITHUB_ORG|$GITHUB_REPOSITORY_OWNER|g" ci/deployment-restart-cronjob.yml
sed -i.bak "s|GITHUB_BRANCH|$GITHUB_REF_NAME|g" ci/service.yml
sed -i.bak "s|GITHUB_BRANCH|$GITHUB_REF_NAME|g" ci/deployment-v2.yml
sed -i.bak "s|GITHUB_BRANCH|$GITHUB_REF_NAME|g" ci/deployment-restart-cronjob.yml
sed -i.bak "s|PROJECT_ID|$PROJECT_ID|g" ci/deployment-v2.yml
sed -i.bak "s|AR_LOCATION|$AR_LOCATION|g" ci/deployment-v2.yml
# Deploy the Docker image to the GKE cluster
- run: |
kubectl apply -n ${{ secrets.KUBERNETES_CLUSTER_NAMESPACE }} -f ci/service.yml && \
kubectl apply -n ${{ secrets.KUBERNETES_CLUSTER_NAMESPACE }} -f ci/deployment-v2.yml
kubectl apply -n ${{ secrets.KUBERNETES_CLUSTER_NAMESPACE }} -f ci/deployment-v2.yml && \
kubectl apply -n ${{ secrets.KUBERNETES_CLUSTER_NAMESPACE }} -f ci/deployment-restart-cronjob.yml
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,9 @@ Temporary Items
/newrelic_agent.log
.idea
static/kuberentes-ca.crt
docker-compose.yml
docker-compose.yml

# Ignore generated credentials from google-github-actions/auth
gha-creds-*.json

node_modules
36 changes: 29 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,40 @@
FROM node:20-alpine
ENV VERSION=v1.29.0
FROM node:23.5-alpine
ENV KUBECTL_VERSION=1.32.0
ENV NODE_ENV=production
ENV SERVICE_ENABLE_SSHD=true
ENV SERVICE_ENABLE_API=true
ENV SERVICE_ENABLE_FIREBASE=false

RUN apk update && apk upgrade && apk add bash

RUN apk add --no-cache git openssh nfs-utils rpcbind curl ca-certificates nano tzdata ncurses make tcpdump \
&& curl -L https://storage.googleapis.com/kubernetes-release/release/$VERSION/bin/linux/amd64/kubectl -o /usr/local/bin/kubectl \
RUN apk update --no-cache && apk upgrade --no-cache && apk add bash tar

# Install build dependencies
RUN apk add --no-cache \
build-base \
linux-headers \
openssl-dev \
zlib-dev \
file \
wget

# Download the latest OpenSSH (9.8p1) source
RUN wget https://ftp.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-9.9p1.tar.gz \
&& tar -xzf openssh-9.9p1.tar.gz \
&& cd openssh-9.9p1 \
# Configure and compile the source
&& ./configure \
&& make \
&& make install

# Cleanup build dependencies and unnecessary files
RUN apk del build-base linux-headers openssl-dev zlib-dev file wget \
&& rm -rf /openssh-9.9p1.tar.gz /openssh-9.9p1

RUN apk add --no-cache nfs-utils rpcbind curl ca-certificates nano tzdata ncurses make tcpdump \
&& curl -L https://dl.k8s.io/release/v$KUBECTL_VERSION/bin/linux/amd64/kubectl -o /usr/local/bin/kubectl \
&& chmod +x /usr/local/bin/kubectl \
&& kubectl version --client \
&& rm -rf /etc/ssh/* \
&& mkdir /etc/ssh/authorized_keys.d \
&& mkdir -p /etc/ssh/authorized_keys.d \
&& cp /usr/share/zoneinfo/America/New_York /etc/localtime \
&& echo "America/New_York" > /etc/timezone \
&& apk del tzdata
Expand Down
10 changes: 6 additions & 4 deletions bin/controller.keys.js
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,12 @@ module.exports.updateKeys = function updateKeys(options, taskCallback) {

singleItem.Labels = _.get(singleItem, 'metadata.labels');

singleItem.Labels['ci.rabbit.name'] = singleItem.Labels['name'];

singleItem.Labels['ci.rabbit.ssh.user'] = singleItem.Labels['ci.rabbit.ssh.user'] || null;
return singleItem;
// Prevents the application from being added to the list if it does not have the required labels
if ( _.get(singleItem.Labels, 'name', false) && _.get(singleItem.Labels, 'ci.rabbit.ssh.user', false) ) {
singleItem.Labels['ci.rabbit.name'] = singleItem.Labels['name'];
singleItem.Labels['ci.rabbit.ssh.user'] = singleItem.Labels['ci.rabbit.ssh.user'] || null;
return singleItem;
}

});

Expand Down
48 changes: 29 additions & 19 deletions bin/controller.ssh.entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,25 +9,35 @@ export USER_LOGIN=$(echo ${ENV_VARS} | cut -d ';' -f 2)

echo "[$(date)] Have a session for [${USER_LOGIN}] : ${USER}, ${SSH_ORIGINAL_COMMAND}, ${SSH_CLIENT}, ${SSH_CONNECTION} and [${CONNECTION_STRING}] command." >> /var/log/sshd.log

## SFTP.
if [[ ${SSH_ORIGINAL_COMMAND} == "internal-sftp" ]]; then

echo "[$(date)] Have SFTP connection [${CONNECTION_STRING}] for [${USER}]." >> /var/log/sshd.log

/usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -i -- /usr/lib/sftp-server

exit;

fi

if [[ ${SSH_ORIGINAL_COMMAND} == "/usr/lib/ssh/sftp-server" ]]; then

echo "[$(date)] Have SFTP connection [${CONNECTION_STRING}] for [${USER}]." >> /var/log/sshd.log

/usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -i -- /usr/lib/sftp-server

exit;

## SFTP handling with Alpine-specific paths
if [[ ${SSH_ORIGINAL_COMMAND} == "internal-sftp" ]] || [[ ${SSH_ORIGINAL_COMMAND} == "/usr/lib/ssh/sftp-server" ]]; then
echo "[$(date)] SFTP connection attempt from [${SSH_CLIENT}] for user [${USER}] to pod [${CONNECTION_STRING}]" >> /var/log/sshd.log

# Check container OS type for better error reporting
CONTAINER_OS="unknown"
if /usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -- which apk >/dev/null 2>&1; then
CONTAINER_OS="Alpine"
elif /usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -- which apt-get >/dev/null 2>&1; then
CONTAINER_OS="Debian/Ubuntu"
elif /usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -- which yum >/dev/null 2>&1; then
CONTAINER_OS="RHEL/CentOS"
fi
echo "[$(date)] Container OS detected: ${CONTAINER_OS}" >> /var/log/sshd.log

# Try common SFTP server paths
for SFTP_PATH in "/usr/lib/ssh/sftp-server" "/usr/lib/sftp-server" "/usr/libexec/sftp-server"; do
echo "[$(date)] Checking for SFTP server at ${SFTP_PATH}" >> /var/log/sshd.log
if /usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -- test -f ${SFTP_PATH} 2>/dev/null; then
echo "[$(date)] Found SFTP server at ${SFTP_PATH}, establishing connection" >> /var/log/sshd.log
exec /usr/local/bin/kubectl exec -n ${CONNECTION_STRING} -i -- ${SFTP_PATH}
exit 0
fi
done

# If we get here, we couldn't find the SFTP server
echo "[$(date)] Error: SFTP server not found in ${CONTAINER_OS} container [${CONNECTION_STRING}]. Client IP: ${SSH_CLIENT}" >> /var/log/sshd.log
echo "Error: SFTP access requires openssh-sftp-server to be installed in the container. Please contact your administrator." >&2
exit 1
fi

## Specific Command, pipe into container.
Expand Down
4 changes: 2 additions & 2 deletions bin/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,8 @@ setInterval(function() {

var _containers = body = _.map(body.items, function(singleItem) {
singleItem.Labels = _.get(singleItem, 'metadata.labels');
singleItem.Labels['ci.rabbit.name'] = singleItem.Labels['name'];
singleItem.Labels['ci.rabbit.ssh.user'] = singleItem.Labels['ci.rabbit.ssh.user'] || null;
singleItem.Labels['ci.rabbit.name'] = _.get(singleItem.Labels,'name', null);
singleItem.Labels['ci.rabbit.ssh.user'] = _.get(singleItem.Labels,'ci.rabbit.ssh.user', null);
return singleItem;
});

Expand Down
Loading