Skip to content

Commit

Permalink
feat(security): Add new implementation for security bootstrapping/ins…
Browse files Browse the repository at this point in the history
…tallation

The scurity container bootstrapping initiates with security-bootstrapper service, in which it bootstraps
the pre-seeded secrets and credentials.

The security-bootstrapper starts with seeding the vault_wait_install.sh script (contains the dockerize utility)
to be available for other containers that needs to wait for the intended done-listener is issued and connected.

The other containers in the security bootstrapping process currently are:
 - Redis bootstrapping
 - Consul bootstrapping
 - Kong bootstrapping
 - Postgres bootstrapping

The dockerize utility is used on those above containers to wait for that security-bootstrapper
tcp listener done signal and then those container can proceed to start up.

Still WIP: security-bootstrapper's entrypoint script to add more bootstrapping steps. eg.: injecting entrypoint scripts to various containers.

Signed-off-by: Jim Wang <[email protected]>
  • Loading branch information
jim-wang-intel committed Jan 4, 2021
1 parent 68a8f91 commit 5afc258
Show file tree
Hide file tree
Showing 47 changed files with 2,646 additions and 2 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ cmd/sys-mgmt-agent/sys-mgmt-agent
cmd/sys-mgmt-executor/sys-mgmt-executor
cmd/security-bootstrap-redis/security-bootstrap-redis
cmd/secrets-config/secrets-config
cmd/security-bootstrapper/security-bootstrapper

docs/_build/

Expand Down
19 changes: 17 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ DOCKERS= \
docker_support_scheduler \
docker_security_proxy_setup \
docker_security_secretstore_setup \
docker_security_bootstrap_redis
docker_security_bootstrap_redis \
docker_security_bootstrapper

.PHONY: $(DOCKERS)

Expand All @@ -35,7 +36,8 @@ MICROSERVICES= \
cmd/security-secretstore-setup/security-secretstore-setup \
cmd/security-file-token-provider/security-file-token-provider \
cmd/security-bootstrap-redis/security-bootstrap-redis \
cmd/secrets-config/secrets-config
cmd/secrets-config/secrets-config \
cmd/security-bootstrapper/security-bootstrapper

.PHONY: $(MICROSERVICES)

Expand Down Expand Up @@ -87,6 +89,9 @@ cmd/security-bootstrap-redis/security-bootstrap-redis:
cmd/secrets-config/secrets-config:
$(GO) build $(GOFLAGS) -o ./cmd/secrets-config ./cmd/secrets-config

cmd/security-bootstrapper/security-bootstrapper:
$(GO) build $(GOFLAGS) -o ./cmd/security-bootstrapper/security-bootstrapper ./cmd/security-bootstrapper

clean:
rm -f $(MICROSERVICES)

Expand Down Expand Up @@ -193,3 +198,13 @@ docker_security_bootstrap_redis:
-t edgexfoundry/docker-security-bootstrap-redis-go:$(GIT_SHA) \
-t edgexfoundry/docker-security-bootstrap-redis-go:$(DOCKER_TAG) \
.

docker_security_bootstrapper:
docker build \
--build-arg http_proxy \
--build-arg https_proxy \
-f cmd/security-bootstrapper/Dockerfile \
--label "git_sha=$(GIT_SHA)" \
-t edgexfoundry/docker-security-bootstrapper-go:$(GIT_SHA) \
-t edgexfoundry/docker-security-bootstrapper-go:$(DOCKER_TAG) \
.
3 changes: 3 additions & 0 deletions cmd/core-command/Attribution.txt
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,6 @@ https://github.com/go-playground/validator/blob/master/LICENSE

leodido/go-urn (MIT) https://github.com/leodido/go-urn
https://github.com/leodido/go-urn

github.com/lib/pq (Unspecified) https://github.com/lib/pq
https://github.com/lib/pq/blob/master/LICENSE.md
3 changes: 3 additions & 0 deletions cmd/core-data/Attribution.txt
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,6 @@ https://github.com/go-playground/validator/blob/master/LICENSE

leodido/go-urn (MIT) https://github.com/leodido/go-urn
https://github.com/leodido/go-urn

github.com/lib/pq (Unspecified) https://github.com/lib/pq
https://github.com/lib/pq/blob/master/LICENSE.md
3 changes: 3 additions & 0 deletions cmd/core-metadata/Attribution.txt
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,6 @@ https://github.com/go-playground/validator/blob/master/LICENSE

leodido/go-urn (MIT) https://github.com/leodido/go-urn
https://github.com/leodido/go-urn

github.com/lib/pq (Unspecified) https://github.com/lib/pq
https://github.com/lib/pq/blob/master/LICENSE.md
3 changes: 3 additions & 0 deletions cmd/secrets-config/Attribution.txt
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,6 @@ https://github.com/go-playground/validator/blob/master/LICENSE

leodido/go-urn (MIT) https://github.com/leodido/go-urn
https://github.com/leodido/go-urn

github.com/lib/pq (Unspecified) https://github.com/lib/pq
https://github.com/lib/pq/blob/master/LICENSE.md
3 changes: 3 additions & 0 deletions cmd/security-bootstrap-redis/Attribution.txt
Original file line number Diff line number Diff line change
Expand Up @@ -173,3 +173,6 @@ https://github.com/go-playground/validator/blob/master/LICENSE

leodido/go-urn (MIT) https://github.com/leodido/go-urn
https://github.com/leodido/go-urn

github.com/lib/pq (Unspecified) https://github.com/lib/pq
https://github.com/lib/pq/blob/master/LICENSE.md
179 changes: 179 additions & 0 deletions cmd/security-bootstrapper/Attribution.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
The following open source projects are referenced by Core Data Go:

pkg/errors (BSD-2) https://github.com/pkg/errors
https://github.com/pkg/errors/blob/master/LICENSE

gorilla/mux (BSD-3) - https://github.com/gorilla/mux
https://github.com/gorilla/mux/blob/master/LICENSE

pebbe/zmq4 (BSD-2) https://github.com/pebbe/zmq4
https://github.com/pebbe/zmq4/blob/master/LICENSE.txt

go-kit/kit (MIT) github.com/go-kit/kit
https://github.com/go-kit/kit/blob/master/LICENSE

go-logfmt/logfmt (MIT) https://github.com/go-logfmt/logfmt
https://github.com/go-logfmt/logfmt/blob/master/LICENSE

robfig/cron (MIT) https://github.com/robfig/cron
https://github.com/robfig/cron/blob/master/LICENSE

dgrijalva/jwt-go (MIT) https://github.com/dgrijalva/jwt-go
https://github.com/dgrijalva/jwt-go/blob/master/LICENSE

google/uuid (BSD-3) https://github.com/google/uuid
https://github.com/google/uuid/blob/master/LICENSE

pelletier/go-toml (MIT) https://github.com/pelletier/go-toml
https://github.com/pelletier/go-toml/blob/master/LICENSE

influxdata/influxdb/client/v2 (MIT) https://github.com/influxdata/influxdb
https://github.com/influxdata/influxdb/blob/master/LICENSE

influxdata/platform (MIT) https://github.com/influxdata/platform
https://github.com/influxdata/platform/blob/master/LICENSE

eclipse/paho.mqtt.golang (Eclipse Public License 1.0) https://github.com/eclipse/paho.mqtt.golang
https://github.com/eclipse/paho.mqtt.golang/blob/master/LICENSE

mattn/go-xmpp (BSD-3) https://github.com/mattn/go-xmpp
https://github.com/mattn/go-xmpp/blob/master/LICENSE

BurntSushi/toml (MIT) https://github.com/BurntSushi/toml
https://github.com/BurntSushi/toml/blob/master/COPYING

mitchellh/consulstructure (MIT) https://github.com/mitchellh/consulstructure
https://github.com/mitchellh/consulstructure/blob/master/LICENSE

mitchellh/mapstructure (MIT) https://github.com/mitchellh/mapstructure
https://github.com/mitchellh/mapstructure/blob/master/LICENSE

mitchellh/copystructure (MIT) https://github.com/mitchellh/copystructure
https://github.com/mitchellh/copystructure/blob/master/LICENSE

mitchellh/reflectwalk (MIT) https://github.com/mitchellh/reflectwalk
https://github.com/mitchellh/reflectwalk/blob/master/LICENSE

cenkalti/backoff (MIT) https://github.com/cenkalti/backoff
https://github.com/cenkalti/backoff/blob/master/LICENSE

hashicorp/consul/api 1.1.0 (Mozilla Public License 2.0) - https://github.com/hashicorp/consul/api
https://github.com/hashicorp/consul/blob/master/LICENSE

hashicorp/go-cleanhttp (Mozilla Public License 2.0) - https://github.com/hashicorp/go-cleanhttp
https://github.com/hashicorp/go-cleanhttp/blob/master/LICENSE

hashicorp/go-rootcerts (Mozilla Public License 2.0) https://github.com/hashicorp/go-rootcerts
https://github.com/hashicorp/go-rootcerts/blob/master/LICENSE

mitchellh/go-homedir (MIT) https://github.com/mitchellh/go-homedir
https://github.com/mitchellh/go-homedir/blob/master/LICENSE

mitchellh/mapstructure (MIT) https://github.com/mitchellh/mapstructure
https://github.com/mitchellh/mapstructure/blob/master/LICENSE

hashicorp/serf (Mozilla Public License 2.0) https://github.com/hashicorp/serf
https://github.com/hashicorp/serf/blob/master/LICENSE

armon/go-metrics (MIT) https://github.com/armon/go-metrics
https://github.com/armon/go-metrics/blob/master/LICENSE

hashicorp/go-immutable-radix (Mozilla Public License 2.0) https://github.com/hashicorp/go-immutable-radix
https://github.com/hashicorp/go-immutable-radix/blob/master/LICENSE

hashicorp/golang-lru (Mozilla Public License 2.0) https://github.com/hashicorp/golang-lru
https://github.com/hashicorp/golang-lru/blob/master/LICENSE

github.com/go-redis/redis/v7 (BSD-2) https://github.com/go-redis/redis
https://github.com/go-redis/redis/blob/master/LICENSE
https://github.com/go-redis/redis/blob/master/LICENSE

gomodule/redigo (Apache 2.0) https://github.com/gomodule/redigo
https://github.com/gomodule/redigo/blob/master/LICENSE

OneOfOne/xxhash (Apache 2.0) https://github.com/OneOfOne/xxhash
https://github.com/OneOfOne/xxhash/blob/master/LICENSE

imdario/mergo (BSD-3) github.com/imdario/mergo
https://github.com/imdario/mergo/blob/master/LICENSE

magiconair/properties (BSD-2) https://github.com/magiconair/properties
https://github.com/magiconair/properties/blob/master/LICENSE

gopkg.in/eapache/queue.v1 (MIT) gopkg.in/eapache/queue.v1
https://github.com/eapache/queue/blob/v1.1.0/LICENSE

bertimus9/systemstat (MIT) https://bitbucket.org/bertimus9/systemstat
https://bitbucket.org/bertimus9/systemstat/src/master/LICENSE

davecgh/go-spew (ISC) https://github.com/davecgh/go-spew
https://github.com/davecgh/go-spew/blob/master/LICENSE

edgexfoundry/go-mod-bootstrap (Apache 2.0) https://github.com/edgexfoundry/go-mod-bootstrap
https://github.com/edgexfoundry/go-mod-bootstrap/blob/master/LICENSE

edgexfoundry/go-mod-configuration (Apache 2.0) https://github.com/edgexfoundry/go-mod-configuration
https://github.com/edgexfoundry/go-mod-configuration/blob/master/LICENSE

edgexfoundry/go-mod-core-contracts (Apache 2.0) https://github.com/edgexfoundry/go-mod-core-contracts
https://github.com/edgexfoundry/go-mod-core-contracts/blob/master/LICENSE

edgexfoundry/go-mod-messaging (Apache 2.0) https://github.com/edgexfoundry/go-mod-messaging
https://github.com/edgexfoundry/go-mod-messaging/blob/master/LICENSE

edgexfoundry/go-mod-registry (Apache 2.0) https://github.com/edgexfoundry/go-mod-registry
https://github.com/edgexfoundry/go-mod-registry/blob/master/LICENSE

edgexfoundry/go-mod-secrets (Apache 2.0) https://github.com/edgexfoundry/go-mod-secrets
https://github.com/edgexfoundry/go-mod-secrets/blob/master/LICENSE

gorilla/context (BSD-3) https://github.com/gorilla/context
https://github.com/gorilla/context/blob/master/LICENSE

kr/logfmt (Unspecified) https://github.com/kr/logfmt
https://github.com/kr/logfmt/blob/master/Readme

pmezard/go-difflib (Unspecified) https://github.com/pmezard/go-difflib
https://github.com/pmezard/go-difflib/blob/master/LICENSE

stretchr/objx (MIT) https://github.com/stretchr/objx
https://github.com/stretchr/objx/blob/master/LICENSE

stretchr/testify (MIT) https://github.com/stretchr/testify
https://github.com/stretchr/testify/blob/master/LICENSE

fxamacker/cbor (MIT) https://github.com/fxamacker/cbor/v2
https://github.com/fxamacker/cbor/blob/master/README.md#license

x448/float16 (MIT) https://github.com/x448/float16
https://github.com/x448/float16/blob/master/LICENSE

golang.org/x/net (Unspecified) https://github.com/golang/net
https://github.com/golang/net/blob/master/LICENSE

gopkg.in/yaml.v2 (Apache 2.0) https://github.com/go-yaml/yaml/
https://github.com/go-yaml/yaml/blob/v2.2.2/LICENSE

gopkg.in/yaml.v3 (MIT) https://github.com/go-yaml/yaml/
https://github.com/go-yaml/yaml/blob/v3/LICENSE

cloudflare/gokey (BSD-3) https://github.com/cloudflare/gokey
https://github.com/cloudflare/gokey/blob/master/LICENSE

golang.org/x/crypto (Unspecified) https://github.com/golang/crypto
https://github.com/golang/crypto/blob/master/LICENSE

go-playground/locales (MIT) https://github.com/go-playground/locales
https://github.com/go-playground/locales/blob/master/LICENSE

go-playground/universal-translator (MIT) https://github.com/go-playground/universal-translator
https://github.com/go-playground/universal-translator/blob/master/LICENSE

github.com/go-playground/validator/v10 (MIT) https://github.com/go-playground/validator
https://github.com/go-playground/validator/blob/master/LICENSE

leodido/go-urn (MIT) https://github.com/leodido/go-urn
https://github.com/leodido/go-urn

github.com/lib/pq (Unspecified) https://github.com/lib/pq
https://github.com/lib/pq/blob/master/LICENSE.md
82 changes: 82 additions & 0 deletions cmd/security-bootstrapper/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# ----------------------------------------------------------------------------------
# Copyright 2021 Intel Corp.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
# SPDX-License-Identifier: Apache-2.0'
# ----------------------------------------------------------------------------------

ARG BUILDER_BASE=golang:1.15-alpine3.12
FROM ${BUILDER_BASE} AS builder

WORKDIR /edgex-go

RUN sed -e 's/dl-cdn[.]alpinelinux.org/nl.alpinelinux.org/g' -i~ /etc/apk/repositories

RUN apk add --update --no-cache make git

COPY go.mod .

RUN go mod download

COPY . .

RUN make cmd/security-bootstrapper/security-bootstrapper \
&& make cmd/security-bootstrap-redis/security-bootstrap-redis

FROM alpine:3.12

RUN apk add --update --no-cache dumb-init openssl

LABEL license='SPDX-License-Identifier: Apache-2.0' \
copyright='Copyright (c) 2021 Intel Corporation'

# Use dockerize utility for services to wait for certain ports to be available
ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
&& tar -C /usr/local/bin -xzvf dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
&& rm dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz

ENV SECURITY_INIT_DIR /edgex-init
ARG BOOTSTRAP_REDIS_DIR=${SECURITY_INIT_DIR}/bootstrap-redis

RUN mkdir -p ${SECURITY_INIT_DIR} \
&& mkdir -p ${BOOTSTRAP_REDIS_DIR} \
&& echo "Copy dockerize executable" \
&& cp /usr/local/bin/dockerize ${SECURITY_INIT_DIR}

WORKDIR ${SECURITY_INIT_DIR}

# copy all entrypoint scripts into shared folder
COPY --from=builder /edgex-go/cmd/security-bootstrapper/entrypoint-scripts/ ${SECURITY_INIT_DIR}/
RUN chmod +x ${SECURITY_INIT_DIR}/*.sh

COPY --from=builder /edgex-go/cmd/security-bootstrapper/security-bootstrapper .
COPY --from=builder /edgex-go/cmd/security-bootstrapper/res/configuration.toml ./res/

# needed for bootstrapping Redis db
COPY --from=builder /edgex-go/cmd/security-bootstrap-redis/security-bootstrap-redis ${BOOTSTRAP_REDIS_DIR}/
COPY --from=builder /edgex-go/cmd/security-bootstrap-redis/res/configuration.toml ${BOOTSTRAP_REDIS_DIR}/res/

# Expose the file directory as a volume since there's long-running state
VOLUME ${SECURITY_INIT_DIR}

# The entry point script uses dumb-init as the top-level process to reap any
# zombie processes
COPY --from=builder /edgex-go/cmd/security-bootstrapper/entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/entrypoint.sh \
&& ln -s /usr/local/bin/entrypoint.sh /
ENTRYPOINT ["entrypoint.sh"]

CMD ["install"]
Loading

0 comments on commit 5afc258

Please sign in to comment.