Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce opencti/platform docker image size #191

Closed
2xyo opened this issue Aug 22, 2019 · 2 comments
Closed

Reduce opencti/platform docker image size #191

2xyo opened this issue Aug 22, 2019 · 2 comments
Assignees
@richard-julien
Labels
feature use for describing a new feature to develop solved use to identify issue that has been solved (must be linked to the solving PR)
Milestone
Release 2.0.0

Comments

@2xyo
Copy link
Contributor

2xyo commented Aug 22, 2019

Problem to Solve

opencti/platform docker image is big:

$ docker images opencti/platform:1.1.1
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
opencti/platform    1.1.1               6ba2d445f688        2 weeks ago         1.58GB


$ docker history opencti/platform:1.1.1
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
6ba2d445f688        2 weeks ago         /bin/sh -c #(nop)  ENTRYPOINT ["/entrypoint.…   0B                  
<missing>           2 weeks ago         /bin/sh -c #(nop)  EXPOSE 4000/tcp              0B                  
<missing>           2 weeks ago         /bin/sh -c chmod +x /entrypoint.sh              1.09kB              
<missing>           2 weeks ago         /bin/sh -c #(nop) COPY file:28f787f6dc9089f6…   1.09kB              
<missing>           2 weeks ago         /bin/sh -c rm -Rf /opt/opencti-build            0B                  
<missing>           2 weeks ago         /bin/sh -c mv /opt/opencti-build/opencti-gra…   189MB               
<missing>           2 weeks ago         /bin/sh -c cd /opt/opencti-build/opencti-gra…   250MB               
<missing>           2 weeks ago         /bin/sh -c cd /opt/opencti-build/opencti-fro…   895MB               
<missing>           2 weeks ago         /bin/sh -c #(nop) COPY dir:257a895a13d3af22a…   689kB               
<missing>           2 weeks ago         /bin/sh -c #(nop) COPY dir:6a7dfa868771c828e…   3.14MB              
<missing>           2 weeks ago         /bin/sh -c DEBIAN_FRONTEND=noninteractive ap…   6.45MB              
<missing>           2 weeks ago         /bin/sh -c echo "deb https://dl.yarnpkg.com/…   47B                 
<missing>           2 weeks ago         /bin/sh -c curl -sS https://dl.yarnpkg.com/d…   14.8kB              
<missing>           2 weeks ago         /bin/sh -c DEBIAN_FRONTEND=noninteractive ap…   82.9MB              
<missing>           2 weeks ago         /bin/sh -c curl -sL https://deb.nodesource.c…   38MB                
<missing>           2 weeks ago         /bin/sh -c DEBIAN_FRONTEND=noninteractive ap…   18.4MB              
<missing>           2 weeks ago         /bin/sh -c DEBIAN_FRONTEND=noninteractive ap…   24MB                
<missing>           2 weeks ago         /bin/sh -c #(nop)  ENV NODE_OPTION=--max_old…   0B                  
<missing>           4 weeks ago         /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B                  
<missing>           4 weeks ago         /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B                  
<missing>           4 weeks ago         /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   933B                
<missing>           4 weeks ago         /bin/sh -c [ -z "$(apt-get indextargets)" ]     985kB               
<missing>           4 weeks ago         /bin/sh -c #(nop) ADD file:e7de0c937513ffc00…   69MB

Current Workaround

Limit RUN commands and clear apt/yarn cache.

Proposed Solution

# Use the official Docker Hub Ubuntu 19.04 base image
FROM ubuntu:19.04

# Build env
ENV NODE_OPTION --max_old_space_size=4096

# Copy work files
COPY opencti-front /opt/opencti-build/opencti-front
COPY opencti-graphql /opt/opencti-build/opencti-graphql
COPY entrypoint.sh /

# Update the base image 
# Install Node 11 and Yarn
# Build frontend && GraphQL API
RUN DEBIAN_FRONTEND=noninteractive \
	apt-get update && \
	apt-get -y upgrade && \
	apt-get -y dist-upgrade && \
	apt-get -y install curl rsyslog wget netcat && \
	curl -sL https://deb.nodesource.com/setup_11.x | bash - && \
	apt-get install -y nodejs && \
	curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
	echo "deb https://dl.yarnpkg.com/debian/ stable main" > /etc/apt/sources.list.d/yarn.list && \
	apt-get update && \
	apt-get -y install yarn && \
	rm -rf /var/lib/apt/lists/* && \
	cd /opt/opencti-build/opencti-front && \
	yarn install && \
	yarn build && \
	yarn cache clean && \
	cd /opt/opencti-build/opencti-graphql && \
	yarn install && \
	yarn build && \
	yarn cache clean && \
	mv /opt/opencti-build/opencti-graphql /opt/opencti && \
	rm -rf /opt/opencti-build

# Expose and entrypoint
RUN chmod +x /entrypoint.sh 
EXPOSE 4000/tcp
ENTRYPOINT ["/entrypoint.sh"]

Additional Information

$ docker images opencti/platform:1.1.1-local-light
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
opencti/platform    1.1.1-local-light   d1db9ca54382        12 minutes ago      456MB

$ docker history opencti/platform:1.1.1-local-light
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
d1db9ca54382        12 minutes ago      /bin/sh -c #(nop)  ENTRYPOINT ["/entrypoint.…   0B                  
efa468bb45eb        12 minutes ago      /bin/sh -c #(nop)  EXPOSE 4000/tcp              0B                  
904f9db83fba        12 minutes ago      |0 /bin/sh -c chmod +x /entrypoint.sh           1.09kB              
ee2539607b2f        12 minutes ago      |0 /bin/sh -c DEBIAN_FRONTEND=noninteractive…   383MB               
6f942aa996b9        28 minutes ago      /bin/sh -c #(nop) COPY file:2d04bc201940d626…   1.09kB              
32a3c6f107f4        28 minutes ago      /bin/sh -c #(nop) COPY dir:dd8882947f54d879d…   689kB               
e6c7fab6ca0b        28 minutes ago      /bin/sh -c #(nop) COPY dir:9edf6f50572675b59…   3.14MB              
b819f5cc99bd        About an hour ago   /bin/sh -c #(nop)  ENV NODE_OPTION=--max_old…   0B                  
9f3d7c446553        7 days ago          /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B                  
<missing>           7 days ago          /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B                  
<missing>           7 days ago          /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   933B                
<missing>           7 days ago          /bin/sh -c [ -z "$(apt-get indextargets)" ]     985kB               
<missing>           7 days ago          /bin/sh -c #(nop) ADD file:fcc7c52abc98c93b7…   69MB

Other solution: use Alpine
@richard-julien
Copy link
Member

Thanks for the issue, its clearly something we need to improve. For me we need to do both:

  • Reduce the footprint of the application. (cleanup cache, ...)
  • Reduce the base image size by moving to Alpine.

@richard-julien richard-julien added feature use for describing a new feature to develop build labels Aug 22, 2019
@richard-julien richard-julien added this to the Release 1.2.0 milestone Sep 5, 2019
@richard-julien richard-julien self-assigned this Sep 5, 2019
@richard-julien
Copy link
Member

First statistic after some works on the platform and worker images.

Platform:

  • version 1.1.2: 1.58GB
  • version 2.0.0: 306MB
    Image reduce by 80%

Worker:

  • version 1.1.2: 544MB
  • version 2.0.0: 122MB
    Image reduce by 78%

@richard-julien richard-julien mentioned this issue Sep 7, 2019
Closed
SamuelHassine pushed a commit that referenced this issue Sep 13, 2019
@richard-julien
[build] Reduce docker image size and let docker restart the containers (
224e2c5 
#191, #212)

* Introducing new Alpine image and adapt schema and migrations to be part of the main process
* Yarn schema is no longer required
* Modification of the schema init for testing
* Bump dependencies versions and adapt healthcheck and logs
* Adapt worker script to check openCTI API before start. Exit the process if script have starting errors
* Align log message on domain reference
* Fix closeReadTx function by removing the log message introduce by previous commit
* Fix hadolint docker warnings
@SamuelHassine SamuelHassine added the solved use to identify issue that has been solved (must be linked to the solving PR) label Sep 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@richard-julien richard-julien

Labels
feature use for describing a new feature to develop solved use to identify issue that has been solved (must be linked to the solving PR)
Projects
None yet
Milestone
Release 2.0.0
Development

No branches or pull requests
3 participants
@2xyo @richard-julien @SamuelHassine