Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unified setup #6

Open
wants to merge 47 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
94e7ab1
WIP
edwinyyyu Sep 4, 2024
00f618e
WIP
edwinyyyu Sep 4, 2024
8cc1ff0
WIP
edwinyyyu Sep 6, 2024
a9d45c9
WIP
edwinyyyu Sep 7, 2024
7be721e
WIP
edwinyyyu Sep 7, 2024
5aa7d22
WIP
edwinyyyu Sep 7, 2024
40872ff
WIP
edwinyyyu Sep 7, 2024
0ff13d3
WIP
edwinyyyu Sep 7, 2024
49f5e34
WIP
edwinyyyu Sep 7, 2024
5accd9f
WIP
edwinyyyu Sep 9, 2024
eec547f
WIP
edwinyyyu Sep 9, 2024
8041a04
WIP
edwinyyyu Sep 9, 2024
c112bea
WIP
edwinyyyu Sep 9, 2024
5a3c55e
WIP
edwinyyyu Sep 9, 2024
cd3993d
WIP
edwinyyyu Sep 9, 2024
9a58a0c
WIP
edwinyyyu Sep 9, 2024
d140ac5
WIP
edwinyyyu Sep 9, 2024
ba241a6
WIP
edwinyyyu Sep 9, 2024
6c72bc4
WIP
edwinyyyu Sep 9, 2024
2861d04
Modify Kubeflow
edwinyyyu Sep 10, 2024
84f5a4e
Improve Ansible inventory prompts
edwinyyyu Sep 10, 2024
db1f26f
Log stdout and stderr to timestamped directory
edwinyyyu Sep 10, 2024
12016b3
curl from this branch
edwinyyyu Sep 10, 2024
2ba4ad5
make curl silent
edwinyyyu Sep 10, 2024
d60ed51
Improve script importing
edwinyyyu Sep 10, 2024
c182140
Improve yes/no prompting
edwinyyyu Sep 12, 2024
542dd8f
Merge branch 'main' into unified-setup
edwinyyyu Sep 12, 2024
4052a93
Update imported script branches
edwinyyyu Sep 13, 2024
73a0d35
Merge branch 'main' into unified-setup
edwinyyyu Sep 13, 2024
7c53c67
Make ensure prerequisites script executable
edwinyyyu Sep 13, 2024
0c76ba4
Merge branch 'main' into unified-setup
edwinyyyu Sep 13, 2024
cdb7fe2
Merge branch 'main' into unified-setup
edwinyyyu Sep 13, 2024
f7a1e51
Add log message about engine cleanup
edwinyyyu Sep 13, 2024
1769d4f
Merge branch 'main' into unified-setup
edwinyyyu Sep 16, 2024
3532601
Add option to ignore setup errors
edwinyyyu Sep 16, 2024
6fddfc0
Separate log files
edwinyyyu Sep 16, 2024
123fdb9
Get labels more reliably
edwinyyyu Sep 16, 2024
2bd7cbd
Unset log file before resetting log file
edwinyyyu Sep 16, 2024
88536ce
Fix Helm login
edwinyyyu Sep 16, 2024
931365e
Setting log file unsets log file first
edwinyyyu Sep 16, 2024
be48e09
Prepend timestamps to each line of output
edwinyyyu Sep 17, 2024
5e20338
Set log file before ensuring prerequisites
edwinyyyu Sep 17, 2024
c6c198e
Minor
edwinyyyu Sep 17, 2024
483d5a2
Minor
edwinyyyu Sep 18, 2024
7ca73e7
Ensure minimum prerequisite versions
edwinyyyu Sep 18, 2024
be84ee3
Define fixed paths for command line tools
edwinyyyu Sep 18, 2024
10269e3
Get information on failure (#18)
edwinyyyu Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 1 addition & 66 deletions kubeflow-setup.sh
Original file line number Diff line number Diff line change
@@ -1,70 +1,5 @@
#!/bin/bash

source logging.sh

div
log "Welcome to the MMC.AI Kubeflow installer!"
log "First, setting sysctl variables across all hosts..."
div

cat > sysctl-playbook.yaml <<EOF
---
- name: Configure sysctl settings across all hosts
hosts: all
become: yes
tasks:
- name: Ensure sysctl configuration file exists
ansible.builtin.file:
path: /etc/sysctl.conf
state: touch

- name: Set fs.inotify.max_queued_events
ansible.builtin.lineinfile:
path: /etc/sysctl.conf
regexp: '^fs.inotify.max_queued_events'
line: 'fs.inotify.max_queued_events = 32384'
create: yes

- name: Set fs.inotify.max_user_instances
ansible.builtin.lineinfile:
path: /etc/sysctl.conf
regexp: '^fs.inotify.max_user_instances'
line: 'fs.inotify.max_user_instances = 4096'
create: yes

- name: Set fs.inotify.max_user_watches
ansible.builtin.lineinfile:
path: /etc/sysctl.conf
regexp: '^fs.inotify.max_user_watches'
line: 'fs.inotify.max_user_watches = 2008100'
create: yes

- name: Set net.core.somaxconn
ansible.builtin.lineinfile:
path: /etc/sysctl.conf
regexp: '^net.core.somaxconn'
line: 'net.core.somaxconn = 8192'
create: yes

- name: Apply sysctl changes
ansible.builtin.sysctl:
name: "{{ item.key }}"
value: "{{ item.value }}"
state: present
reload: yes
loop:
- { key: 'fs.inotify.max_queued_events', value: '32384' }
- { key: 'fs.inotify.max_user_instances', value: '4096' }
- { key: 'fs.inotify.max_user_watches', value: '2008100' }
- { key: 'net.core.somaxconn', value: '8192' }
EOF

ansible-playbook sysctl-playbook.yaml

div
log "Second, installing kubeflow..."
div

## Use newer kubeflow
sed -i 's/v1.7.0/v1.8.0/g' ./scripts/k8s/deploy_kubeflow.sh

Expand All @@ -77,4 +12,4 @@ sed -i 's:istio-1-16:istio-1-17:g' ./scripts/k8s/deploy_kubeflow.sh
# sed -i '/^source /i\source git-clone.sh' ./scripts/k8s/deploy_kubeflow.sh
# sed -i 's:git clone:git_clone:g' ./scripts/k8s/deploy_kubeflow.sh

./scripts/k8s/deploy_kubeflow.sh
./scripts/k8s/deploy_kubeflow.sh
Loading