Added support for Swarm Mode #344

filopedraz · 2023-09-01T13:28:23Z

Description

Added support for Swarm Mode which starts the necessary processes in order to be part of the Petals Swarm. The User can Enable the mode from Settings. Tauri will spawn a thread that installs the dependencies and runs the python process.

The PR contains all the building blocks in order to implement the complete POC functionality.

Limitations

UI/UX must be drastically improved. No CSS changes have been made.
It currently works only if the User has Python installed on its machine.

@tiero @casperdcl @sekulicd @Janaka-Steph please have a look.

what-the-diff · 2023-09-01T13:28:52Z

PR Summary

Introduction of a checking mechanism for Python
A new functionality is added to verify that Python is correctly installed on the machine.
Enhanced support for Swarm Mode
New functions have been introduced to allow for better handling of Docker's Swarm Mode. Specifically, these functions enable us to check the running status of Swarm Mode, start it, and stop it when needed.
Addition of SwarmMode Component
A new component named SwarmMode is created to incorporate Docker's Swarm Mode features into our application UI. This component is added as part of our Settings module, empowering users with more operational flexibility.
Utilities for Swarm Mode
Additional utility functions for handling Docker Swarm Mode are implemented. These help us to verify the operational status of Swarm Mode, stop it, or start it based on user input or application requirements.

src-tauri/src/main.rs

casperdcl · 2023-09-02T07:34:38Z

src-tauri/src/main.rs

+            println!("🚀 Starting the Swarm...");
+
+            let _ = Command::new("/usr/bin/python3")
+                .args(&["-m", "pip", "install", "git+https://github.com/bigscience-workshop/petals",])


Potentially fallback to pip install --user before giving up?

Ideally, we should install python, conda, create a conda env, and then install, but I want your opinion on that on how to make this installation super smooth.

I'd suggest:

try to find mamba on $PATH

try to find conda on $PATH

try to find python3 on $PATH

install https://github.com/conda-forge/miniforge#mambaforge

for (1), (2) and (4): create/activate an env called prem before installing

What' the Tauri problem with $PATH?

src-tauri/src/main.rs

Janaka-Steph

LGTM

filopedraz · 2023-09-04T20:08:06Z

@casperdcl @sekulicd @Janaka-Steph @tiero instead of passing "--public_name", "prem-app" we should pass the User username/identifier (like a nostr pubkey) :)

casperdcl · 2023-09-04T20:24:49Z

could just use current username

filopedraz · 2023-09-05T06:12:02Z

Good point. I ll apply the changes you suggested above.

Petals swarm

casperdcl

lgtm but maybe open follow-up issues for:

filopedraz · 2023-09-20T11:54:05Z

I ll address the remaining points in this PR soon. I want first the new services to be working and then I ll move on with this. :)

filopedraz · 2023-09-23T13:48:35Z

Missing

Give the User the possibility to decide which model to contribute to.
Check if the User can run Swarm mode (only available for Mac users), otherwise don't enable the functionality.
Give the User the possibility to decide how many blocks to contribute giving some tooltips in terms of how much memory is required per block.

… feat/swarm-mode

nsosio · 2023-10-12T12:07:10Z

Give the User the possibility to decide which model to contribute to.
Check if the User can run Swarm mode (only available for Mac users), otherwise don't enable the functionality.
Give the User the possibility to decide how many blocks to contribute giving some tooltips in terms of how much memory is required per block.

@casperdcl can you check it? @Janaka-Steph can you enhance the style of swarm settings?

nsosio · 2023-10-16T16:30:56Z

@filopedraz, @tiero should we update it as user_name@premAI or keep it as user_name? Where user name is what specified by the user (with default prem-app)

tiero · 2023-10-16T16:34:04Z

src-tauri/petals/build_petals_sidecar.sh

+pip install -r $REQUIREMENTS_FILE_PATH
+
+# Package the Python script
+pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-x86_64-apple-darwin


This will not work on all architectures, please build one for each architecture

Suggested change

pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-x86_64-apple-darwin

pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-x86_64-apple-darwin

pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-aarch64-apple-darwin

I think this will not be enough. If is not possible to make an universal binary with single name, we must package both in the App

dont trust, verify (its the AI) https://chat.openai.com/share/2a996ae8-abe3-4b50-a13a-3a6c8c41afbd

If we manage to make an universal binary, we should them call it petals-universal-apple-darwin otherwsie when you will be up to compile

Error failed to bundle project: Failed to copy external binaries: Failed to copy external binaries: path matching bin/python/petals-universal-apple-darwin not found.: path matching bin/python/petals-universal-apple-darwin not found.

if not possible to have universal binary, we must also change how we build the desktop app and file naming (splitting Apple SIlicon and Apple Intel) on website, documentations etc..

universal bin may or may not be possible¹:

Suggested change

pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-x86_64-apple-darwin

pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean --target-arch universal2 -n petals-universal-apple-darwin

plus tauri build --target universal-apple-darwin²³

Note, however, that this increases your bundle size significantly⁴

Footnotes

https://pyinstaller.org/en/stable/feature-notes.html#macos-multi-arch-support ↩

https://tauri.app/v1/api/cli/#build ↩

https://github.com/premAI-io/prem-app/blob/a7df7ed5c915dd0fcd79525cb78314a3e7b19faf/.github/workflows/on-tag.yaml#L97 ↩

https://tauri.app/v1/guides/building/macos/#binary-targets ↩

package.json

tiero · 2023-10-16T16:46:34Z

@filopedraz, @tiero should we update it as user_name@premAI or keep it as user_name? Where user name is what specified by the user (with default prem-app)

We should start with a default name taken from user-space ie. $whoami@premAI, and not sure how much worth is to allow user to change it 🤔

filopedraz · 2023-10-16T17:11:19Z

@tiero it will be a public name. User should be able to provide what they want.

tiero · 2023-10-16T17:20:17Z

I did little tweaks to UI and compiled also aarch64 version to test for Apple SIlicon, please @nsosio pull before commiting more work

tiero · 2023-10-16T17:20:58Z

@tiero it will be a public name. User should be able to provide what they want.

Ok I will reintroduce the Input, but once they start the swarm

tiero · 2023-10-16T17:29:37Z

Starting the Swarm on any model crashes once click on Start
https://www.loom.com/share/43213eaf03b14a8a840488cf91150a1b

nsosio · 2023-10-17T15:37:32Z

enabled swarm-mode only for aarch64
petals sidecar only for aarch64
added get_username Tauri command retrieving whoami
moved script for petals binary to scripts

CC @filopedraz @casperdcl @tiero

filopedraz · 2023-10-17T18:45:00Z

@nsosio LGTM. @tiero can you update the UI accordingly?

…ersal-apple-darwin

tiero · 2023-10-18T12:41:30Z

Currently does not work on my Mac M1, investigating. @filopedraz If you have time to test on your device let me know if you experience the same.

filopedraz · 2023-10-18T14:05:30Z

Just tested, to me works.

casperdcl · 2023-10-18T14:58:53Z

working on this in a separate repo (https://github.com/premAI-io/test-app/actions)...

any idea why tauri build fails with "path matching bin/python/petals-aarch64-apple-darwin not found" despite src-tauri/bin/python/petals-aarch64-apple-darwin existing?

Janaka-Steph · 2023-10-19T07:56:18Z

It would be nice to move the Petals rust code into its own file rather than in main.rs

casperdcl · 2023-10-19T09:53:01Z

follow-up: apparently tauri build --target universal-apple-darwin wants ALL THREE of:

petals-aarch64-apple-darwin
petals-x86_64-apple-darwin
petals-universal-apple-darwin

Just one or two of the above isn't enough.

casperdcl · 2023-10-19T12:31:22Z

CI-built *.dmg + binaries if anyone wants to test things :)

added support for swarm mode

8e17313

casperdcl reviewed Sep 2, 2023

View reviewed changes

Janaka-Steph approved these changes Sep 4, 2023

View reviewed changes

stable petals release fixed version

29b1cdd

filopedraz mentioned this pull request Sep 9, 2023

Nomad as Orchestrator premAI-io/prem-daemon#140

Open

5 tasks

filopedraz and others added 3 commits September 19, 2023 13:23

solved conflicts

f9e02fd

Fix style + add PETALS tag

f1f029b

Merge pull request #1 from Janaka-Steph/petals-swarm

66ec10a

Petals swarm

casperdcl approved these changes Sep 20, 2023

View reviewed changes

filopedraz added 3 commits October 9, 2023 18:25

Merge branch 'main' into feat/swarm-mode

e4f2516

Merge branch 'feat/swarm-mode' of github.com:filopedraz/prem-app into…

ed72704

… feat/swarm-mode

updated small fix readme

daa95ee

casperdcl assigned nsosio Oct 9, 2023

casperdcl added the enhancement New feature or request label Oct 9, 2023

nsosio added 8 commits October 10, 2023 09:53

show swarm mode only on macos

2dc2f49

added num_blocks parameter; start petals does not work

35576fb

clean up

5a05001

bugfix

eb72a0f

added hardcoded model selection

7315bfe

added model selection

6d375d3

added username from whomai (with prem-app as fallback)

e353f94

removed max val for num blocks

e3117bd

build fixed

81611a6

tiero reviewed Oct 16, 2023

View reviewed changes

package.json Outdated Show resolved Hide resolved

fix: supports Apple Silicon and little UI tweaks

d733d93

nsosio added 3 commits October 16, 2023 23:10

moved petals sidecar script to scripts; bugfix in SwarmMode

cf2b920

added get_username tauri command

1f46717

added swarm-mode only for aarch64

8dbca3e

renamed petals binary from petals-aarch64-apple-darwin to petals-univ…

e6aea1d

…ersal-apple-darwin

casperdcl added 2 commits October 18, 2023 14:47

CI: workflow tidy

c6670fe

build_petals: parameterise for arches

e2a23d4

casperdcl force-pushed the feat/swarm-mode branch from 66102ba to 293d52d Compare October 18, 2023 14:11

casperdcl force-pushed the main branch from 6589bf4 to a7df7ed Compare October 18, 2023 14:14

casperdcl added 2 commits October 18, 2023 15:15

CI: build petals

1075901

CI: temp PR build

dfccace

casperdcl force-pushed the feat/swarm-mode branch from 293d52d to dfccace Compare October 18, 2023 14:16

nsosio mentioned this pull request Oct 23, 2023

Swarm mode with python env #393

Merged

tiero closed this Oct 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support for Swarm Mode #344

Added support for Swarm Mode #344

filopedraz commented Sep 1, 2023

what-the-diff bot commented Sep 1, 2023

casperdcl Sep 2, 2023

filopedraz Sep 5, 2023

casperdcl Sep 7, 2023

Janaka-Steph left a comment

filopedraz commented Sep 4, 2023

casperdcl commented Sep 4, 2023

filopedraz commented Sep 5, 2023

casperdcl left a comment •

edited

Loading

filopedraz commented Sep 20, 2023

filopedraz commented Sep 23, 2023

nsosio commented Oct 12, 2023

nsosio commented Oct 16, 2023

tiero Oct 16, 2023 •

edited

Loading

tiero Oct 16, 2023

tiero Oct 16, 2023

tiero Oct 16, 2023

tiero Oct 16, 2023

casperdcl Oct 16, 2023 •

edited

Loading

tiero commented Oct 16, 2023

filopedraz commented Oct 16, 2023

tiero commented Oct 16, 2023 •

edited

Loading

tiero commented Oct 16, 2023

tiero commented Oct 16, 2023

nsosio commented Oct 17, 2023

filopedraz commented Oct 17, 2023

tiero commented Oct 18, 2023

filopedraz commented Oct 18, 2023

casperdcl commented Oct 18, 2023 •

edited

Loading

Janaka-Steph commented Oct 19, 2023

casperdcl commented Oct 19, 2023

casperdcl commented Oct 19, 2023 •

edited

Loading

	pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-x86_64-apple-darwin
	pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-x86_64-apple-darwin
	pyinstaller --onefile $PYTHON_SCRIPT_PATH --distpath $DIST_PATH --clean -n petals-aarch64-apple-darwin

Added support for Swarm Mode #344

Added support for Swarm Mode #344

Conversation

filopedraz commented Sep 1, 2023

Description

Limitations

what-the-diff bot commented Sep 1, 2023

PR Summary

casperdcl Sep 2, 2023

Choose a reason for hiding this comment

filopedraz Sep 5, 2023

Choose a reason for hiding this comment

casperdcl Sep 7, 2023

Choose a reason for hiding this comment

Janaka-Steph left a comment

Choose a reason for hiding this comment

filopedraz commented Sep 4, 2023

casperdcl commented Sep 4, 2023

filopedraz commented Sep 5, 2023

casperdcl left a comment • edited Loading

Choose a reason for hiding this comment

filopedraz commented Sep 20, 2023

filopedraz commented Sep 23, 2023

Missing

nsosio commented Oct 12, 2023

nsosio commented Oct 16, 2023

tiero Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

tiero Oct 16, 2023

Choose a reason for hiding this comment

tiero Oct 16, 2023

Choose a reason for hiding this comment

tiero Oct 16, 2023

Choose a reason for hiding this comment

tiero Oct 16, 2023

Choose a reason for hiding this comment

casperdcl Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

Footnotes

tiero commented Oct 16, 2023

filopedraz commented Oct 16, 2023

tiero commented Oct 16, 2023 • edited Loading

tiero commented Oct 16, 2023

tiero commented Oct 16, 2023

nsosio commented Oct 17, 2023

filopedraz commented Oct 17, 2023

tiero commented Oct 18, 2023

filopedraz commented Oct 18, 2023

casperdcl commented Oct 18, 2023 • edited Loading

Janaka-Steph commented Oct 19, 2023

casperdcl commented Oct 19, 2023

casperdcl commented Oct 19, 2023 • edited Loading

casperdcl left a comment •

edited

Loading

tiero Oct 16, 2023 •

edited

Loading

casperdcl Oct 16, 2023 •

edited

Loading

tiero commented Oct 16, 2023 •

edited

Loading

casperdcl commented Oct 18, 2023 •

edited

Loading

casperdcl commented Oct 19, 2023 •

edited

Loading