Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs/SK-958 | Improve docs #675

Merged
merged 13 commits into from
Aug 30, 2024
Merged

Docs/SK-958 | Improve docs #675

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
94a0897
Update headings in quickstart to align with the typical user-flow in …
Aug 19, 2024
e0200e7
Improved introduction section
Aug 25, 2024
854aae6
Move APIClient examples from Quickstart to APIClient page
Aug 25, 2024
1403c6a
Work in progress on APIClient page
Aug 25, 2024
2275d11
Move auth into developer guide
Aug 25, 2024
5872de7
latest wip
Aug 25, 2024
c6b3abc
Merge branch 'master' into docs/SK-958
Aug 25, 2024
5576c04
Removed prerequisite in projects page
Aug 25, 2024
4495dc9
wip
Aug 27, 2024
263d1fe
Rework the project page, add code example
Aug 28, 2024
7ce942c
Fixed test compute package section
Aug 28, 2024
110c188
Removed unused file
Aug 28, 2024
8eec540
Fixed API page and some headings
Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 35 additions & 27 deletions docs/quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,24 @@ Getting started with FEDn
- `A FEDn Studio account <https://fedn.scaleoutsystems.com/signup>`__


1. Start a FEDn Studio Project
------------------------------
1. Start the server
--------------------

Start by creating an account in Studio. Head over to `fedn.scaleoutsystems.com/signup <https://fedn.scaleoutsystems.com/signup/>`_ and sign up.
The first step is to start the server side (aggregator, controller). We do this by setting up a new Project in FEDn Studio.
Start by creating an account in Studio: `fedn.scaleoutsystems.com/signup <https://fedn.scaleoutsystems.com/signup/>`_.

Logged into Studio, create a new project by clicking on the "New Project" button in the top right corner of the screen.
You will see a Studio project similar to the image below. The Studio project provides all the necessary server side components of FEDn.
We will use this project in a later stage to run the federated experiments. But first, we will set up the local client.

You will see a Studio project similar to the image below. The Studio project provides a secure and managed deployment of all the necessary server side components.

.. image:: img/studio_project_overview.png

2. Prepare the clients
-----------------------

Next, we will prepare and package the ML code to be executed by each client and create a first version of the global model (seed model).
We will work with one of the pre-defined projects in the FEDn repository, ``mnist-pytorch``.

2. Install FEDn on your client
-------------------------------
First install the FEDn API on your local machine (client):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API and client-side libraries.


**Using pip**

Expand All @@ -49,10 +52,7 @@ It is recommended to use a virtual environment when installing FEDn.

.. _package-creation:

Next, we will prepare the client. We will use one of the pre-defined projects in the FEDn repository, ``mnist-pytorch``.

3. Create the compute package and seed model
--------------------------------------------
**Create the compute package and seed model**

In order to train a federated model using FEDn, your Studio project needs to be initialized with a ``compute package`` and a ``seed model``. The compute package is a code bundle containing the
code used by the client to execute local training and local validation. The seed model is a first version of the global model.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to give them the instructions to generate compute package and seed model:

Locate into the fedn/examples/mnist-pytorch folder.

fedn package create --path client

This will create a file called package.tgz in the root of the project.

Next, create the seed model:

fedn run build --path client

This will create a file called seed.npz in the root of the project.

Expand Down Expand Up @@ -90,14 +90,13 @@ This will create a file called ``seed.npz`` in the root of the project.

Next will now upload these files to your Studio project:

4. Initialize your FEDn Studio Project
--------------------------------------
**Initialize your FEDn Studio Project**

In the Studio UI, navigate to the project you created above and click on the "Sessions" tab. Click on the "New Session" button. Under the "Compute package" tab, select a name and upload the generated package file. Under the "Seed model" tab, upload the generated seed file:
In the Studio UI, navigate to the project you created in step one and click on the "Sessions" tab. Click on the "New Session" button. Under the "Compute package" tab, select a name and upload the generated package file. Under the "Seed model" tab, upload the generated seed file:

.. image:: img/upload_package.png

**Upload the package and seed model using the Python APIClient**
** (Alternative) Upload the package and seed model using the Python APIClient**

It is also possible to upload a package and seed model using the Python API Client.

Expand All @@ -116,8 +115,13 @@ To upload the package and seed model using the APIClient:
>>> client.set_active_model("seed.npz")


5. Configure and attach clients
-------------------------------
3. Start clients
-----------------

Now we are ready to start FEDn clients on your local machine. There are two steps involved:

1. Register a new client in your Studio project, issuing an access token.
2. Start up a client process on your local host (using the token to connect securely)

**Generate an access token for the client (in Studio)**

Expand All @@ -138,7 +142,7 @@ A normal laptop should be able to handle several clients for this example.

**Modifying the data split (multiple-clients, optional):**

The default traning and test data for this particular example (mnist-pytorch) is for convenience downloaded and split automatically by the client when it starts up (see the 'startup' entrypoint).
The default traning and test data for this particular example (mnist-pytorch) is for convenience downloaded and split automatically by the client when it starts up.
The number of splits and which split to use by a client can be controlled via the environment variables ``FEDN_NUM_DATA_SPLITS`` and ``FEDN_DATA_PATH``.
For example, to split the data in 10 parts and start a client using the 8th partiton:

Expand All @@ -161,15 +165,19 @@ For example, to split the data in 10 parts and start a client using the 8th part
fedn client start -in client.yaml --secure=True --force-ssl


6. Start a training session
---------------------------
4. Training
--------------

With clients connected, we are now ready to train the global model. This can be done using either the Studio dashboard or the Python API. In FEDn, training is organised
in Sessions. One training session consists of a configurable number of training rounds (local model updates and aggregation).

In Studio click on the "Sessions" link, then the "New session" button in the upper right corner. Click the "Start session" tab and enter your desirable settings (the default settings are good for this example) and hit the "Start run" button.
In the terminal where your are running your client you should now see some activity. When a round is completed, you can see the results on the "Models" page.

**Watch the training progress**
**Watch real-time updates of training progress**

Once a training session is started, you can monitor the progress of the training by navigating to "Sessions" and click on the "Open" button of the active session. The session page will list the models as soon as they are generated.
Once a training session is started, you can monitor the progress by clicking the drop-down button for the active Sessions and the clicking on the "View session" button. The session page will show
metrics related to the training progress (accuracy, loss etc), as well as performance data such as total round times and individual client training times. A list of models in the session is updated as soon as new models are generated.
To get more information about a particular model, navigate to the model page by clicking the model name. From the model page you can download the model weights and get validation metrics.

.. image:: img/studio_model_overview.png
Expand All @@ -179,7 +187,7 @@ To get more information about a particular model, navigate to the model page by
Congratulations, you have now completed your first federated training session with FEDn! Below you find additional information that can
be useful as you progress in your federated learning journey.

**Control training sessions using the Python APIClient**
**Run training sessions using the Python APIClient**

You can also issue training sessions using the APIClient:

Expand Down Expand Up @@ -214,7 +222,7 @@ You can also access global model updates via the APIClient:
**Where to go from here?**
--------------------------

With you first FEDn federated project set up, we suggest that you take a close look at how a FEDn project is structured
With you first FEDn federated project set up, we suggest that you take a closer look at how a FEDn project is structured
and how you develop your own FEDn projects:

- :ref:`projects-label`
Expand All @@ -223,8 +231,8 @@ You can also dive into the architecture overview to learn more about how FEDn is

- :ref:`architecture-label`

For developers looking to cutomize FEDn and develop own aggregators, check out the local development guide.
This page also has instructions for using Docker to run clients:
For developers looking to customize FEDn and develop own aggregators, check out the local development guide
to learn how to set up an all-in-one deployment. This page also has instructions for using Docker to run clients:

- :ref:`developer-label`

Expand Down
Loading