Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
fstagni committed Oct 4, 2024
1 parent 6a5c986 commit 32b5695
Show file tree
Hide file tree
Showing 5 changed files with 89 additions and 52 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/create.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:

- name: Install dependencies and build
run: |
npm install @iconify-json/clarity @iconify-json/devicon @slidev/cli @slidev/theme-default slidev-theme-hep slidev-theme-neversink
npm install @iconify-json/clarity @iconify-json/devicon @slidev/cli @slidev/theme-default slidev-theme-hep slidev-theme-neversink playwright-chromium
npm run build -- --base /${{ github.event.repository.name }}/
npm list
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "1.0.0",
"main": "index.js",
"scripts": {
"build": "slidev build",
"build": "slidev build --download",
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
Expand Down
Binary file added public/images/WebApp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added public/images/fcc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
137 changes: 87 additions & 50 deletions slides.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,26 +55,57 @@ color: gray-light
---

<div style="display: flex; flex-wrap: wrap; justify-content: space-around; gap: 20px;">
<img id="DIRAC" src="/public/images/Juno.jpeg" alt="Juno" style="width: 80px;">
<img id="DIRAC" src="/public/images/Belle2_logo.png" alt="Belle2" style="width: 80px;">
<img id="DIRAC" src="/public/images/CTA.jpg" alt="CTA" style="width: 80px;">
<img id="DIRAC" src="/public/images/ilc.png" alt="ILC" style="width: 80px;">
<img id="DIRAC" src="/public/images/LHCb.png" alt="LHCb" style="width: 80px;">
<img id="DIRAC" src="/public/images/nica.jpg" alt="nica" style="width: 80px;">
<img id="DIRAC" src="/public/images/BES3.png" alt="BES3" style="width: 80px;">
<img id="DIRAC" src="/public/images/GridPP-Logo.png" alt="gridPP" style="width: 80px;">
<img id="DIRAC" src="/public/images/pierre-auger-logo.png" alt="pierre-auger" style="width: 80px;">
<img id="DIRAC" src="/public/images/EGI.png" alt="EGI" style="width: 80px;">
<img id="DIRAC" src="/public/images/cepc.png" alt="cepc" style="width: 80px;">
<img id="DIRAC" src="/public/images/na62.jpeg" alt="na62" style="width: 80px;">
<img id="DIRAC" src="/public/images/t2k.png" alt="t2k" style="width: 80px;">
<img id="DIRAC" src="/public/images/WeNMR-logo.png" alt="weNMR" style="width: 80px;">
<img id="DIRAC" src="/public/images/LogoHyperK.png" alt="hyperk" style="width: 80px;">
<img id="DIRAC" src="/public/images/euclid.jpeg" alt="euclid" style="width: 80px;">
<img id="DIRAC" src="/public/images/LZ.png" alt="lz" style="width: 80px;">
<img id="Juno" src="/public/images/Juno.jpeg" alt="Juno" style="width: 80px;">
<img id="Belle2" src="/public/images/Belle2_logo.png" alt="Belle2" style="width: 80px;">
<img id="CTA" src="/public/images/CTA.jpg" alt="CTA" style="width: 80px;">
<img id="ilc" src="/public/images/ilc.png" alt="ILC" style="width: 80px;">
<img id="LHCb" src="/public/images/LHCb.png" alt="LHCb" style="width: 80px;">
<img id="nica" src="/public/images/nica.jpg" alt="nica" style="width: 80px;">
<img id="BES3" src="/public/images/BES3.png" alt="BES3" style="width: 80px;">
<img id="GriPP" src="/public/images/GridPP-Logo.png" alt="gridPP" style="width: 80px;">
<img id="PA" src="/public/images/pierre-auger-logo.png" alt="pierre-auger" style="width: 80px;">
<img id="EGI" src="/public/images/EGI.png" alt="EGI" style="width: 80px;">
<img id="cepc" src="/public/images/cepc.png" alt="cepc" style="width: 80px;">
<img id="na62" src="/public/images/na62.jpeg" alt="na62" style="width: 80px;">
<img id="t2k" src="/public/images/t2k.png" alt="t2k" style="width: 80px;">
<img id="weNMR" src="/public/images/WeNMR-logo.png" alt="weNMR" style="width: 80px;">
<img id="hyperk" src="/public/images/LogoHyperK.png" alt="hyperk" style="width: 80px;">
<img id="euclid" src="/public/images/euclid.jpeg" alt="euclid" style="width: 80px;">
<img id="LZ" src="/public/images/LZ.png" alt="lz" style="width: 80px;">
<img id="fcc" src="/public/images/fcc.png" alt="lz" style="width: 80px;">
</div>


---
layout: top-title
color: gray-light
align: l
title: history
---

:: title ::

# DIRAC timeline

:: content ::

```mermaid
timeline
section LHCb software
around 2000 : MC production system: bash scripts running at production sites
2002 : DIRAC2 <br> Rewritten in Python, using xml-rpc, interfacing to EDG
Data Challenge 04 : First successful grid usage ever.
: First use of pilot jobs based WMS
2006-2007 : DIRAC3<br> Full rewriting, development of the DISET protocol -- still in use today!
: the current DIRAC framework is still based on this work
section Open sourced, wider adoption
2008 : Large-ish reshuffling to become multi-VO
: LHCbDIRAC extension separated from Vanilla DIRAC code
2009 : CLIC community adopts DIRAC
2011 : France-Grilles is the first multi-VO DIRAC installation
2012 : Belle2, BES3, CTA adopt DIRAC
2021 : Python3 full support
```


---
Expand All @@ -86,7 +117,7 @@ title: WMS

:: title ::

# Workload Management System
# WMS: Workload Management System
- Pull model based on Pilot jobs
- Also "Push" solution for HPCs that do not support pilots (because of limited internet access).
- Will integrate [CWL (Common Workflow Language)](https://www.commonwl.org) as a way of defining jobs (replacing JDL)
Expand Down Expand Up @@ -126,15 +157,14 @@ titlewidth: is-5

:: title ::

# Data Management System
# DMS: Data Management System
It’s about **files**:​ placing, replicating, removing files​

- there are **LFNs** (logical file names) → and users ONLY work with these​
- **LFNs** are registered in *catalog(s)​*
- where are the LFNs? (in the DIRAC File Catalog (DFC), or in Rucio)​
- what are their metadata? (in the DFC, or in the LHCb Bookkeeping, or in AMGA)​
- LFNs *may* have **PFNs**, stored in **SEs**.​
- You can access those PFNs with several protocols.​
- LFNs *may* have **PFNs**, stored in **SEs**, that can accessed with several protocols.​

:: content ::

Expand Down Expand Up @@ -166,12 +196,12 @@ title: Productions

- A `production` is a *Data Processing transformation* (e.g. Simulation, Merge, DataReconstruction…). A Production creates jobs in the WMS (and re-sumbit them, and destroy them).​

- (Unless you are using Rucio) a *Data Manipulation transformation* replicates, or remove, data from storage elements.
- (Unless you are using Rucio) a *Data Manipulation* `transformation` replicates, or remove, data from storage elements.

:: content ::

<span class="bg-cyan-100 text-cyan-600 p-4 border-l-6 border-2 border-cyan-400 rounded-lg pl-8 pr-8 w-full block">
The **Transformation System (TS)** is used to automate common tasks related to production activities
The Transformation System (TS) is used to automate common tasks related to production activities
</span>

&nbsp;
Expand All @@ -193,35 +223,30 @@ TransformationSystem((TransformationSystem))
```

---
layout: top-title
layout: two-cols
align: rm-lm
color: gray-light
align: l
title: history
title: webapp
---

:: title ::
:: left ::

# Brief history of DIRAC

:: content ::
# Tech stack:

- DIRAC is written in python3
- Services are exposed at urls like `dips://box.some.where:9132/WorloadManagement/`
- `dips` stands for "DIRAC protocol"
- The DIRAC framework provides also "Agents" and "Executors" to animate the system
- As backends, MySQL and OpenSearch are supported (for different purposes)
- DIRAC also provides a WebApp

:: right ::

<div>
<img id="WebApp" src="/public/images/WebApp.png"">
</div>

```mermaid
timeline
section LHCb software
around 2000 : MC production system: bash scripts running at production sites
2002 : DIRAC2 <br> Rewritten in Python, using xml-rpc, interfacing to EDG
Data Challenge 04 : First successful grid usage ever.
: First use of pilot jobs based WMS
2006-2007 : DIRAC3<br> Full rewriting, development of the DISET protocol -- still in use today!
: the current DIRAC framework is still based on this work
section Open sourced, wider adoption
2008 : Large-ish reshuffling to become multi-VO
: LHCbDIRAC extension separated from Vanilla DIRAC code
2009 : CLIC community adopts DIRAC
2011 : France-Grilles is the first multi-VO DIRAC installation
2012 : Belle2, BES3, CTA adopt DIRAC
2021 : Python3 full support
```

---
layout: side-title
Expand All @@ -237,7 +262,7 @@ title: issues

:: content ::

<ul class="text-sm mx-auto">
<ul class="font-sans text-sm mx-auto">
<li> complex, with high entrance bar</li>
<li> somewhat cumbersome deployment</li>
<li> late on “standards”
Expand Down Expand Up @@ -293,11 +318,11 @@ color: gray-light

# DiracX Web API

- DIRAC Web APIs are developed using FastAPI
- Nicely documented in Swagger (or Redoc), this what you see on the right
- DIRAC Web APIs are developed using FastAPI <span class="iconify" data-icon="devicon:fastapi-wordmark" data-inline="false"></span>
- Nicely documented in Swagger (or Redoc), this what you see on the right <span class="iconify" data-icon="devicon:swagger-wordmark" data-inline="false"></span>

<AdmonitionType type='caution' >
What is on the right is the certification WebApp, loaded live. Use with caution!
What is on the right is the certification Web API, loaded live. Use with caution!
</AdmonitionType>

---
Expand Down Expand Up @@ -487,11 +512,23 @@ title: Extensions

:: content ::


Dirac has to support different Communities with different workflows and requirements. For example:
- LHCb ...
- Belle2 ...
- CTA ...
- Astrophysics exp
-


The ol'good DIRAC is extendable: from the main code (DIRAC) to the Web and the Pilot, communites can create/run extensions.

DiracX follows the same path, including DiracX-Web. We also provide a reference extension, dubbed "gubbins"


Extensiona also for the Web


---
layout: top-title
color: gray-light
Expand Down

0 comments on commit 32b5695

Please sign in to comment.