Skip to content
This repository has been archived by the owner on Aug 15, 2022. It is now read-only.

Add All hands reference to the homepage #361

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
b2f140b
Update deployment
CannonLock Mar 31, 2022
eba682c
Fire Action
CannonLock Mar 31, 2022
0160c02
Fire Action
CannonLock Mar 31, 2022
3e2339c
Fire Action
CannonLock Mar 31, 2022
ad84548
Fire Action
CannonLock Mar 31, 2022
65c6cbd
Add All hands reference to the homepage
CannonLock Apr 5, 2022
54c5d31
Add missing scss file
CannonLock Apr 5, 2022
b85f62b
Merge remote-tracking branch 'opensciencegrid/master'
CannonLock Apr 5, 2022
745fa66
Add link to presentation materials
CannonLock Apr 5, 2022
5c04582
Text update
CannonLock Apr 6, 2022
c64c2db
Update the index page
CannonLock Apr 6, 2022
03ba749
Update the event page organization
CannonLock Apr 6, 2022
878b80c
Add a preview branch
CannonLock Apr 6, 2022
663e859
Update text
CannonLock Apr 6, 2022
8544da5
Merge remote-tracking branch 'opensciencegrid/master'
CannonLock Apr 7, 2022
73050dc
Populate the CNAME of opensciencegrid
CannonLock Apr 7, 2022
a87bde3
Update the existing OSPool page and redirect a now defunct page
CannonLock Apr 7, 2022
721060b
Update preview deployment
CannonLock Apr 7, 2022
72ee27d
Change URL Pointer
CannonLock Apr 7, 2022
43a6835
Update deploy-production-opensciencegrid.yml
CannonLock Apr 8, 2022
f6bc1a9
Update _about/open_science_pool.md
CannonLock Apr 8, 2022
0c2979a
Merge pull request #1 from osg-htc/preview-ospool-changes
CannonLock Apr 8, 2022
cc1b4b2
Merge in opensciencegrid master
CannonLock Apr 11, 2022
95f54f7
Merge remote-tracking branch 'upstream/master'
CannonLock Apr 11, 2022
d0205b5
Merge pull request #4 from CannonLock/add-josies-news
CannonLock Apr 11, 2022
dd498cc
Add News
CannonLock Apr 11, 2022
4812ac0
Add date to news items
CannonLock Apr 11, 2022
dd6f36d
Add functionality to remove after a date
CannonLock Apr 11, 2022
4e1fe35
Merge pull request #5 from CannonLock/job-posting
CannonLock Apr 11, 2022
4cb6d98
Add Employees
CannonLock Apr 15, 2022
74220d8
Update CC* page
CannonLock Apr 15, 2022
6292916
Update CC* page
CannonLock Apr 15, 2022
cbe9a04
Update CC* page
CannonLock Apr 15, 2022
6883848
Update CC* page
CannonLock Apr 15, 2022
6483161
Update CC* page
CannonLock Apr 15, 2022
6ac504c
Update CC* page
CannonLock Apr 15, 2022
9207165
Update the User School Event page and post News
CannonLock Apr 18, 2022
f222a5f
Merge pull request #7 from osg-htc/preview-ccstar-update
CannonLock Apr 18, 2022
12f32c8
Merge pull request #8 from osg-htc/preview-school-close
CannonLock Apr 19, 2022
a7641ca
Merge in Master
CannonLock Apr 19, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/delete-preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ on:

jobs:
Delete-Preview-Branch:
if: ${{ github.repository_owner == 'opensciencegrid' }}
if: ${{ github.repository_owner == 'osg-htc' }}
runs-on: ubuntu-18.04
env:
preview_url: https://github.com/opensciencegrid/web-preview.git
preview_url: https://github.com/osg-htc/web-preview.git

steps:
- uses: actions/checkout@v2
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy-preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ on:

jobs:
Deploy-Web-Preview:
if: ${{ github.repository_owner == 'opensciencegrid' }}
if: ${{ github.repository_owner == 'osg-htc' }}
runs-on: ubuntu-18.04
env:
preview_url: https://github.com/opensciencegrid/web-preview.git
preview_url: https://github.com/osg-htc/web-preview.git
steps:
- uses: actions/checkout@v2
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ on:

jobs:
Build-And-Deploy-Production:
if: ${{ github.repository_owner == 'opensciencegrid' }}
if: ${{ github.repository_owner == 'osg-htc' }}
runs-on: ubuntu-18.04
env:
CNAME: osg-htc.org
production_url: https://github.com/opensciencegrid/osg-htc.git
OVERRIDE_CONFIG: _config_osg-htc_org.yml
CNAME: opensciencegrid.org
production_url: https://github.com/opensciencegrid/opensciencegrid.github.io
OVERRIDE_CONFIG: _config_opensciencegrid_org.yml

steps:
- uses: actions/checkout@v2
Expand All @@ -24,11 +24,11 @@ jobs:
- name: Setup SSH Keys and known_hosts
uses: webfactory/[email protected]
with:
ssh-private-key: ${{ secrets.OSG_HTC_DEPLOY_KEY }}
ssh-private-key: ${{ secrets.OPENSCIENCEGRID_ORG_DEPLOY_KEY }}

- name: Initialize GH User
run: |
git config user.name "GitHub Actions"
git config user.name "GitHub Actions "
git config user.email "[email protected]"

- uses: ruby/setup-ruby@v1
Expand All @@ -50,4 +50,4 @@ jobs:
git checkout -b split-branch `git subtree split --prefix _site master`

# Push to Production
git push ${{ env.production_url }} HEAD:main --force
git push ${{ env.production_url }} HEAD:from-osg-htc --force
4 changes: 2 additions & 2 deletions .github/workflows/deploy-production-same-repo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ on:

jobs:
Build-And-Deploy-Production:
if: ${{ github.repository_owner == 'opensciencegrid' }}
if: ${{ github.repository_owner == 'osg-htc' }}
runs-on: ubuntu-18.04
env:
CNAME: opensciencegrid.org
CNAME: osg-htc.org
OVERRIDE_CONFIG: _config_opensciencegrid_org.yml

steps:
Expand Down
111 changes: 16 additions & 95 deletions _about/computation-ideal-for-OSPool.md
Original file line number Diff line number Diff line change
@@ -1,99 +1,20 @@
---
title: Computation on the OSPool
layout: blank
redirect_url: "/about/open_science_pool"
---
<figure class="figure">
<img src="{{site.baseurl}}/assets/images/osg_pool_lab.png" class="figure-img img-fluid rounded" alt="Open Science Pool Users">
</figure>

## What size of computations are ideal for the OSPool?

> For problems that can be run as many independent jobs, as in the first two columns of the table below, the OSPool provides computing capacity that can transform the types of questions that researchers are able to tackle. Importantly, many compute tasks that may appear to not be a good fit can be modified in simple ways to take advantage, and we'd love to discuss options with you!

<table class="table table-bordered table-striped">
<tbody>
<tr>
<th></th>
<th>Ideal Jobs!</th>
<th>Still very advantageous</th>
<th>Maybe not, but get in touch!</th>
</tr>
<tr>
<th>Expected Throughput, per user</th>
<td>1000s concurrent cores</td>
<td>100s concurrent cores</td>
<td>Let's discuss!</td>
</tr>
<tr>
<th>CPU</th>
<td>1</td>
<td>< 8</td>
<td>> 8 (or MPI)</td>
</tr>
<tr>
<th>Walltime</th>
<td>< 10 hrs*</td>
<td>< 20 hrs*</td>
<td>> 20 hrs</td>
</tr>
<tr>
<th>RAM</th>
<td>< few GB</td>
<td>< 40 GB</td>
<td>> 40 GB</td>
</tr>
<tr>
<th>Input</th>
<td>< 500 MB</td>
<td>< 10 GB</td>
<td>> 10 GB**</td>
</tr>
<tr>
<th>Output</th>
<td>< 1 GB</td>
<td>< 10 GB</td>
<td>> 10 GB**</td>
</tr>
<tr>
<th>Software</th>
<td>pre-compiled binaries, containers</td>
<td>Most other than -></td>
<td>Licensed Software, non-Linux</td>
</tr>
</tbody>
</table>

<span class="text-muted">or checkpointable *per job; you can work with a large dataset on OSG if it can be split into pieces</span>

## Some examples of work that has been a good fit for the OSG and benefited from using its resources include

- image analysis (including MRI, GIS, etc.)
- text-based analysis, including DNA read mapping and other bioinformatics
- parameter sweeps
- Monte Carlo methods and other model optimization

[Learn more and chat with a Research Computing Facilitator](https://connect.osg-htc.org/) by requesting an account.

## Running jobs on the OSPool

> Users submitting jobs can specify their own requirements on what machines to use. You can require the use of one particular machine, or any from a particular cluster, or any with a certain amount of memory or processor type.

> We recommend submitting lots of jobs and taking advantage of all the idle cycles, wherever they may be. If you are doing simulation work, don't submit one simulation, submit one hundred or one thousand variations at once. We cannot guarantee that any single job will finish quickly, but it will allow you to accomplish more overall work than if you used just your own machines.

## Where do I go if I need help or have questions?

> We have complete knowledge base materials here and an active and supportive facilitation team.

## What about Campuses that want to Support Researchers and dHTC locally?

> Campuses can

- Build ‘Local’ dHTC capacity with HTCondor, which provides service orchestration services
- Share local resources and excess capacity via OSG
- Take advantage of dHTC and dHTC Facilitation training
- Provide Local submission points into OSPools
- These Services are all Free and Open with Facilitation for Campuses

> Most campuses prefer the hosted Compute Entry (CE) option wherein the OSG team will host and operate the HTCondor Compute Entrypoint, and configure it for the communities the campus chooses to support.

> Other campuses choose to set up their compute entry point locally and use an OSG hosted Compute CE that provides the entry point for resource requests coming from the OSG; it handles authorization and delegation of resource requests to campus’ existing HPC/HTC cluster.
<meta http-equiv="refresh" content="1; url={{ page.redirect_url }}" />
<script>
window.location.href = '{{ page.redirect_url }}';
</script>
<div style="display:flex; height:100vh;">
<div style="margin:auto;">
<div>
<a href="{{ page.redirect_url }}"><img style="width: 100%;" alt="OSG Logo" src="{{ '/assets/images/logos/OSG-logo.svg' | relative_url }}"></a>
</div>
<div style="text-align: center; margin-top: 20px;">
<a href="{{ page.redirect_url }}">Click Here to be Redirected to Office Hours!</a>
</div>
</div>
</div>

105 changes: 94 additions & 11 deletions _about/open_science_pool.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,32 +6,115 @@ title: Open Science Pool
<img src="{{site.baseurl}}/assets/images/OSGPools.jpg" class="figure-img img-fluid rounded" alt="Open Science Pool on the OSG">
</figure>

## What is the Open Science Pool?
# What is the Open Science Pool?

> The Open Science Pool (OSPool) is a virtual cluster operated by the OSG, with shared computing and data resources using distributed high-throughput computing (dHTC). The pool aggregates computing resources contributed by clusters across and beyond the US, making them usable by the US-based open science community, from individual researchers and research groups at campuses, to multi-institutional collaborations with a US constituency.
The Open Science Pool (OSPool) is a virtual cluster operated by the OSG,
with shared computing and data resources via distributed high-throughput
computing (dHTC) technologies. The pool aggregates mostly opportunistic
(“backfill”) computing resources from contributing clusters at campuses
and other organizations, making them available to the US-based open science
community.

Researchers can submit computational work to the OSPool via access points operated locally to their campuses, or via access points operated as part of the [OSG Connect](https://connect.osg-htc.org/) service, which serves researchers affiliated with projects at US-based academic, non-profit, and government institutions.
Researchers can submit computational work to the OSPool via access points
operated locally to their campuses, or via access points operated as part
of the [OSG Connect](https://connect.osg-htc.org/) service, which serves researchers affiliated with
projects at US-based academic, non-profit, and government institutions.
Sign up for an account on [OSG Connect](https://connect.osg-htc.org/), today!

## Who can use the OSPool?

> Access is free and open to
**Access is free and open to**
- any researcher affiliated with a project at a US-based academic, government, or non-profit institution (via the OSG Connect service).
- any researcher affiliated with such an institution or project with its own access point.
- all areas of research including social sciences, humanities, life sciences, engineering, medicine, chemistry and physics.

>- any researcher affiliated with a project at a US-based academic, government, or non-profit institution (via the OSG Connect service).
>- any researcher affiliated with an organization that has its own access point (not limited to US affiliation).
>- all areas of research including social sciences, humanities, life sciences, engineering, medicine, chemistry and physics.
[View projects](https://gracc.opensciencegrid.org/d/000000037/payload-jobs-summary?orgId=1&var-ReportableVOName=osg&var-Project=All&var-Facility=All&var-User=All&var-ExitCode=All&var-Probe=All&var-interval=1d)
using the OSPool in OSG’s Accounting System (GRACC).

## What types of work run well on the OSPool?

> Work that can be executed as independent laptop-sized computations and individually complete in less than 20 hours run well in the OSPool. A wide range of research problems and computational methods can be broken up or otherwise executed this way, including:
For problems that can be run as numerous, self-contained jobs, the OSPool provides computing capacity that can transform the types of questions researchers are able to tackle (see the table below). A wide range of research problems and computational methods can be broken up or otherwise executed in this high-throughput computing (HTC ) approach, including:

- image analysis (including MRI, GIS, etc.)
- text-based analysis, including DNA read mapping and other bioinformatics
- parameter sweeps
- model optimization approaches, including Monte Carlo methods
- Machine learning and AI executed with multiple independent training tasks, different parameters, and/or data subsets
- machine learning and AI executed with multiple independent training tasks, different parameters, and/or data subsets

>Learn more and chat with a Research Computing Facilitator by [requesting an account](https://connect.osg-htc.org/).
The OSPool is made up of mostly opportunistic capacity - contributing clusters may interrupt jobs at any time. Thus, the OSPool supports workloads of numerous jobs that individually complete or checkpoint within 20 hours.

__Importantly, many compute tasks can take advantage of the OSPool with simple modifications, and we’d love to discuss options with you!__

<table class="table table-bordered table-striped">
<tbody>
<tr>
<th></th>
<th>Ideal Jobs!</th>
<th>Still very advantageous</th>
<th>Maybe not, but get in touch!</th>
</tr>
<tr>
<th>Expected Throughput, per user</th>
<td>1000s concurrent cores</td>
<td>100s concurrent cores</td>
<td>Let's discuss!</td>
</tr>
<tr>
<th>CPU</th>
<td>1 per job</td>
<td>< 8 per job</td>
<td>> 8 per job</td>
</tr>
<tr>
<th>Walltime</th>
<td>< 10 hrs*</td>
<td>< 20 hrs*</td>
<td>> 20 hrs</td>
</tr>
<tr>
<th>RAM</th>
<td>< few GB</td>
<td>< 40 GB</td>
<td>> 40 GB</td>
</tr>
<tr>
<th>Input</th>
<td>< 500 MB</td>
<td>< 10 GB</td>
<td>> 10 GB**</td>
</tr>
<tr>
<th>Output</th>
<td>< 1 GB</td>
<td>< 10 GB</td>
<td>> 10 GB**</td>
</tr>
<tr>
<th>Software</th>
<td>pre-compiled binaries, containers</td>
<td>Most other than &#8594;</td>
<td>Licensed Software, non-Linux</td>
</tr>
</tbody>
</table>

*or checkpointable

** per job; you can work with a large dataset on OSG if it can be split into pieces


Learn more and chat with a Research Computing Facilitator by [requesting an account](https://connect.osg-htc.org/).

## Learning to use the OSPool

We have a complete [knowledge base of user documentation](https://support.opensciencegrid.org/support/home)
and an active and supportive [facilitation team](https://support.opensciencegrid.org/support/solutions/articles/12000084585),
who support all users on [OSG Connect](https://connect.osg-htc.org/) access points.

Users submitting jobs can specify their own requirements for per-job compute resources (e.g. CPU cores, memory, etc.) and any special server requirements. We recommend submitting lots of jobs and taking advantage of all the idle cycles, wherever they may be. We cannot guarantee that any single job will finish quickly, but the OSPool will allow you to accomplish a tremendous amount of work across jobs.

## Who contributes capacity to the OSPool?

> The computing resources for the OSPool are contributed by members of the OSG Compute Federation, typically campuses, government supported supercomputing centers or research collaborations. The members individually determine their policies for contributing resources, including the amount of resources it contributes and when these resources are available. In addition, some resource providers decide to share their resources with a specific research project, or they may choose to contribute resources to all in the OSPool.
The computing resources for the OSPool are contributed by members of the OSG Compute Federation, typically campuses, government-supported supercomputing centers, and research collaborations. The members individually determine their policies for contributing resources, including the amount of resources it contributes and when these resources are available. In addition, some resource providers decide to share their resources with specific research projects, or they may choose to contribute resources to all in the OSPool.

[View facility contributions](https://gracc.opensciencegrid.org/d/000000043/pilot-jobs-summary?orgId=1&var-VOName=osg&var-Facility=All&var-ExitCode=All&var-Probe=All&var-interval=1d) to the OSPool in OSG’s Accounting System (GRACC).
4 changes: 4 additions & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ include:
- .well-known
- .nojekyll


exclude:
- _spotlights/images

defaults:
- scope:
path: ""
Expand Down
1 change: 0 additions & 1 deletion _data/about.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
docs:
- introduction
- open_science_pool
- computation-ideal-for-OSPool
- osdf
- organization
- team
Expand Down
4 changes: 4 additions & 0 deletions _data/people/patrick-lubben.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
name: Patrick Lubben
image: "/assets/images/team/patrick_lubben.jpeg"
title: Assistant System Administrator
institution: "University of Wisconsin-Madison"
4 changes: 4 additions & 0 deletions _data/people/taylor-halvensleben.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
name: Taylor Halvensleben
image: "/assets/images/team/taylor_halvensleben.png"
title: Office and Communications Assistant
institution: "Morgridge Institute for Research"
Loading