2024.08.07

[seanmcilroy29]

---

2024.08.07 Agenda/Minutes

---

Time: Bi-weekly @ 1700 (BST) - See the time in your timezone

• Co-Chair - Chris Xie- (Futurewei)
• Co-Chair - Vacant
• Convener – Sean Mcilroy (Linux Foundation)

Antitrust Policy

Joint Development Foundation meetings may involve participation by industry competitors, and the Joint Development Foundation intends to conduct all of its activities in accordance with applicable antitrust and competition laws. It is, therefore, extremely important that attendees adhere to meeting agendas and be aware of and not participate in any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

If you have questions about these matters, please contact your company counsel or counsel to the Joint Development Foundation, DLA Piper.

Recordings

WG agreed to record all Meetings. This meeting recording will be available until the next scheduled meeting.
Meeting recording link

Roll Call

Please add 'Attended' to this issue during the meeting to denote attendance.

Any untracked attendees will be added by the GSF team below:

• Full Name, Affiliation, (optional) GitHub username

Agenda

• Approve agenda
• Approve previous Meeting Minutes

Introductions

• Any new members?

Discussion

• How can SCER play a significant role in the new EU AI Act? #78

Issue Feedback review

• UC: Discourage Transparency #67
• UC: Use of Offsets over Efficiency #66
• UC: Category Optimization #65
• UC: Manipulative Benchmark Optimization #64
• UC: Functional Unit Redefinition #63

Standard update submissions

• Proposed Roadmap for SCER #73
• How would this impact the environment: Nvidia Stock Will Reach $10 Trillion Market Cap By 2030[Question] #71
• UC: Software Boundary Redefinition #62

Project Dashboard

• Review and close stale issues

Future meeting Agenda submissions

• If you have any Agenda submissions, please let Sean know via email [email protected] or open an Issue

Next Meeting

• 21st Aug Jul @ 9 am P/T

Adjourn

• Motion to adjourn

---

Previous meeting AI's

• Explore potential extensions of the standard framework to other software categories like databases
• Participate in the new Green AI Committee to represent the interests of the SCER standard

[metacosm]

Attended.

[seanmcilroy29]

Attended

[chrisxie-fw]

attended

[seanmcilroy29]

MoM
Chris opens the meeting at 1700 (BST) - Recording

Creating a software certification lab for carbon efficiency.
Chris highlights a proposed software carbon efficiency standard that is currently in the draft stage. He explains the Share Certification for AI models, a simple system designed to evaluate models' carbon efficiency. Chris and Eddie discuss the creation of a green software certification lab, complete with a rating and labelling system.

Creating a software plugin for carbon efficiency in AI development.
Adi plans to create a folder with a read me and work on a private branch to break down the spec and fill it in. Chris proposes using Kepler to streamline Hugging Face's benchmarking process.

Categorizing and benchmarking large language models.
Chris explains the importance of categorizing software into specific categories for certification purposes. Chris discusses categorization certification for large language models, comparing model size, application type, and energy data. Chris and Tamar discuss guidelines for categorizing language models based on their size, life cycle phase, and type.

Measuring energy consumption and carbon emissions for AI models.
Tamar asks about how hugging face deals with differences in accelerators, and Chris explains that they use a unified hardware setup for measuring energy consumption. Tamar clarifies that she wants to know if the share is related to SCR, and Chris says that people have different opinions on this matter. Tamar and Chris discuss challenges in defining units of management for AI models, with a focus on industry collaboration. Tamar and Chris discuss community involvement in defining efficiency ranges for carbon emissions in token generation.

Using carbon efficiencies to measure language models' environmental impact.
Chris explains the rationale behind using carbon efficiencies instead of kilowatt hours to measure language model carbon emissions. Chris explains how a language model's carbon efficiency can impact business value.

AI model certification and standardization.
Chris explains how organizations can achieve carbon efficiency standards through share specification. Chris highlights the benefits of adopting the Share guidelines for organizations, including IBM and Red Hat. Tamar suggests incorporating domain-specific experts in AI specification, and Sean to share Asim's comments.

Open-source project for software standardization.
Sean McIlroy and Chris Xie discuss UC submissions and potential contentions. Chris emphasizes that the project is open source and up for adoption, but the GSF won't endorse anything until it's approved. Sean clarifies that the draft specification is liable to change and won't be endorsed by the GSF until approved. Chris emphasizes the open-source nature of projects and community involvement.

Action Items

• Chris/Adi - Start setting up an SCR certification lab folder and documentation.
• All - Review Asim's previous comments and suggestions on strengthening the SCR standard.
• All - Provide direct feedback and comments on the SCR documentation.