Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2nd Community Diligence Review of Hash Big Data #174

Closed
hash889900 opened this issue Sep 25, 2024 · 5 comments
Closed

2nd Community Diligence Review of Hash Big Data #174

hash889900 opened this issue Sep 25, 2024 · 5 comments
Assignees
Labels
Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards Refresh Applications received from existing Allocators for a refresh of DataCap allowance

Comments

@hash889900
Copy link

hash889900 commented Sep 25, 2024

Latest Compliance Report: https://compliance.allocator.tech/report/f03014732/1727222529/report.md
Previous Compliance Review Comments: #99

Datacap continues to be assigned to two existing clients

hash889900/HashTeam#3

hash889900/HashTeam#8

Generates CID reports on an ongoing basis and asks additional questions based on the generated reports.

I found that the retrieval rate plummeted a few days ago, and when I asked why, the customers answered that the problem was caused by a program failure on SPARK's side.
hash889900/HashTeam#8 (comment)
hash889900/HashTeam#3 (comment)

Based on a few days of follow-up, retrieval rates are now continuing to rise

@Kevin-FF-USA Kevin-FF-USA self-assigned this Sep 26, 2024
@Kevin-FF-USA Kevin-FF-USA added Refresh Applications received from existing Allocators for a refresh of DataCap allowance Awaiting Governance/Watchdog Comment DataCap Refresh requests awaiting a public verification of the metrics outlined in Allocator App. labels Sep 26, 2024
@filecoin-watchdog
Copy link
Collaborator

Allocator Application
First Review
Compliance Report
2nd allocation - 17.09

Issue 3
The client used all the DC requested in the original application and then requested an increase in DC and the number of replicas. The client added 6 new SPs to the application.

The retrieval rate could be better, as all 11 SPs hover between 50-75%.

Below SPs might be using a VPN:
F02866666 Tokyo (actual location: Shenzhen, China, IP: 113.96.22.203)
F02588263 Hong Kong (actual location: Shenzhen, China, IP: 113.96.22.204)

Issue 8
The client has updated the SP list several times and changed the declared number of replicas.

The retrieval rate is very uneven across 12 SPs, oscillating from 0 to 72%. The allocator should ask the client about this inconsistency.

In general, the allocator ensures that the data clients provide is correct. They ask questions when detect discrepancies and regularly check CID reports, highlighting when the retrieval rate is too low.

@Kevin-FF-USA Kevin-FF-USA added Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards and removed Awaiting Governance/Watchdog Comment DataCap Refresh requests awaiting a public verification of the metrics outlined in Allocator App. labels Oct 1, 2024
@hash889900
Copy link
Author

Thank you for pointing this out, I will continue to follow up on this issue, thank you!

@galen-mcandrew
Copy link
Collaborator

Based on the investigation and details provided, we are seeing mixed compliance results. Some areas for this allocator to focus on should include:

  • Clients changing replicas/project sizes without sufficient justification. For example, would it be better for a client to open a new application for these new SPs and replicas?
  • SPs utilizing VPNs without increased diligence
  • Highly variable retrieval rates
  • Data preparation details from clients

Given this review, we are requesting that the allocator verify that they will uphold all aspects & requirements of their initial application, with some extra focus on these areas. If so, we will request an additional 10PiB of DataCap from RKH, to allow this allocator to show increased diligence and alignment.

@hash889900
Copy link
Author

Follow up on the issues you mentioned, thanks for reviewing!

@Kevin-FF-USA
Copy link
Collaborator

DataCap has been refilled.
https://datacapstats.io/notaries/f03014732

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Diligence Audit in Process Governance team is reviewing the DataCap distributions and verifying the deals were within standards Refresh Applications received from existing Allocators for a refresh of DataCap allowance
Projects
None yet
Development

No branches or pull requests

4 participants