Application for DataCap Refilling from Genesis
Closed this issue · 15 comments
Allocator Application: filecoin-project/notary-governance#1019
- Since becoming an Allocator, we have received a total of 5PiB DataCap, At present , we have issued about 4.5 PiB. To better serve the Clients, we are submitting a refilling application.
- Our entire approval process manages customer applications based on the governance guidelines submitted in the V5 Allocator Application. We first check whether the customer's GitHub account registration days are greater than 180 days, and then review the customer's entity, data content, data demand, SP location, and CID retrieval success rate. If it meets the approval process, the initial approved DC will not exceed 100TiB, and the maximum DC in the entire process will not exceed 1PiB. The reasonable use of DC is ensured by controlling the maximum value of DC. Finally, the approval is completed based on the CID Checker Report.
- During our initial small-scale DC distribution, some customer applications failed to trigger the CID Checker Report. We also submitted a BUG issue for this. fidlabs/allocator-tooling#61 Because it is still can't trigger the report, we stop approving DC to these applications temporary to keep transparency.
- Finally, in the process of acting as an allocator, we actively participated in every governance meeting, paid attention to community dynamics, and did a good job of strict control. At the same time, in order to avoid a one-size-fits-all approach to Spark retrieval issues, we adopted a flexible review method. We support multiple retrieval methods, such as graphsync, http, legacy lotus market vs. boost, etc. And support different systems such as Lotus/Venus/Curio, etc.
- In a word, we hope to refill the datacap through compliance review in order to better build and serve the filecoin ecosystem.
There are 5 applications in my pathway.
1st Allocation
Client：https://github.com/lidunhan The github account registered for more than 180 days.
Application: Chuangshi1/Genesis#30
Total Approved: 100TiB
The application can't trigger the report. Stop approving DC to these applications temporary to keep transparency.
fidlabs/allocator-tooling#61
2nd Allocation
Client：https://github.com/Finonaa The github account registered for more than 180 days.
Application: Chuangshi1/Genesis#20
Total Approved: 1st round 100TiB, 2nd round 500TiB, 3rd round 512TiB.
CID Checker Report https://check.allocator.tech/report/Chuangshi1/Genesis/issues/20/1718700900895.md
5 SPs in the application and 3 match
3rd Allocation
Client：https://github.com/Cloudyia The github account registered for more than 180 days.
Application: Chuangshi1/Genesis#6
Total Approved: 1st round 50TiB, 2nd round 500TiB, 3rd round 500TiB 4th round 1PiB
The application can't trigger the report. Stop approving DC to these applications temporary to keep transparency.
fidlabs/allocator-tooling#61
But we ask client to post retrieval evidence。
4th Allocation
Client：https://github.com/nike-mp The github account registered for more than 180 days.
Application: Chuangshi1/Genesis#14
Total Approved:100TiB
CID Checker Report https://check.allocator.tech/report/Chuangshi1/Genesis/issues/14/1718590802412.md
No one SP march, and no spark retrieval. So i stop approving DC temporarily. Waiting for explanation from client
5th Allocation
Client：https://github.com/Tom88881025 The github account registered for more than 180 days.
Application: Chuangshi1/Genesis#8
Total Approved: 1st round 500TiB 2nd round 512TiB
15 SPs offered and 7 SPs match
CID Checker Report https://check.allocator.tech/report/Chuangshi1/Genesis/issues/8/1718591267400.md
We ask client to post retrieval evidence. And the Spark retrieval are improving gradually.
Please let me know if you have any questions and i will try my best to answer . It will be the most direct way to help us became a great Allocator.
@Kevin-FF-USA @galen-mcandrew When will we get response ?
First Example:
SPs provided
HK f01730296
Guangdong f02825338 f02639675
SIchuan f0109903
Sichuan f02201915
SPs taking deals https://datacapstats.io/clients/f03069150/breakdown
f03074587 | 20.11% | 417.03 TiB
f03074583 | 20.11% | 417.03 TiB
f03074589 | 19.91% | 412.88 TiB
f03074586 | 19.85% | 411.69 TiB
f03074592 | 20.03% | 415.38 TiB
2PiBs given to SPs that do not match and have 0% retrievals
Also, same string of SPs taking equal distribution - flagging as one entity
Second example:
Chuangshi1/Genesis#30
SPs taking deals
Storage Provider ID | % of total client used DataCap | Total size
f03126600 | 45.62% | 26.72 TiB
f01660008 | 1.01% | 608 GiB
f03098989 | 53.36% | 31.25 TiB
0 Retrievals Chuangshi1/Genesis#30 (comment)
2 of 4 SPs match those listed.
f03096886 Guangdong province of China
f03098989 HK
f01660008 Jakarta, Indonesia
f02824216 Zhejiang province of China
Example 3: Chuangshi1/Genesis#8
SPs declared up front:
f03028310 Fly Tokyo
f03028318 Seven Singapore
f02062851 Mike USA
f03028325 HK_Fil HK
f02235190 WHR Shenzhen
f03028326 HKblockchain US
f03028315 Datastone Guangdong
f039940 Cabrina China
f02289399 Bailey HK
SPs taking deals
Provider | Location | Total Deals Sealed | Percentage | Unique Data | Duplicate Deals | Mean Spark Retrieval Success Rate 7d
f03074592 | Los Angeles, California, USCNSERVERS LLC | 93.75 TiB | 9.40% | 93.75 TiB | 0.00% | 0.00%
f03028326 | Shenzhen, Guangdong, CNCTGNet | 576.00 GiB | 0.06% | 576.00 GiB | 0.00% | 0.00%
f03074586 | Hong Kong, Hong Kong, HKHong Kong Beecloud System Technology Services Limited | 156.25 TiB | 15.66% | 156.25 TiB | 0.00% | 0.00%
f03074589 | Hong Kong, Hong Kong, HKHong Kong Beecloud System Technology Services Limited | 93.75 TiB | 9.40% | 93.75 TiB | 0.00% | 0.00%
f02953066 | Los Angeles, California, USIPTELECOM ASIA | 44.94 TiB | 4.50% | 44.94 TiB | 0.00% | 2.17%
f02956073 | Singapore, Singapore, SGIPTELECOM Global | 93.63 TiB | 9.38% | 93.63 TiB | 0.00% | 47.84%
f02837684 | Hong Kong, Hong Kong, HKIPTELECOM Global | 25.63 TiB | 2.57% | 25.63 TiB | 0.00% | 0.00%
f03074587 | Seoul, Seoul, KRMOACK.Co.LTD | 53.13 TiB | 5.32% | 53.13 TiB | 0.00% | 0.00%
f03074583 | Tokyo, Tokyo, JPNearoute Limited | 156.25 TiB | 15.66% | 156.25 TiB | 0.00% | 0.00%
f02199999new | Singapore, Singapore, SGSimplerCloud Pte Ltd | 92.56 TiB | 9.28% | 92.56 TiB | 0.00% | -
f03080852 | Toronto, Ontario, CAThe Constant Company, LLC | 93.66 TiB | 9.39% | 93.66 TiB | 0.00% | 56.04%
f03080854 | Piscataway, New Jersey, USThe Constant Company, LLC | 93.66 TiB | 9.39% | 93.66 TiB | 0.00% | 69.14%
No SP matches from declared SPs. 3-4 SPs have retreivals
Several SPs from first example above also taking deals here - showing in varied locations but likely same entity in same location
f03074583
f03074587
f03074589
f03074586
f03074592
Example 4 Chuangshi1/Genesis#20
SPs provided:
f01660008 India
f03126600 CN
f03010701 CN
f03099903 HK
f01730296 HK
SPs taking deals:
Provider | Location | Total Deals Sealed | Percentage | Unique Data | Duplicate Deals | Mean Spark Retrieval Success Rate 7d
f03099903 | Hong Kong, Hong Kong, HKANYUN INTERNET TECHNOLOGY (HK) CO.,LIMITED | 423.38 TiB | 39.43% | 423.38 TiB | 0.00% | -
f03098989new | Shenzhen, Guangdong, CNChina Mobile communications corporation | 450.91 TiB | 41.99% | 450.91 TiB | 0.00% | -
f03139960new | Chongqing, Chongqing, CNChina Mobile Communications Group Co., Ltd. | 1.25 TiB | 0.12% | 1.25 TiB | 0.00% | -
f03096886 | Jiangmen, Guangdong, CNCHINANET-BACKBONE | 59.72 TiB | 5.56% | 59.72 TiB | 0.00% | 0.00%
f02370792 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 45.63 TiB | 4.25% | 45.63 TiB | 0.00% | 0.00%
f03126600 | Changsha, Hunan, CNCT-HuNan-Changsha-IDC | 92.91 TiB | 8.65% | 92.91 TiB | 0.00% | -
2 SPs match. 2 SPs sealing 85% of deals. No retrievals.
I can't show you the screenshot by this account, so i will use another account. @1ane-1
- About retrieval: we always ask our client to proof that.But about spark, wait the update from government.
If the client couldn’t offer these, it won’t get DC.
2. About Chuangshi1/Genesis#6
We already closed the application many days ago, because no checker report and reply. We pull the bug issue here:fidlabs/allocator-tooling#61
Besides, we will ask the information of SP location to check if they are one entity. If it is, we will mark them and refuse to allocate DC to them. Location will be gathered and post here.
3 .About the SP match: Some client add more SPs gradually, and change their SPs if wait for a long time.
4 .According to our rules, the first round is 50TiB and the next round gradually added but no more than 2PiB once. And we check every client github account if more 180 days.
The report is the standard of our audit. We are asking the client to be retrieved by Spark.
After participating in the governance meetings of the past two weeks, we requested clients to contact SPs and research the Spark retrieval principle. It can be seen that the Spark retrieval rate has improved, which is about to meet the governance team's requirement of 85%.
First, as you said, our 5PiB is allocated to 4 customers and 26 SPs, and the maximum DC obtained by each SP is no more than 451TiB, accounting for 8.8% of 5PiB. So it is difficult for me to agree with your conclusion that it is allocated to the same entity.
Second, as you can see, 4 of the SPs we cooperate with support Spark, and we may be the SP that supports Spark the most.
Third, some of the SPs we cooperate with use the Venus system, which leads to unsatisfactory data supporting Spark. In order to solve this problem, we will refuse to use Venus SPs until Spark supports Venus.
Fourth, regarding the disclosure of SPs, our communication with customers is not only in GitHub, but also on TG, zoom and phone. What we must do before approval is due diligence and review the basic information of customers and miners.
Finally, I hope we can be refreshed 10PiB and continue to provide better services to customers, and we also welcome everyone's supervision.
Update: Another application 3 SPs support SPARK retrieve and improve the retrieval rate gradually.
Hi @Chuangshi1,
Appreciate the detailed responses. Wanted to keep you updated and thanks for your patience while most of the team was out for Dev Summit last week. Tomorrow I'm connecting with the rest of the Governance Team to finalize any remaining issues before a decision for renewal will push out.
Will update ticket around Noon pst.
Hi @Chuangshi1,
Appreciate the detailed responses. Wanted to keep you updated and thanks for your patience while most of the team was out for Dev Summit last week. Tomorrow I'm connecting with the rest of the Governance Team to finalize any remaining issues before a decision for renewal will push out.
Will update ticket around Noon pst.
Thank you so much.
@Kevin-FF-USA Is there any update news here?
Based on an additional compliance review, it appears this allocator is attempting to work primarily with public open dataset clients.
However, the data associated with this pathway is showing mixed results for retrieval testing. The allocator makes claims above about requiring more Spark retrieval and coaching clients to work with compliant SPs.
As a reminder, the allocator team is responsible for verifying, supporting, and intervening with their clients. If a client is NOT providing accurate deal-making info (such as incomplete or inaccurate SP details) or making deals with noncompliant unretrievable SPs, then the allocator needs to intervene and require client updates before more DataCap should be awarded.
Given this mixed review, we are requesting that the allocator verify that they will uphold all aspects & requirements of their initial application. If so, we will request an additional 5PiB of DataCap from RKH, to allow this allocator to show increased diligence and alignment.
Chuangshi1 can you verify that you will enforce retrievability requirements, such as through Spark?
Please reply here with acknowledgement and any additional details for our review.
@galen-mcandrew Yes. We always ask the client to obey the retrieval requirement. It aims to 85%. And before the allocating, we will ask the SP information and check if they are marked by us because their bad performance. Now we can see the SP retrieval rate is improving gradually.The retrieval rate increase to 66% from 41%. Other SP will also be like this gradually. In the future we will pay more attention and supervise SPs.
