FIP Discussion: BatchBalancer & BatchDiscount Post-HyperDrive Adjustment
zixuanzh opened this issue · 18 comments
Simple Summary
Adjust BatchBalancer and BatchDiscount to match observed network growth rate post-HyperDrive & apply mechanism consistently to both ProveCommitAggregate & PreCommitBatch.
Abstract
BatchBalancer and BatchDiscount were introduced in FIP13 HyperDrive to align participants' incentives with the long-term health and success of the network. At that time, the parameter values were set to accommodate an onboarding rate of up to 1 EiB/day. However, given that the network is growing at ~60PiB/day nearly 3 months after HyperDrive (up roughly 2x from ~30-35PiB/day prior to the upgrade), these parameters must be re-calibrated for the long-term success of the network and its participants. This FIP proposes to increase BatchBalancer to 5 nanoFIL.
In addition, BatchBalancer and BatchDiscount were only applied at ProveCommitAggregate, not at PreCommitBatch. The protocol should also apply the same mechanism at PreCommitBatch to be in line with the spirit and considerations in FIP13.
Change Motivation
BatchBalancer. The initial parameter values of BatchBalancer and BatchDiscount were set after considering storage onboarding expectations, equilibrium network BaseFee, return on providing storage on Filecoin, cost of PublishStorageDeals, and protocol revenue. HyperDrive unlocked 10-25x storage onboarding capacity and the parameters were provisioning for growth on the order of onboarding 1 EiB/day. However, the network is not growing at that level, resulting in a loss in protocol revenue that hurts all participants long-term. Hence, the protocol needs to adjust its parameters accordingly based on current network growth rate (~60PiB/day) and current growth projections to >150PiB/day. The protocol may re-calibrate this parameter once the network significantly surpasses that growth rate. Work is being done to design mechanisms to set these protocol parameters algorithmically based on on-chain states in the future, so that the protocol re-calibrates automatically based on participants’ behavior.
PreCommitBatch. During HyperDrive, BatchBalancer and BatchDiscount were initially only applied at ProveCommitAggregate to simplify implementation. However, storage onboarding is a two-step process and the same mechanism should be applied at PreCommitBatch.
For the incentive consideration part of the initial FIP draft, could you please:
- clarify how PSD network fee is calculated? More specifically, how many deals in a single message was taken?
- What does the
Estimated 32 GiB sector network feeinclude? is it precommit gas cost + provecommit gas cost? Does it include collateral? And is it for CC sector or the Deal sector?
Note that Network Fee is halved for 64 GiB sectors and the above unit cost numbers go down further as the number of aggregation increases.
Could you please specify which network fee is referred here? Is it only for the X GiB sector network fee?
Hi everyone, we did some research and analysis on Filecoin protocol revenue. You can see relevant data and visualization here for more context.
https://observablehq.com/@starboard/filecoin-protocol-revenue-analysis
clarify how PSD network fee is calculated? More specifically, how many deals in a single message was taken?
This is calculated by multiplying the crossover network BaseFee with the gas usage of PSD. This could be lower or higher if the network BaseFee is lower or higher than the equilibrium BaseFee. This does not take into the account of the number of deals that are batched in PSD. In other words, per-deal network fee could be a lot lower, 1/10 for example if 10 deals are batched in a single PSD message.
What does the Estimated 32 GiB sector network fee include? is it precommit gas cost + provecommit gas cost? Does it include collateral? And is it for CC sector or the Deal sector?
Estimated 32 GiB sector network fee includes network fee that consists of gas cost + batch gas charge (assuming batching ~10 sectors) for Pre and ProveCommit. It does not include collateral since collateral is a fraction of the return and returned at the end of the sector lifetime. This network fee is applied to every sector and hence it is not relevant if it's CC sector or sector with deals. Note that network fee per-unit storage goes down even further with 64 GiB sectors or greater aggregation when the network BaseFee increases.
What is protocol revenue? The income of f099?
What is protocol revenue? The income of f099?
There are different ways to define protocol revenue but in general it is the revenue that the protocol generates from enabling activities of its participants. Providing and consuming decentralized and verifiable storage on Filecoin are some examples of using the protocol. Network transaction fee aligns token supply with usage of the network. As long as there is any action or utility on the network, network transaction fee will be paid to the protocol. This can be measured by the balance in f099 but Starboard team did a breakdown of its constituents in the link above. One can also visit TokenTerminal to see the comparison across different protocols at different timescales.
Basically this approach will benefit Filecoin hodlers while the protocol revenue is getting higher, and encourage more investors to join Filecoin ecosystem, at the same time, the new coming storage providers is to pay the cost. However, the storage providers earns block rewards, which will also benefit from the protocol revenue.
I would expect there will be different opinions from different people, I would suggest the SPWGs can discuss this, and provide feedback.
Just increasing the BatchBalancer will just make everything more expensive just for the sake of making things more expensive.
Most notably, this makes deals significantly more expensive. Publish storage deals is currently on the border of what is an acceptable expensive, on the order of $0.10 USD per deal, with a base fee of 0.1 nanoFIL. If we increase the base fee to 5 nanoFIL via this FIP, that seems like it would make deals cost $5 each to publish. $5 per (max) 32gb deal that lasts up to 1.5 years? s3 is very expensive storage, and to store 32gb for 18 months it would cost about $13. For that money, you get 3x replication, and good bandwidth, fast retrieval of your data, and pretty much zero chance of data loss. To publish three storage deals puts us more expensive than s3 with a worse quality of service (and thats more expensive just for the fees of publishing the deal, this doesnt even account for the miner charging any money for their service, or any of the other fees associated with storing data on the network).
If we want to burn more FIL, we should make just precommits and prove commits burn more, and leave the storage market (which must be on chain..) out of it. Otherwise we can't really do the thing that Filecoin is supposed to do, and store files for people.
I agree with @whyrusleeping the cost for storage deal is very high at the moment, without verified deal, we barely can find storage provider at the moment, with this extra cost we will have more difficulties to push the industry usage.
Agree it's a hard tradeoff - however I think diverging the gas model for PublishStorageDeals vs committing new CC Sectors would be a much larger / more involved change than what is proposed here. This FIP simply recalibrates existing network-wide cryptoecon parameters to adjust to the observed onboarding rate since HyperDrive (at still a much much lower rate than prior to HyperDrive).
With the scaling of large DataCap distribution as part of the FilecoinPlus program, I think the vast majority of new deals made in the network by storage clients are/should be verified deals -- giving storage providers an additional 10x power incentive to offset deal-making costs. In practice, I think that 10x multiplier helps subsidize the storage costs passed on to the client - at least until the network can land optimizations directly to make PublishStorageDeals less gas expensive / work for larger deals to amortize across multiple sectors. ((Note that I believe the FIP's estimate for PublishStorageDeals cost is actually without aggregation, which should make observed costs 10x cheaper than reported))
Also - I think your math is off @whyrusleeping. Adjusting BatchBalancer to 5nFIL should only move crossover BaseFee from 0.12nFIL to ~0.32 nanoFIL -- so storage deals aren't going to become 500x more expensive... As the FIP mentions, this limited recalibration should only cause a ~.009 FIL (currently ~$0.5) total increase in the gas costs of PublishStorageDeal messages.
To reiterate the estimates in the FIP since there might be some confusion:
When increasing the BatchBalancer (not the base fee) to 5 nanoFIL, the equilibrium crossover BaseFee becomes ~0.32 nanoFIL. This means that each PublishStorageDeals costs about 0.016 FIL (about <$1 at current FIL price). Note that storage providers are strongly encouraged to aggregate deals before publishing. Assuming 10 deals are batched in a PublishStorageDeals message, each deal will only cost about 0.0016 FIL (< $0.1/deal at current FIL price, and storage providers can be shielded from token price volatility with loans).
Of course, calculation could be wrong too, happy to cross validate if folks have different calculation methods/parameter estimates
@momack2 to clarify I didnt say 500x, I said 50x, $0.10 to $5.
When increasing the BatchBalancer (not the base fee) to 5 nanoFIL, the equilibrium crossover BaseFee becomes ~0.32 nanoFIL.
Ah, thats the trick then. I had the concepts of BatchBalancer and the crossover BaseFee mixed up. That makes significantly more sense. So we are still making everything else more expensive, but only about 3x, which doesnt entirely blow up the storage cost math (but does make it harder). I hope we see some good FIPs soon to make dealmaking cheaper.
Yes, research is under way to make BatchBalancer dynamic but the team's plate is very full. We already have some ideas on how it might work but it takes more testing and validation (projecting a few months of work and delay since we are working on some new priority). We would also like to apply a direct discount on PublishStorageDeals but it may become an attack vector and implementation may not be straightforward. Hence, as Molly mentioned this FIP simply recalibrates existing network-wide cryptoecon parameters to adjust to the observed onboarding rate since HyperDrive (at still a much much lower rate than prior to HyperDrive) given all the tradeoffs. We also look forward to FIPs that make dealmaking cheaper. We can also explore software improvement and creative business models to reduce the cost of dealmaking, just like how it costs more to make a trade on DeFi and people still want to do it.
Work is being done to design mechanisms to set these protocol parameters algorithmically based on on-chain states in the future, so that the protocol re-calibrates automatically based on participants’ behavior.
Is there a timeline when we can be free from manual intervention of protocol params?
In general, we should be prepared to tune protocol parameters. However, they need to be done carefully, with good intention, reasoning, and validation and we try not to do it unless it is necessary or brings significant value to the network. We can go further into what a good process to do so is and how we can do it better as a community but that is beyond the scope of this FIP. Nonetheless, there is also value in figuring out how to do things manually before automating. For this particular parameter, we have mentioned in this FIP and the FIP before that it will be great to adjust it automatically with on-chain estimators. However, there are some serious risks with automation (if not done well) and new tradeoffs to evaluate. There is no concrete timeline yet but we are expecting a few months of work.
- Given the current crypto currency ban in a major country, the storage growth may slow down. Should we wait a bit and see?
- Is it possible to adjust this parameter dynamically according to the network condition(from stats of last week, i.e.) so we eliminate the need to manually adjust the fee in the future?
Given the current crypto currency ban in a major country, the storage growth may slow down. Should we wait a bit and see?
The motivation for this FIP comes from the observation that the network growth rate is way below the level provisioned by the protocol. If we are projecting a slower network growth rate, it will actually argue for an even higher BatchBalancer value (10 nanoFIL for example). However, given all the tradeoffs described, we still recommend sticking with 5 nanoFIL as initially proposed in the FIP.
Is it possible to adjust this parameter dynamically according to the network condition(from stats of last week, i.e.) so we eliminate the need to manually adjust the fee in the future?
It is possible and the team is working on design and validation. However, quoting from @mzargham in this tweet, “Despite our preference for automation in decentralized protocols, automation is not entirely free of manual inputs. Those inputs move up the stack to the metaparameters the automation relies on; they encode invariants and goals set by human participants. As events play out and as priorities shift, interventions may be required to update these inputs.” We decided to adjust parameters at the current level given the timeline, complexity, and risk tradeoff.
Closing due to FIP acceptance!