dagstore: lotus mount should avoid file copy into transients when locally attached storage is used

[raulk]

Checklist

• This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
• This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
• I have a specific, actionable, and well motivated feature request to propose.

Lotus component

lotus miner/market - storage deal

What is the motivation behind this feature request? Is your feature request related to a problem? Please describe.

We confirmed that the miner and markets node can share storage paths if (a) they are running on the same machine, (b) they are running on separate machines but the storage dirs are attached as network mounts. However, the dagstore is not able to recognise when local storage is being used, and thus is not able to avoid a copy into transients.

Describe the solution you'd like

Solution comprises various steps:

1. In the dagstore Mount abstraction, remove the static Info() method and move the Kind and Access patterns fields to the dynamic Stat() method. This enables mounts to evaluate and report those fields dynamically based on whether the piece is available through a local or a remote store. eliminate Info() and merge access patterns and kind into Stat() dagstore#96
2. Make stores.Remote#CheckIsUnsealed report whether the answer is coming from the local store or the remote store, so that the Mount can determine whether the file is accessible or not locally.
3. Implement those changes in the LotusMount.

Describe alternatives you've considered

No response

Additional context

No response

[rjan90]

Closing this issue as outdated on the lotus-miner side of things. This issue is related to the legacy markets code in the Lotus-Miner which will be deprecated.

Boost has been working on designing, building, and testing a new Local Index Directory which will replace the dagstore which this issue is related to. X-posting the message from their announcement channel:

📣 Coming soon: Yugabyte database for the new Local Index Directory (LID) 📣 **
As you may know, we have been working on designing, building, and testing a new Local Index Directory which will replace the dagstore. Many of you have reported problems with the dagstore, and migrating to use LID will help resolve these issues. The LID introduces a horizontally scalable backend database for storing the data - YugabyteDB. We would love to get your input and feedback on these changes - feel free to raise them in Boost discussion or reach out in this slack channel.

How this benefits SPs!

• We are no longer relying on the dagstore!
  • Increased visibility into the state of your sectors and indexes
• Improved coordination between Lotus and Boost (Lotus will also leverage YugabyteDB)
  • built-in UI and admin panel for YugabyteDB for easy management
• Enables scalability for SPs storing client data.
  • Read more details on how this works: https://boost.filecoin.io/experimental-features/local-index-directory
  • Ensuring your data can be reliably retrieved will become increasingly more important as the ecosystem continues pushing on retrieval incentives and reputation systems, both of which have teams actively working on them.

What’s next?

• The Boost team has completed migration of our two production miners to LID, and we will be monitoring for any issues that come up. 📊
  • We will work with other early adopter SPs and Lotus TSEs to try this out! 🎉
• We are working on documentation and migration tools, and will announce when it is ready for all SPs to migrate to LID.
• Again, we’d love to hear directly from you. Please let us know your input and feedback in Boost discussion, or reach out in #boost-help.