SPX Index Provider Testing #8087

jennijuju · 2022-02-14T14:03:52Z

jennijuju
Feb 14, 2022
Maintainer

💙 Huge shot out to datasystem team (@masih, @willscott ,@gammazero @rvagg , @MarcoPolo , @hannahhoward) & the ignite team (@aarshkshah1992, @nonsense, @dirkmc) for building this part of the interplanetary network content addressing system and bringing it into lotus so that Filecoin storage providers that use lotus may participate as the early content & retrieval providers in the network with ease!

note: for the initial rollout, we are doing testing with cohort outer space, with 5 fellows.

Table of Contexts

Overview
Expectations
⚠️ Action Items, Reminders and Risk Mitigations
🛰️ How to Become an Index Provider
- Index Provider Configuration
...Almost Done

Overview

Interplanetary network indexer nodes aim to store the indexes that provide a content routing sub-system to identify which providers in the interplanetary network(IPFS, Filecoin, etc.) are able to provide what content.

To build such content addressing system, index providers will act as the entities that advertise the content to index nodes and serve retrieval requests over graph sync. Index providers may be run as a standalone service or ported into an existing Golang application. In which, in lotus cases, it's embedded into the market subsystem so that Filecoin storage providers may serve as the index provider (later retrieval providers) in the Interplanetary network (IPFS & Filecoin).

To become a index provider involves three main parts:

A DagStore migration
Indexing announcement of available content
Content routing verification

DagStore Migration

The DagStore migration is required to roll out the new CARv2 indexing format, MultihashIndexSorted. It regenerates the indices with the latest CARv2 index format that includes the full multihash of CIDs in each DagStore shard. For more inform see DagStore Lotus documentation.

Indexing Announcement

The indexing announcement publishes advertisements about the content available for retrieval onto a gossipsub topic, which is listened to by a set of indexer nodes.

The advertisements are then processed by the indexer in order to provide an endpoint that allows clients to lookup where and how to retrieve data for a given multihash. For more information, see Indexer Node Design.

Content Routing Verification

The content routing verification involves checking that the advertisements published by the Lotus instance are ingested by the indexer nodes and for a given stored multihash the correct provider is returned by the indexer node.

Note that ingestion of advertisements by the indexer nodes is progressive; total ingestion time depends on the number of multhihashes being advertised. Therefore, verification needs to be performed with some delay after indexing announcement is made.

Expectations

New CAR index format in DagStore

The new index format stores the multihash code as well as the digest. The size of DagStore index files as a result are slightly larger. The size increase is negligible.

Index Provider GossipSub Announcements

The index provider integration announces changes to the chain of advertisements onto a gossipsub topic, named /indexer/ingest/mainnet, which is propagated through the Lotus daemon onto the indexer nodes.

Index Provider GraphSync Server

The index provider integration exposes a GraphSync server which serves requests from indexer nodes to sync the list of advertised multihashes provided by the SP.

Note that the server is exposed on the address configured under IndexProvider configuration section, with key ListenAddresses. The advertisements will include the address configured under the same section with key AnnounceAddresses. You must make sure both sets of address are reachable publicly. For more information see Indexer Provider Configuration.

Index Provider Storage Usage

The index provider integration shares a datastore with markets process, wrapped under the namespace index-provider. The datastore entries stored include:

internal mappings used by the index provider engine, and
cache of chained multihash chunks.

The storage used by the internal mappings is negligible.

The storage used by caching is bound by an LRU cache, the maximum size of which is configured to 1024, i.e. the number of chunks cached. The maximum length of each chunk configured to 16,384.

The exact storage usage, then, depends on the number of multihashes stored in a single chunk and the size of each multihash, and can be calculated as:

1024 * 16384 * <multihash-length>

For example, caching 128-bit long multihashes will result in chunk sizes of 0.25MiB with maximum cache growth of 256 MiB.

Note that the LRU cache may grow beyond its max size if the generated chain of chunks is longer than the configured LinkChunkSize. This is to avoid partial caching of chunks within a single advertisement. The cache expansion is logged in INFO level at provider/engine logging subsystem and can be monitored for diagnosis purposes.

⚠️ Action Items, Reminders and Risk Mitigations

Make yourself a very first Index Provider(IP) of the InterPlanetary network by following the steps listed in Steps to Become an Index Provider (Finish reading this section first before you head there 😉 )!
Leave a comment in this disussion with the results of provider verify-ingest .
Monitor the Index Provider storage and system usage. This would be most effective if you can pause other market operations(deal making) and only run IP process on your markets node for a couple hours , and collect the following data points:
- number of deals your have
- size of the dagstore repo as it’s shipping with a new top level index.
- growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size
  - This is to monitor the disk usage of datatrasfers used by the index provider to sync advertisements with indexer nodes.
- growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size
  - This is to monitor the disk usage of index provider integration.
- any outstanding IO/mem usage, with different value of MaxSimultaneousTransfers
Questions are welcomed in this discussion, the team will be monitoring this discussion and check-in regularly.
Issues can be reported in the https://github.com/filecoin-project/index-provider. Issues should be submitted with:
- The multiaddr of your markets node
- the step at which the problem occurred
- The stack trace or logs as relevant. As always, for large log files, you may upload it in this gdrive folder and link it in the issue!
❗ If you suspect there is an issue with the DagStore migration or being an index provider, and it’s impacting your SP operation severely, please save all the logs to describe what went wrong, then you may follow the steps below to rollback:
1. Downgrade the previously installed lotus version.
2. Stop ****daemon, miner and markets processes.
3. Restore the backed up DagStore repository.
4. Start daemon and miner processes first.
5. Wait for the following commands to succeed:
  1. lotus-miner storage-deals list
  2. lotus-miner sectors list
6. Start the markets process.

Steps to Become an Index Provider

Stop the daemon miner and markets processes

Stop all lotus processes to suspend any changes to the state of DagStore during the rollout.

The daemon needs to be stoped in order to roll out a change to the API that protects connections between markets and daemon libp2p node.
Back up the existing DagStore repository

The DagStore repository is located at $LOTUS_MARKETS_PATH/dagstore by default. Make a copy of that folder. This is necessary for:
1. verifying that the expected shard indices are re-generated after migration, and
2. ❗rolling back the changes in case of an error.
Delete the existing DagStore repository

Delete the DagStore repository located at $LOTUS_MARKETS_PATH/dagstore by default. The absence of the repository signals to the Lotus instance that a DagStore migration is needed and will automatically trigger one upon markets instance start-up.
Rotate any existing Lotus log files and adjust log level

For easier debugging rotate any existing logs so that the new logs only include output generated by the target release.
Deploy daemon and miner processes

Deploy the index provider tag - master-spx.idxprov.rc-1 to the daemon and miner processes and await until they are fully started and ready.

Note: This tag is based off release/v1.15.0, thus it also supports the upcoming OhSnap network v15 upgrade!

They are ready when the following commands succeed:

lotus-miner storage-deals list

lotus-miner sectors list
Deploy the target release on markets process

Download and deploy master-spx.idxprov.rc-1to the market process.
Start the markets process

Start only the markets process and wait for the following log line in the markets process logs:

dagstore migration completed successfully

This indicates that the list of shards that require initialisation have been queued for processing. See [DagStore First-time Migration](https://lotus.filecoin.io/docs/storage-providers/dagstore/#first-time-migration) for more information. See Indexer Provider Config to customize the configuration of the subsystem that announces indexes.
Configure logging subsystems

Make sure your lotus installation persists the log files for future debugging.
Set the log level for the following subsystems on market node to INFO :
- go-legs-gpubsub
- provider/engine
- dagstore
To do this run the following command:
```
lotus-miner --call-on-markets log set-level --system provider/engine --system go-legs-gpubsub --system dagstore info
```
Initialise the the DagStore shards

To start the initialisation of DagStore shards, run:

lotus-miner dagstore initialize-all --concurrency=N if you run a monolith miner process or lotus-miner --call-on-market dagstore initialize-all --concurrency=N if you have split your market subsystem.

⚠️ Initialization places IO workload on your storage system. N controls the number of deals that are concurrently initialised. See DagStrore Force Bulk Initialisation docs for more information.

Wait for the initialisation to complete. The initialisation time is a factor of the volume of data stored, since it involves re-indexing the data blocks.
Verify re-creation of DagStore repository

The successful completion of the previous step should recreate the DagStore repository, located at $LOTUS_MARKETS_PATH/dagstore . Navigate to that director. Under the subfolder index verify that matching *.full.idx files can be found for all files under the same sub-directory in the backup of DagStore taken in step 2.
✨ Announce all indices to the indexers

To announce all the indices in bulk to the indexers, run:

lotus-miner index announce-all if you run a monolith miner process or lotus-miner --call-on-market index announce-all if you have split your market subsystem.

This command generates advertisements and publishes indices onto the indexer gossipsub channel. In the markets logs look for a series of logs that include deal announcement sent to index provider . You should see one such log per deal. The log line also includes advertisement CID, the deal proposal CID to which it belongs and the shardKey from which its multihash entries are generated. The logs should also include logs that provide information about the number of multihash entries each advertisement includes. For example:
```
deal announcement sent to index provider	{"advertisementCid": "baguqeeqqvr2irdrq45d7npj7elogzpaaam", "shard-key": "baga6ea4seaqegic2h4qoao4urcwhin7tgwlb4cguqymtriheqoyjjaabz6viegq", "proposalCid": "bafyreihhqszkcv3egsb7xkuhyswqjdy3oa2kboi2zjvrbkuj3jgq2g4d4m"}
Generated linked chunks of multihashes	{"totalMhCount": 32449, "chunkCount": 2}
```
Note that the bulk advertisement only announces deals that are not expired and handed over to the sealing subsystem. The expired deals will not be advertised. For any remaining deals the advertisement will occur after they are handed over to the sealing subsystem.

Wait for the bulk indexing announcement to complete. The bulk announcement is complete when finished announcing active deals to index provider is logged.

Verify indices in DagStore repository are ingested by indexer nodes

To verify ingestion, download and install the latest provider CLI tool from:

https://github.com/filecoin-project/index-provider/releases/latest

The built binaries can be found under assets attached for each target platform.

Once installed, download the following script.

Script below to be provided as a downloadable .sh file; for now pasted below for review purposes.

#!/usr/bin/env sh

MINER_ID="${1:?miner peer ID must be specified as the first argument}" 
DAGSTORE_REPO="${2:?dagstore repo location must be specified as the second argument}" 
SAMPLING_PROB="${3-0.05}"

echo "MINER_ID:      ${MINER_ID}"
echo "DAGSTORE_REPO: ${DAGSTORE_REPO}"
echo "SAMPLING_PROB: ${SAMPLING_PROB}"
echo ""

for idx in ${DAGSTORE_REPO}/index/*.full.idx
do
   echo "Verifying ${idx}"
   provider verify-ingest \
      --to cid.contact:80 \
      --print-unindexed-mhs \
      --provider-id "${MINER_ID}" \
      --from-car-index "${idx}" \
      --sampling-prob "${SAMPLING_PROB}" 
   echo ""
done

The script takes two mandatory argument:

Miner peer ID as the first argument, and
Path to dagstore repository as the second argument, e.g. $LOTUS_MARKETS_PATH/dagstore

A third argument may optionally be specified as a number between 0.0>=1.0 which sets the selection probability of the multihash sample that is verified for ingestion, set to 5% by default.

You can find out what your peer ID is using lotus-miner net id

This script will:

iterate over all index files that match *.full.idx name,
selects 5% of the multihashes at random from each index, and
verifies that the indexer node has those multihash associated with the given MINER_ID as the provider.

A verification result is printed for each file. Verify that verification is successful for each of the files. See provider verify-ingest -h for example output.

Indexer Provider Configuration

You can adjust the values under the IndexProvidersession in the config.toml of your market process to configure indexes announcement to the indexer.

If the session doesn't exist, you can manually add it:

[IndexProvider]
  # The maximum number of multihash chunk links that index provider cache can store before
  # LRU eviction. If chunks belonging to a single advertisement are larger than the cache can
  # hold, the cache is resized to be able to hold all links. The actual disk usage depends on
  # LinkedChunkSize and the length of multihashes. For example, for 128-bit long multihashes
  # with the default LinkedChunkSize, and LinkCacheSize the cache size can grow to 256MiB. 
  #
  # type int
  # env var: LOTUS_INDEXPROVIDER_LINKCACHESIZE
  #LinkCacheSize = 1024

  # The number of multihashes in each chunk of the
  # advertised multihash entries linked list.  If multihashes are 128-bit, then
  # setting LinkedChunkSize = 16384 will result in blocks of 0.25MiB when
  # full.
  #
  # type int
  # env var: LOTUS_INDEXPROVIDER_LINKEDCHUNKSIZE
  #LinkedChunkSize = 16384

  # The gossipsub topic name used to publish change to the advertised content.
  #
  # env var: LOTUS_INDEXPROVIDER_PUBSUBTOPIC
  #PubSubTopic = "/indexer/ingest/mainnet"

  # Whether to purge all cached entries on start-up.
  #
  # env var: LOTUS_INDEXPROVIDER_PURGELINKCACHE
  #PurgeLinkCache = false

  # Binding address for the libp2p host contacted by indexer nodes to sync the list of advertised 
  # multihashes. Note that when port is set to 0 a random port is generated at runtime and may be 
  # different on every restart. The format of the strings specified must conform to multiaddress; 
  # see https://multiformats.io/multiaddr/
  #
  # type: []string
  # env var: LOTUS_INDEXPROVIDER_LISTENADDRESSES
  #ListenAddresses = ["/ip4/0.0.0.0/tcp/0", "/ip6/::/tcp/0"]

  # The address the endpoints at which the data associated to the advertised
  # multihashes can be retrieved. If not specified, the ListenAddresses are used instead. The format
  # of the strings specified must conform to multiaddress; see https://multiformats.io/multiaddr/
  #
  # type: []string
  # env var: LOTUS_INDEXPROVIDER_ANNOUNCEADDRESSES
  #AnnounceAddresses = []

  # The maximum number of simultaneous requests syncing the list of advertised multihashes between 
  # the indexers and the index provider.
  #
  # type: uint64
  # env var: LOTUS_INDEXPROVIDER_MAXSIMULTANEOUSTRANSFERS
  #MaxSimultaneousTransfers = 20

Almost Done

In cace you forgot the actions items listed in the previous session, here's a reminder to collect the requested metrics after you successfully deploy and become an Index Provider! As always, you may use Ozzy's Report as a template 😉 . We are looking forward to hearing back from you, good luck and have fun!

willscott · 2022-02-14T14:08:44Z

willscott
Feb 14, 2022
Collaborator

One small note: because only a small number of storage providers will initially be running this version, the notifications of indexing generated at step 11 are not very likely to actually reach our network indexer.

Fear not!

We will have the indexer poll for new index availability regularly. If you get to step 11, feel free to ping us, and we can initiate an additional poll to pick up your provider.

As long as your markets peer ID and multiaddr are associated with your provider's on-chain activity, your indexing will be found and ingested by the network indexer.

2 replies

tmyuu Feb 16, 2022

Hi
I have advertised all deals. (Step 11) But most of verification check failed❌ (Step 12)
Does this mean ingesting indexes is just delayed due to the few index providers in operation?
verify-ingest.log

Perhaps I've completed indexing and advertising, so please pick me up.
peerId: 12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq

willscott Feb 16, 2022
Collaborator

right now there's a manual step from our side between step 11 and 12 because the full lotus gossip mesh hasn't been upgraded to support the new type of messages yet.
We'll run our manual step again today to pick up your index.

jennijuju · 2022-02-14T14:16:21Z

jennijuju
Feb 14, 2022
Maintainer Author

Ozzy's IP Report Example

Results of `provider verify-ingest`

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	333
size of the dagstore repo as it’s shipping with a new top level index	7GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	5 MiB / 10 min
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	3 MiB / 10 min
IO/mem usage	with `MaxSimultaneousTransfer` = 10, usage is..

0 replies

f8-ptrk · 2022-02-14T15:03:24Z

f8-ptrk
Feb 14, 2022

Monitor the Index Provider storage and system usage. This would be most effective if you can pause other market operations(deal making) and only run IP process on your markets node for a couple hours , and collect the following data points:
number of deals your have
size of the dagstore repo as it’s shipping with a new top level index.
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size
    This is to monitor the disk usage of datatrasfers used by the index provider to sync advertisements with indexer nodes.
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size
    This is to monitor the disk usage of index provider integration.
any outstanding IO/mem usage, with different value of MaxSimultaneousTransfers

is there a sampling rate you want for this? this will be easy to script and given to the community for broader testing later on

0 replies

rjan90 · 2022-02-15T10:14:41Z

rjan90
Feb 15, 2022
Maintainer

WIP 🚧

Phi-rjan´s IP Report Example

Result of `provider verify-ingest`

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	8398
size of the dagstore repo as it’s shipping with a new top level index	8.1GB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	n/a / n/a
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	n/a / n/a
IO/mem usage	with MaxSimultaneousTransfer = 10, usage is..

WIP 🚧

1 reply

rjan90 Mar 8, 2022
Maintainer

Folder Size Change Report

Dagstore PRE bug fix: 9.2 GB
6.5 M datastore
9.2 G index
4.0 K transients

Dagstore POST bug fix: 8.1 GB
6.6 G datastore
2.2 G index
4.0 K transients

TippyFlitsUK · 2022-02-15T11:49:15Z

TippyFlitsUK
Feb 15, 2022
Maintainer

@TippyFlits IP Report

Results of `provider verify-ingest` - TBD

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	4145
size of the dagstore repo as it’s shipping with a new top level index	8.3GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	TBD
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	TBD
IO/mem usage	with MaxSimultaneousTransfer = 20, usage is normal

1 reply

TippyFlitsUK Mar 8, 2022
Maintainer

Folder Size Change Report

Dagstore PRE bug fix: 47 GiB
9.3M datastore
47G index
4.0K transients

Dagstore POST bug fix: 31 GiB
23G datastore
8.3G index
4.0K transients

Daemon:  1.15.1-dev+mainnet+git.77a954c7c+api1.4.0
Local: lotus-miner version 1.15.1-dev+mainnet+git.77a954c7c

tmyuu · 2022-02-16T00:57:53Z

tmyuu
Feb 16, 2022

@tmyuu IP Report

Results of `provider verify-ingest` - TBD

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	1564
size of the dagstore repo as it’s shipping with a new top level index	6.7GB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	TBD
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	TBD
IO/mem usage	usage is TBD

2 replies

willscott Feb 16, 2022
Collaborator

@tmyuu the indexer has indexed your node now, you should be able to verify

tmyuu Feb 17, 2022

@willscott
I checked the log.
the indexer getting the advertised index!

2022-02-17T01:16:47.474+0900    DEBUG   swarm2  [email protected]/swarm_listen.go:103     swarm listener accepted connection: <stream.Conn[TCP] /ip4/192.168.0.10/tcp/55555 (12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq) <-> /ip4/18.116.210.184/tcp/3003 (12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU)>
2022-02-17T01:16:47.474+0900    DEBUG   swarm2  [email protected]/swarm.go:336    [12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq] opening stream to peer [12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU]
2022-02-17T01:16:47.508+0900    DEBUG   basichost       basic/basic_host.go:414 protocol negotiation took 35.907µs
2022-02-17T01:16:47.549+0900    DEBUG   basichost       basic/basic_host.go:414 protocol negotiation took 31.949µs
2022-02-17T01:16:47.559+0900    DEBUG   basichost       basic/basic_host.go:389 protocol mux failed: stream reset (took 265.861283ms)
...
2022-02-17T01:16:47.791+0900    DEBUG   basichost       basic/basic_host.go:414 protocol negotiation took 159.258029ms
2022-02-17T01:16:47.885+0900    DEBUG   basichost       basic/basic_host.go:414 protocol negotiation took 39.033µs
2022-02-17T01:16:47.954+0900    DEBUG   basichost       basic/basic_host.go:414 protocol negotiation took 355.342µs
2022-02-17T01:16:47.954+0900    INFO    dt_graphsync    graphsync/graphsync.go:558      12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993, pull request queued, req_id=0
2022-02-17T01:16:47.955+0900    INFO    graphsync       responsemanager/server.go:218   graphsync request initiated     {"request id": 0, "peer": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU", "root": "baguqeeqqudhoe5gn6flt72ghlvggfeynn4"}
2022-02-17T01:16:47.955+0900    INFO    graphsync       responsemanager/server.go:250   graphsync response processing begins    {"request id": 0, "peer": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU", "total time": 0.0000214}
2022-02-17T01:16:47.955+0900    INFO    dt-impl impl/events.go:352      channel 12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993: received new channel request from 12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU
2022-02-17T01:16:47.955+0900    INFO    dt-impl impl/events.go:431      data-transfer request validated, will create & start tracking channel   {"channelID": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993", "payloadCid": "baguqeeqqudhoe5gn6flt72ghlvggfeynn4"}
2022-02-17T01:16:47.959+0900    INFO    dt_graphsync    graphsync/graphsync.go:1081     incoming graphsync request      {"peer": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU", "graphsync request id": 0, "data transfer channel id": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993"}
2022-02-17T01:16:47.959+0900    DEBUG   swarm2  [email protected]/swarm.go:336    [12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq] opening stream to peer [12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU]
2022-02-17T01:16:47.959+0900    INFO    provider/engine engine/linksystem.go:41 Retrieved advertisement from datastore  {"cid": "baguqeeqqudhoe5gn6flt72ghlvggfeynn4", "size": 718}
...
2022-02-17T01:16:49.099+0900    INFO    provider/engine engine/linksystem.go:41 Retrieved advertisement from datastore  {"cid": "baguqeeqqccecypzj7ew4rtyu4grmldny6m", "size": 718}
2022-02-17T01:16:49.103+0900    INFO    graphsync       responsemanager/server.go:301   graphsync response processing complete (messages stil sending)  {"request id": 0, "peer": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU", "total time": 1.148704676}
2022-02-17T01:16:49.105+0900    INFO    dt-impl impl/events.go:289      received OnChannelCompleted, will send completion message to initiator  {"chid": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993"}
2022-02-17T01:16:49.105+0900    INFO    dt-impl impl/events.go:296      sending completion message to initiator {"chid": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993"}
2022-02-17T01:16:49.105+0900    DEBUG   swarm2  [email protected]/swarm.go:336    [12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq] opening stream to peer [12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU]
2022-02-17T01:16:49.105+0900    INFO    dt-impl impl/events.go:303      successfully sent completion message to initiator       {"chid": "12D3KooWC7kMRLFT2kqv5XkhD7cR2Nw2UbKE5rwWk11VCZ22undU-12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq-1644861040879517993"}
...
2022-02-17T01:16:55.397+0900    INFO    provider/engine engine/linksystem.go:103        Found cache entry for CID       {"cid": "baguqeeqqudujikmgcyspko5ll5mdk6roxe"}

jennijuju · 2022-02-16T14:29:24Z

jennijuju
Feb 16, 2022
Maintainer Author

New tag is out!

https://github.com/filecoin-project/lotus/releases/tag/master-spx.idxprov.rc-3 is out! Update your nodes at your convenience!

cc @masih to share high-level changes!

1 reply

masih Feb 16, 2022
Maintainer

`master-spx-idxprov.rc-3`

This tag uses the same libp2p host as the markets process. As a result, graphsync/datatransfer instance is also shared with markets and index provider. This means the markets process libp2p host now also supports indexing protocol IDs.

The following parameters are now removed and no longer present under [IndexProvider] section of markets config:

AnnounceAddresses
ListenAddresses
MaxSimultaneousTransfers

The changes only impact the instantiation of the index provider engine in the internals of markets process. This means no impact on migration if you have done so already. If you are in the middle of migration, there is no need to restart the process either. Simply upgrade your installation to this task when finished.

If you have not started the assignment yet, then please use the tag above as target release.

Meest-NAM · 2022-02-17T08:35:05Z

Meest-NAM
Feb 17, 2022

#@Meest NAM IP Report
##Results of provider verify-ingest - TBD
##Monitoring Metrics

Are you only running IP process on your market subsystem (no deal making) when collecting the data below? ---> YES
number of the deals made --->2747
size of the dagstore repo as it’s shipping with a new top level index ---> 2.4G
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size ---> TBD
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size ---> TBD
IO/mem usage ----> with MaxSimultaneousTransfer = 20, usage is..

0 replies

benjaminh83 · 2022-02-17T13:54:10Z

benjaminh83
Feb 17, 2022

##BenjaminH IP Report
##Result of provider verify-ingest: Waiting for GO to run

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	5796
size of the dagstore repo as it’s shipping with a new top level index	47GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	Static 8KiB
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	1.6GiB->5.5GiB->4.5GiB
IO/mem usage	with MaxSimultaneousTransfer = 10, nothing special

1 reply

benjaminh83 Mar 8, 2022

##BenjaminH IP Report
##Result of provider verify-ingest: Waiting for GO to run
##Update on the size:

growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	27G total

| 20G /mnt/md12/.lotusmarkets/dagstore/datastore
| 6.9G /mnt/md12/.lotusmarkets/dagstore/index
| 4.0K /mnt/md12/.lotusmarkets/dagstore/transients

masih · 2022-02-23T15:01:56Z

masih
Feb 23, 2022
Maintainer

RC-4 Tag is out

The new tag master-spx.idxprov.rc-4 is out. This tag introduces the following changes:

The DagStore repo bloating issue is now fixed.
Retrieval of removal advertisements with no entries fixed.
Several bug fixes and improvements from dependency upgrades, including sync from DNS multiaddr.

Going forward please use this tag as the release target for the testing assignment.

Already ran migration with earlier tags?

You will need to re-run the dagstore migration if your installation used any of the previous tags.

However, if you installed this commit (or any commits after it) from the indexing integration PR, then there is no need to re-run the migration or re-announce indices to the indexers again. Simply install the RC-4 tag when you get a chance.

Massive thanks to @aarshkshah1992 for resolving the dagstore issue and SPX folks whom helped verify the fix.

0 replies

SBudo · 2022-03-03T09:07:29Z

SBudo
Mar 3, 2022

@stephane's IP Report

Results of `provider verify-ingest - TBD`

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	2847
size of the dagstore repo as it’s shipping with a new top level index	26GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	TBD
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	TBD
IO/mem usage	TBD

0 replies

cryptowhizzard · 2022-03-03T22:19:24Z

cryptowhizzard
Mar 3, 2022

@cryptowhizzard Report

Results of `provider verify-ingest`

TBD

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	703
size of the dagstore repo as it’s shipping with a new top level index	20 GB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	10G
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	4.5G
IO/mem usage	with `MaxSimultaneousTransfer` = 10, usage has not changed

0 replies

SBudo · 2022-03-04T00:18:58Z

SBudo
Mar 4, 2022

Step 11 (announcement) worked as I got the message:
2022-03-04T11:02:07.841+1100 INFO storagemarket_impl impl/provider.go:542 finished announcing active deals to index provider {"number of deals": 2942, "number of shards": {"baga6ea4seaqa23f45da7lrsyl3oi3jkyfiea4hsj3vunq5xqblsnlj7awqtrcfi":{},"baga6ea4seaqa2637jzz4nf2avcf6fwxkpppxdsdjwofcnclmku2eemplh7ea6da":{},"baga6ea4seaqa2mhlhvov5iysce2qriqyywbdlpnmjhnlce5nz2bwzwgy3jqvudq":{},"baga6ea4seaqa3eysewrw7cam3gbqnvw4vbxcnuuiuxf44mrgq4rkqzz2cyokw...........

but, I'm getting the following error on some index files not found:

2022-03-04T11:02:07.843+1100    WARN    rpc     [email protected]/handler.go:279        error in RPC call to 'Filecoin.IndexerAnnounceAllDeals': 10 errors occurred:
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqh46u4payuomw55arjgdmhkqmgeidchlvdihjlkoxbhe45iknkkoy.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqb2yzx7eeeeedovtzmps7nrkeyoz3gmier6bdycrukg44qjsfzkpi.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqoi2j3ipmfjvziwtptivgqot7d4yy6xfmz3quohizlnmuub4dkymq.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqdvhn7uovj5rqr2cppl3crhq54zaomtme73qmicgxfkpu7orw4ygi.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqetj4rzl7uxtzy5n667eph33xllagr2d56goua2am6fo4zt4tlway.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqgrkjf7b3lbixmnvhzaqpigdxt7odobgfuia2zyhw345xx5m6heiq.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqatyeoan3qwkzgsvjzaaqquwjrefqbagvr52e54vr757yl4tn7ify.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqeqehsjfq2huwbtqu6hgtmccau36s7bai4wutqxwbu4zmezlcy2cq.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqiohxybg2bochyqfok6r3ars4kgedzqfdndk4izshjbk56lnogecq.full.idx: no such file or directory
        * failed to get iterable index: failed to get iterable index: open /mnt/data/.lotusmarkets/dagstore/index/baga6ea4seaqj2vbf6jxbzi2qbl46v5c7wrstaj3yk4ag2yk6tvfjqaml33q6uoa.full.idx: no such file or directory

Those index files were not in the original dagstore index (from backup)
Can I ignore and move on?

If so, could you run your magic @willscott and run the manual step on your end?

9 replies

SBudo Mar 8, 2022

I ran the script which does the "verify-ingest", but each test returns a "failed verification check".
From what I read above, there seems to be a manual step to be completed on you side?
Just wondering if that's it or if it should work without this now?

`Hi
I have advertised all deals. (Step 11) But most of verification check failed❌ (Step 12)
Does this mean ingesting indexes is just delayed due to the few index providers in operation?
verify-ingest.log

Perhaps I've completed indexing and advertising, so please pick me up.
peerId: 12D3KooWSPtnggJL5wkoAmtgGCosViEC6Z2heTmt7ZTF98GYPyKq

@willscott
willscott
20 days ago
Collaborator
right now there's a manual step from our side between step 11 and 12 because the full lotus gossip mesh hasn't been upgraded to support the new type of messages yet.
We'll run our manual step again today to pick up your index.`

rjan90 Mar 8, 2022
Maintainer

@SBudo I can see your PeerID in here, http://18.116.210.184:3000/providers, so your node has been successfully synced by the indexer. As for the step.12, I think we are still waiting for the production instance to be ready before we are able to get ✅ on the verify-ingest-script.

masih Mar 8, 2022
Maintainer

Hi @SBudo, thank you for pointing this out and apologies for the late reply. Looking at the providers list on the production indexer instance I can see that the peer ID you provided is present.

As @rjan90 pointed out, a small body of work is remaining to get the gossip propagation working such that the sync from indexer happens automatically.

You can safely ignore the verification check failures for now.
I can verify and trigger manual sync from indexer side as needed.

SBudo Mar 8, 2022

awesome, thank you :-)

masih Mar 9, 2022
Maintainer

@SBudo Can I ask you to confirm:

the markets multiaddr here,
whether it is publicly accessible, and
your miner ID

please?

neondragon · 2022-03-05T22:36:30Z

neondragon
Mar 5, 2022

@alex Fox 's IP Report

Results of `provider verify-ingest`

TBA

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	TBA
size of the dagstore repo as it’s shipping with a new top level index	TBA
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	TBA
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	TBA
IO/mem usage	with MaxSimultaneousTransfer = 10, usage is.. TBA

3 replies

neondragon Mar 5, 2022

I'm seeing a markets process repeatedly fail to start and logging this error
{"level":"error","ts":"2022-03-05T22:34:59.651Z","logger":"storagemarket_impl","caller":"impl/provider.go:196","msg":"failed to connect index provider host with the full node: failed to connect index provider host with the full node: failed to dial 12D3KooWGNLxbsJXQYfaSnMRemgVaVYFCBr9w9NTkYaKmLipY6g5:\n * [/ip6/2a04:7340:0:1002::7/tcp/1337] failed to negotiate security protocol: read tcp6 [2a04:7340:0:1002::15]:50430->[2a04:7340:0:1002::7]:1337: read: connection reset by peer\n * [/ip4/185.37.216.112/tcp/1337] failed to negotiate security protocol: read tcp4 185.37.217.6:41344->185.37.216.112:1337: read: connection reset by peer"}

neondragon Mar 6, 2022

It started after about 30 minutes but has this problem again on startup.

benjaminh83 Mar 6, 2022

@neondragon there is a problem with libp2p that has been identified and working on a fix. This is the issue #8129
For now, restart the daemon (only), and after a bit, start the market.

Angelo-gh3990 · 2022-03-06T07:25:37Z

Angelo-gh3990
Mar 6, 2022

Results of `provider verify-ingest - master-spx.idxprov.rc-5`

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	662
size of the dagstore repo as it’s shipping with a new top level index	4.2GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	x MiB / 10 min
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	x MiB / 10 min
IO/mem usage	with MaxSimultaneousTransfer = 10, usage is..

3 replies

Angelo-gh3990 Mar 8, 2022

@willscott , is there a way to verify if the index process is working correctly or that everything got indexed the right way? Thanks.

rjan90 Mar 8, 2022
Maintainer

You should have been indexed if you can see your PeerID/Multiaddrs in this link: http://18.116.210.184:3000/providers :)

Angelo-gh3990 Mar 8, 2022

Well looky looky I'm in that list, lol :-D ... Thanks @rjan90
Ok. Well, great :-)

Meest-NAM · 2022-03-08T03:48:47Z

Meest-NAM
Mar 8, 2022

Question

STEP 12

#!/usr/bin/env sh

MINER_ID="${12D3KooWG66fpxbAxbn4d4Tj3ewY6CceHmuShmQrSvAk9sA1qC9Z}"
DAGSTORE_REPO="${/miner/.lotusmarket/dagstore}"
SAMPLING_PROB="${3-0.05}"

echo "MINER_ID: ${MINER_ID}"
echo "DAGSTORE_REPO: ${DAGSTORE_REPO}"
echo "SAMPLING_PROB: ${SAMPLING_PROB}"
echo ""

for idx in ${DAGSTORE_REPO}/index/*.full.idx
do
echo "Verifying ${idx}"
provider verify-ingest
--to cid.contact:80
--print-unindexed-mhs
--provider-id "${MINER_ID}"
--from-car-index "${idx}"
--sampling-prob "${SAMPLING_PROB}"
echo ""
done

./scrip.sh
scrip.sh: 3: Bad substitution

It can't be implemented, but can I know what the problem is?

2 replies

tmyuu Mar 8, 2022

Please correct it in this way.

MINER_ID="12D3KooWG66fpxbAxbn4d4Tj3ewY6CceHmuShmQrSvAk9sA1qC9Z"
DAGSTORE_REPO="/miner/.lotusmarket/dagstore"
SAMPLING_PROB="0.05"

Meest-NAM Mar 8, 2022

Thx..
but...
./scrip.sh

Verifying /miner/.lotusmarket/dagstore/index/*.full.idx
./scrip.sh: 15: provider: not found

jamerduhgamer · 2022-03-09T01:16:21Z

jamerduhgamer
Mar 9, 2022

James' IP Report

Results of `provider verify-ingest`

Monitoring Metrics

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	2240
size of the dagstore repo as it’s shipping with a new top level index	7.7 GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	x MIB / 10 min
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	x MIB / 10 min
IO/mem usage	with MaxSimultaneousTransfer = 10, usage is (WIP)

0 replies

Meest-NAM · 2022-03-09T09:38:45Z

Meest-NAM
Mar 9, 2022

Results of provider verify-ingest - master-spx.idxprov.rc-5

Monitoring Metrics : f024184

Title	Metric
Are you only running IP process on your market subsystem (no deal making) when collecting the data below?	[x] yes [ ] no
number of the deals made	1836
size of the dagstore repo as it’s shipping with a new top level index	7.7 GiB
growth rate of the $LOTUS_MARKETS_PATH/index-provider folder’s size	x MiB / 10 min
growth rate of the $LOTUS_MARKETS_PATH/datastore/metadata folder’s size	x MiB / 10 min
IO/mem usage	with MaxSimultaneousTransfer = 10, usage is (WIP)

0 replies

Meest-NAM · 2022-03-09T23:37:12Z

Meest-NAM
Mar 9, 2022

2022-03-10T07:57:10.621+0900 INFO dt-impl impl/events.go:303 successfully sent completion message to initiator {"chid": "12D3KooWRKk7RMYvs5NQKmk5zRAPHPSxvNuEKeAVqmexfVZXi33u-12D3KooWM4wsQ3kdd8CDHiVDQthU9JZ9KqsxSdSQT2xj6TAdDth5-1646858319637830640"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x1d394a0]

goroutine 3578830 [running]:
github.com/filecoin-project/go-statemachine/fsm.eventProcessor.Generate(0x41bb140, 0x3b142a0, 0x3ba0391, 0x6, 0xc001a3be90, 0xc001a3bec0, 0x41770d0, 0xc000052968, 0x3684ae0, 0x70873f0, ...)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/eventprocessor.go:113 +0x240
github.com/filecoin-project/go-statemachine/fsm.(*stateGroup).Send(0xc0002ec700, 0x39752c0, 0xc0b9ef1dd0, 0x3684ae0, 0x70873f0, 0xc1189ca000, 0x1, 0x1, 0xc0b9ef1dd0, 0xc0b9ef1dd0)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/fsm_group.go:35 +0x9a
github.com/filecoin-project/go-ds-versioning/pkg/fsm.(*migratedFsm).Send(0xc00025bb20, 0x39752c0, 0xc0b9ef1dd0, 0x3684ae0, 0x70873f0, 0xc1189ca000, 0x1, 0x1, 0x6, 0x6)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/pkg/fsm/fsm.go:57 +0xcc
github.com/filecoin-project/go-data-transfer/channels.(*Channels).send(0xc001a434c0, 0xc01a6573e0, 0x26, 0xc001a7c1b0, 0x26, 0x16dad0ea9168dbf0, 0x6, 0xc1189ca000, 0x1, 0x1, ...)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/channels/channels.go:396 +0x342
github.com/filecoin-project/go-data-transfer/channels.(*Channels).Error(...)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/channels/channels.go:330
github.com/filecoin-project/go-data-transfer/impl.(*manager).OnChannelCompleted(0xc001c40480, 0xc01a6573e0, 0x26, 0xc001a7c1b0, 0x26, 0x16dad0ea9168dbf0, 0x0, 0x0, 0x16dad0ea9168dbf0, 0xc001a41b01)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/impl/events.go:311 +0x46b
github.com/filecoin-project/go-data-transfer/transport/graphsync.(*Transport).gsCompletedResponseListener(0xc001cb8000, 0xc11bf064e0, 0x26, 0x418c2a8, 0xc127a07d40, 0x14)
/root/go/pkg/mod/github.com/filecoin-project/[email protected]/transport/graphsync/graphsync.go:710 +0x18b
github.com/ipfs/go-graphsync/listeners.completedResponseDispatcher(0x38e5b20, 0xc0b9ada3c0, 0x37758a0, 0xc017d5c8e0, 0x0, 0xc0555d7898)
/root/go/pkg/mod/github.com/ipfs/[email protected]/listeners/listeners.go:24 +0xe3
github.com/hannahhoward/go-pubsub.(*PubSub).Publish(0xc001a43080, 0x38e5b20, 0xc0b9ada3c0, 0x0, 0x0)
/root/go/pkg/mod/github.com/hannahhoward/[email protected]/pubsub.go:76 +0xd0
github.com/ipfs/go-graphsync/listeners.(*CompletedResponseListeners).NotifyCompletedListeners(0xc00063c5e0, 0xc11bf064e0, 0x26, 0x418c2a8, 0xc127a07d40, 0x14)
/root/go/pkg/mod/github.com/ipfs/[email protected]/listeners/listeners.go:40 +0xb9
github.com/ipfs/go-graphsync/responsemanager.(*subscriber).OnNext(0xc0db8a6280, 0x3686560, 0xc0b98d04f0, 0x38e5d60, 0xc0b98b7cb0)
/root/go/pkg/mod/github.com/ipfs/[email protected]/responsemanager/subscriber.go:50 +0x39e
github.com/ipfs/go-graphsync/notifications.(*subscriberRegistry).send(0xc0555d7c98, 0x3686560, 0xc0b98d04f0, 0x38e5d60, 0xc0b98b7cb0)
/root/go/pkg/mod/github.com/ipfs/[email protected]/notifications/publisher.go:175 +0x102
github.com/ipfs/go-graphsync/notifications.(*publisher).start(0xc11a40fbc0)
/root/go/pkg/mod/github.com/ipfs/[email protected]/notifications/publisher.go:141 +0x245
created by github.com/ipfs/go-graphsync/notifications.(*publisher).Startup
/root/go/pkg/mod/github.com/ipfs/[email protected]/notifications/publisher.go:46 +0x3f

"Process" shutdown after 3 hours of running "provider-index".

1 reply

willscott Mar 10, 2022
Collaborator

This looks like a generic graphsync issue rather than with anything in the index provider directly. Perhaps it has to do with our ramp-up in testing active retrievals. cc @rvagg to see if you can get a sense of what might be up in graphsync here.

SPX Index Provider Testing #8087

jennijuju Feb 14, 2022 Maintainer

Table of Contexts

Overview

DagStore Migration

Indexing Announcement

Content Routing Verification

Expectations

New CAR index format in DagStore

Index Provider GossipSub Announcements

Index Provider GraphSync Server

Index Provider Storage Usage

⚠️ Action Items, Reminders and Risk Mitigations

Steps to Become an Index Provider

Indexer Provider Configuration

Almost Done

Replies: 19 comments · 26 replies

willscott Feb 14, 2022 Collaborator

willscott Feb 16, 2022 Collaborator

jennijuju Feb 14, 2022 Maintainer Author

Ozzy's IP Report Example

Results of provider verify-ingest

Monitoring Metrics

rjan90 Feb 15, 2022 Maintainer

Phi-rjan´s IP Report Example

Result of provider verify-ingest

Monitoring Metrics

rjan90 Mar 8, 2022 Maintainer

Folder Size Change Report

TippyFlitsUK Feb 15, 2022 Maintainer

@TippyFlits IP Report

Results of provider verify-ingest - TBD

Monitoring Metrics

TippyFlitsUK Mar 8, 2022 Maintainer

Folder Size Change Report

@tmyuu IP Report

Results of provider verify-ingest - TBD

Monitoring Metrics

willscott Feb 16, 2022 Collaborator

jennijuju Feb 16, 2022 Maintainer Author

New tag is out!

masih Feb 16, 2022 Maintainer

masih Feb 23, 2022 Maintainer

RC-4 Tag is out

Already ran migration with earlier tags?

@stephane's IP Report

Results of provider verify-ingest - TBD

Monitoring Metrics

@cryptowhizzard Report

Results of provider verify-ingest

Monitoring Metrics

rjan90 Mar 8, 2022 Maintainer

masih Mar 8, 2022 Maintainer

masih Mar 9, 2022 Maintainer

@alex Fox 's IP Report

Results of provider verify-ingest

Monitoring Metrics

Results of provider verify-ingest - master-spx.idxprov.rc-5

Monitoring Metrics

rjan90 Mar 8, 2022 Maintainer

jennijuju
Feb 14, 2022
Maintainer

Replies: 19 comments 26 replies

willscott
Feb 14, 2022
Collaborator

willscott Feb 16, 2022
Collaborator

jennijuju
Feb 14, 2022
Maintainer Author

Results of `provider verify-ingest`

rjan90
Feb 15, 2022
Maintainer

Result of `provider verify-ingest`

rjan90 Mar 8, 2022
Maintainer

TippyFlitsUK
Feb 15, 2022
Maintainer

Results of `provider verify-ingest` - TBD

TippyFlitsUK Mar 8, 2022
Maintainer

Results of `provider verify-ingest` - TBD

willscott Feb 16, 2022
Collaborator

jennijuju
Feb 16, 2022
Maintainer Author

masih Feb 16, 2022
Maintainer

masih
Feb 23, 2022
Maintainer

Results of `provider verify-ingest - TBD`

Results of `provider verify-ingest`

rjan90 Mar 8, 2022
Maintainer

masih Mar 8, 2022
Maintainer

masih Mar 9, 2022
Maintainer

Results of `provider verify-ingest`

Results of `provider verify-ingest - master-spx.idxprov.rc-5`

rjan90 Mar 8, 2022
Maintainer