Run data collection for Clay v0.2

[yellowcap]

We can use the current pipeline, but probably with the following changes:

• Reduce chip size to 256x256 pixels
• Add more time steps (can do all available years)
• Increase the number of MGRS tiles by 3-5 times.

Regarding the MGRS tile increase, the question is if we want to change the ratio of the input. I discussed with @srmsoumya yesterday that we should mabye increase the fraction of the landcover classes with human footprint, i.e. Urban and Agriculture. Presumably that is what users will be most interested in for search. So we could increase the fraction of that to give this more weight.

[brunosan]

I suspect we are also dropping a very substantial share of inputs due a single no-data pixel invalidating the whole set.

model/scripts/tile.py

Lines 39 to 51 in ae70345

	if int(tile.sel(band="B02").isin([NODATA]).sum()):
	print("Too much no-data in B02")
	return False
	bands_to_check = ["vv", "vh", "dem"]
	for band in bands_to_check:
	if int(np.isnan(tile.sel(band=band)).sum()):
	print(f"Too much no-data in {band}")
	return False
	# Check for cloud coverage
	cloudy_pixel_count = int(tile.sel(band="SCL").isin(SCL_FILTER).sum())
	if cloudy_pixel_count / PIXELS_PER_TILE >= BAD_PIXEL_MAX_PERCENTAGE:

aoi = gpd.GeoDataFrame(
    pd.DataFrame(["CDL Test Region"], columns=["Region"]),
    crs="EPSG:4326",
    geometry=[box(-92.30926, 32.17581, -90.01114, 38.63658)],  # using lower left and upper right coordinates
)

See: https://github.com/Clay-foundation/office/issues/170#issuecomment-1914173261

[brunosan]

For the latlon coordinates embeddings to capture the intended global structure, I believe we must include full global coverage on the training set, which in my opinion means to add full coverage from MODIS, either composite or several times raw images.

Perhaps even train first with modis only to warm up a general latlon embeddings?

[yellowcap]

For Clay v0.2 we are not planning to change the input platforms. Adding MODIS would require changes in architecture. The idea for v0.2 was to use the same datasources but with a much larger sample.

[yellowcap]

Ran data collection with code from #173

We have 2535 MGRS tiles successfully processed, the data sits in s3://clay-tiles-04-sample-v02