Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify specifying collection properties #246

Open
jdries opened this issue Oct 5, 2021 · 7 comments
Open

simplify specifying collection properties #246

jdries opened this issue Oct 5, 2021 · 7 comments

Comments

@jdries
Copy link
Collaborator

jdries commented Oct 5, 2021

Currently, in load_collection, property filtering only seems to work with callbacks:

    s2 = connection.load_collection(s["SENTINEL2"],                                 
                                temporal_extent=temp_ext,
                                spatial_extent=spat_ext,
                                bands=["B03","B04","B05","B06","B07","B08","B11","B12","SCL"],
                                properties={"cloudcover":lambda x: x.eq(90)}

This increase the learning curve somewhat, and a lot of property filters are based on equality. So something like:

    properties={"cloudcover":90}

could be translated into a correct callback by the client?

@m-mohr
Copy link
Member

m-mohr commented Oct 6, 2021

What would properties={"cloudcover":90, "platform": "S1a"} translate to? Are they connected with "and" or "or"?

Long-term it might make sense to adopt something like CQL2 text (see OGC APIs) as a simplified version instead of objects, which are more human-readable and flexible (e.g. cloudcover < 90 AND platform = 's1a'). Assuming that a CQL2 text parser will be implemented soon.

PS: cloud cover equality is probably not the best example...

@jdries
Copy link
Collaborator Author

jdries commented Oct 6, 2021

yes, multiple properties would imply an AND condition in my interpretation
cloudcover is indeed a special case: most search catalogs somehow don't seem to have a problem with specifying this as an equals and interpreting it as smaller than, but we may need to check the details on that!

@soxofaan
Copy link
Member

soxofaan commented Oct 8, 2021

or introduce a minimal (SQL-like) query language

properties="cloudcover<90 and platform=s1a"

that is converted by python client appropriately

@m-mohr
Copy link
Member

m-mohr commented Oct 8, 2021

CQL2 text would be exactly that, which then you can also re-use for STAC API queries etc. And we could hope that we get a Python library from OGC API implementors so that we don't need to implement it ourself.

@soxofaan
Copy link
Member

related API issue: Open-EO/openeo-api#396 (what properties can be filtered on?)

@soxofaan
Copy link
Member

soxofaan commented Nov 3, 2021

or introduce a minimal (SQL-like) query language

such a mini language could even be useful in other places, e.g.:

cube.reduce_dimension("bands", reducer=" (B08 - B05) / (B08 - B05)")

mask = cube.apply("x == 4")

that's actually a generalization of what's under construction under #255

@soxofaan
Copy link
Member

In #331 I experimented with this approach:

from openeo import collection_property

cube = con100.load_collection(
    "S2",
    properties=[
        collection_property("eo:cloud_cover") <= 75,
        collection_property("platform") == "Sentinel-2B",
    ]
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants