Create workflow to check query forms

[andrewtavis]

Terms

• I have searched open and closed feature requests
• I agree to follow Scribe-Data's Code of Conduct

Description

This issue is to document work that I've been doing to create a check for form identifiers within all Scribe-Data queries. The check goes through and derives the properties of all forms and create a new metadata file for lexeme forms from which a naming criteria is derived.

Contribution

I'm working on this. Current status at time of writing is that I need to add in a check for whether all forms are actually returned in the query, that each returned form is unique, and that returned forms are ordered in the same way as the metadata file. From there also a minor check to see that the query docstring is correct would be good.

From there I need to go through and rename all the forms in the queries that have issues :)

[DeleMike]

Hello @andrewtavis 👋🏾, well done for #451 !!
I'm happy to assist you on this issue. Is there any task I can help with?

[andrewtavis]

Thanks for your offer to help, @DeleMike :) Maybe I can solve the current issues with the queries later tonight, and from there we can make some issues for the extra features?

[DeleMike]

Sure! I will ping you later, then!

[andrewtavis]

Ok @DeleMike, with #451 in we now need the following:

• Add in a check for whether all forms are actually returned in the query (i.e. are there optional selections that are not brought up to the select statement)
• Add a check that each returned form is unique (never ?plural ?plural)
• Add a check that the returned forms are ordered in the same way as the metadata file and that their optional statements match (this is a simple as checking the top against the labels for the generated form labels)
• General testing and error handling

CC @axif0 who's also looking for Python work to do :) Do you all want to plan out the work here?

[DeleMike]

Thanks @andrewtavis!

We will look into it. @axif0 how about we break this into parts?

---

Some questions:
For these checks, how do we wanna verify outputs? Are we gonna make network calls for all the checks?

Also, I believe these checks will be part of our local tests, yes?

[andrewtavis]

Honestly let's keep this to expanding the lexeme_form_metadata.json file. Yes it's more work for us, but doing network calls for all of this each time a PR commit is made would be way to much usage of a common database :)

And ya we really should do a single Python or shell script that runs all of the checks we've been working on locally and add a note on that into the testing section of the contriving guide 😊

[axif0]

I'm done with Add a check that each returned form is unique (never ?plural ?plural) and working on Add in a check for whether all forms are actually returned in the query (i.e. are there optional selections that are not brought up to the select statement)

[andrewtavis]

This would be great, @axif0! This can form the basis of some of the other work here. So as with getting all the form texts, we should also parse and get all forms that are being returned from the query. This first check will be to make sure that each is unique, and from there we can work towards making sure that what's being returned matches what's in the forms 😊

[DeleMike]

[OFF TOPIC]
Hey everyone, I'm currently not strong enough. I fell ill.

But hoping to come back soon and resolve this issue! They said I need rest.

EDIT: I had to drop this update so that it doesn't seem I am silent.

[andrewtavis]

No stress, @DeleMike! Please take care and feel better soon!

[OmarAI2003]

[OFF TOPIC] Hey everyone, I'm currently not strong enough. I fell ill.

But hoping to come back soon and resolve this issue! They said I need rest.

EDIT: I had to drop this update so that it doesn't seem I am silent.

Thank you for the update.

I hope you feel better soon! If there’s anything I can do to assist with this issue while you’re away, please let me know, and I’m happy to coordinate with @andrewtavis and @axif0 if needed.

Take care!

[KesharwaniArpita]

Get well soon @DeleMike :)

Hi @andrewtavis, @DeleMike , @axif0 and @OmarAI2003 I also want to help here. Should we all do one task each?

[andrewtavis]

Ok @axif0 finished the first two :) @OmarAI2003, do you want to do the edit to this process that checks to make sure that the order of the returned forms is the same as how they appear below in the query? And @KesharwaniArpita, can you check the functions and make sure that they have the needed checks included?

[KesharwaniArpita]

Sure @andrewtavis

[OmarAI2003]

Sure, thanks, Andrew.

[KesharwaniArpita]

And @KesharwaniArpita, can you check the functions and make sure that they have the needed checks included?

@andrewtavis I am a little confused by like what functions are we talking about? The one in the check_query_forms.py or the files for the scribe data functions of list, total etc?

[andrewtavis]

Would be the functions in check_query_forms.py, @KesharwaniArpita :)

[DeleMike]

Thank you so much everyone! ✨
This means a lot!

Feel a bit better :)

[andrewtavis]

Great to hear, @DeleMike!

[andrewtavis]

So last thing here is to check that the actual QIDs that make up the form after wikibase:grammaticalFeature are in order, right? :) Thanks all so much for the amazing work here!

[OmarAI2003]

So last thing here is to check that the actual QIDs that make up the form after wikibase:grammaticalFeature are in order, right? :) Thanks all so much for the amazing work here!

Yeah, and the ordering of the labels in sELECT to be based on the lexeme forms JSON.

[KesharwaniArpita]

Yess!!