Add workflow to check project structure

[andrewtavis]

Terms

• I have searched open and closed feature requests
• I agree to follow Scribe-Data's Code of Conduct

Description

Along with #339 and #340 another check that's needed for the project is one that validates that some directories are being created in an appropriate way. Specifically within the src/scribe_data/language_data_extraction we need all sub directories of languages to be consistently named, where as of now we have some directories that are different like noun instead of nouns. Ideally this would take the form of a workflow check_project_structure.yaml within .github/workflows that would call a script check_project_structure.py that would make sure that all directories under languages are data types of the project.

Contribution

Happy to support with this and review when a PR is ready 😊

[axif0]

want to assign.. if there is no problem.. @andrewtavis I'm learning github action thanks to you 😅 .

[andrewtavis]

Sure thing, @axif0 :) Check #350 for another one and my comment for how to make it run on all commits to a PR 😊

[andrewtavis]

Another idea for a workflow, @axif0: How about something that tells us which queries we're missing? So goes through and checks all languages and makes sure there's a directory for all the data types? Might fail for a quick moment, but that would assure that we have good support in the package 😊

[axif0]

@andrewtavis Ok. what I understand is we need to create an workflow action that shows annotations (warning type) for missing queries. This workflow go through all the language folders and sub_language, check if all data_types or not, if not append the warning. Also if folder exist, then check if query file exist or not.

reference

Am I right ? can you please elaborate a bit if you don't mind?

[andrewtavis]

Roughly what you're describing, so make sure that all the directories are there and that in each of them is the query file. With that being said, we don't need this to be a warning. Let's let it fail and then that forces us to write the missing queries :)

[andrewtavis]

We can similarly add it to the current workflow, I'd say @axif0. We're already parsing all the directories, so we might as well just add them to the output? Or would you say a different workflow would be better?

[axif0]

ya @andrewtavis I think so. Adding this feature in this workflow would be a better option. let's do it.

Should we search for all data-types of specific language?
like for English we need to verify the data-types are there or not?

adjectives",
"adverbs",
"articles",
"autosuggestions",
"conjunctions",
"emoji_keywords",
"nouns",
"personal_pronouns",
"postpositions",
"prepositions",
"pronouns",
"proper_nouns",
"verbs",

[andrewtavis]

I think that we're generally fine to just get all data types and the user can see that there's no returned data and then check to see whether the language has or doesn't have say postpositions, which might not be in every language. More important that we have tests that we have the functionality for all of them :)

[andrewtavis]

Closed by #432 :) We should look into simplifying all this down to maybe three checks, @axif0:

• check_project_metadata makes sure the metadata files are correct
• check_project_structure makes sure the structure is valid given the files
• check_project_queries the combination of check_query_identifiers and check_query_forms

Let me know how this sounds and we can make another issue!