Add "filetypes" metadata to rules

[tstromberg]

To improve performance and reduce false positives, I'd like to add metadata to our rules to constrain which file types they are associated with. For example:

meta:
   filetypes = "python,javascript"

I expect that most rules will be associated with a single file type. As far as standardization goes, I recommend basing it loosely on MIME types, specifically the second half of it. Looking at https://www.sitepoint.com/mime-types-complete-list/ - I would say our filetypes would look something like:

• python
• javascript
• shell
• c

For examples where the mime types don't exist or are excessively long, we can make up something meaningful, like:

• macho
• elf

[tstromberg]

cc @egibs for his thoughts.

This is related to #382

[egibs]

I'm all for this. The hard part will be going through each rule and adding the metadata and we still don't have a way to layer customizations on top of third party rules so TBD how complete our coverage will end up being.

[tstromberg]

I’m considering third party rules out of scope for this at the moment. There may be however some interesting heuristics we can do to autogenerate this metadata based on tags or other metadata fields supported by 3rd party rules. But yes, adding metadata to each rule will be painful, but I’m down to do it as part of a general cleanup.

…

On Mon, Sep 16, 2024 at 7:53 AM Evan Gibler ***@***.***> wrote: I'm all for this. The hard part will be going through each rule and adding the metadata and we still don't have a way to layer customizations on top of third party rules so TBD how complete our coverage will end up being. — Reply to this email directly, view it on GitHub <#454 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAYYMDAURIFFSNJMFU4Q4DZW3BEXAVCNFSM6AAAAABOIFG6N2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJSG4YDINBUGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[tstromberg]

A number of rules have this now: I think the next step will be updating our engine to respect this file type. Then we will be more likely to implement it everywhere.

For example, .py files should only run rules without a filetype defined, or those with "py" in the file types.