Support searching with regular expressions

[lyz-code]

I'd like to use regular expressions to find elements of a dictionary with grep.

(Pdb) entity_attributes
{'alone': {'id_': 'alone', 'name': 'Timothy', 'last_name': 'Jackson', 'country': 'Korea', 'rating': 5491}, 'film': {'id_': 'film', 'name': 'David', 'last_name': 'Howell', 'country': 'Moldova', 'rating': 9731}, 'thought': {'id_': 'thought'
, 'name': 'Jason', 'last_name': 'Roberts', 'country': 'Western Sahara', 'rating': 5484}}
(Pdb) entity_attributes | grep('Da')
{'matched_values': ["root['film']['name']"]}
(Pdb) entity_attributes | grep('Da.*')
{}

If you give me some guidance on how to do it, I can make a PR

[seperman]

Hi @lyz-code
Thanks a lot for the PR. I left a comment. We can continue discussing in the PR. :)

[lyz-code]

Hi @seperman, I've added a new PR to close this issue, so you can review it when you have time

[seperman]

Hi lyz-code, Thanks for making the PR. Im not next to my laptop right now so can’t leave a thorough review yet. Im not sure if we should expand regex matches to include numbers. In the current code when the user is searching for numbers it only matches with numbers and not strings. Should we not keep the behavior consistent? Sep Dehpour

…

On Apr 19, 2021, at 7:01 AM, lyz-code ***@***.***> wrote:  Hi @seperman, I've added a new PR to close this issue, so you can review it when you have time — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[lyz-code]

It will greatly help me in my use case. I'm developing repository-orm a library to act as an abstraction over persistent storage, allowing the user to decouple the model layer from the data layer.

In one of the implementations I use grep to search on the attributes of pydantic objects. The library is already being used by other programs that need to do regular expression searches on numbers of these object attributes, but pydantic correctly saves them as integers, so they don't show up in the results.

If we don't implement it, I will need to manually go through each attribute of each object stored, and if it's an int, convert it to string and see if the regular expression matches. Basically rewrite part of DeepSearch, so I'll be glad if you accepted the change.

I can see that having a different outcome whether use_regexp is enabled is not desirable, so if you want, I can implement that also for the regular use. I think it's safe to assume that if a user is searching for '1234', 1234 should also show up. If you don't agree maybe we can use a strict_checking flag?

Thanks anyway for reviewing it :)

[seperman]

Thanks for explaining! That makes sense now. Sorry it took me a while to get back to you. Im not next to my laptop this week often. Yes the strict_checking flag would be great! Keeping the libraries outcome consistent between different versions is important. If you could implement that flag with a default of True and have it use the same logic when use_regexp=False, that would be great. Basically when strict_checking is False, in your example of searching for ‘1234’, it should match 1234 whether use_regexp=True or False. Im curious, do you think it should match 1123456 too? Thanks, Sep Dehpour

…

On Apr 19, 2021, at 12:39 PM, lyz-code ***@***.***> wrote:  It will greatly help me in my use case. I'm developing repository-orm a library to act as an abstraction over persistent storage, allowing the user to decouple the model layer from the data layer. In one of the implementations I use grep to search on the attributes of pydantic objects. The library is already being used by other programs that need to do regular expression searches on numbers of these object attributes, but pydantic correctly saves them as integers, so they don't show up in the results. If we don't implement it, I will need to manually go through each attribute of each object stored, and if it's an int, convert it to string and see if the regular expression matches. Basically rewrite part of DeepSearch, so I'll be glad if you accepted the change. I can see that having a different outcome whether use_regexp is enabled is not desirable, so if you want, I can implement that also for the regular use. I think it's safe to assume that if a user is searching for '1234', 1234 should also show up. If you don't agree maybe we can use a strict_checking flag? Thanks anyway for reviewing it :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[lyz-code]

Hi, don't worry about the delay, that's the thing with open source, sometimes you have more time, sometimes you have less.

I'll implement the strict_checking flag these days. Regarding your question, my first thought is that if strict_checking == False and use_regexp == False then 1123456 should not match as I would expect only to match 1234, but if you use use_regexp == True then yes, as you're not searching for the exact 1234 content but strings that match that regular expression. How does that sound?

[seperman]

You are right about opensource! I totally agree with what you described regarding the 1234 matching other integers when regexp is True. Thanks! Sep Dehpour

…

On Apr 21, 2021, at 2:46 PM, lyz-code ***@***.***> wrote:  Hi, don't worry about the delay, that's the thing with open source, sometimes you have more time, sometimes you have less. I'll implement the strict_checking flag these days. Regarding your question, my first thought is that if strict_checking == False and use_regexp == False then 1123456 should not match as I would expect only to match 1234, but if you use use_regexp == True then yes, as you're not searching for the exact 1234 content but strings that match that regular expression. How does that sound? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[lyz-code]

I've made the changes, they are ready for review in the PR.

[lyz-code]

Thanks for merging the PR, should we close then this issue when #245 is merged?

[seperman]

Yep! I will ping you once I cut a release. Thanks! Sep Dehpour

…

On Apr 27, 2021, at 8:42 AM, lyz-code ***@***.***> wrote:  Thanks for merging the PR, should we close then this issue when #245 is merged? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[seperman]

Hi @lyz-code
Thanks for contributing to DeepDiff. Your feature is released: https://zepworks.com/deepdiff/current/#new-in-deepdiff-5-3-0