Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit information collected by group-members list command #64

Closed
wants to merge 1 commit into from

Conversation

malacupa
Copy link

Dear BloodHound team,

This is possibly breaking change to only collect user IDs when collecting group members.

Reason for this is that if you run collection of this type in large environment you'd need VERY beefy machine to collect it. The I suppose you'd need even beefier machine to import it to neo4j. This happens because plenty of additional information are collected for each member while for most cases collecting group ID and matching user IDs should be enough. It happened to me that I was not even able to collect whole group memberships and the JSON file on disk was already over 200GB.

If this is unacceptable for list group-members, how about using this behavior at least for list az-ad? The list az-ad should collect information about users in each group anyway.

Copy link

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


malacupa seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request

@malacupa
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

@sircodemane
Copy link
Contributor

Hi @malacupa thanks for bringing this up. We have had some reports from our BloodHound slack of possibly unintentional/malformed data that is severely bloating AzureHound collection files and we have an internal ticket to track and investigate the issue. I will bring this suggestion up with the team, but we may hold off looking into this change until we can rule out the previous problem as the main source of bloat.

@sircodemane
Copy link
Contributor

sircodemane commented Dec 8, 2023

Hello @malacupa, we have another PR up to address empty data being encoded and bloating the collection files. If you're interested, I'd love to have you check out that branch and see if it has an impact on your collections: #67

@1njected
Copy link

Had problems with a large capture, 15GB. This PR fixed the issues and drastically reduced the size of the capture and we were able to successfully import it to BloodHound.

@StephenHinck
Copy link
Contributor

Hey all, this issue is resolved in #67. This will be included in next week's release. Thank you for your effort and contributions!

@github-actions github-actions bot locked and limited conversation to collaborators Feb 7, 2024
@StephenHinck StephenHinck reopened this May 1, 2024
@StephenHinck
Copy link
Contributor

@malacupa - thank you very much for your effort on this. Even though we did not end up including your code within our fixes, we would still like to send you a swag package to show our gratitude.

If you'd be interested, please email me at shinck [at] specterops [dot] io with your shipping address and t-shirt size, and I'll get that on its way!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants