Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Edge refactoring to DAL and minor PIP improvements #671

Merged
merged 5 commits into from
Jun 8, 2020

Conversation

VakarisZ
Copy link
Contributor

What is this?

Fixes part of #662

Add any further explanations here.

Checklist

  • Have you successfully tested your changes locally?
  • Is the TravisCI build passing?

Changes

Refactored edges to use DAL
Minor PIP improvements

@VakarisZ VakarisZ requested a review from ShayNehmad May 29, 2020 09:07
@codecov
Copy link

codecov bot commented May 29, 2020

Codecov Report

Merging #671 into develop will increase coverage by 2.05%.
The diff coverage is 75.63%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #671      +/-   ##
===========================================
+ Coverage    56.62%   58.68%   +2.05%     
===========================================
  Files          120      127       +7     
  Lines         4044     4216     +172     
===========================================
+ Hits          2290     2474     +184     
+ Misses        1754     1742      -12     
Impacted Files Coverage Δ
.../monkey_island/cc/services/reporting/pth_report.py 31.64% <ø> (ø)
...nfo_collectors/system_info_telemetry_dispatcher.py 89.65% <ø> (ø)
monkey/monkey_island/cc/services/node.py 29.46% <16.66%> (-0.86%) ⬇️
monkey/monkey_island/cc/services/edge/edge.py 62.85% <62.85%> (ø)
...y/monkey_island/cc/services/edge/displayed_edge.py 84.44% <84.44%> (ø)
monkey/monkey_island/cc/models/edge.py 100.00% <100.00%> (ø)
...key_island/cc/services/edge/test_displayed_edge.py 100.00% <100.00%> (ø)
monkey/monkey_island/cc/services/edge/test_edge.py 100.00% <100.00%> (ø)
...ollectors/test_system_info_telemetry_dispatcher.py 100.00% <100.00%> (ø)
monkey/monkey_island/cc/testing/IslandTestCase.py 100.00% <100.00%> (ø)
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d430b91...4c83196. Read the comment docs.

displayed_edge["os"] = os
# we need to deepcopy all mutable edge properties, because weak-reference link is made otherwise,
# which is destroyed after method is exited and causes an error later.
displayed_edge["exploits"] = deepcopy(edge['exploits'])
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edge is a weak-reference link to mongoengine document. This link is destroyed once we exit the method where edge = Edge.objects.get(). I'm not sure how we can make this less error prone, maybe refactor exploits into embedded document...

Copy link
Contributor

@ShayNehmad ShayNehmad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have some questions, mostly about static functions

It looks a LOT better


@staticmethod
def get_or_create_edge(src_node_id, dst_node_id, src_label, dst_label):
edge = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'll be better to start with None and in the finally use if edge is None

Comment on lines 23 to 25
edge.src_label = src_label
edge.dst_label = dst_label
edge.save()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we reuse update_label here?

monkey/monkey_island/cc/services/edge/edge.py Show resolved Hide resolved
monkey/monkey_island/cc/services/edge/edge.py Show resolved Hide resolved
monkey/monkey_island/cc/services/edge/edge.py Outdated Show resolved Hide resolved
monkey/monkey_island/cc/services/edge/edge.py Outdated Show resolved Hide resolved
monkey/monkey_island/cc/services/edge/edge.py Outdated Show resolved Hide resolved
edge.tunnel = False
edge.save()
except DoesNotExist:
pass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log a warning?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not exactly, but solved

displayed_edge = DisplayedEdgeService.edge_to_displayed_edge(edge, for_report)
return displayed_edge

@staticmethod
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a static method and not a regular method of Edge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to separate the data structure/schema of Edge document from methods, in order not to have a single huge file. But you're right - we can do it without static method :) And I should make it into a boundary.

else:
return [x + ": " + (services[x]['name'] if 'name' in services[x] else 'unknown') for x in services]

@staticmethod
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this a static method and not a regular method of Edge?

Copy link
Contributor

@ShayNehmad ShayNehmad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I 100% agree with separating data structure and behaviour, but this looks good nonetheless. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants