Adding the Microsoft.Security NetworkData API #6860

DanaPeled · 2019-08-04T08:58:41Z

NetworkData is a new feature in Azure Security Center.
Security Center's Network Map displays a visualization of the various network elements such as VNets, Subnets, VM's etc. connectivity and traffic information. The network traffic REST API presents the underlying information upon which the map is constructed, including the allowed connections and actual traffic data for each VM.

merging master

merge from master

azure into cloned repo

master to develop

…specs

openapi-sdkautomation · 2019-08-04T08:58:44Z

In Testing, Please Ignore

[Logs] (Generated from 1712a86, Iteration 46)

.NET: test-repo-billy/azure-sdk-for-net [Logs] [Diff]

Microsoft.Azure.Management.SecurityCenter [Logs] [Instructions]
- Microsoft.Azure.Management.SecurityCenter.1.0.0.nupkg
- Microsoft.Azure.Management.SecurityCenter.1.0.0.snupkg

Go: test-repo-billy/azure-sdk-for-go [Logs] [Diff]

preview/security/mgmt/v1.0 [Logs]
preview/security/mgmt/v2.0 [Logs]
preview/security/mgmt/v3.0 [Logs]

Python: test-repo-billy/azure-sdk-for-python [Logs] [Diff]

azure-mgmt-security [Logs] [Instructions] [Generation PR]
- azure-mgmt-security-0.3.0.zip
- azure_mgmt_security-0.3.0-py2.py3-none-any.whl

JavaScript: test-repo-billy/azure-sdk-for-js [Logs] [Diff]

@azure/arm-security [Logs] [Instructions]
- azure-arm-security-1.2.0.tgz

AutorestCI · 2019-08-04T09:00:02Z

Automation for azure-sdk-for-python

Encountered an unknown error: (azure-sdk-for-python)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 093067a4b5aa7c7a5478d3a14df7132404e97ed3
  stderr: 'fatal: reference is not a tree: 093067a4b5aa7c7a5478d3a14df7132404e97ed3'

AutorestCI · 2019-08-04T09:02:03Z

Automation for azure-sdk-for-go

Encountered an unknown error: (azure-sdk-for-go)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout 093067a4b5aa7c7a5478d3a14df7132404e97ed3
  stderr: 'fatal: reference is not a tree: 093067a4b5aa7c7a5478d3a14df7132404e97ed3'

azuresdkci · 2019-08-04T09:05:38Z

Can one of the admins verify this patch?

DanaPeled · 2019-08-05T08:47:37Z

@chlahav , Review will be much appreciated :)

yanivn3 · 2019-09-04T06:38:11Z

A few questions for the ARM review. How is this data used? Are clients expected to poll it frequently? What is the refresh interval of the data?

Hi @KrisBash ,
he data is used as part of network map feature in Azure Security Center. The feature shows all possible vs allowed connections in Azure in order to recommend what connections can be closed (in security this is called segmentation). The backend process updates the data all the data but for this Rest API we keep it in hourly buckets (there is no importance to actual traffic just for the summary of it - if user wants to see the actual traffic, our UI redirect him to Azure Monitor blade).
Regards clients, The Ibize blade of network map will call the API upon blade initialization and upon refresh - we do not refresh the data regularly since this is not relevant for our scenario.

merged from azure-master

...ion/security/resource-manager/Microsoft.Security/preview/2019-01-01-preview/networkData.json

…specs

pilor

Some follow-up comments

...ion/security/resource-manager/Microsoft.Security/preview/2019-01-01-preview/networkData.json

anuchandy · 2019-09-11T00:57:32Z

...ion/security/resource-manager/Microsoft.Security/preview/2019-01-01-preview/networkData.json

+        "tcpPorts": {
+          "readOnly": true,
+          "type": "string",
+          "description": "The connections tcp ports"


what is the delimiter for the ports? can we update the description to include that info.

anuchandy · 2019-09-11T00:58:09Z

...ion/security/resource-manager/Microsoft.Security/preview/2019-01-01-preview/networkData.json

+        "udpPorts": {
+          "readOnly": true,
+          "type": "string",
+          "description": "The connections udp ports"


same comment as above on including delimiter info in the description.

anuchandy · 2019-09-11T01:01:44Z

...ion/security/resource-manager/Microsoft.Security/preview/2019-01-01-preview/networkData.json

+        "tags": [
+          "NetworkData"
+        ],
+        "description": "Get the network data on all your scanned resources inside a scope",


Get the network data on all your scanned resources inside a scope - how can we specify the scope parameter?

anuchandy · 2019-12-01T21:49:52Z

Closing this PR since another recent PR with similar content opened.

DanaPeled and others added 7 commits October 16, 2018 15:19

Merge pull request #1 from Azure/master

61f5a59

merging master

Merge pull request #2 from Azure/master

556ffa5

merge from master

Merge pull request #3 from Azure/master

b2ac914

merge from master

Merge pull request #4 from Azure/master

faa2211

azure into cloned repo

Merge pull request #5 from Azure/master

4f1e974

master to develop

added asc network trafic api

6ff019d

Merge branch 'master' of https://github.com/DanaPeled/azure-rest-api-…

7ab778a

…specs

azuresdkci assigned anuchandy Aug 4, 2019

azuresdkci requested a review from anuchandy August 4, 2019 09:00

DanaPeled added 11 commits August 4, 2019 12:20

fixed errors

770cd79

fixed errors

740c0c4

fixed errors

9cdaa74

fixed errors

5a478c4

fixed path

bd26635

fixed examples

5b0291d

fixed errors

6bbca92

fixed errors

cec7f3c

fixed errors

a0e2f58

fixed errors

659e402

fixed errors

e7ea903

This was referenced Aug 5, 2019

[AutoPR security/resource-manager] Adding the Microsoft.Security NetworkData API Azure/azure-sdk-for-python#6661

Closed

[AutoPR security/resource-manager] Adding the Microsoft.Security NetworkData API Azure/azure-sdk-for-go#5432

Closed

chlahav self-requested a review August 5, 2019 11:14

chlahav added DoNotMerge <valid label in PR review process> use to hold merge after approval WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required labels Aug 5, 2019

KrisBash added the ARMReviewInProgress label Sep 4, 2019

Merge pull request #6 from Azure/master

0ba80dc

merged from azure-master