Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AI examples extension #1196

Merged
merged 22 commits into from
Feb 25, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
6cfdacd
Initial extension creation
mirdaki-ms Dec 26, 2019
043ea9a
Add ability to add Aladdin examples to the default help ones
mirdaki-ms Dec 27, 2019
fae34bf
Removed unused boilerplate for Aladdin extension
mirdaki-ms Dec 30, 2019
467d8f3
Add test and clean up Aladdin extension code
mirdaki-ms Dec 31, 2019
1afbd22
Rename extension from Aladdin to "ai-examples"
mirdaki-ms Jan 4, 2020
3cc1cf4
Add a min version of 2.0.80, which should have the hook
mirdaki-ms Jan 8, 2020
5f98770
Update the de-duplication to clean text first
mirdaki-ms Jan 13, 2020
7848e78
Change the default operation to replace the existing examples
mirdaki-ms Jan 13, 2020
d8add28
Update summary to be more compliant with guidelines
mirdaki-ms Jan 13, 2020
424d83e
Add ai-examples to index.json as part of publishing
mirdaki-ms Jan 13, 2020
46b7e6c
Update README, remove cli-core dependency, change version for now
mirdaki-ms Jan 15, 2020
8f23d83
Merge branch 'master' into ai-examples
Jan 19, 2020
277163f
Update Python and max version number to be accurate
mirdaki-ms Jan 22, 2020
c321740
Merge branch 'ai-examples' of https://github.com/mirdaki/azure-cli-ex…
mirdaki-ms Jan 22, 2020
8a08d57
Merge branch 'master' into ai-examples
Feb 5, 2020
253ba25
Update to accurate minimum CLI version
mirdaki-ms Feb 10, 2020
ede70af
Merge branch 'ai-examples' of https://github.com/mirdaki/azure-cli-ex…
mirdaki-ms Feb 10, 2020
ead3587
Merge branch 'master' into ai-examples
mirdaki Feb 10, 2020
53205d5
Update index with new hash and other metadata
mirdaki-ms Feb 11, 2020
f8f3a61
Merge branch 'ai-examples' of https://github.com/mirdaki/azure-cli-ex…
mirdaki-ms Feb 11, 2020
d9c3666
Switch to CLI download
mirdaki-ms Feb 24, 2020
0836330
Merge branch 'master' into ai-examples
Feb 25, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@

/src/ip-group/ @haroldrandom

/src/ai-examples/ @mirdaki

/src/notification-hub/ @fengzhou-msft

/src/connection-monitor-preview/ @haroldrandom
Expand Down
10 changes: 10 additions & 0 deletions src/ai-examples/HISTORY.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. :changelog:

Release History
===============

0.1.0
++++++
* Initial release.
* Add AI generated examples to the "--help" examples
* Check to see if the client can connect to the AI example service
6 changes: 6 additions & 0 deletions src/ai-examples/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Microsoft Azure CLI 'AI Examples' Extension
==========================================

Improve user experince by adding AI powered examples to command help content.
haroldrandom marked this conversation as resolved.
Show resolved Hide resolved

This extension changes the default examples provided when calling `-h` or `--help` on a command, such as `az vm create -h`, with ones selected by an AI powered service. The service provides examples based on Azure usage, internet sources, and other factors.
38 changes: 38 additions & 0 deletions src/ai-examples/azext_ai_examples/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# --------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# --------------------------------------------------------------------------------------------


from azure.cli.core import AzCommandsLoader

from azext_ai_examples._help import helps # pylint: disable=unused-import


def inject_functions_into_core():
# Replace the default examples from help calls
from azure.cli.core._help import AzCliHelp
from azext_ai_examples.custom import provide_examples
AzCliHelp.example_provider = provide_examples


class AiExamplesCommandsLoader(AzCommandsLoader):

def __init__(self, cli_ctx=None):
from azure.cli.core.commands import CliCommandType
ai_examples_custom = CliCommandType(
operations_tmpl='azext_ai_examples.custom#{}')
super(AiExamplesCommandsLoader, self).__init__(cli_ctx=cli_ctx,
custom_command_type=ai_examples_custom)
inject_functions_into_core()

def load_command_table(self, args):
from azext_ai_examples.commands import load_command_table
load_command_table(self, args)
return self.command_table

def load_arguments(self, command):
pass


COMMAND_LOADER_CLS = AiExamplesCommandsLoader
19 changes: 19 additions & 0 deletions src/ai-examples/azext_ai_examples/_help.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# coding=utf-8
# --------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# --------------------------------------------------------------------------------------------


from knack.help_files import helps # pylint: disable=unused-import


helps['ai-examples'] = """
type: group
short-summary: Add AI powered examples to help content.
"""

helps['ai-examples check-connection'] = """
type: command
short-summary: Check if the client can connect to the AI example service.
"""
4 changes: 4 additions & 0 deletions src/ai-examples/azext_ai_examples/azext_metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"azext.isPreview": true,
"azext.minCliCoreVersion": "2.0.81"
}
13 changes: 13 additions & 0 deletions src/ai-examples/azext_ai_examples/commands.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# --------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# --------------------------------------------------------------------------------------------


def load_command_table(self, _):

with self.command_group('ai-examples') as g:
g.custom_command('check-connection', 'check_connection_aladdin')

with self.command_group('ai-examples', is_preview=True):
pass
139 changes: 139 additions & 0 deletions src/ai-examples/azext_ai_examples/custom.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# --------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# --------------------------------------------------------------------------------------------


import json
import re
import requests
from pkg_resources import parse_version

from azure.cli.core import telemetry as telemetry_core
from azure.cli.core import __version__ as core_version
from azure.cli.core._help import HelpExample


# Commands
def check_connection_aladdin():
response = ping_aladdin_service()
if response.status_code == 200:
print('Connection was successful')
else:
print('Connection failed')


# Replacements for core functions
def provide_examples(help_file):
return replace_examples(help_file)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we provide two options for users to select?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is an experiment, we were planning on getting feedback for one scenario at a time. The append code is there, because that was the original plan, but we've decided to have just the Aladdin examples to better test them.



# Provide two options for changing the examples

# Replace built in examples wirh Aladdin ones
def replace_examples(help_file):
# Specify az to coerce the examples to be for the exact command
lookup_term = "az " + help_file.command
return get_generated_examples(lookup_term)


# Append Aladdin examples to the built in ones
def append_examples(help_file):
# Specify az to coerce the examples to be for the exact command
lookup_term = "az " + help_file.command
aladdin_examples = get_generated_examples(lookup_term)
return concat_unique_examples(help_file.examples, aladdin_examples)


# Support functions
def get_generated_examples(cli_term):
examples = []
response = call_aladdin_service(cli_term)

if response.status_code == 200:
for answer in json.loads(response.content):
examples.append(clean_from_http_answer(answer))

return examples


def concat_unique_examples(first_list, second_list):
for first_item in first_list:
for second_item in second_list:
if are_examples_equal(first_item, second_item):
second_list.remove(second_item)
return first_list + second_list


def are_examples_equal(first, second):
return clean_string(first.short_summary) == clean_string(second.short_summary) \
or clean_string(first.command) == clean_string(second.command)


def clean_string(text):
return text.strip()


def clean_from_http_answer(http_answer):
example = HelpExample()
example.short_summary = http_answer['title'].strip()
example.command = http_answer['snippet'].strip()
if example.short_summary.startswith("az "):
example.short_summary, example.command = example.command, example.short_summary
example.short_summary = example.short_summary.split('\r\n')[0]
elif '```azurecli\r\n' in example.command:
start_index = example.command.index('```azurecli\r\n') + len('```azurecli\r\n')
example.command = example.command[start_index:]
example.command = example.command.replace('```', '').replace(example.short_summary, '').strip()
example.command = re.sub(r'\[.*\]', '', example.command).strip()
# Add a '\n' to comply with the existing examples format
example.command = example.command + '\n'
return example


# HTTP calls
def ping_aladdin_service():
api_url = 'https://app.aladdin.microsoft.com/api/v1.0/monitor'
headers = {'Content-Type': 'application/json'}

response = requests.get(
api_url,
headers=headers)

return response


def call_aladdin_service(query):
client_request_id = ''
if telemetry_core._session.application: # pylint: disable=protected-access
client_request_id = telemetry_core._session.application.data['headers']['x-ms-client-request-id'] # pylint: disable=protected-access

session_id = telemetry_core._session._get_base_properties()['Reserved.SessionId'] # pylint: disable=protected-access
subscription_id = telemetry_core._get_azure_subscription_id() # pylint: disable=protected-access
client_request_id = client_request_id # pylint: disable=protected-access
installation_id = telemetry_core._get_installation_id() # pylint: disable=protected-access
version = str(parse_version(core_version))

context = {
"sessionId": session_id,
"subscriptionId": subscription_id,
"clientRequestId": client_request_id,
"installationId": installation_id,
"versionNumber": version
}

api_url = 'https://app.aladdin.microsoft.com/api/v1.0/examples'
headers = {'Content-Type': 'application/json'}

response = requests.get(
api_url,
params={
'query': query,
'clientType': 'AzureCli',
'context': json.dumps(context),
'commandOnly': True,
'numberOfExamples': 5
},
headers=headers)

return response
5 changes: 5 additions & 0 deletions src/ai-examples/azext_ai_examples/tests/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# -----------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# -----------------------------------------------------------------------------
5 changes: 5 additions & 0 deletions src/ai-examples/azext_ai_examples/tests/latest/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# -----------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# -----------------------------------------------------------------------------
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# --------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for license information.
# --------------------------------------------------------------------------------------------


import json
import unittest
import mock
import requests

from azure_devtools.scenario_tests import AllowLargeResponse
from azure.cli.testsdk import (ScenarioTest, ResourceGroupPreparer)
from azext_ai_examples.custom import (call_aladdin_service, ping_aladdin_service,
clean_from_http_answer, get_generated_examples)


def create_valid_http_response():
mock_response = requests.Response()
mock_response.status_code = 200
data = [{
'title': 'RunTestAutomation',
'snippet': 'az find'
}, {
'title': 'az test',
'snippet': 'The title'
}]
mock_response._content = json.dumps(data)
return mock_response


def create_empty_http_response():
mock_response = requests.Response()
mock_response.status_code = 200
data = []
mock_response._content = json.dumps(data)
return mock_response


def create_failed_http_response():
mock_response = requests.Response()
mock_response.status_code = 500
data = []
mock_response._content = json.dumps(data)
return mock_response


class AiExamplesCustomCommandTest(unittest.TestCase):

# Test the Aladdin check connection command
def test_ai_examples_ping_aladdin_service_success(self):
mock_response = create_empty_http_response()

with mock.patch('requests.get', return_value=(mock_response)):
response = ping_aladdin_service()

self.assertEqual(200, response.status_code)

def test_ai_examples_ping_aladdin_service_failed(self):
mock_response = create_failed_http_response()

with mock.patch('requests.get', return_value=(mock_response)):
response = ping_aladdin_service()

self.assertEqual(500, response.status_code)

# Test the Aladdin examples
def test_ai_examples_call_aladdin_service(self):
mock_response = create_valid_http_response()

with mock.patch('requests.get', return_value=(mock_response)):
response = call_aladdin_service('RunTestAutomation')
self.assertEqual(200, response.status_code)
self.assertEqual(2, len(json.loads(response.content)))

def test_ai_examples_example_clean_from_http_answer(self):
cleaned_responses = []
mock_response = create_valid_http_response()

for response in json.loads(mock_response.content):
cleaned_responses.append(clean_from_http_answer(response))

self.assertEqual('RunTestAutomation', cleaned_responses[0].short_summary)
self.assertEqual('az find\n', cleaned_responses[0].command)
self.assertEqual('The title', cleaned_responses[1].short_summary)
self.assertEqual('az test\n', cleaned_responses[1].command)

def test_ai_examples_get_generated_examples_full(self):
examples = []
mock_response = create_valid_http_response()

with mock.patch('requests.get', return_value=(mock_response)):
examples = get_generated_examples('RunTestAutomation')

self.assertEqual('RunTestAutomation', examples[0].short_summary)
self.assertEqual('az find\n', examples[0].command)
self.assertEqual('The title', examples[1].short_summary)
self.assertEqual('az test\n', examples[1].command)

def test_ai_examples_get_generated_examples_empty(self):
examples = []
mock_response = create_empty_http_response()

with mock.patch('requests.get', return_value=(mock_response)):
examples = get_generated_examples('RunTestAutomation')

self.assertEqual(0, len(examples))
2 changes: 2 additions & 0 deletions src/ai-examples/setup.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[bdist_wheel]
universal=1
Loading