Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smoke test for Python #6412

Merged
merged 26 commits into from
Jul 25, 2019
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
10f4e09
Smoke Test Sample for Track 2 libraries
JonathanCrd Jul 17, 2019
d6e763c
simpleQuery method added
JonathanCrd Jul 17, 2019
45e9699
Method's names updated
JonathanCrd Jul 17, 2019
1b5ef9a
Create README.md
JonathanCrd Jul 19, 2019
aee43e1
Update README.md
JonathanCrd Jul 19, 2019
97e4f60
Commented lines deleted
JonathanCrd Jul 19, 2019
c587a82
README.md moved to correct folder
JonathanCrd Jul 19, 2019
7909abd
Create requirements.txt
JonathanCrd Jul 22, 2019
1d393e6
Update README.md
JonathanCrd Jul 22, 2019
81d389e
Update README.md
JonathanCrd Jul 23, 2019
ab8fa9d
Imports changed
JonathanCrd Jul 23, 2019
e7f7346
Use of literals instead of append
JonathanCrd Jul 23, 2019
7066da0
Database Name variable to class level.
JonathanCrd Jul 23, 2019
81adc4c
Use of Pythonic with statements
JonathanCrd Jul 23, 2019
4b79c6a
Update requirements.txt
JonathanCrd Jul 23, 2019
a3df531
Revert "Update requirements.txt"
JonathanCrd Jul 23, 2019
27b2a2d
Revert "Use of Pythonic with statements"
JonathanCrd Jul 23, 2019
4c2d73d
Revert "Revert "Use of Pythonic with statements""
JonathanCrd Jul 23, 2019
15e025e
requiriments.txt encoded as a txt file
JonathanCrd Jul 23, 2019
3d9ce90
requirements.txt as text file
JonathanCrd Jul 23, 2019
22927ad
Misspelling in "Key concepts"
JonathanCrd Jul 23, 2019
17a53cb
Update .docsettings.yml to match the tittle of Smoke Test
JonathanCrd Jul 23, 2019
35c6223
Went trought Suyog comments
JonathanCrd Jul 24, 2019
ada5a74
Revert "Went trought Suyog comments"
JonathanCrd Jul 24, 2019
ff419ff
Gone trought Suyog comments
JonathanCrd Jul 24, 2019
e2190c9
use of snake case in file names
JonathanCrd Jul 24, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 116 additions & 0 deletions samples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Azure Smoke Tests for Python
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
This sample code is a smoke test to ensure that Azure Preview for Python work while loaded into the same process by performing 2 or more actions with them.

Libraries tested:
* keyvault-secrets
* identity
* storage-blob
* event-hubs
* cosmos

## Getting started
### Setup Azure resources
For this sample, it is necessary to create/have the following resources in the [Azure Portal](https://portal.azure.com/):
* **App registration**: Register a new app or use an existing one.
* Under _Certificates & secrets_ create a new **client secret** and store the value in a safe place.
* **Key Vaults**: Create a new Key Vault resource or use an existing one.
* Under _Access policies_, add the app registrated in the previous step.
* **Storage acounts**: Create a container in a new or existing storage account. The container in this sample is named "mycontainer", if you want to use other name you can change the value in `BlobStorage.ts` file:
`const containerName = "mycontainer";`
* **Event Hubs**: Create an event hub inside a new or existing Event Hubs Namespace. The container in this sample is named "myeventhub", if you want to use other name you can change the value in `EventHubsTest.ts` file: `let eventHubName = "myeventhub";`
* **Azure Cosmos DB**: Create a new account or use an existing one.

### Azure credentials
The following environment variables are needed:
* From **App Registration**, in the _Overview_ section:
* AZURE_TENANT_ID: The directory tentant ID.
* AZURE_CLIENT_ID: The application ID.
* AZURE_CLIENT_SECRET: The client secret stored previusly when creating the _client secret_.

* From **Key Vault**, in the _Overview_ section:
* AZURE_PROJECT_URL: The DNS Name

* From **Event Hubs**, in _Shared access policies_ section:
* EVENT_HUBS_CONNECTION_STRING: Connection string from a policy

* From **Storage Account**, in the _Access Keys_ section:
* STORAGE_CONNECTION_STRING : A connection strings.

* From **Azure Cosmos DB**, in the _Keys_ section, select the _Read-Write Keys_ tab:
* COSMOS_ENDPOINT: URI.
* COSMOS_KEY: Primary or secondary key.

```
//Bash code to create the environment variables
export AZURE_CLIENT_ID=""
export AZURE_CLIENT_SECRET=""
export AZURE_TENANT_ID=""
export EVENT_HUBS_CONNECTION_STRING=""
export AZURE_PROJECT_URL=""
export STORAGE_CONNECTION_STRING=""
export COSMOS_ENDPOINT=""
export COSMOS_KEY=""
```

### Running the console app
[Python](https://nodejs.org/en/) version 3.7.4 was used to run this sample.

In the \SmokeTest\ directory, run Program.py
```
python .\Program.py
```

## Key Concepts


## Examples
All the classes in this sample has a `Run()` method as entry point, and do not depend on each other.

It is possible to run them individually:
```python
from KeyVaultSecrets import KeyVault

KeyVault().Run()
```

They can be included in other projects by moving the class in it:
```python
from KeyVaultSecrets import KeyVault

...

def myTests():
console.log("Smoke Test imported from other project")
KeyVault().Run()

myTests()
otherFunction()
...
```

The classes can be used as base code and be changed to satisfied specific needs. For example, the method `EventHub().SendAndReceiveEvents()` can be change to only send events from an array given from a parameter:
```python
def SendAndReceiveEvents(self, partitionID, events):
producer = self.client.create_producer(partition_id=partitionID)
producer.send(events)
producer.close()
```

**Note:** The methods in the classes are not necessary independent on each other, and the order matters. For example, in order to run `BlobStorage().DeleteBlob();`, the method `BlobStorage().UploadBLob();` must be run before, since in the other way it will fail because there is not going to be a blob to delete.

## Troubleshooting

### Authentification
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
Be sure to set the environment variables and credentials required before running the sample.

## Next steps
Check the [Azure SDK for Python Repository](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk) for more samples inside the sdk folder.

## Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

If you'd like to contribute to this library, please read the contributing guide to learn more about how to build and test the code.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.
97 changes: 97 additions & 0 deletions samples/smoketest/CosmosDB.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import azure.cosmos.cosmos_client as cosmos_client
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
import azure.cosmos.errors as errors
from azure.cosmos.partition_key import PartitionKey
import os

class CosmosDB:
def __init__(self):
URL = os.environ["COSMOS_ENDPOINT"]
KEY = os.environ["COSMOS_KEY"]
self.client = cosmos_client.CosmosClient(URL,{'masterKey': KEY})

def CreateDatabase(self):
dbName="pySolarSystem"
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
print("Creating '{0}' database...".format(dbName))
return self.client.create_database(dbName)

def CreateContainer(self, db):
collectionName = "Planets"
print("Creating '{0}' collection...".format(collectionName))
partition_key = PartitionKey(path='/id', kind='Hash')
return db.create_container(id="Planets", partition_key=partition_key)

def CreateDocuments(self, container):
planets = []
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved

# Cosmos will look for an 'id' field in the items, if the 'id' is not specify Cosmos is going to assing a random key.
planets.append({
'id' : "Earth",
'HasRings' : False,
'Radius' : 3959,
'Moons' :
[
{
'Name' : "Moon"
}
]
})

planets.append({
"id" : "Mars",
"HasRings" : False,
"Radius" : 2106,
"Moons" :
[
{
"Name" : "Phobos"
},
{
"Name" : "Deimos"
}
]
})

print("Inserting items in the collection...")
for planet in planets:
container.create_item(planet)
print("\t'{0}' created".format(planet['id']))
print("\tdone")

def SimpleQuery(self, container):
print("Quering the container...")
items = list(container.query_items(
query="SELECT c.id FROM c",
enable_cross_partition_query = True
))
print("\tdone: {0}".format(items))

def DeleteDatabase(self):
print("Cleaning up the resource...")
self.client.delete_database("pySolarSystem")
print("\tdone")

def Run(self):
print()
print("------------------------")
print("Cosmos DB")
print("------------------------")
print("1) Create a Database")
print("2) Create a Container in the database")
print("3) Insert Documents (items) into the Container")
print("4) Delete Database (Clean up the resource)")
print()

# Ensure that the database does not exists
try:
self.DeleteDatabase()
except:
pass

try:
db = self.CreateDatabase()
container = self.CreateContainer(db=db)
self.CreateDocuments(container=container)
self.SimpleQuery(container=container)
finally:
# if something goes wrong, the resource should be cleaned anyway
self.DeleteDatabase()
52 changes: 52 additions & 0 deletions samples/smoketest/EventHubs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
from azure.eventhub import EventHubClient, EventData, EventPosition
from datetime import datetime
import os

class EventHub:
def __init__(self):
# This test requires a previusly created Event Hub.
# In this example the name is "myeventhub", but it could be change below
connectionString = os.environ["EVENT_HUBS_CONNECTION_STRING"]
eventHubName = 'myeventhub'
self.client = EventHubClient.from_connection_string(connectionString, eventHubName)

def GetPartitionIds(self):
print("Getting partitions id...")
partition_ids = self.client.get_partition_ids()
print("\tdone")
return partition_ids

def SendAndReceiveEvents(self, partitionID):
consumer = self.client.create_consumer(consumer_group="$default", partition_id=partitionID, event_position=EventPosition(datetime.utcnow()))
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved

print("Sending events...")
producer = self.client.create_producer(partition_id=partitionID)
event_list = [EventData(b"Test Event 1 in Python"),EventData(b"Test Event 2 in Python"),EventData(b"Test Event 3 in Python")]
producer.send(event_list)
producer.close()
print("\tdone")

print("Receiving events...")
received = consumer.receive(max_batch_size=len(event_list), timeout=2)
for event_data in received:
print("\tEvent Received: " + event_data.body_as_str())
consumer.close()
print("\tdone")

if(len(received) != len(event_list)):
raise Exception("Error, expecting {0} events, but {1} were received.".format(str(len(event_list)),str(len(received))))

def Run(self):
print()
print("------------------------")
print("Event Hubs")
print("------------------------")
print("1) Get partition ID")
print("2) Send Events")
print("3) Consume Events")
print()

partitionID = self.GetPartitionIds()
#In this sample the same partition id is going to be used for the producer and consumer,
#It is the first one, but it could be any (is not relevant as long as it is the same in both producer and consumer)
self.SendAndReceiveEvents(partitionID[0])
43 changes: 43 additions & 0 deletions samples/smoketest/KeyVaultSecrets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
import os

class KeyVault:
def __init__(self):
# DefaultAzureCredential() expects the following environment variables:
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
# * AZURE_CLIENT_ID
# * AZURE_CLIENT_SECRET
# * AZURE_TENANT_ID
credential = DefaultAzureCredential()
self.secret_client = SecretClient(vault_url=os.environ["AZURE_PROJECT_URL"], credential=credential)

def SetSecret(self):
print("Setting a secret...")
self.secret_client.set_secret("secret-name", "secret-value")
print("\tdone")

def GetSecret(self):
print("Getting a secret...")
secret = self.secret_client.get_secret("secret-name")
print("\tdone: " + secret.name)

def DeleteSecret(self):
print("Deleting a secret...")
deleted_secret = self.secret_client.delete_secret("secret-name")
print("\tdone: " + deleted_secret.name)

def Run(self):
print()
print("------------------------")
print("Key Vault - Secrets\nIdentity - Credential")
print("------------------------")
print("1) Set a secret")
print("2) Get that secret")
print("3) Delete that secret (Clean up the resource)")
print()

try:
self.SetSecret()
self.GetSecret()
finally:
self.DeleteSecret()
13 changes: 13 additions & 0 deletions samples/smoketest/Program.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from KeyVaultSecrets import KeyVault
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
from StorageBlobs import StorageBlob
from EventHubs import EventHub
from CosmosDB import CosmosDB

print("==========================================")
print(" AZURE TRACK 2 SDKs SMOKE TEST")
print("==========================================")

KeyVault().Run()
StorageBlob().Run()
EventHub().Run()
CosmosDB().Run()
47 changes: 47 additions & 0 deletions samples/smoketest/StorageBlobs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
from azure.storage.blob import BlobClient
import os

class StorageBlob:
def __init__(self):
connectionString = os.environ["STORAGE_CONNECTION_STRING"]
self.blob = BlobClient.from_connection_string(connectionString, container="mycontainer", blob="pyTestBlob.txt")

def UploadBLob(self):
print("uploading blob...")
self.data = "This is a sample data for Python Test"
self.blob.upload_blob(self.data)
print("\tdone")

def DownloadBlob(self):
print("downloading blob...")
with open("./downloadedBlob.txt", "wb+") as my_blob:
my_blob.writelines(self.blob.download_blob())

print("\tdone")

def DeleteBlob(self):
print("Cleaning up the resource...")
self.blob.delete_blob()
print("\tdone")

def Run(self):
print()
print("------------------------")
print("Storage - Blob")
print("------------------------")
print("1) Upload a Blob")
print("2) Download a Blob")
print("3) Delete that Blob (Clean up the resource)")
print()

#Ensure that the blob does not exists before the tests
try:
self.DeleteBlob()
except:
JonathanCrd marked this conversation as resolved.
Show resolved Hide resolved
pass

try:
self.UploadBLob()
self.DownloadBlob()
finally:
self.DeleteBlob()