Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Added "Set Entities" script for Azure Table storage #222

Merged

Conversation

Annelotte-Mons
Copy link
Contributor

Added new "AzTable-SetEntities" script.

This script will first connect to the specified Azure Storage Account / Table.
Then it will first delete all existing entities on the specified table.
Finally it will insert new entities based on a JSON configuration file.

An example of this configuration file looks like:

{
  "entities": [
    {
      "PartitionKey": "RDS-Fileshare",
      "Active": "Yes",
      "Domain": "Truck",
      "Service": "TruckOrder-UpdateConfirmation",
      "Action": "Process",
      "Version": "v1",
      "ContentType": "text/xml",
      "MinimumFileAgeInSeconds": "60",
      "Recursive": "No",
      "Sender": "Truckers",
      "ServerDirectory": "/Truckers/orders/in",
      "ServerId": "RDS-Fileshare"
    },
    {
      "PartitionKey": "RDS-Fileshare",
      "Active": "Yes",
      "Domain": "Messaging",
      "Service": "STM",
      "Action": "RoutingSlipIngest",
      "Version": "v2",
      "ContentType": "text/plain",
      "MinimumFileAgeInSeconds": "60",
      "Recursive": "No",
      "Sender": "Truckers",
      "ServerDirectory": "/Truckers/STM/in",
      "ServerId": "RDS-Fileshare"
    }
  ]
}``

The fields PartitionKey & RowKey are optional, if not provided a random GUID will be used.
Other fields can be any key-value you'd like, 

Script uses 'AzTable' module

@netlify
Copy link

netlify bot commented Sep 14, 2021

Deploy Preview for arcus-scripting ready!

Name Link
🔨 Latest commit 01058af
🔍 Latest deploy log https://app.netlify.com/sites/arcus-scripting/deploys/64770d34e8d64600087516b6
😎 Deploy Preview https://deploy-preview-222--arcus-scripting.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@Annelotte-Mons
Copy link
Contributor Author

Todo: Add docs? If so, what folder/files exactly?

@mbraekman mbraekman linked an issue Sep 14, 2021 that may be closed by this pull request
@mbraekman
Copy link
Contributor

Haven't had a thorough look yet, but can you:

  • make sure to change the name of the command to use the approved verbs -> see docs
    This should align with the existing commands, for example: Set-AzStorageTableEntities instead of AzTable-SetEntities.
  • add documentation, see the contribution guide on how/where.
  • include some unit/integration tests, as per the existing test-projects.

If anything is unclear, let us know and we'll point you in the right direction!

By default this script is going to delete all entities and add those configured in the JSON-file. Would it be an idea to use a switch-param to indicate whether the existing entries should be removed? This way this can be created to simply add new entries as well?

@stijnmoreels stijnmoreels changed the title Added "Set Entities" script for Azure Table storage feat: Added "Set Entities" script for Azure Table storage Sep 16, 2021
@stijnmoreels
Copy link
Member

Great stuff! Thanks for your PR! 🎉
Like Maxim said: maybe a quick rundown of our contribution guide will already help you a lot. I'll also wait for my review. Please contact us if you're having trouble with something.

Thanks again for your enthusiasm!

@stijnmoreels stijnmoreels marked this pull request as draft September 23, 2021 05:41
@stijnmoreels stijnmoreels changed the title feat: Added "Set Entities" script for Azure Table storage feat: Added "Set Entities" script for Azure Table storage (🚧) Sep 23, 2021
@pim-simons pim-simons self-assigned this May 25, 2023
@stijnmoreels stijnmoreels removed the request for review from tomkerkhove May 26, 2023 04:23
Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for these changes already! 🎉🎉 Looks great. Super job on providing the necessary logs and error handling for missing resources or values. Will bring great user experience to our library. Much appreciated!

Added some suggestions, but the biggest missing link here are tests (both unit and integration tests).

Unit tests could check for regular success/failure paths, while the integration tests can actually run this functionality against a real storage account. The tests would have to setup/teardown entities before/after running the script to make it stable.
Are you up for this, or should we help out?
I'll first let my buddy @pim-simons have a look at this too.

@stijnmoreels stijnmoreels marked this pull request as ready for review May 26, 2023 04:46
@stijnmoreels stijnmoreels requested a review from pim-simons as a code owner May 26, 2023 04:46
@pim-simons
Copy link
Contributor

pim-simons commented May 26, 2023

Unit and integration tests have been added, but annoyingly I cannot figure out how to run the DevOps pipeline for this PR 🙈
@stijnmoreels any idea on how to trigger this?

@stijnmoreels
Copy link
Member

/azp run CI - Arcus.Scripting

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@stijnmoreels
Copy link
Member

Unit and integration tests have been added, but annoyingly I cannot figure out how to run the DevOps pipeline for this PR 🙈 @stijnmoreels any idea on how to trigger this?

Triggered it: seems like the built-in PowerShell JSON deserialization expects (always?) a JSON object, while we're testing with arrays.

@pim-simons
Copy link
Contributor

Unit and integration tests have been added, but annoyingly I cannot figure out how to run the DevOps pipeline for this PR 🙈 @stijnmoreels any idea on how to trigger this?

Triggered it: seems like the built-in PowerShell JSON deserialization expects (always?) a JSON object, while we're testing with arrays.

I had the same issue on my machine, had to update the Powershell version to 7.3.4 and the issue was fixed.
It seems the DevOps agents (windows-latest and ubuntu-latest) are running Powershell version 7.2.11, I will look into how to update to the latest version.

@pim-simons
Copy link
Contributor

/azp run CI - Arcus.Scripting

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@pim-simons
Copy link
Contributor

/azp run CI - Arcus.Scripting

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@pim-simons
Copy link
Contributor

/azp run CI - Arcus.Scripting

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@stijnmoreels
Copy link
Member

Strange that this pipeline is not triggered automatically. Could it be that the main branch on the PR-branch is still master? 🤔

@pim-simons
Copy link
Contributor

Strange that this pipeline is not triggered automatically. Could it be that the main branch on the PR-branch is still master? 🤔

Not really sure, but at least the tests are now all working correctly and finished successfully 😉

Copy link
Member

@stijnmoreels stijnmoreels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super. This looks ready!
Thx for both validation+tests. ❤️

@pim-simons pim-simons merged commit 1cb37f2 into arcus-azure:main May 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide script to set all entities in a specified Azure Table Storage
4 participants