New utility /Action=SaveSettings and /SettingsFile argument for simpl…

…e reuse of settings

Updated Nuget package versions

AssemblyVersioning.cs

@@ -27,5 +27,5 @@
 //   2) Update the installer version to match the AssemblyVersion below.
 //   These can be found in SqlBuildManager.Setup -> Organize Your Setup -> General Information
 // ** Also, don't forget to update the change_notes.xml and .html files!
-[assembly: AssemblyVersion("10.4.2.*")]
-[assembly: AssemblyFileVersion("10.4.2")]
+[assembly: AssemblyVersion("10.4.3.*")]
+[assembly: AssemblyFileVersion("10.4.3")]

Docs/AzureBatch.md

@@ -95,18 +95,23 @@ The following command line assumes that the Batch and Storage settings (except f
 
 ----
 ## Azure Batch - Pre-Stage Batch nodes (/Action=BatchPreStage)
+_Note:_ You can also leverage the [SettingsFile](commandline.md#save-settings-actionsavesettings) option to reuse most of the arguments
 - `/BatchNodeCount="##"` - Number of nodes to provision to run the batch job  (default is 10)
 - `/BatchVmSize="<size>"` - Size key for VM size required (see https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general) [can also be set via BatchVmSize app settings key]
 - `/BatchAccountName="<batch acct name>"` - String name of the Azure Batch account  [can also be set via BatchAccountName app settings key]
 - `/BatchAccountKey="<batch acct key>"` - Account Key for the Azure Batch account [can also be set via BatchAccountKey app settings key]
 - `/BatchAccountUrl="<batch acct url>"` - URL for the Azure Batch account [can also be set via BatchAccountUrl app settings key]
 - `/PollBatchPoolStatus=(true|false)` - Whether or not you want to get updated status (true, default) or fire and forget (false)
+
 ----
 ## Azure Batch Execution (/Action=Batch)
 
-In addition to the [authentication](commandline.md#General-Authentication-settings) and [runtime](commandline.md#General-Runtime-settings) arguments above, these are specifically needed for Azure Batch executions
+In addition to the [authentication](commandline.md#General-Authentication-settings) and [runtime](commandline.md#General-Runtime-settings) arguments above, these are specifically needed for Azure Batch executions.
+\
+_Note:_ 
+1. You can also leverage the [SettingsFile](commandline.md#save-settings-actionsavesettings) option to reuse most of the arguments
+2. either /PlatinumDacpac _or_ /PackageName are required. If both are given, then /PackageName will be used.
 
-*Note:* either /PlatinumDacpac _or_ /PackageName are required. If both are given, then /PackageName will be used.
 - `/PlatinumDacpac="<filename>"` - Name of the dacpac containing the platinum schema
 - `/PackageName="<filename>"` - Name of the .sbm or .sbx file to execute save execution logs 
 - `/BatchAccountName="<batch acct name>"` - String name of the Azure Batch account  [can also be set via BatchAccountName app settings key]
@@ -125,6 +130,8 @@ If you don't run the `/Action=BatchPreStage`  and `Action=BatchCleanup` command
 
 ----
 ## Azure Batch Clean Up (delete) nodes (/Action=BatchCleanUp)
+
+_Note:_ You can also leverage the [SettingsFile](commandline.md#save-settings-actionsavesettings) option to reuse most of the arguments
 - `/BatchAccountName="<batch acct name>"` - String name of the Azure Batch account  [can also be set via BatchAccountName app settings key]
 - `/BatchAccountKey="<batch acct key>"` - Account Key for the Azure Batch account [can also be set via BatchAccountKey app settings key]
 - `/BatchAccountUrl="<batch acct url>"` - URL for the Azure Batch account [can also be set via BatchAccountUrl app settings key]

Docs/change_notes.md

@@ -1,7 +1,10 @@
-
+
 # SQL Build Manager Change Notes
       
 					
+### Version 10.4.3
+- *ADDED:* New utility /Action=SaveSettings and /SettingsFile argument for simple reuse of settings.
+
 ### Version 10.4.2
 - *ADDED:* Added Batch pool handling action 'BatchPreStage' so that Pools can be created in advance of needing the compute. Avoids the waiting "cold start" of a batch job
 - *ADDED:* Added Batch pool handling action  'BatchCleanup' so that Pools can be deleted separately from a build run (useful if you need to re-run jobs)

Docs/commandline.md

@@ -31,18 +31,21 @@ The basics of running the command line is the /Action argument. This is required
 ##### Utility actions 
 - `BatchPreStage` - Pre-stage Azure Batch VM nodes
 - `BatchCleanUp` - Deletes Azure Batch VM nodes
+- `SaveSettings` - Creates a Json file that will save your userName and password as well as your Batch settings so that you can reference them later.
 - `Package` - Create an SBM package from an SBX configuration file and scripts
 - `PolicyCheck` - Performs a script policy check on the specified SBM package
 -  `GetHash` - Calculates the SHA-1 hash fingerprint value for the SBM package (scripts + run order)
 - `CreateBackout` - Generates a backout package (reversing stored procedure and scripted object changes)
 - `GetDifference` - Determines the difference between SQL Build run histories for two databases. Calculate and list out packages that need to be run between /Database and /GoldDatabase
-- `Syncronize` - Performs a database syncronization between between /Database and /GoldDatabase
+- `Synchronize` - Performs a database synchronization between between /Database and /GoldDatabase
 - `ScriptExtract` - Extract a SBM package from a source `/PlatinumDacPac`
 
+
 #### General Authentication settings 
 
 Used to provide authentication to databases during a build or for script extraction/ DACPAC creation\
-Applies to: `/Action={Build|Threaded|Batch|Remote|GetDifference|Syncronize`
+Applies to: `/Action={Build|Threaded|Batch|Remote|GetDifference|Synchronize`\
+_Note:_ Is using username/password authentication, you can also leverage the [SettingsFile](#save-settings-actionsavesettings) option
 - `/AuthType="<type>"` - Values: `AzureADIntegrated`, `AzureADPassword`, `Password` (default if Username/Password set), `Windows` (default if no Username/Password set), 
 - `/UserName="<username>"` - The username to authenticate against the database if not using integrate auth (required when `RemoteServers="azure"`)
 - `/Password="<password>"` - The password to authenticate against the database if not using integrate auth (required when `/RemoteServers="azure`")
@@ -100,6 +103,18 @@ Applies to: `/Action={Threaded|Batch|Remote}`
 - `/PlatinumDbSource="<database name>"` - Instead of a formally built Platinum Dacpac, target this database as having the desired state schema
 - `/PlatinumServerSource="<server name>"` - Instead of a formally built Platinum Dacpac, target a database on this server as having the desired state schema
 
+#### Save Settings (/Action=SaveSettings)
+This utility action will save a reusable Json file to make running the command line easier (especially for Batch processing). To save the settings, create the command line as you normally would, and set the `/Action=SaveSettings` and add a `/SettingsFile="<file path>`" argument. 
+
+The next time you run a build action, use the `/SettingsFile="<file path>` in place of the arguments below.
+
+- Authentication: `/UserName`, `/Password`
+- Azure Batch: `BatchNodeCount`, `/BatchAccountName`, `/BatchAccountKey`, `/BatchAccountUrl`, `/StorageAccountName`, `/StorageAccountKey`, `/BatchVmSize`, `/DeleteBatchPool`, `/DeleteBatchJob`, `/PollBatchPoolStatus`
+- Run time settings: `/RootLoggingPath`, `/LogAsText`
+
+_Note:_ 
+1. the values for `/UserName`, `/Password`, `/BatchAccountKey` and `/StorageAccountKey` will be encrypted. 
+2. If there are duplicate values in the `/SettingsFile` and the command line, the command line argument will take precedence. 
 
 #### Script Extraction from Dacpac (/Action=ScriptExtract)
 - `/PlatinumDacpac="<filename>"` - Name of the dacpac containing the platinum schema

SqlBuildManager.Console/App.config

@@ -8,17 +8,7 @@
     <add key="DynamicTcpEndpointTemplate" value="net.tcp://[[ServerName]]:8676/SqlBuildManager.Services/BuildService" />
     <add key="DynamicAzureHttpEndpointTemplate" value="http://[[ServerName]]/BuildService.svc" />
     <add key="AzureDnsName" value="sqlbuildmanager.cloudapp.net" />
-
-    <add key="BatchAccountName" value="sqlbuildmanager" />
-    <add key="BatchAccountKey" value="" />
-    <add key="BatchAccountUrl" value="https://sqlbuildmanager.eastus.batch.azure.com" />
-    <add key="BatchVmSize" value="Standard_DS1_v2" /> <!-- Note: list of sizes availabe here: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general -->
-    <add key="StorageAccountName" value="sqlbuildmanager" />
-    <add key="StorageAccountKey" value="" />
-
-
-
-
+
   </appSettings>
 	<log4net debug="false">
 		<appender name="EventLogAppender" type="log4net.Appender.EventLogAppender">

SqlBuildManager.Console/ConsoleHelp2.txt

@@ -8,7 +8,7 @@ Action value options (/Action=<value>)
     Threaded                 For updating multiple databases simultaneously from the current machine
     Remote                   For updating multiple databases simultaneously using remote execution servers to spread the processing 
     Batch                    For updating multiple databases simultaneously using Azure batch services
-	BatchPreStage            Pre-stage the Azure Batch VM nodes
+    BatchPreStage            Pre-stage the Azure Batch VM nodes
     Package                  Creates an SBM package from an SBX configuraiton file and scripts
     PolicyCheck              Performs a script policy check on the specified SBM package
     GetHash                  Calculates the SHA-1 hash fingerprint value for the SBM package (scripts + run order)
@@ -43,7 +43,7 @@ Azure Batch Execution (/Action=Batch)
     /PackageName="<filename>"                 Name of the .sbm or .sbx file to execute
     /RootLoggingPath="<directory>"            Directory to save execution logs 
     /DeleteBatchPool=(true|false)             Whether or not to delete the batch pool servers after an execution (default is false)
-	/DeleteBatchJob=(true|false)              Whether or not to delete the batch job after an execution (default is true)
+    /DeleteBatchJob=(true|false)              Whether or not to delete the batch job after an execution (default is true)
     /BatchNodeCount="##"                      Number of nodes to provision to run the batch job  (default is 10)
     /BatchVmSize="<size>"                     Size key for VM size required (see https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general) [can also be set via BatchVmSize app settings key]
     /BatchAccountName="<batch acct name>"     String name of the Azure Batch account  [can also be set via BatchAccountName app settings key]

SqlBuildManager.Console/Program.cs

@@ -16,6 +16,9 @@
 using Microsoft.WindowsAzure.Storage.Blob;
 using Microsoft.WindowsAzure.Storage.RetryPolicies;
 using System.Threading.Tasks;
+using System.Runtime.Serialization.Json;
+using System.Web.Script.Serialization;
+using Newtonsoft.Json;
 
 namespace SqlBuildManager.Console
 {
@@ -36,9 +39,11 @@ static async Task Main(string[] args)
             try
             {
                 string joinedArgs = string.Join(",", args).ToLower();
+                string[] helpRequest = new string[] {"/?", "-?", "--?", "-h", "-help", "/help", "--help", "--h", "/h" };
+
                 var cmdLine = CommandLine.ParseCommandLineArg(args);
 
-                if (args.Length == 0 || joinedArgs.Contains("/?") || joinedArgs.Contains("/help"))
+                if (args.Length == 0 || helpRequest.Any(h => joinedArgs.Contains(h))) 
                 {
                     log.Info(Properties.Resources.ConsoleHelp2);
                     Environment.Exit(0);
@@ -75,8 +80,8 @@ static async Task Main(string[] args)
                     case CommandLineArgs.ActionType.ScriptExtract:
                         ScriptExtraction(cmdLine);
                         break;
-                    case CommandLineArgs.ActionType.Encrypt:
-                        EncryptCreds(cmdLine);
+                    case CommandLineArgs.ActionType.SaveSettings:
+                        SaveAndEncryptSettings(cmdLine);
                         break;
                     case CommandLineArgs.ActionType.Batch:
                         retVal = await RunBatchExecution(cmdLine, start);
@@ -157,33 +162,44 @@ private static async Task<int> RunBatchExecution(CommandLineArgs cmdLine, DateTi
             return retVal;
         }
 
-        private static void EncryptCreds(CommandLineArgs cmdLine)
+        private static void SaveAndEncryptSettings(CommandLineArgs cmdLine)
         {
-            string username = string.Empty;
-            string password = string.Empty;
 
-            if (!string.IsNullOrWhiteSpace(cmdLine.AuthenticationArgs.UserName))
+            if(string.IsNullOrWhiteSpace(cmdLine.SettingsFile))
             {
-                //System.Console.WriteLine(string.Format("<UserName>{0}</UserName>",
-                username = Cryptography.EncryptText(cmdLine.AuthenticationArgs.UserName, ConnectionHelper.ConnectCryptoKey);
+                log.Error("When /Action=SaveSettings is specified the /SettingsFile argument is also required");
             }
-            if (!string.IsNullOrWhiteSpace(cmdLine.AuthenticationArgs.Password))
-            {
-                //System.Console.WriteLine(string.Format("<Password>{0}</Password>", 
 
-                password = Cryptography.EncryptText(cmdLine.AuthenticationArgs.Password, ConnectionHelper.ConnectCryptoKey);
-            }
+            cmdLine = Cryptography.EncryptSensitiveFields(cmdLine);
+            var mystuff = JsonConvert.SerializeObject(cmdLine, Formatting.Indented, new JsonSerializerSettings
+            {
+                NullValueHandling = NullValueHandling.Ignore
+            });
 
-            if (!string.IsNullOrWhiteSpace(cmdLine.Server))
+            try
             {
-                System.Console.WriteLine(
-                    string.Format("<ServerConfiguration Name=\"{0}\" LastAccessed=\"2000-01-01T00:00:00.0000-04:00\">\r\n    <UserName>{1}</UserName>\r\n    <Password>{2}</Password>\r\n</ServerConfiguration>", cmdLine.Server, username, password));
+                string write = "y";
+                if(File.Exists(cmdLine.SettingsFile))
+                {
+                    System.Console.WriteLine($"The settings file '{cmdLine.SettingsFile}' already exists. Overwrite (Y/N)?");
+                    write = System.Console.ReadLine();
+                }
+
+                if (write.ToLower() == "y")
+                {
+                    File.WriteAllText(cmdLine.SettingsFile, mystuff);
+                    log.Info($"Settings file saved to '{cmdLine.SettingsFile}'");
+                }
+                else
+                {
+                    log.Info("Settings file not saved");
+                }
             }
-            else
+            catch (Exception exe)
             {
-                System.Console.WriteLine(string.Format("<UserName>{0}</UserName>", username));
-                System.Console.WriteLine(string.Format("<Password>{0}</Password>", password));
+                log.Error($"Unable to save settings file.\r\n{exe.ToString()}");
             }
+
         }
 
         private static void ScriptExtraction(CommandLineArgs cmdLine)

SqlBuildManager.Console/SqlBuildManager.Console.csproj

@@ -79,7 +79,7 @@
       <HintPath>..\packages\WindowsAzure.Storage.9.3.3\lib\net45\Microsoft.WindowsAzure.Storage.dll</HintPath>
     </Reference>
     <Reference Include="Newtonsoft.Json, Version=12.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
-      <HintPath>..\packages\Newtonsoft.Json.12.0.2-beta1\lib\net45\Newtonsoft.Json.dll</HintPath>
+      <HintPath>..\packages\Newtonsoft.Json.12.0.2\lib\net45\Newtonsoft.Json.dll</HintPath>
     </Reference>
     <Reference Include="System" />
     <Reference Include="System.Core">
@@ -105,6 +105,7 @@
     </Reference>
     <Reference Include="System.ServiceModel" />
     <Reference Include="System.Web" />
+    <Reference Include="System.Web.Extensions" />
     <Reference Include="System.Xml" />
   </ItemGroup>
   <ItemGroup>
@@ -197,7 +198,6 @@
       <Link>Microsoft_SqlDB_DAC\%(RecursiveDir)%(FileName)%(Extension)</Link>
       <CopyToOutputDirectory>Always</CopyToOutputDirectory>
     </Content>
-    <None Include="AzureBatch.md" />
     <None Include="Dockerfile" />
     <None Include="packages.config">
       <SubType>Designer</SubType>

SqlBuildManager.Console/Validation.cs

@@ -41,42 +41,6 @@ public static int ValidateUserNameAndPassword(ref CommandLineArgs cmdLine, out s
                 }
             }
 
-            if(cmdLine.AuthenticationArgs.SavedCreds)
-            {
-                if(string.IsNullOrWhiteSpace(cmdLine.Server))
-                {
-                    error = "The /Server argument is required when /SavedCreds is used";
-                    errorMessages = new string[] { error, "Returning error code: " + (int)ExecutionReturn.FinishingWithErrors};
-                    log.Error(error);
-                    return (int)ExecutionReturn.FinishingWithErrors;
-                }
-
-                ServerConnectConfig.ServerConfigurationDataTable tmpTbl;
-                bool found = false;
-                UtilityHelper.GetRecentServers(out tmpTbl);
-                if(tmpTbl != null)
-                {
-                    string server = cmdLine.Server.Trim().ToLowerInvariant();
-                    var row = tmpTbl.Where(r => r.Name.Trim().ToLowerInvariant() == server);
-                    if (row.Any())
-                    {
-                        cmdLine.AuthenticationArgs.UserName = Cryptography.DecryptText(row.First().UserName, ConnectionHelper.ConnectCryptoKey);
-                        cmdLine.AuthenticationArgs.Password = Cryptography.DecryptText(row.First().Password, ConnectionHelper.ConnectCryptoKey);
-                        found = true;
-                        log.InfoFormat("Saved credentials found. Server={2}: UserName={0} and Password={1}", cmdLine.AuthenticationArgs.UserName, "*".PadRight(cmdLine.AuthenticationArgs.Password.Length, '*'),server);
-                    }
-                }
-                if(!found)
-                {
-                    error = string.Format("The /SavedCreds could not be retrieved for /Server={0}", cmdLine.Server);
-                    errorMessages = new string[] { error, "Returning error code: " + (int)ExecutionReturn.FinishingWithErrors };
-                    log.Error(error);
-                    return (int)ExecutionReturn.FinishingWithErrors;
-                }
-
-
-            }
-
             return 0;
         }
         /// <summary>

SqlBuildManager.Console/packages.config

@@ -5,7 +5,7 @@
   <package id="Microsoft.Azure.KeyVault.Core" version="3.0.3" targetFramework="net461" />
   <package id="Microsoft.Rest.ClientRuntime" version="2.3.20" targetFramework="net461" />
   <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.19" targetFramework="net461" />
-  <package id="Newtonsoft.Json" version="12.0.2-beta1" targetFramework="net461" />
+  <package id="Newtonsoft.Json" version="12.0.2" targetFramework="net461" />
   <package id="System.IO.Compression" version="4.3.0" targetFramework="net461" />
   <package id="WindowsAzure.Storage" version="9.3.3" targetFramework="net461" />
 </packages>

SqlBuildManager.Services.Host/SqlBuildManager.Services.Host.csproj

@@ -66,7 +66,7 @@
       <HintPath>..\packages\Microsoft.WindowsAzure.ConfigurationManager.3.2.3\lib\net40\Microsoft.WindowsAzure.Configuration.dll</HintPath>
     </Reference>
     <Reference Include="Newtonsoft.Json, Version=12.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
-      <HintPath>..\packages\Newtonsoft.Json.12.0.2-beta1\lib\net45\Newtonsoft.Json.dll</HintPath>
+      <HintPath>..\packages\Newtonsoft.Json.12.0.2\lib\net45\Newtonsoft.Json.dll</HintPath>
     </Reference>
     <Reference Include="System" />
     <Reference Include="System.Configuration" />

SqlBuildManager.Services.Host/packages.config

@@ -4,6 +4,6 @@
   <package id="Microsoft.Azure.KeyVault.Core" version="3.0.3" targetFramework="net461" />
   <package id="Microsoft.Data.Edm" version="5.8.4" targetFramework="net461" />
   <package id="Microsoft.WindowsAzure.ConfigurationManager" version="3.2.3" targetFramework="net461" />
-  <package id="Newtonsoft.Json" version="12.0.2-beta1" targetFramework="net461" />
+  <package id="Newtonsoft.Json" version="12.0.2" targetFramework="net461" />
   <package id="System.IO.Compression" version="4.3.0" targetFramework="net461" />
 </packages>