Skip to content

Commit

Permalink
Install LME in the testbed from a single script (#150)
Browse files Browse the repository at this point in the history 
* Adding the configure scripts

* Add scripts to zip and copy to a container for downloading in the server

* Grab the expiry time properly in copy file

* Overwrite the blob if it exists

* Add the script to download file into DC

* Script that unzips the files in a container

* Adds username argument to download files

* Add script to run scripts in container

* Adds username argument to gpo script

* Modifies the url name in the client GPO

* Adds the functionality for chapter 1 and first half of chapter 2

* Imports the sysmon GPO

* Update the variables for sysmon gpo

* Name the scripts so they are grouped together in a listing

* Echos the file download url

* Expands the domain name correctly in create ou

* Write the url output of copy file to container to a different output stream

* Create a new LME folder for our scripts and files

* Set path for extract to lme

* Update paths for scripts to /lme

* Fix the wec server name setting

* Adds the scripts to install chapter 1 and 2

* Allows azure to download in linux and windows

* Adds linux install scripts.

* Adds winlogbeat installer

* emove garbage in update server name

* Tweak several scripts to get the scp of files_for_windows

* Adds installer script to run all the scripts

* Fixes the formatting method for az output

* Clean up the scripts and add documentation

* Fixes outputting format errors

* Fixes hanging on adding ls1 to domain

* Fix formatting errors on responses

* Update linux expect script for different prompts.

* Handle the reboot message for linux expect script

* Echos the file download url

* Create a new LME folder for our scripts and files

* Set path for extract to lme

* Update paths for scripts to /lme

* Update paths for scripts to /lme

* Fix the wec server name setting

* Adds the scripts to install chapter 1 and 2

* Allows azure to download in linux and windows

* Adds linux install scripts.

* Adds winlogbeat installer

* emove garbage in update server name

* Tweak several scripts to get the scp of files_for_windows

* Adds installer script to run all of the scripts

* Fixes the formatting method for az output

* Clean up the scripts and add documentation

* Fixes outputting format errors

* Fixes hanging on adding ls1 to domain

* Fix formatting errors on responses

* Update linux expect script for different prompts.

* Handle the reboot message for linux expect script

* Adds InstallTestbed instructions to Readme.md

* Modifies parameters to be pascal case

* ls1 not being set on DC1

* Adds Linux Only install to SetupTestbed

* Remove separate linux only script

* Update testing/Readme.md

Co-authored-by: Alden Hilton <[email protected]>

* Make number of clients consisten between scripts

* Add ports for elk stack for testing

* Update readmes to change ResourceGroupName to ResourceGroup

* Adds a switch to install linux only

* Adds simple tests to check install

* Removes the error if the old configure zip is not found.

* Adds variables to linux tests run command

* Move credential extraction to lib for use by other scripts.

* Adds npm for other testing

* Adds latest version of nodejs for testing

* Make output.log readable for tests

* Add the -m parameter in the testing readme

* Download the latest version or a specified version

* Reboot for 1.3.0

* Notes that we could have different expect scripts

* Put back in the restart after all of the domain updates

* Scp uses ls1 instead of ls1.lme.local

* Up the timeout of the adding ls1.lme.local

* Up the timeout of the adding ls1.lme.local

* Fixes chmod of the output.log for tests

* Adds venv to the gitignore

* Adds the ability to pass a branch to the installer

* Remove node installer

* Change timeout in expect script for slow connections

* Make shell files executable

---------

Co-authored-by: Clint Baxley <[email protected]>
Co-authored-by: Alden Hilton <[email protected]>
  • Loading branch information
@cbaxley @adhilto @mitchelbaker-cisa
3 people authored and mitchelbaker-cisa committed Feb 21, 2024
1 parent 39b1efd commit 3046a4c
Show file tree
Hide file tree
Showing 38 changed files with 2,377 additions and 208 deletions.
2 changes: 2 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,5 @@ dashboard_update.sh
files_for_windows.zip
lme.conf
lme_update.sh
/testing/tests/.env
/testing/tests/venv/
386 changes: 386 additions & 0 deletions testing/InstallTestbed.ps1
View file

Large diffs are not rendered by default.

67 changes: 51 additions & 16 deletions testing/Readme.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,29 +13,31 @@ Using the Azure CLI, it creates the following:
This script does not install LME; it simply creates a fresh environment that's ready to have LME installed.

## Usage
| **Parameter** | **Alias** | **Description** | **Required** |
|------------------------|-----------|----------------------------------------------------------------------------------------|---------------------------------------|
| $ResourceGroup | -g | The name of the resource group that will be created for storing all testbed resources. | Yes |
| $NumClients | -n | The number of Windows clients to create; maximum 16; defaults to 1 | No |
| $AutoShutdownTime | | The auto-shutdown time in UTC (HHMM, e.g. 2230, 0000, 1900); auto-shutdown not configured if not provided | No |
| $AutoShutdownEmail | | An email to be notified if a VM is auto-shutdown. | No |
| $AllowedSources | -s | Comma-Separated list of CIDR prefixes or IP ranges, e.g. XX.XX.XX.XX/YY,XX.XX.XX.XX/YY,etc..., that are allowed to connect to the VMs via RDP and ssh. | Yes |
| $NoPrompt | -y | Switch, run the script with no prompt (useful for automated runs). By default, the script will prompt the user to review paramters and confirm before continuing. | No |
| **Parameter** | **Alias** | **Description** | **Required** |
|--------------------|-----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|
| $ResourceGroup | -g | The name of the resource group that will be created for storing all testbed resources. | Yes |
| $NumClients | -n | The number of Windows clients to create; maximum 16; defaults to 2 | No |
| $AutoShutdownTime | | The auto-shutdown time in UTC (HHMM, e.g. 2230, 0000, 1900); auto-shutdown not configured if not provided | No |
| $AutoShutdownEmail | | An email to be notified if a VM is auto-shutdown. | No |
| $AllowedSources | -s | Comma-Separated list of CIDR prefixes or IP ranges, e.g. XX.XX.XX.XX/YY,XX.XX.XX.XX/YY,etc..., that are allowed to connect to the VMs via RDP and ssh. | Yes |
| $Location | -l | The region you would like to build the assets in. Defaults to westus | No |
| $NoPrompt | -y | Switch, run the script with no prompt (useful for automated runs). By default, the script will prompt the user to review paramters and confirm before continuing. | No |
| $LinuxOnly | -m | Run a minimal install of only the linux server | No |

Example:
```
./SetupTestbed.ps1 -ResourceGroup Example1 -NumClients 2 -AutoShutdownTime 0000 -AllowedSources "1.2.3.4,1.2.3.5" -y
```

## Running Using Azure Shell
| **#** | **Step** | **Screenshot** |
|-------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------|
| 1 | Open a cloud shell by navigating to portal.azure.com and clicking the shell icon. | ![image](/docs/imgs/testing-screenshots/shell.png) |
| 2 | Select PowerShell. | ![image](/docs/imgs/testing-secreenshots/shell2.png) |
| 3 | Upload `SetupTestbed.ps1` by clicking the "Upload/Download files" icon | ![image](/docs/imgs/testing-screenshots/shell3.png) |
| 4 | Run the script, providing values for the parameters when promoted (see [Usage](#usage)). The script will take ~20 minutes to run to completion. | ![image](/docs/imgs/testing-screenshots/shell4.png) |
| 5 | Save the login credentials printed to the terminal at the end. At this point you can login to each VM using RDP (for the Windows servers) or SSH (for the Linux server). | ![image](/docs/imgs/testing-screenshots/shell5.png) |
| 6 | When you're done testing, simply delete the resource group to clean up all resources created. | ![image](/docs/imgs/testing-screenshots/delete.png) |
| **#** | **Step** | **Screenshot** |
|-------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------|
| 1 | Open a cloud shell by navigating to portal.azure.com and clicking the shell icon. | ![image](/docs/imgs/testing-screenshots/shell.png) |
| 2 | Select PowerShell. | ![image](/docs/imgs/testing-secreenshots/shell2.png) |
| 3 | Clone the repo `git clone https://github.com/cisagov/LME.git` and then `cd LME\testing` | |
| 4 | Run the script, providing values for the parameters when promoted (see [Usage](#usage)). The script will take ~20 minutes to run to completion. | ![image](/docs/imgs/testing-screenshots/shell4.png) |
| 5 | Save the login credentials printed to the terminal at the end (They will also be in a file called `<$ResourceGroup>.password.txt`). At this point you can login to each VM using RDP (for the Windows servers) or SSH (for the Linux server). | ![image](/docs/imgs/testing-screenshots/shell5.png) |
| 6 | When you're done testing, simply delete the resource group to clean up all resources created. | ![image](/docs/imgs/testing-screenshots/delete.png) |

# Extra Functionality:

Expand All @@ -55,3 +57,36 @@ Flags:
- enable: deletes the DENYINTERNET/DENYLOADBALANCER rules
- NSG: sets NSG to a custom NSG if desired [NSG1 default]

## Install LME on the cluster:
### InstallTestbed.ps1
## Usage
| **Parameter** | **Alias** | **Description** | **Required** |
|-------------------|-----------|----------------------------------------------------------------------------------------|--------------|
| $ResourceGroup | -g | The name of the resource group that will be created for storing all testbed resources. | Yes |
| $NumClients | -n | The number of Windows clients you have created; defaults to 2 | No |
| $DomainController | -w | The name of the domain controller in the cluster; defaults to "DC1" | No |
| $LinuxVm | -l | The name of the linux server in the cluster; defaults to "LS1" | No |
| $LinuxOnly | -m | Run a minimal install of only the linux server | No |
| $Version | -v | Optionally provide a version to install if you want a specific one. `-v 1.3.2` | No |
| $Branch | -b | Optionally provide a branch to install if you want a specific one `-b your_branch` | No |

Example:
```
./InstallTestbed.ps1 -ResourceGroup YourResourceGroup
# Or if you want to save the output to a file
./InstallTestbed.ps1 -ResourceGroup YourResourceGroup | Tee-Object -FilePath "./YourResourceGroup.output.log"
```
| **#** | **Step** | **Screenshot** |
|-------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------|
| 1 | Open a cloud shell by navigating to portal.azure.com and clicking the shell icon. | ![image](/docs/imgs/testing-screenshots/shell.png) |
| 2 | Select PowerShell. | ![image](/docs/imgs/testing-secreenshots/shell2.png) |
| 3.a | If you have already cloned the LME repo then make sure you are in the `LME\testing` directory and run git pull before changing to the testing directory. | |
| 3.b | If you haven't cloned it, clone the github repo in the home directory. `git clone https://github.com/cisagov/LME.git` and then `cd LME\testing`. | |
| 4 | Now you can run one of the commands from the Examples above. | |
| 5 | Save the login credentials printed to the terminal at the end. *See note* | |
| 6 | When you're done testing, simply delete the resource group to clean up all resources created. | |

Note: When the script finishes you will be in the azure_scripts directory, and you should see the elasticsearch credentials printed to the terminal.
You will need to `cd ../../` to get back to the LME directory. All the passwords should also be in the `<$ResourceGroup>.password.txt` file.


Loading

0 comments on commit 3046a4c

Please sign in to comment.