Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LAWS-3514: added module for s3 bucket in apex #3667

Merged
merged 95 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
fe11bd8
LAWS-3514: added module for s3 bucket in apex
tmahmood72 Oct 16, 2023
26cf532
LAWS-3514: updated bucket name in s3 module
tmahmood72 Oct 16, 2023
8d4f414
LAWS-3514: added lambda for dbsnapshot
tmahmood72 Oct 16, 2023
a957411
LAWS-3514: updated snapshotDBFunction with correct iam role name
tmahmood72 Oct 16, 2023
317d1c7
LAWS-3514: adding lambda funtion as a module - v1.00
tmahmood72 Oct 16, 2023
9b5048b
create lambda module
tajewole-moj Oct 17, 2023
8e5c50f
removed the variable file
tajewole-moj Oct 17, 2023
dd12e1f
forgot to add the count.index
tajewole-moj Oct 17, 2023
505e39c
typo on count index
tajewole-moj Oct 17, 2023
1620799
wrong values
tajewole-moj Oct 17, 2023
312139d
LAWS-3514: adding 2 lambda funtions using variables
tmahmood72 Oct 17, 2023
fce2b9d
added data function for iam roles
tajewole-moj Oct 17, 2023
a8bcbc3
create iam module
tajewole-moj Oct 17, 2023
cf75f91
typo on filename
tajewole-moj Oct 17, 2023
070311f
remove duplicate resource
tajewole-moj Oct 17, 2023
c0088df
file name error
tajewole-moj Oct 17, 2023
c681797
rename module
tajewole-moj Oct 17, 2023
d12c2dd
change backup role name
tajewole-moj Oct 17, 2023
2e1d577
rename the lambda functions
tajewole-moj Oct 17, 2023
6954495
change runtime
tajewole-moj Oct 17, 2023
7bde437
adding extra vars
tajewole-moj Oct 17, 2023
8b575cb
adding extra vars v1
tajewole-moj Oct 17, 2023
3e8ca1e
change runtime
tajewole-moj Oct 17, 2023
38bc11c
add 3rd function
tajewole-moj Oct 17, 2023
ca5a4c3
remove double quotes
tajewole-moj Oct 17, 2023
73638ef
tidy up
tajewole-moj Oct 18, 2023
0ba836f
re-tidy v1
tajewole-moj Oct 18, 2023
5edd770
event rules
tajewole-moj Oct 18, 2023
a4d9567
added count index
tajewole-moj Oct 18, 2023
b9a86cb
amend the count indexes
tajewole-moj Oct 18, 2023
71daa8e
only attaching the first lambda function to event rules
tajewole-moj Oct 18, 2023
f03e4c1
3 is not require just one rule for now
tajewole-moj Oct 18, 2023
cc929ff
changed scheduled expressions
tajewole-moj Oct 18, 2023
7a472a3
rebuild ec2 with new ami and merge ecs code
tajewole-moj Oct 19, 2023
c780215
Merge branch 'main' into LAWS-3514-Backup-Lambda
tajewole-moj Oct 19, 2023
de12d86
change cron time for event bridge
tajewole-moj Oct 19, 2023
7cdbb94
change cron time for event bridge
tajewole-moj Oct 19, 2023
3dbfb3d
change event bridge cron time
tajewole-moj Oct 19, 2023
1b7b1e5
added input paramater and change cron time
tajewole-moj Oct 20, 2023
aa56696
uncomment event rules
tajewole-moj Oct 20, 2023
6f8efb5
deploying everything again
tajewole-moj Oct 20, 2023
4394676
LAWS-3514: changed cron entry time for Lambda backup
tmahmood72 Oct 20, 2023
3ebcba4
LAWS-3514: removed cron Lambda backup entry
tmahmood72 Oct 20, 2023
db8bd5b
Merge branch 'main' of https://github.com/ministryofjustice/modernisa…
tajewole-moj Oct 23, 2023
5aef983
remove event rule and module folder
tajewole-moj Oct 23, 2023
b905f5f
deleted module folder
tajewole-moj Oct 23, 2023
1460eaa
typo on module folder
tajewole-moj Oct 23, 2023
a45d078
re-added the event rules
tajewole-moj Oct 23, 2023
b24cf1e
allow json values
tajewole-moj Oct 23, 2023
53f946b
create sg and attach sg and created env
tajewole-moj Oct 23, 2023
41bf25b
included extra variables for the module
tajewole-moj Oct 23, 2023
3175b95
add vpc id
tajewole-moj Oct 23, 2023
270fb41
forgot to save adding vpc
tajewole-moj Oct 23, 2023
8c78453
missed []
tajewole-moj Oct 23, 2023
28e084b
change cron time
tajewole-moj Oct 23, 2023
de69a8f
change cron time v1
tajewole-moj Oct 23, 2023
41c016c
create s3 bucket key and for lambda function
tajewole-moj Oct 23, 2023
13fe178
delete s3.tf
tajewole-moj Oct 23, 2023
cea300b
forgot to save file
tajewole-moj Oct 23, 2023
a5887b1
remove tag vars
tajewole-moj Oct 23, 2023
ddea4eb
reduce the count to 1
tajewole-moj Oct 23, 2023
5ce9bf1
added count index to the s3
tajewole-moj Oct 23, 2023
59e771b
amend the vars and made them list
tajewole-moj Oct 23, 2023
977468c
remove the key entry
tajewole-moj Oct 23, 2023
ba59ce1
remove unwanted vars
tajewole-moj Oct 23, 2023
71237cf
increase the count to 3
tajewole-moj Oct 23, 2023
c2c1617
test trigger
tajewole-moj Oct 23, 2023
916b61b
uncomment out var
tajewole-moj Oct 23, 2023
e93934e
uncomment out var v1
tajewole-moj Oct 23, 2023
c5e1a28
change time
tajewole-moj Oct 23, 2023
96eeb32
amend vars and change cron time
tajewole-moj Oct 24, 2023
478a144
change s3 key name
tajewole-moj Oct 24, 2023
0c280db
deploying lambda out of module
tajewole-moj Oct 24, 2023
6cbdc82
forgot to save the local.tf file
tajewole-moj Oct 24, 2023
bb52d30
typo
tajewole-moj Oct 24, 2023
4f9bb1b
change cron time
tajewole-moj Oct 24, 2023
535f7df
change event bus name
tajewole-moj Oct 24, 2023
9db31c4
remove s3 bucket
tajewole-moj Oct 24, 2023
6c2cac7
include file name
tajewole-moj Oct 24, 2023
b55b806
change cron time
tajewole-moj Oct 24, 2023
a792dee
add function layers
tajewole-moj Oct 24, 2023
7635b03
change runtime
tajewole-moj Oct 24, 2023
b490a5e
change runtime v1
tajewole-moj Oct 24, 2023
07a2cf1
change bucket name
tajewole-moj Oct 24, 2023
eb9ccbe
use arn
tajewole-moj Oct 24, 2023
77ff9e7
change filename on local vars
tajewole-moj Oct 24, 2023
d2dbb82
LAWS-3514: changed nodeks version for function and ssh layer
tmahmood72 Oct 25, 2023
c3a44f0
LAWS-3514: changed nodejs version for function and ssh layer to versi…
tmahmood72 Oct 25, 2023
8e9a10b
LAWS-3514: changed event rule time
tmahmood72 Oct 25, 2023
7a9980b
LAWS-3514: changed nodejs from version 12 to 14
tmahmood72 Oct 25, 2023
7c7ef6e
change cron time
tajewole-moj Oct 26, 2023
4af6538
change time
tajewole-moj Oct 26, 2023
09da645
utc time
tajewole-moj Oct 26, 2023
d1cfc4c
tidy up the code and remove variables no more required
tajewole-moj Oct 26, 2023
592a692
tidy up part2
tajewole-moj Oct 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions terraform/environments/apex/application_variables.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@
"accounts": {
"development": {
"example_var": "dev-data",
"ec2amiid": "ami-057de645e240e87c1",
"ec2amiid": "ami-0a5b0804d9efdeed0",
"ec2instancetype": "t3.medium",
"mp_vpc_cidr": "10.26.56.0/21",
"lz_shared_nonprod_mgmt_vpc_cidr": "10.200.0.0/20",
"lz_shared_prod_mgmt_vpc_cidr": "10.200.16.0/20",
"u01_orahome_size": "40",
"u01_orahome_snapshot": "snap-0fb84db7af842db4b",
"u01_orahome_snapshot": "snap-0dbe4e9eb3ae8c8e2",
"u02_oradata_size": "100",
"u02_oradata_snapshot": "snap-002a82aef2628b621",
"u02_oradata_snapshot": "snap-08eb47b98646c0002",
"u03_redo_size": "50",
"u03_redo_snapshot": "snap-08499cdd392b9b7dc",
"u03_redo_snapshot": "snap-02864a98e6392f7ba",
"u04_arch_size": "50",
"u04_arch_snapshot": "snap-03b6eb947cfe779d8"
"u04_arch_snapshot": "snap-0bc63d7b569e9eb3b"
},
"test": {
"example_var": "test-data"
Expand Down
173 changes: 173 additions & 0 deletions terraform/environments/apex/dbconnect.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
// CWA automated backup script
// - Makes call to lambda which connects to EC2 instance and put
// DB in backup mode
// - Call Oracle SQL scripts as Oracle user
//
// version: 0.1
// auth: phil h
/////////////////////////////////////////////////////////////////////

const SSH = require("simple-ssh");
const AWS = require("aws-sdk");

//SSM object with temp parms
const ssm = new AWS.SSM({ apiVersion: "2014-11-06" });

// Environment variables
const pem = "MGMT_EC2_KEY_DEFAULT";
const username = "ec2-user";

//Set date format
var today = new Date();
var dd = today.getDate();
var mm = today.getMonth() + 1;
var yyyy = today.getFullYear();

if (dd < 10) {
dd = "0" + dd;
}

if (mm < 10) {
mm = "0" + mm;
}
today = dd + "-" + mm + "-" + yyyy;

//EC2 object
let ec2 = new AWS.EC2({ apiVersion: "2014-10-31" });

//Get private IP address for EC2 instances tagged with Name:{ appname }
// May return more than 1 instance if there are multiple instances with the same name
async function getInstances(appname) {
console.log("Getting all instances tagged with Name:", appname);
return ec2
.describeInstances({ Filters: [{ Name: "tag:Name", Values: [appname] }] })
.promise();
}

async function getIPaddress(appname) {
var instance_ip_list = [];
var instance_data = await getInstances(appname);
for (const res of instance_data["Reservations"]) {
for (const instance of res["Instances"]) {
instance_ip_list.push(instance["PrivateIpAddress"]);
}
}
console.log("Found ", instance_ip_list.length, " instances");
return instance_ip_list;
}


// Get SSH key from param store

async function getSSMparam() {
return await ssm.getParameter({ Name: pem, WithDecryption: true }).promise();
}

// Trigger SSH connection to the EC2 instance
// Run SSH command

async function connSSH(action, appname) {
//get ssm key
const key = await getSSMparam();

const myKey = key["Parameter"]["Value"];

const addresses = await getIPaddress(appname);
// all this config could be passed in via the event
for(var address of addresses){
const ssh = new SSH({
host: address,
port: 22,
user: username,
key: myKey,
});

let prom = new Promise(function (resolve, reject) {
if (action == "begin") {
console.log("[+] Trying connecting to EC2 ==>> " + address);
console.log(`[+] Running "begin backup commands" as Oracle`);

ssh
.exec(
'sudo su - oracle -c "sqlplus / as sysdba <<EOFUM' +
"\n" +
"alter system switch logfile;" +
"\n" +
"alter system switch logfile;" +
"\n" +
"alter database begin backup;" +
"\n" +
"exit;" +
"\n" +
'EOFUM"',
{
pty: true,
out: console.log.bind(console),
exit: function (code, stdout, stderr) {
console.log("operation exited with code: " + code);
console.log(stdout);
console.log(stderr);
},
}
)
.start();
} else if (action == "end"){
console.log(`[+] Running "end backup commands" as Oracle`);

ssh
.exec(
'sudo su - oracle -c "sqlplus / as sysdba <<EOFUM' +
"\n" +
"alter database end backup;" +
"\n" +
"alter system switch logfile;" +
"\n" +
"alter system switch logfile;" +
"\n" +
"exit;" +
"\n" +
'EOFUM"',
{
pty: true,
out: console.log.bind(console),
exit: function (code, stdout, stderr) {
console.log("operation exited with code: " + code);
console.log(stdout);
console.log(stderr);
},
}
)
.start();
}
});
try {
await prom;

const response = {
statusCode: 200,
};

console.log(`[+] Completed DB alter state: ${action} ==>> ` + address);
console.log("[+] Returned response: " + response);

ssh.end();

return response;
} catch (e) {
console.log(e);
context.fail();
}
}
}

exports.handler = async (event, context) => {
try {
console.log("[+} Received event:", JSON.stringify(event, null, 2));
await connSSH(event.action, event.appname);

context.done();
} catch (error) {
console.error(error);
context.fail();
}
};
Loading
Loading