SPIKE - Explore what logs are available on AWS + Incubator

[JasonEb]

Overview

As an Ops member, I'd like to know what logs does Incubator currently have. Once a good understanding of what's available, we then can document it better on Incubator or Ops' wiki.

Action Items

• Explore

  • ELB
  • RDS
  • ECR
  • ECS
  • Cloudwatch

• Record and share results

Resources/Instructions

• https://github.com/hackforla/incubator/wiki/AWS-Resources

[ale210]

Available logs:

• ALB/ELB: When enabled, logs are published to s3 as gzip files for individual web requests. Within the gzip file, log entries appear as: https 2018-07-02T22:23:00.186641Z app/my-loadbalancer/50dc6c495c0c9188 192.168.131.39:2817 10.0.0.1:80 0.086 0.048 0.037 200 200 0 57 "GET https://www.example.com:443/ HTTP/1.1" "curl/7.46.0" ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 arn:aws:elasticloadbalancing:us-east-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067 "Root=1-58337281-1d84f3d73c47ec4e58577259" "www.example.com" "arn:aws:acm:us-east-2:123456789012:certificate/12345678-1234-1234-1234-123456789012" 1 2018-07-02T22:22:48.364000Z "authenticate,forward" "-" "-" "10.0.0.1:80" "200" "-" "-" TID_123456
• RDS: Normal logs for either MySQL or PostreSQL are exposed via RDS. They contain errors, open/closed connections, queries, etc. RDS logs can also be sent to cloudwatch.
• ECR: Logging is available via cloudtrail for new image upload, security scanning results, etc
• ECS: Docker logging can be configured to go into cloudwatch. Additionally, it may be possible via cloudtrail to get ECS events like failed deployments, etc

[sudhara]

per discussion with John (Food Oasis project), there are currently no logs for troubleshooting application issues - Heroku has ? logs that can be reviewed. DevOps team need to come up with a framework for various apps to do the application logging. ECS enable log could be an option to leverage in other projects.

[sudhara]

closing this issue as the research work is complete.