Skip to content

Latest commit

 

History

History
115 lines (91 loc) · 3.44 KB

README.md

File metadata and controls

115 lines (91 loc) · 3.44 KB
Raw

End-to-End Machine Learning Project

Project Overview

This project demonstrates a complete pipeline for building, deploying, and maintaining a machine learning model in production.

Prerequisites

  • Python 3.9+
  • Virtual Environment: Create a virtual environment using venv or conda

Installation

  1. Clone the repository: 
    git clone https://github.com/AbdullahMansoor123/ml_end2end
  2. Navigate to the project directory: 
    cd end-to-end-ml-project
  3. Create and activate a virtual environment: 
    python -m venv venv
source venv/bin/activate  # For Windows: venv\Scripts\activate
  4. Install the required dependencies: 
    pip install -r requirements.txt

Usage

  1. Run Flask App:
    • Run the app by running python app.py in the terminal with virtual environment enabled.

CICD with AWS

Steps

  1. Clone the GitHub Repo

    git clone https://github.com/AbdullahMansoor123/ml_end2end
cd ml_end2end

  2. Create a Dockerfile

    • Define the Dockerfile to build the application.

  3. Create a GitHub Workflow

    • Set up GitHub Actions workflow for continuous integration.

  4. Create IAM User and Download Access Key

    • Create an IAM user in AWS with necessary permissions and download the access key and secret key.

  5. Create AWS ECR Repo and Copy URL

    • Create a repository in AWS Elastic Container Registry (ECR) and copy the repository URL.

  6. Set Up EC2 Instance

    • Launch an EC2 instance and configure it as follows: 
      sudo apt-get update -y
sudo apt-get update
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker ubuntu
newgrp docker
      check if the setup completed successfully by running docker images on the EC2 instance

  7. Create Runner on GitHub to Connect Repo with EC2

    • Create a Folder for the Runner

      mkdir actions-runner && cd actions-runner

    • Download the Latest Runner Package

      curl -o actions-runner-linux-x64-2.319.1.tar.gz -L https://github.com/actions/runner/releases/download/v2.319.1/actions-runner-linux-x64-2.319.1.tar.gz

    • Optional: Validate the Hash

      echo "3f6efb7488a183e291fc2c62876e14c9ee732864173734facc85a1bfb1744464  actions-runner-linux-x64-2.319.1.tar.gz" | shasum -a 256 -c

    • Extract the Installer

      tar xzf ./actions-runner-linux-x64-2.319.1.tar.gz

    • Configure the Runner

      ./config.sh --url https://github.com/AbdullahMansoor123/ml_end2end --token <your_github_token>

    • Run the Runner

      ./run.sh

    • Update GitHub Workflow YAML
      Use the following in your GitHub Actions workflow file:

      runs-on: self-hosted

  8. Add GitHub Secrets for AWS

    • Add AWS credentials (access key and secret key) as GitHub secrets for your workflow to access AWS services.

  9. Edit Inbound Rules for EC2 Instance

    • Go to the Security settings of your EC2 instance.
    • Navigate to Security Groups.
    • Edit the Inbound Rules by adding a Custom TCP rule with the port number 4242 as specified in the GitHub workflow file

  10. Initiate CICD github workflow

  • Commit and push any change in the git repo to intiate cicd workflow.