02_git_configuration_HiPerGator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2ab10303-851b-47bf-a2c1-23e98a8a99ad",
   "metadata": {},
   "source": [
    "![Practicum AI Logo image](https://github.com/PracticumAI/practicumai.github.io/blob/main/images/logo/PracticumAI_logo_250x50.png?raw=true) <img src='https://github.com/PracticumAI/practicumai.github.io/blob/main/images/icons/practicumai_python.png?raw=true' align='right' width=50>\n",
    "\n",
    "# *Practicum AI: Computing for AI*\n",
    "## Module 3: Git and GitHub\n",
    "## GitHub Account Configuration on HiPerGator\n",
    "\n",
    "\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfae84fd-91d8-4c0a-9e39-ca9eeca2bcc5",
   "metadata": {},
   "source": [
    "This notebook will help you get your GitHub account configured if you are using HiPerGator. **If you plan to use Google Colab, [use this notebook instead](02_git_configuration_Google_Colab.ipynb).**\n",
    "\n",
    "The code blocks below may seem complicated, though if you read through them, you can probably make some good guesses about what they are doing. You will learn much of the code later, but for now, know that first we check if you have an ed25519 key, and if you do, offer to use that. If not, we create the key, and provide the contents for the next step.\n",
    "\n",
    "<div style=\"padding: 10px;margin-bottom: 20px;border: thin solid #30335D;border-left-width: 10px;background-color: #fff\">\n",
    "<p><strong>Note:</strong> The SSH key generated with this code does not have a passphrase. Adding a passphrase to an SSH key is a best practice. Unfortunately, Jupyter doesn't support them, and you wouldn't be able to use the SSH key with the built in `git` support in Jupyter if we put a passphrase on the key.\n",
    "</div>\n",
    "\n",
    "There are five steps for this notebook. Except for step 2, which is done in your GitHub account, there is a code block to execute--**All you should need to do is run each cell *without modification***. Some code blocks are long, and you probbly won't undestand everything, though you can probably follow what is happening if your read the comments and code. If you would rather not run code that you don't understand, for each step there are also alternative step-by-step instructions for running a few commands on the Linux, or Bash, Command Line. Again, you may not understand those, though we provide links to GitHub's explanation and our own explanations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c0cd615-1c42-425e-aae0-e22ceb89d3cc",
   "metadata": {},
   "source": [
    "## Step 1: Create your ssh key pair\n",
    "\n",
    "### What is an SSH Key??\n",
    "\n",
    "<div style=\"padding: 10px;margin-bottom: 20px;border: thin solid #30335D;border-left-width: 10px;background-color: #fff\">\n",
    "<p>SSH keys are a more secure method of authenticating a user than usernames and passwords. For this reason, GitHub has stopped using username/password authentication for pushing content to repos.</p>\n",
    "    \n",
    "<p>SSH is the secure shell application, and how many computers talk to each other. SSH keys consist of two files, one, the private key, should be protected and remain in computers you trust (for example, HiPerGator, or your own computer), the second, the public key, can be placed on any computer you want to login to.</p>\n",
    "\n",
    "<p>When you try to login to a computer, like GitHub, messages encrypted and decrypted with the public and private keys are passed between the servers. Only someone in possession of the private key file will be able to login to a server that has had the corresponding public key added to it.</p>\n",
    "\n",
    "<p>So, the process of setting up SSH key authentication involves three steps:</p>\n",
    "<ol>\n",
    "    <li>Create the SSH key pair</li>\n",
    "    <li>Add the public key to the server you want to log into</li>\n",
    "    <li>Test the connection</li>\n",
    "    </ol>\n",
    "</div>\n",
    "\n",
    "**Run the cell below and copy the last line of the output**. It should start with \"ssh-ed25519\" and be followed by a string of letters and numbers.\n",
    "\n",
    "If you would rather run the commands in the Bash terminal yourself, you can skip down to the Alternative Bash Command Line Method section. You do not need to do both."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd257fa5-89ea-4b43-9e81-513fb0fc41f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Do not change this code block\n",
    "\n",
    "import os, sys\n",
    "\n",
    "# Get user home and set paths for private and public keys\n",
    "home = os.path.expanduser('~')\n",
    "private_key_path=os.path.join(home, '.ssh', 'id_ed25519')\n",
    "public_key_path=os.path.join(home, '.ssh', 'id_ed25519.pub')\n",
    "\n",
    "# Test if there is an existing ed25519 key \n",
    "if os.path.isfile(private_key_path):\n",
    "    print(f\"Your account already has an ed25519 key at {private_key_path}.\")\n",
    "    use_existing=input(\"Do you want to use this key?\")\n",
    "    print('')\n",
    "    \n",
    "    if use_existing.lower() == \"y\":\n",
    "        # Read private key contents into pub_key\n",
    "        f = open(public_key_path, \"r\")\n",
    "        pub_key = f.read()\n",
    "        f.close()\n",
    "        print_key=1\n",
    "    else:\n",
    "        print(\"GitHub needs the private key file to be located in your private key to be located at ~/.ssh/id_ed25519.\")\n",
    "        print(\"This file already exists but you did not want to use it.\")\n",
    "        print(\"You will need to move the file or find a different way.\")\n",
    "        print_key=0\n",
    "       \n",
    "else:\n",
    "    print(f\"Creating a key pair at: {private_key_path} and {public_key_path}\")\n",
    "    os.system(f\"ssh-keygen -t ed25519 -C 'My GitHub key' -N '' -f {private_key_path}\")\n",
    "            # -t specifies the key type\n",
    "            # -C is a comment, can be anything\n",
    "            # -N is the pasphrase, a blank string is used here for no passphrase\n",
    "            # -f is the file to use for the key\n",
    "            # Providing the passphrase and file means no user interaction is needed\n",
    "    print('\\n\\n')\n",
    "    # Read private key contents into pub_key\n",
    "    f = open(public_key_path, \"r\")\n",
    "    pub_key = f.read()\n",
    "    f.close()\n",
    "    print_key=1\n",
    "    \n",
    "if print_key:\n",
    "    print(\"Below is the public key file contents. This is what you want to copy for the next step.\\n\")\n",
    "    print(pub_key)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e22babd0-39cf-4850-87ba-04a5e9148237",
   "metadata": {},
   "source": [
    "**Copy the last line of output from above.** It should look something like \"ssh-ed25519 AAAAC3daakfmDI1NTE5AAAAIPoySr7ZDbMvqAgpGRFfmC0DlEVMs//pEF2PYgHqn6Cs My GitHub key\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea16a806-52b2-4c61-bafb-6d3a75917452",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Alternative Bash Command Line Method\n",
    "\n",
    "**You do not need to do this if you ran the cell above. This is an alternative method to do the step.**\n",
    "\n",
    "><img src='images/hacker_terminal.gif' alt='gif of green terminal screen with 1s and 0s scrolling by' width=200 align='right' hspace=15>As an alternative to the code below, you can also take a step back in time and open a terminal (File menue > New > Terminal) and type the following commands. **You do not need to do both--if you ran the cell above, you can skip to step 2.**\n",
    ">\n",
    ">This is a simplified version of the [details outlined on GitHub.com](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent?platform=linux).\n",
    ">\n",
    ">* First make sure you don't have an existing key file at: `~/.ssh/id_ed25519.pub`:\n",
    "> \n",
    ">    `cat ~/.ssh/id_ed25519.pub`\n",
    "> \n",
    ">   * That should return an error say that no such file or directory exitsts.\n",
    ">    * If it does return \"ssh-ed25519\" followed by a string of letters and numbers, then you can either use that key file, or move/delete the key and its private pair and make a new one.\n",
    ">\n",
    ">* Make the ed25519 key pair with no passphrase:\n",
    ">\n",
    ">   `ssh-keygen -t ed25519 -C 'My GitHub key' -N '' -f ~/.ssh/id_ed25519`\n",
    ">\n",
    ">* Display the contents of the public key so you can **copy it**:\n",
    ">\n",
    ">   `cat ~/.ssh/id_ed25519.pub`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19844553-fc41-4d0a-812c-2a2b56aa31bc",
   "metadata": {},
   "source": [
    "## Step 2: Add your public key to your GitHub account\n",
    "\n",
    "1. Go to your GitHub Settings at: [https://github.com/settings/keys](https://github.com/settings/keys).\n",
    "\n",
    "1. As outlined [on GitHub.com](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account), click the New SSH Key button. [![Screenshot of the New SSH Key button](images/new_key.png)](https://github.com/settings/ssh/new)\n",
    "\n",
    "1. Give the key a name, \"HiPerGator\" for example, and **paste the public key text** you copied above into the Key box.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c9e3929-cc83-43c7-8733-0e74d94ed9ac",
   "metadata": {},
   "source": [
    "## Step 3: Configure `git` on HiPerGator for your user\n",
    "\n",
    "The following code cell will ask you to enter your name and then your email. It will then setup git on HiPerGator using the values you provide. Run this cell unchanged and enter the information in the propt box. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "948b70f8-1117-4f82-84c8-e2a1c739890b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Do not change this code block\n",
    "\n",
    "# Ask user for their name and email\n",
    "name = input(\"Please enter your name:\")\n",
    "email = input(\"Please enter your email:\")\n",
    "\n",
    "print(f\"Thank you {name}, now configuring your git account with your name and email\")\n",
    "\n",
    "# Run git config to set user.name and user.email\n",
    "os.system(f\"git config --global user.name '{name}'\")\n",
    "os.system(f\"git config --global user.email {email}\")\n",
    "\n",
    "print(f\"\\nVerifying the configuration has been set.\")\n",
    "\n",
    "# Get current git config list and use grep to show only the user lines\n",
    "config_list = os.system(\"git config --list | grep user\")\n",
    "\n",
    "print(f\"\\nAssuming user.name and user.email look correct, you should be set.\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccab460c-2edb-4b15-a021-f344474465ae",
   "metadata": {},
   "source": [
    "### Alternative Bash Command Line Method\n",
    "\n",
    "**You do not need to do this if you ran the cell above. This is an alternative method to do the step.**\n",
    "\n",
    ">\n",
    ">Run the following 2 commands using your Name (inside quotes) and email address:\n",
    ">\n",
    ">  `git config --global user.name 'Your Name'`\n",
    ">  \n",
    ">  `git config --global user.email your_email@example.com`\n",
    "> \n",
    ">The lines configure git for your user account."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fce5a7f-c94a-4072-b67f-ba7ffea14102",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Step 4: Test your SSH key\n",
    "\n",
    "**IMPORTANT: Do not skip this step** Test the key and add github.com to your known hosts. Back in the Jupyter terminal, type:\n",
    "\n",
    "The cell below adds the known information about the GitHub server to your known_hosts file (located at `~/.ssh/known_hosts`) and tests the connection to GitHub. \n",
    "\n",
    "You may see a warning about permenantly adding the ECDSA host key for an IP address. This is Ok.\n",
    "You should then see a greating from GitHub with your GitHub username. It will say \"You've successfully authenticated, but GitHub does not provide shell access.\" That is the expected output and means everything is setup correctly!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "966f1bb9-5d3d-4e45-84bb-cec41becc828",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Do not change this code block\n",
    "\n",
    "# Add github host information to known hosts file.\n",
    "!echo 'github.com,140.82.112.3 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=' \\\n",
    "  >> ~/.ssh/known_hosts\n",
    "\n",
    "# Test ssh connection to GitHub using the SSH keys setup above.\n",
    "!ssh -T git@github.com\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ac625b6-af75-4c20-8dfc-9df281d2b90b",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Alternative Bash Command Line Method\n",
    "\n",
    "**You do not need to do this if you ran the cell above. This is an alternative method to do the step.**\n",
    "\n",
    ">* As [outlined on GitHub.com](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/testing-your-ssh-connection): In the terminal type:\n",
    ">\n",
    ">   `ssh -T git@github.com`\n",
    ">\n",
    ">   (Do NOT replace the username, type it as git@github.com). \n",
    ">  \n",
    ">   * Since this is likely the first time you have connected to github.com from the compute your are on, you will be asked if you want to trust the unknown host. Type: **`yes`**  \n",
    ">   * You should then see a reply like this:\n",
    ">\n",
    ">     ```\n",
    ">     [magitz@login6 ~]$ ssh -T git@github.com\n",
    ">     Hi magitz! You've successfully authenticated, but GitHub does not provide shell access.\n",
    ">     ```\n",
    ">\n",
    ">   * This is the expected reply, your GitHub.com user name will be shown along with a note that you have successfully authenticated (using the setup ssh keys), but that GitHub does not provide shell access."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d4ad84b-ebc0-426e-81a4-081b7d02b316",
   "metadata": {},
   "source": [
    "## Step 5: Creat symbolic links to your `/blue` directory\n",
    "\n",
    "On HiPerGator, each user has a home directory (located at `/home/username`), which is where Jupyter starts and is likely where you are now, and one or more directories on the `/blue` filesystem (our high-performance filesystem). UFIT Research Computing policy is that all data read or written while running applications should be on the `/blue` filesystem, not in `/home/`. \n",
    "\n",
    "Unfortunately, Jupyter doesn't have an easy way to navigate to your `blue` directories. This cell will check the group used when you submitted the job, and make a symbolic link to the `/blue/group` directory, calling it `blue_group`. This will then appear as folders in your home directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8a92986-40ff-4c7c-ba69-6e671a21cbb7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Do not change this code block\n",
    "\n",
    "# Get the group used to submit the job running this notebook and user's home directory\n",
    "group = os.getenv(\"SLURM_JOB_ACCOUNT\")\n",
    "home = os.getenv(\"HOME\")\n",
    "\n",
    "# Set filesystem to make link to: 'blue' on HiPerGator\n",
    "filesystem = 'blue'\n",
    "\n",
    "# Get path to link to: /blue/group on HiPerGator and set a name for the link, blue_group by convention\n",
    "link_path = os.path.join(os.sep, filesystem, group)\n",
    "link_name = filesystem + '_' + group\n",
    "\n",
    "if os.path.islink(os.path.join(os.sep, home, link_name)) or os.path.isfile(os.path.join(os.sep, home, link_name)):\n",
    "    print(f\"Symbolic link, or file, already exists at {os.path.join(os.sep, home, link_name)}, doing nothing.\")\n",
    "else:\n",
    "    print(f\"Making symbolic link to {link_path} at {os.path.join(os.sep, home, link_name)}\")\n",
    "    os.system(f\"ln -s {link_path} {os.path.join(os.sep, home, link_name)}\") \n",
    "\n",
    "print(\"\\nDone!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c8032e6-faed-4abe-a1b1-57da4f9dc197",
   "metadata": {},
   "source": [
    "### Alternative Bash Command Line Method\n",
    "\n",
    "**You do not need to do this if you ran the cell above. This is an alternative method to do the step.**\n",
    "\n",
    ">First, determine the group where your `/blue` folder is located. The `id` command will show all your groups. For example:\n",
    ">\n",
    ">```bash\n",
    ">[albert@c1000a-s29 ~]$ id\n",
    ">uid=91234(albert) gid=9000(gator) groups=9000(gator),7917(practicum-ai)\n",
    ">```\n",
    ">\n",
    ">This shows that the user albert has the primary group of \"gator\" and is also in the \"practicum-ai\" group. They should most likely use the \"gator\" group for the next step.\n",
    ">\n",
    ">Second, create the link to `/blue/group` in your home directory. For the user albert above, making a link to the group gator, they would type:\n",
    ">\n",
    ">```ln -s /blue/gator ~/blue_gator```\n",
    ">\n",
    ">**Note**: You need to replace \"gator\" with **your** group name! The `ln -s` part of the command is to create a symbolic link, the `/blue/gator` part is what you want to link to (and why you need to change that so that the link points to your group directory), and the `~/blue_gator` is what you want to name the link (`~` is short for your home directory). We use the `blue_group` convention, because that makes it clear where the link is going to take you."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f161a0d7-ed0e-4655-8a70-3675bdc5181c",
   "metadata": {},
   "source": [
    "## That's it!!\n",
    "\n",
    "Your account shoudl be setup and ready to go!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "UFRC Python-3.10",
   "language": "python",
   "name": "python3-3.10-ufrc"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}