8 Working_with_files_and_folders.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 8. Working with files and folders\n",
    "\n",
    "\n",
    "## Reading a file\n",
    "\n",
    "\n",
    "If the data that you need to work with in a reseaech project is saved as a file on your computer, you can write a program to read this file and to make its contents available within the context of your code.\n",
    "\n",
    "In Python, files can be read using the `open()` function. The result of this function is a new object which is called a file handler (or, more specifically, a `TextIOWrapper` object). Simply put, a file handler is an object which establishes a connection to the text file on your disk. You are free to give this file handler object any name you like. \n",
    "\n",
    "When you use the `open()` function, you are also recommended to specify the character encoding scheme that has been used in the text file, using the `encoding` parameter. This will help Python to process all the characters correctly. \n",
    "\n",
    "Once the connection is established via the `open()` fuction, you can access the contents of the file in a variety of ways. A first option is to read the contents on a line-by-line or a paragraph-by-paragraph basis. This first approach can be followed when units such as lines or paragraphs in the text are delineated using the hard return or the newline character. If this is the case, the file handler that is created for the file, using `open()`, also becomes iterable: the `for` keyword can then be used to iterate across the various units represented by this file handler. \n",
    " \n",
    "The code below demonstrates how you can read and display the full contents of a text file, paragraph by paragraph. It assumes that there is a file named \"BraveNewWorld.txt\", saved in the same directory as the code. It also assumes that the various paragraphs are separated using a hard return. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = open(\"BraveNewWorld.txt\" , encoding = 'utf-8' )\n",
    "\n",
    "for paragraph in text:\n",
    "    print(paragraph)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the case of short texts, you can also make use of the `read()` function. When you do this, the entire text will not be divided into smaller units. The full contents of the text file will become available as one long string. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = open(\"tweet.txt\" , encoding = 'utf-8' )\n",
    "fullText = text.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we have run this code, we can manipulate the string that is created by the `read()` function just like any other string.\n",
    "\n",
    "It is good practice to close the file handler when you are done working on it, using the `close()` method. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next to the options that have been discussed so far, you can also read files by making use of a mechanism that is referred to as a context handler. \n",
    "\n",
    "Context handlers are created using the `with` keyword. After `with`, yu need to use the `open()` function. This `open()` function needs to be followed by the words `as` and the name you would like to give to the file handler. In the code block underneath `with`, you can access the contents of this file handler. It is generally useful to assign the contents of the text file to a variable. When the code block underneath `with` ends, the file handler is closed automatically. This is actually a great advantage of a context handler. You don't risk forgetting to include the `close()` function.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = \"ATaleOfTwoCities.txt\"\n",
    "\n",
    "with open(file, encoding = 'utf-8') as text:\n",
    "    contents = text.read()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing to a file\n",
    "\n",
    "When you run a Python program using the Command Prompt, the full output will normally be printed on the Command Prompt as well. The output of code created in a Jupyter notebook will normally be shown directly underneath the code cell. \n",
    "\n",
    "When the program has many lines to print, it can be very difficult to read the output. In such cases, it can useful to create a new text file which will receive all the output. The results of the program can then be inspected by opening this new file in a text editor. \n",
    "\n",
    "The function `open()`, which can be used to read files, can also be invoked to create a new file. Instead of referencing a file which already exists on your disk, you need to provide a new file name. Next to this, you also need to supply a second parameter, the character “w”, which stands for “write”. This “w” character makes it clear to Python that you want to write to a file. The `open()` function used with the “w” parameter similarly creates a file handler.\n",
    "\n",
    "This handler has a `write()` method, which functions very similarly to the `print()` function. The crucial difference, however, is that the output is not sent to the default output device (e.g. the Command Prompt or Jupyter Notebook), but to the file that is associated with this file handler. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "out = open('data.txt' , 'w')\n",
    "\n",
    "out.write( \"This text is in a file named 'data.txt' \" )\n",
    "\n",
    "out.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading the contents of a folder\n",
    "\n",
    "The `open()` function can be used, as discussed, to read individual files. It is also possible to read the contents of folders with multiple files. For this purpose, you can make use of the `os` library . The two letters in the name of this library stand for 'operating system'. The library includes various functions that can help you to work with files and folders. One useful method is `listdir()`, which, expectedly, enables you to list all the files in a given directory. \n",
    "\n",
    "To make use of `os`, this library needs to be imported first.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "dir = 'Corpus'\n",
    "\n",
    "for fileName in os.listdir( dir ):\n",
    "    print( fileName )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function `join()`, which is part of the `path` module of `os`, can be used to create a string representing the path to a certain file. If you have one variable which records the base directory of a file, and a second variable which captures the filename, the `join()` function can concatenate the values of these two variables to create the full path to this file. The `join()` function always follows the conventions that are in place on a given operating system for representing paths. There can often be certain differences. While Mac OS uses forward slashes, for instance, Windows uses back slashes. Working with `join()` makes your code more platform-independent.        \n",
    "\n",
    "Another useful function in `os` is `isfile()`. As you list the files in a certain directory, using `listdir()`, you can apply this function to check whether you are dealing with a file or with something else ( e.g. a subdirectory).\n",
    "\n",
    "The code below offers a demostration of these two functions. It lists all the files in the directory that is mentiond, but it ignores all the subdirctories. Note that the first line imports the two functions that have been discussed above. As a result of this, it is no longer necessary to use the period syntax for `isfile()` and `join()`.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from os.path import isfile , join\n",
    "\n",
    "dir = 'Corpus'\n",
    "\n",
    "for fileName in os.listdir( dir ):\n",
    "    if isfile( join( dir , fileName ) ):\n",
    "        print( fileName ) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 8.1.\n",
    "\n",
    "Using the `listdir()` function from the `os` library, write code which can list all the files in a given folder on your own computer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 8.2.\n",
    "\n",
    "Download the following file:\n",
    "\n",
    "[https://edu.nl/ggwtf](https://edu.nl/ggwtf)\n",
    "\n",
    "The file contains the text of a poem. \n",
    "\n",
    "Print the content of the poem that you have downloaded for exercises 8.2 on your screen, and make sure that you also add line numbers, as follows:\n",
    "    \n",
    "```\n",
    "1. [line1]\n",
    "2. [line2]\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.get('https://edu.nl/ggwtf')\n",
    "\n",
    "if response.status_code == 200:\n",
    "    response.encoding = 'utf-8'\n",
    "    contents = response.text\n",
    "    with open('sonnet116.txt' , 'w' , encoding = 'utf-8') as fh:\n",
    "        fh.write(contents)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 8.3\n",
    "\n",
    "\n",
    "Create a new file named ‘data.txt’, which gives information about the following aspects of this poem you have downloaded for exercise 8.2:\n",
    "\n",
    "* The number of lines\n",
    "* The total number of characters in the file\n",
    "* The average number of characters per line\n",
    "* The longest line of the sonnet (i.e. the line with the highest number of characters)\n",
    "\n",
    "The average number of characters per line can be calculated by dividing the total number of characters by the number of lines.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}