Lab Project 1: Building a Tokenizer

Introduction ヾ(⌐□_□)ノ♪

Welcome, weary traveller. My objective for the first lab project is to build a tokenizer that maintains a history. Tokenizing is the process of dividing a string into tokens by some delimeter (e.g. the string librarys split() function). Consider tokenizing the string “The cake is a lie!” using the space character. This would result in: [“The”, “cake”, “is”, “a”, “lie!”]. This lab isn't much, but it's honest work.

Compiling & Running the Code

To compile & execute the user interface program, open the command prompt and enter

cd tokenizer-GigaMatt-1\
sudo make
./tokenizer.o

Dependencies

1. The SystemsVM virtual machine provided by UTEP CS 3432 OR
   
2. A Linux-based virtual machine.
   

Contributions

1. Daniel Cervantes (pointing me to use fgets() for user input) & booleans in C are integer 0s/1s.
   
2. Wan Koo for teaching me the differences between <.h> & ".h" imports.
   

Warnings

1. Although the dependencies call for the use of a virual machine, you may use a local Linux OS to compile & execute this code. WARNING This should be reserved for advanced users.