This is a node console application that takes a single URL, fetches the HTML content, and parses out all internal links to discover the website's structure.
Node >= 20.10.0
Run nvm use
To install the appropriate version run npm install.
To run tests on code run npm test
To have the compiler listen for updates and auto compile on save run npm run dev. By default it will execute main.js within the dist directory on successful compilation of TS code within the src directory.
Run npm start https://www.example.com. This will start the WebCrawler and process the provided URL.