Skip to content

Scrape the MDPI Open Access Journal using Selenium and BeautifulSoup4

Notifications You must be signed in to change notification settings

neeeal/mdpi-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Scrape the MDPI Open Access Journal System

Used for scraping and text processing practice.

Six (6) Journals are scraped with a total of 12,000+ article references.

How to use

Open terminal from repository folder. Assuming you have python installed or am in a virtual environment, run the following on the terminal:

  1. install requirements pip install -r requirements.txt

  2. run scraper python main.py

  3. run csv processor python process_csv.py

Note:
The labels for each of the journals stands for:
1 === Social Science
2 === Education
3 === Engineering and Technology

About

Scrape the MDPI Open Access Journal using Selenium and BeautifulSoup4

Topics

Resources

Stars

Watchers

Forks

Languages