Skip to content

Latest commit

 

History

History
44 lines (37 loc) · 1.34 KB

README.md

File metadata and controls

44 lines (37 loc) · 1.34 KB
Raw

Find inspiration in practice

magnet

Practice building a server-end & data-end through python.

Get Started

  • Step 1. you need a Python3 environment.
  • Step 2. you need a pip3 environment.
  • Step 3. you need to create virtualenvs.
  • Step 4. pip install following libs.
(magnet) [root@izuf69ibpdra8ui5a6edjtz magnet]# pipenv graph
cfscrape==2.0.7
  - requests [required: >=2.0.0, installed: 2.22.0]
    - certifi [required: >=2017.4.17, installed: 2019.6.16]
    - chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
    - idna [required: >=2.5,<2.9, installed: 2.8]
    - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.3]
Flask==1.1.1
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10.1, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.15, installed: 0.15.4]
gunicorn==19.9.0
lxml==4.3.4
selenium==3.141.0
  - urllib3 [required: Any, installed: 1.25.3]
  • Step 5. run the following command to launch server.
$ . run.sh

Content

  • Douban: web crawler from douban.com.
  • Thunder: web crawler from 22tu.cc.
  • Beiwo: web crawler from beiwo888.com.
  • Piaohua: web crawler from piaohua.com.
  • Baidu: web crawler from baidu.com.