Popular repositories Loading

1. MNBVC MNBVC Public

   MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

   3.5k 246

2. deduplication_mnbvc deduplication_mnbvc Public

   Forked from aplmikex/deduplication_mnbvc

   专业对大文本进行文本去重的工具

   Python 7

3. pytorch pytorch Public

   Forked from pytorch/pytorch

   Tensors and Dynamic neural networks in Python with strong GPU acceleration

   Python

4. publicRepos_mnbvc publicRepos_mnbvc Public

   Forked from washing1127/publicRepos_mnbvc

   Python

5. jsonlbugfix_mnbvc jsonlbugfix_mnbvc Public

   Forked from imgingroot/jsonl_check_fillter

   修复爬虫jsonl的bug

   Python

6. WikiHowQAExtractor-mnbvc WikiHowQAExtractor-mnbvc Public

   Forked from wanicca/WikiHowQAExtractor-mnbvc

   Extract Chinese/English QA Data from WikiHow pages.

   Python