clean data for ldbc

[heroicNeZha]

some script to clean date for ldbc gen

clean-data.py will generate .copy file next to the csv file, clean-data.py can be repeated many times before executing copy-data.py

# this command gennerate xxx.csv.copy file
$ python3 clean-data.py -j 10 -i ../target/data/test_data/social_network/
handler-Thread-1 handler dynamic/person.csv.
handler-Thread-2 handler dynamic/post.csv.
handler-Thread-3 handler dynamic/person_workAt_organisation.csv.
handler-Thread-4 handler dynamic/person_likes_post.csv.
handler-Thread-5 handler dynamic/person_likes_comment.csv.
handler-Thread-6 handler dynamic/post_hasCreator_person.csv.
handler-Thread-7 handler dynamic/comment.csv.
handler-Thread-8 handler dynamic/forum_hasMember_person.csv.
handler-Thread-9 handler dynamic/person_hasInterest_tag.csv.
handler-Thread-10 handler dynamic/forum_containerOf_post.csv.
handler-Thread-1 handler dynamic/comment_hasTag_tag.csv.
handler-Thread-3 handler dynamic/post_hasTag_tag.csv.
handler-Thread-9 handler dynamic/comment_replyOf_comment.csv.
handler-Thread-3 handler dynamic/comment_isLocatedIn_place.csv.
handler-Thread-10 handler dynamic/person_isLocatedIn_place.csv.
handler-Thread-10 handler dynamic/post_isLocatedIn_place.csv.
handler-Thread-6 handler dynamic/comment_replyOf_post.csv.
handler-Thread-9 handler dynamic/forum.csv.
handler-Thread-4 handler dynamic/comment_hasCreator_person.csv.
handler-Thread-9 handler dynamic/forum_hasModerator_person.csv.
handler-Thread-9 handler dynamic/person_email_emailaddress.csv.
handler-Thread-9 handler dynamic/person_speaks_language.csv.
handler-Thread-9 handler dynamic/person_knows_person.csv.
handler-Thread-9 handler dynamic/person_studyAt_organisation.csv.
handler-Thread-9 handler dynamic/forum_hasTag_tag.csv.
handler-Thread-10 handler static/place.csv.
handler-Thread-10 handler static/tag.csv.
handler-Thread-10 handler static/tag_hasType_tagclass.csv.
handler-Thread-10 handler static/tagclass.csv.
handler-Thread-10 handler static/organisation.csv.
handler-Thread-10 handler static/tagclass_isSubclassOf_tagclass.csv.
handler-Thread-10 handler static/place_isPartOf_place.csv.
handler-Thread-10 handler static/organisation_isLocatedIn_place.csv.
all task done! please run copy-data.py to recover csv file

$ python3 copy-data.py  -i ../target/data/test_data/social_network/

[jackwener]

Need add new line

[jackwener]

It should be obtained through input parameters