Parse_wikidump: programmatic access #28
Conversation
…ality, e.g., initialize db, and initiate downloads via code.
| help='Download snapshot if it does not exist as snapshot.xml.bz2. The corpus file name should match that of snapshot.') | ||
| parser.add_argument('-N', '--ngram', dest='ngram', default=7, type=int, | ||
| help='Maximum order of ngrams, set to None to disable [default: 7].') | ||
| args = parser.parse_args() |
There was a problem hiding this comment.
I think @IsaacHaze will be very unhappy when he sees his darling docopt replaced by ArgumentParser...
There was a problem hiding this comment.
Oh, I didn't know. I was inspired from how things are being done in xtas.
There was a problem hiding this comment.
That was written before I knew about docopt :)
There was a problem hiding this comment.
OK, so docopt is the current default for parsing args?
There was a problem hiding this comment.
Alright. I'll revert the arg parsing logic to use docopt, then.
|
I see how this is a useful feature, but I think it should be implemented differently, viz. by moving functionality from |
|
Good points. Let me work on it a bit more and will hear from me soon. |
…unction in __init__ that takes care of downloading.
|
But apart from the sadness, i'm all for this change (i did something similar in my joblib branch.) |
Refactored parse_wikidump to allow programmatic access to its functionality, e.g., initialize db, and initiate downloads via code.