Named Entity Chunker

[DarkmatterVale]

This PR adds a named entity chunker to Chatterbot's utilities.

[DarkmatterVale]

@gunthercox Any ideas why the travis-ci build isn't working? I tested it on my end and it works fine on my machine.

[gunthercox]

Not sure at the moment, looks like the issue is with numpy https://travis-ci.org/gunthercox/ChatterBot/jobs/105661593

[DarkmatterVale]

It is a weird error because it works fine on Python 3.4, 3.3, 3, and 2.7. Can you reproduce the error on your machine?

[gunthercox]

I cannot, it appears to work correctly for me as well (although I already have numpy installed).

What was the reason for adding numpy to the requirements file? I don't actually see it being imported anywhere.

[DarkmatterVale]

I read online that numpy was required to be on the system but not imported into the program you are using the chunker in. And I believe I got an error with out it in requirements.

[gunthercox]

I don't believe it's needed. I do see that it is included in NLTK's requirements, but they don't appear to be installing it on setup (https://github.com/nltk/nltk/blob/develop/pip-req.txt#L5). Their documentation does say that numpy is optional (http://www.nltk.org/install.html).

[DarkmatterVale]

When I remove numpy from requirements, it throws a "no module named numpy" error. But this is only on the pypy tests, so maybe it is something wrong on their end? I'm not really sure why it works with python 3.4, 3.3, 3, and 2.7 while it doesn't work with pypy; do they each run different tests?

[gunthercox]

Ah, interesting just found a note in the documentation for travis-ci (https://docs.travis-ci.com/user/ci-environment/#Preinstalled-pip-packages)

On all versions except pypy and pypy3 have numpy as well.

So the "issue" is occurring on travis's side. Travis normally has numpy preinstalled so that test builds can be created faster in the virtual environment. So far I have seen a lot of remarks that manually installing numpy and other scientific python packages significantly increases build time.

It should also be noted that there is currently known issues that will occur in a Windows environment when installing python modules with c extensions as requirements. This is the reason for instance that the python-Levenshtein package isn't required in this module.

There may be a way to include numpy in the .travis configuration file (similar to how it is done for mongodb: https://github.com/gunthercox/ChatterBot/blob/master/.travis.yml (Although I have not been able to find evidence of this in any of the documentation yet).

---

Also, I just noticed the test case for the named entity taggger (https://github.com/gunthercox/ChatterBot/pull/120/files#diff-87010571bbeeee0b9a13279c493670f3R42). This looks like a good negative test case (the case that no named entities are found). It is important to include a positive test case as well to show the case of the named entity tagger succeeding.

[gunthercox]

@DarkmatterVale I believe you can add numpy to the travis builds manually by manually installing it:

 install:
+  - sudo apt-get install python-numpy
   - pip install coveralls
-  - pip install nltk
   - pip install -r requirements.txt

(you can also remove the pip install nltk line as it is already included in the requirements.txt file.)

[DarkmatterVale]

Sorry for all the spam emails from Travis-Ci...Trying different things to see if numpy will install. I have not been able to find a solution so far.

@gunthercox Using sudo throws a permission error. If I remove sudo, apt-get doesn't work. I'll leave it at what you asked, however, so you can see the output of the most recent build yourself.

[gunthercox]

Sorry, my mistake.
The package should have been apt-get install python-numpy without the sudo. I believe travis while lists it so it can be installed this way without sudo.

[DarkmatterVale]

That does not work either, as is visible by the travis build.

[gunthercox]

The first line in .travis.yml is sudo: false, want to try removing that and adding back the sudo to the apt-get install line?

[DarkmatterVale]

@gunthercox Still doesn't work...

[kevin-brown]

In order to install packages using APT on the container based builds (so, by default) you need to use the APT addon.

As far as getting numpy installed under pypy, you may want to look at travis-ci/travis-ci#2650.

[DarkmatterVale]

@kevin-brown I added the APT addon as suggested, but it still doesn't build. Am I doing it incorrectly?