IOError: [Errno 32] Broken pipe #14

sunningboy · 2015-12-31T06:27:10Z

OS is mac air OX.

How can solve it?

$ cat test.1k.txt | python python/ner/extractEntities2.py
/bin/sh: .//python/cap/cap_classify: cannot execute binary file
Traceback (most recent call last):
File "python/ner/extractEntities2.py", line 131, in
goodCap = capClassifier.Classify(words) > 0.9
File ".//python/cap/cap_classifier.py", line 33, in Classify
self.capClassifier.stdin.write("%s\n" % self.fe.Extract(' '.join(words)))
IOError: [Errno 32] Broken pipe

mdtareque · 2016-03-05T17:11:36Z

Even on Ubuntu 15.10

$ cat test.1k.txt | python python/ner/extractEntities2.py 
.//python/cap/cap_classify: 1: .//python/cap/cap_classify: Syntax error: ")" unexpected
Traceback (most recent call last):
  File "python/ner/extractEntities2.py", line 131, in <module>
    goodCap = capClassifier.Classify(words) > 0.9
  File ".//python/cap/cap_classifier.py", line 33, in Classify
    self.capClassifier.stdin.write("%s\n" % self.fe.Extract(' '.join(words)))
IOError: [Errno 32] Broken pipe

sruteesh · 2016-03-08T06:35:04Z

Any solutions??
Have the same issue

aritter · 2016-03-08T15:51:54Z

Maybe try running build.sh

-Alan

napsternxg · 2016-03-10T18:41:40Z

@sunningboy, @mdtareque and @sruteesh can you try using my branch mentioned in Pull request #16, it has a much simpler interface for using an input file instead of piping the input from cat.

sruteesh · 2016-03-11T10:24:00Z

Hi @napsternxg and @aritter
Thanks for the prompt response. But the issue still seems to be there. I'm still getting the same error. I tried trying it out on my friend's laptop. Same issue.

`$ python python/ner/extractEntities.py test.1k.txt

Starting with the following configuration

Input file: test.1k.txt
Text Position: 0
Output file: None
Chunk: False
POS: False
Event: False
Classify: False

Mallet Memory: 256m

No output file given. Will write to STDOUT.
Error: Could not find or load main class cc.mallet.fst.SimpleTaggerStdin
/bin/sh: .//python/cap/cap_classify: cannot execute binary file: Exec format error
Finished loading all models. Now reading from test.1k.txt and writing to None
Traceback (most recent call last):
File "python/ner/extractEntities.py", line 167, in
goodCap = capClassifier.Classify(words) > 0.9
File ".//python/cap/cap_classifier.py", line 33, in Classify
self.capClassifier.stdin.write("%s\n" % self.fe.Extract(' '.join(words)))
IOError: [Errno 32] Broken pipe
`
Please look into the matter.

chandra589 · 2016-03-17T16:36:15Z

@napsternxg and @aritter
I'm getting the same error as @sruteesh is having.
Can you please look into it.

napsternxg · 2016-03-17T16:59:57Z

@sruteesh @chandra589
I tried it just now on my machine. It works fine. Can you give details about your system ?

sruteesh · 2016-03-17T17:02:39Z

@napsternxg I am using pyhton 3.4 and running it on Cygwinx64 terminal from windows 10.

napsternxg · 2016-03-17T17:15:13Z

I think this code is compatible with only Python 2.7 because of the print statement formats. Try with Python 2.7.

sidewallme · 2016-04-11T01:34:43Z

I have the same issue. Any solutions?

joanaz · 2016-04-26T18:45:03Z

I'm having the same issue.

/bin/sh: .//python/cap/cap_classify: No such file or directory
Finished loading all models. Now reading from test.1k.txt and writing to output.txt
Traceback (most recent call last):
  File "python/ner/extractEntities.py", line 167, in <module>
    goodCap = capClassifier.Classify(words) > 0.9
  File ".//python/cap/cap_classifier.py", line 33, in Classify
    self.capClassifier.stdin.write("%s\n" % self.fe.Extract(' '.join(words)))
IOError: [Errno 32] Broken pipe

I also got an error when running the build.sh:

warning: optimization level '-O9' is not supported; using '-O3' instead
In file included from param.cpp:28:
In file included from ./common.h:76:
./getopt.h:131:12: error: conflicting types for 'getopt'
extern int getopt ();
           ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/unistd.h:508:6: note: 
      previous declaration is here
int      getopt(int, char * const [], const char *) __DARWIN_ALIAS(getopt);
         ^
param.cpp:217:17: warning: conversion from string literal to 'char *' is
      deprecated [-Wc++11-compat-deprecated-writable-strings]
    char *tmp = "TinySVM::Param::set";
                ^
2 warnings and 1 error generated.
make[2]: *** [param.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive-am] Error 2
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I'm using python 2.7.

rksaxena · 2016-06-03T11:23:13Z

Anybody got a solution? Facing the same issue

vikotse · 2016-07-22T09:16:18Z

I have the same issue.

`

Starting with the following configuration

Input file: test.1k.txt
Text Position: 0
Output file: output.txt
Chunk: False
POS: False
Event: False
Classify: False

Mallet Memory: 256m

/bin/sh: 1: java: not found
Finished loading all models. Now reading from test.1k.txt and writing to output.txt
Traceback (most recent call last):
File "python/ner/extractEntities.py", line 197, in
ner.stdin.write(("\t".join(seq_features) + "\n").encode('utf8'))
IOError: [Errno 32] Broken pipe
`

After I tried to run build.sh, this issue is still existing.

My python version: 2.7.12

DerekZH · 2016-10-27T01:19:33Z

same issue here, python version is Python 2.7.10. Would be great to see a fix. @napsternxg @aritter

DaehanKim · 2017-06-28T10:31:00Z

I have some similar issue.

Anonymous:~/twitter_nlp$ python python/ner/extractEntities.py test.1k.txt -o output.txtStarting with the following configuration
----------------------------------------
Input file: test.1k.txt
Text Position: 0
Output file: output.txt
Chunk: False
POS: False
Event: False
Classify: False
Mallet Memory: 256m
----------------------------------------
/bin/sh: 1: java: not found
Finished loading all models. Now reading from test.1k.txt and writing to output.txt
Traceback (most recent call last):
  File "python/ner/extractEntities.py", line 197, in <module>
    ner.stdin.write(("\t".join(seq_features) + "\n").encode('utf8'))
IOError: [Errno 32] Broken pipe

And I found out that running ./build.sh cause some errors like :

...

rm -f .libs/param.lo
c++ -DHAVE_CONFIG_H -I. -I. -I.. -Wall -O9 -funroll-all-loops -finline -ffast-math -c param.cpp  -fPIC -DPIC -o .libs/param.lo
param.cpp: In member function 'int TinySVM::Param::set(const char*)':
param.cpp:217:17: warning: deprecated conversion from string constant to 'char*' [-Wwrite-strings]
     char *tmp = "TinySVM::Param::set";
                 ^
c++ -DHAVE_CONFIG_H -I. -I. -I.. -Wall -O9 -funroll-all-loops -finline -ffast-math -c param.cpp -o param.o >/dev/null 2>&1
mv -f .libs/param.lo param.lo

...

rm -f .libs/model.lo
c++ -DHAVE_CONFIG_H -I. -I. -I.. -Wall -O9 -funroll-all-loops -finline -ffast-math -c model.cpp  -fPIC -DPIC -o .libs/model.lo
model.cpp: In member function 'virtual int TinySVM::Model::read(const char*, const char*, int)':
model.cpp:230:57: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%s Version %s%*[^\n]\n", tmpbuf, version);
                                                         ^
model.cpp:231:50: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%d%*[^\n]\n",  &param.kernel_type);
                                                  ^
model.cpp:232:45: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%d%*[^\n]\n",  &param.degree);
                                             ^
model.cpp:233:46: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%lf%*[^\n]\n", &param.param_g);
                                              ^
model.cpp:234:46: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%lf%*[^\n]\n", &param.param_s);
                                              ^
model.cpp:235:46: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%lf%*[^\n]\n", &param.param_r);
                                              ^
model.cpp:236:38: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%s%*[^\n]\n",  tmpbuf);
                                      ^
model.cpp:249:34: warning: ignoring return value of 'int fscanf(FILE*, const char*, ...)', declared with attribute warn_unused_result [-Wunused-result]
   fscanf (fp, "%lf%*[^\n]\n", &b);
                                  ^

jamestch · 2017-07-28T02:37:47Z

I hava met the similar problem, has anyone get a solution?
when I run "python python/ner/extractEntities.py test.1k.txt -o output.txt", it got an exception as below:

Starting with the following configuration

Input file: test.1k.txt
Text Position: 0
Output file: output.txt
Chunk: False
POS: False
Event: False
Classify: False
Mallet Memory: 256m

Exception in thread "main" java.lang.NoClassDefFoundError: bsh/Interpreter
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at cc.mallet.util.CommandOption.(CommandOption.java:62)
at cc.mallet.util.CommandOption$Double.(CommandOption.java:483)
at cc.mallet.fst.SimpleTaggerStdin.(SimpleTaggerStdin.java:173)
Caused by: java.lang.ClassNotFoundException: bsh.Interpreter
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more
Finished loading all models. Now reading from test.1k.txt and writing to output.txt
Traceback (most recent call last):
File "python/ner/extractEntities.py", line 197, in
ner.stdin.write(("\t".join(seq_features) + "\n").encode('utf8'))
IOError: [Errno 32] Broken pipe

Johnsonxiong · 2017-12-04T05:18:10Z

Hey, guys, good news for Mac users!

I ended up being able to build on Mac (10.12.6) by doing the following:

Download TinySVM and unzip it to the folder of twitter_nlp-master:
https://github.com/shogo82148/TinySVM

Change build.sh in twitter_nlp-master as below:
cd hbc/models
gcc -O3 labels.c stats.c samplib.c LabeledLDA_infer_stdin.c -o LabeledLDA_infer_stdin.out -lm
cd ../../TinySVM
./configure --prefix=pwd/../ && make && make install
cd ../python/cap
./build.sh

Then change the python/cap/build.sh into like below:

c++ -o cap_classify cap_classify.cpp -ltinysvm

Then run
$ ./build.sh
$ export TWITTER_NLP=./
$ cat test.1k.txt | python2 python/ner/extractEntities2.py

Dont forget to install tinysvm anyway using :
$ brew install tinysvm

This works!
Below is part of the result:
@Jessica_Chobot/O did/O you/O see/O the/O yakuza/B-ENTITY vs/O zombies/O ..../O smh/O but/O cool/O at/O the/O same/O time/O
RT/O @daviddesrosiers/O :/O Happy/O birthday/O @chuckcomeau/O !/O have/O fun/O in/O vancouver/B-ENTITY tonight/O !/O
Spotted/O :/O Kanye/B-ENTITY West/I-ENTITY Celebrates/O LAMB/B-ENTITY With/O Gwen/B-ENTITY Stefani/I-ENTITY :/O New/B-ENTITY York/I-ENTITY Fashion/I-ENTITY Week/I-ENTITY is/O coming/O to/O a/O close/O ,/O but/O not/O before/O .../O http://bit.ly/cSyZUi/O
@zeeDOTi/O i/O might/O join/O in/O if/O I/O make/O it/O home/O in/O time/O ./O :)/O

Hope this helps.

vrnmthr · 2019-06-17T18:50:48Z

@Johnsonxiong worked for me -- thanks! maybe someone can create a PR for this.

the only thing I had to change was in the $PWD reference in ./build.sh:

cd hbc/models
gcc -O3 labels.c stats.c samplib.c LabeledLDA_infer_stdin.c -o LabeledLDA_infer_stdin.out -lm
cd ../../TinySVM
./configure --prefix=$PWD/../ && make && make install
cd ../python/cap
./build.sh

chandrasg · 2020-08-14T22:56:59Z

TinySVM no longer has configure file it seems:
https://github.com/shogo82148/TinySVM

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IOError: [Errno 32] Broken pipe #14

IOError: [Errno 32] Broken pipe #14

sunningboy commented Dec 31, 2015

mdtareque commented Mar 5, 2016

sruteesh commented Mar 8, 2016

aritter commented Mar 8, 2016

napsternxg commented Mar 10, 2016

sruteesh commented Mar 11, 2016

chandra589 commented Mar 17, 2016

napsternxg commented Mar 17, 2016

sruteesh commented Mar 17, 2016

napsternxg commented Mar 17, 2016

sidewallme commented Apr 11, 2016

joanaz commented Apr 26, 2016 •

edited

Loading

rksaxena commented Jun 3, 2016

vikotse commented Jul 22, 2016 •

edited

Loading

DerekZH commented Oct 27, 2016 •

edited

Loading

DaehanKim commented Jun 28, 2017 •

edited

Loading

jamestch commented Jul 28, 2017 •

edited

Loading

Johnsonxiong commented Dec 4, 2017 •

edited

Loading

vrnmthr commented Jun 17, 2019 •

edited

Loading

chandrasg commented Aug 14, 2020

IOError: [Errno 32] Broken pipe #14

IOError: [Errno 32] Broken pipe #14

Comments

sunningboy commented Dec 31, 2015

How can solve it?

mdtareque commented Mar 5, 2016

sruteesh commented Mar 8, 2016

aritter commented Mar 8, 2016

napsternxg commented Mar 10, 2016

sruteesh commented Mar 11, 2016

Starting with the following configuration

Mallet Memory: 256m

chandra589 commented Mar 17, 2016

napsternxg commented Mar 17, 2016

sruteesh commented Mar 17, 2016

napsternxg commented Mar 17, 2016

sidewallme commented Apr 11, 2016

joanaz commented Apr 26, 2016 • edited Loading

rksaxena commented Jun 3, 2016

vikotse commented Jul 22, 2016 • edited Loading

Starting with the following configuration

Mallet Memory: 256m

DerekZH commented Oct 27, 2016 • edited Loading

DaehanKim commented Jun 28, 2017 • edited Loading

jamestch commented Jul 28, 2017 • edited Loading

Starting with the following configuration

Input file: test.1k.txt Text Position: 0 Output file: output.txt Chunk: False POS: False Event: False Classify: False Mallet Memory: 256m

Johnsonxiong commented Dec 4, 2017 • edited Loading

vrnmthr commented Jun 17, 2019 • edited Loading

chandrasg commented Aug 14, 2020

joanaz commented Apr 26, 2016 •

edited

Loading

vikotse commented Jul 22, 2016 •

edited

Loading

DerekZH commented Oct 27, 2016 •

edited

Loading

DaehanKim commented Jun 28, 2017 •

edited

Loading

jamestch commented Jul 28, 2017 •

edited

Loading

Input file: test.1k.txt
Text Position: 0
Output file: output.txt
Chunk: False
POS: False
Event: False
Classify: False
Mallet Memory: 256m

Johnsonxiong commented Dec 4, 2017 •

edited

Loading

vrnmthr commented Jun 17, 2019 •

edited

Loading