Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS lambda and levelDB #37

Open
ghost opened this issue Feb 6, 2018 · 4 comments
Open

AWS lambda and levelDB #37

ghost opened this issue Feb 6, 2018 · 4 comments

Comments

@ghost
Copy link

ghost commented Feb 6, 2018

Hello,

I want to use this code in AWS lambda function but the problen is that the levelDB doesn't store the db in the folder. it can store the following files:

000003.log size: 8830932
CURRENT size: 16
IndexControllerDB.json size: 118
LOCK size: 0
LOG size: 57
MANIFEST-000002 size: 50

but it can't save .ldb file. is there any way to full print the debugging stack to see what is going wrong.

thanks

@larsvoigt
Copy link
Owner

Hi,
okay I have not tried to run epub-search on AWS yet :-).

  • Which distribution is used?
  • Which node version?
  • npm i was run without errors?

Can you please have a look for this file EPUB-search.log on your instance.

Or you can start the service manually (first stop running service epub-search stop)

node  {your_install_path}/epub-full-text-search/lib/WebService.js

in order to see the log output...

Lars

@ghost
Copy link
Author

ghost commented Feb 7, 2018

hello,
thanks for reply.
I am using AWS lambda which use Docker container, and this docker is "Amazon Linux AMI" and kernel version is 3.10.0-693.11.1.el7.x86_64, but it may change because I have no control to choose a distribution .
node --version: v6.10.3
All files will be deleted after the AWS lambda timeout so I have local AWS and I test it and after indexing EPUB-search.log is:

bash-4.2$ cat EPUB-search.log

{"meta":"","level":"info","message":"","timestamp":"2018-01-30T13:41:23.001Z"}
{"level":"info","message":"EPUB-search Copyright (c) 2015-2017 Lars Voigt","timestamp":"2018-01-30T13:41:23.006Z"}
{"level":"info","message":"This program comes with ABSOLUTELY NO WARRANTY.","timestamp":"2018-01-30T13:41:23.006Z"}
{"level":"info","message":"This is free software, and you are welcome to redistribute it under certain conditions.","timestamp":"2018-01-30T13:41:23.007Z"}
{"level":"info","message":"For the full license, please visit: https://opensource.org/licenses/MIT \n","timestamp":"2018-01-30T13:41:23.007Z"}
{"level":"info","message":"Tue Jan 30 2018 14:41:23 GMT+0100 (W. Europe Standard Time): EPUB search started on undefined:8085 ... \n","timestamp":"2018-01-30T13:41:23.025Z"}
{"level":"info","message":"client request search","timestamp":"2018-01-30T13:41:45.584Z"}
{"level":"info","message":"REQ: [object Object]","timestamp":"2018-01-30T13:41:45.587Z"}
{"level":"info","message":"RES: [object Object]","timestamp":"2018-01-30T13:41:45.589Z"}
{"level":"info","message":"\u001b[34mrequest search\u001b[39m -> query: math -> bookTitle: * -> uuid: -1","timestamp":"2018-01-30T13:41:45.590Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml","level":"error","timestamp":"2018-01-30T13:41:46.175Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml","level":"error","timestamp":"2018-01-30T13:41:46.179Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li6.html'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li6.html'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li6.html","level":"error","timestamp":"2018-01-30T13:41:46.186Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02.xhtml","level":"error","timestamp":"2018-01-30T13:41:46.190Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s02.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s02.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s02.xhtml","level":"error","timestamp":"2018-01-30T13:41:46.190Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li66.html'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li66.html'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li66.html","level":"error","timestamp":"2018-01-30T13:41:46.195Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li65.html'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li65.html'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li65.html","level":"error","timestamp":"2018-01-30T13:41:46.199Z"}
{"level":"info","message":"","timestamp":"2018-01-30T13:42:14.197Z"}
{"level":"info","message":"Start normalize EPUB content","timestamp":"2018-01-30T13:42:14.197Z"}
{"level":"info","message":"\u001b[34mIndexing local path\u001b[39m","timestamp":"2018-01-30T13:42:14.198Z"}
{"message":"Cant index empty local: epub3-samples\\linear-algebra","stack":"Error: Cant index empty local: epub3-samples\linear-algebra\n at Object.Normalize.local (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\Normalize.js:34:35)\n at Object.SearchEngine.indexing (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\SearchEngine.js:77:40)\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\Rest-API\AddToIndex.js:23:12\n at _fulfilled (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:854:54)\n at self.promiseDispatch.done (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:883:30)\n at Promise.promise.promiseDispatch (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:816:13)\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:624:44\n at runSingle (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:137:13)\n at flush (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:125:13)\n at _combinedTickCallback (internal/process/next_tick.js:73:7)","level":"error","timestamp":"2018-01-30T13:42:14.207Z"}
{"message":"IO error: LockFile C:\Users\daoud\.epub-full-text-search/LOCK: The process cannot access the file because it is being used by another process.\r\n","type":"OpenError","name":"OpenError","cause":{},"stack":"OpenError: IO error: LockFile C:\Users\daoud\.epub-full-text-search/LOCK: The process cannot access the file because it is being used by another process.\r\n\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\levelup\lib\levelup.js:117:34\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\abstract-leveldown\abstract-leveldown.js:39:16","level":"error","timestamp":"2018-01-30T13:42:48.206Z"}
{"message":"IO error: LockFile C:\Users\daoud\.epub-full-text-search/LOCK: The process cannot access the file because it is being used by another process.\r\n","type":"OpenError","name":"OpenError","cause":{},"stack":"OpenError: IO error: LockFile C:\Users\daoud\.epub-full-text-search/LOCK: The process cannot access the file because it is being used by another process.\r\n\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\levelup\lib\levelup.js:117:34\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\abstract-leveldown\abstract-leveldown.js:39:16","level":"error","timestamp":"2018-01-30T13:42:53.460Z"}
{"meta":"","level":"info","message":"","timestamp":"2018-01-30T13:43:11.361Z"}
{"level":"info","message":"EPUB-search Copyright (c) 2015-2017 Lars Voigt","timestamp":"2018-01-30T13:43:11.365Z"}
{"level":"info","message":"This program comes with ABSOLUTELY NO WARRANTY.","timestamp":"2018-01-30T13:43:11.369Z"}
{"level":"info","message":"This is free software, and you are welcome to redistribute it under certain conditions.","timestamp":"2018-01-30T13:43:11.369Z"}
{"level":"info","message":"For the full license, please visit: https://opensource.org/licenses/MIT \n","timestamp":"2018-01-30T13:43:11.369Z"}
{"level":"info","message":"Tue Jan 30 2018 14:43:11 GMT+0100 (W. Europe Standard Time): EPUB search started on undefined:8085 ... \n","timestamp":"2018-01-30T13:43:11.393Z"}
{"level":"info","message":"
","timestamp":"2018-01-30T13:43:15.115Z"}
{"level":"info","message":"Start normalize EPUB content","timestamp":"2018-01-30T13:43:15.116Z"}
{"level":"info","message":"\u001b[34mIndexing local path\u001b[39m","timestamp":"2018-01-30T13:43:15.124Z"}
{"message":"Cant index empty local: node_modules\\epub3-samples\\linear-algebra","stack":"Error: Cant index empty local: node_modules\epub3-samples\linear-algebra\n at Object.Normalize.local (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\Normalize.js:34:35)\n at Object.SearchEngine.indexing (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\SearchEngine.js:77:40)\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\Rest-API\AddToIndex.js:23:12\n at _fulfilled (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:854:54)\n at self.promiseDispatch.done (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:883:30)\n at Promise.promise.promiseDispatch (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:816:13)\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:624:44\n at runSingle (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:137:13)\n at flush (C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\q\q.js:125:13)\n at _combinedTickCallback (internal/process/next_tick.js:73:7)","level":"error","timestamp":"2018-01-30T13:43:15.147Z"}
{"message":"IO error: LockFile C:\Users\daoud\.epub-full-text-search/LOCK: The process cannot access the file because it is being used by another process.\r\n","type":"OpenError","name":"OpenError","cause":{},"stack":"OpenError: IO error: LockFile C:\Users\daoud\.epub-full-text-search/LOCK: The process cannot access the file because it is being used by another process.\r\n\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\levelup\lib\levelup.js:117:34\n at C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\abstract-leveldown\abstract-leveldown.js:39:16","level":"error","timestamp":"2018-01-30T13:44:07.458Z"}
{"meta":"","level":"info","message":"","timestamp":"2018-01-30T13:44:13.053Z"}
{"level":"info","message":"EPUB-search Copyright (c) 2015-2017 Lars Voigt","timestamp":"2018-01-30T13:44:13.060Z"}
{"level":"info","message":"This program comes with ABSOLUTELY NO WARRANTY.","timestamp":"2018-01-30T13:44:13.060Z"}
{"level":"info","message":"This is free software, and you are welcome to redistribute it under certain conditions.","timestamp":"2018-01-30T13:44:13.060Z"}
{"level":"info","message":"For the full license, please visit: https://opensource.org/licenses/MIT \n","timestamp":"2018-01-30T13:44:13.072Z"}
{"level":"info","message":"Tue Jan 30 2018 14:44:13 GMT+0100 (W. Europe Standard Time): EPUB search started on undefined:8085 ... \n","timestamp":"2018-01-30T13:44:13.095Z"}
{"level":"info","message":"","timestamp":"2018-01-30T13:44:15.526Z"}
{"level":"info","message":"Start normalize EPUB content","timestamp":"2018-01-30T13:44:15.553Z"}
{"level":"info","message":"\u001b[34mIndexing local path\u001b[39m","timestamp":"2018-01-30T13:44:15.569Z"}
{"level":"info","message":"EPUB data local: \u001b[31mC:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\node_modules\epub3-samples\u001b[39m","timestamp":"2018-01-30T13:44:15.582Z"}
{"level":"info","message":"Analyse local:","timestamp":"2018-01-30T13:44:15.949Z"}
{"level":"info","message":"\t--> EPUB title \u001b[31mAccessible EPUB 3\u001b[39m already indexed","timestamp":"2018-01-30T13:44:15.983Z"}
{"level":"info","message":"\t--> EPUB title \u001b[31mA First Course in Linear Algebra\u001b[39m already indexed","timestamp":"2018-01-30T13:44:15.984Z"}
{"level":"info","message":"Ready with normalize EPUB","timestamp":"2018-01-30T13:44:15.994Z"}
{"level":"info","message":"DONE! Nothing to do, EPUBs already indexed.","timestamp":"2018-01-30T13:44:16.001Z"}
{"level":"info","message":"DONE! EPUB is indexed.","timestamp":"2018-01-30T13:44:16.032Z"}
{"level":"info","message":"client request search","timestamp":"2018-01-30T13:44:33.891Z"}
{"level":"info","message":"REQ: [object Object]","timestamp":"2018-01-30T13:44:33.899Z"}
{"level":"info","message":"RES: [object Object]","timestamp":"2018-01-30T13:44:33.899Z"}
{"level":"info","message":"\u001b[34mrequest search\u001b[39m -> query: math -> bookTitle: * -> uuid: -1","timestamp":"2018-01-30T13:44:33.903Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml","level":"error","timestamp":"2018-01-30T13:44:34.018Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s03.xhtml","level":"error","timestamp":"2018-01-30T13:44:34.023Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li6.html'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li6.html'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li6.html","level":"error","timestamp":"2018-01-30T13:44:34.050Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02.xhtml","level":"error","timestamp":"2018-01-30T13:44:34.058Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s02.xhtml'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s02.xhtml'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\accessible_epub_3\EPUB\ch02s02.xhtml","level":"error","timestamp":"2018-01-30T13:44:34.073Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li66.html'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li66.html'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li66.html","level":"error","timestamp":"2018-01-30T13:44:34.077Z"}
{"message":"ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li65.html'","stack":"Error: ENOENT: no such file or directory, open 'C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li65.html'\n at Error (native)","errno":-4058,"code":"ENOENT","syscall":"open","path":"C:\MyData\epub_search\searchEngine\epub-full-text-search-original - Copy\target\node_modules\epub3-samples\linear-algebra\EPUB\xhtml\fcla-xml-2.30li65.html","level":"error","timestamp":"2018-01-30T13:44:34.089Z"}
{"meta":"","level":"info","message":"","timestamp":"2018-01-30T13:46:45.989Z"}
{"level":"info","message":"EPUB-search Copyright (c) 2015-2017 Lars Voigt","timestamp":"2018-01-30T13:46:45.993Z"}
{"level":"info","message":"This program comes with ABSOLUTELY NO WARRANTY.","timestamp":"2018-01-30T13:46:45.994Z"}
{"level":"info","message":"This is free software, and you are welcome to redistribute it under certain conditions.","timestamp":"2018-01-30T13:46:45.995Z"}
{"level":"info","message":"For the full license, please visit: https://opensource.org/licenses/MIT \n","timestamp":"2018-01-30T13:46:45.995Z"}
{"level":"info","message":"Tue Jan 30 2018 14:46:46 GMT+0100 (Mitteleuropäische Zeit): EPUB search started on undefined:8085 ... \n","timestamp":"2018-01-30T13:46:46.014Z"}
{"level":"info","message":"
","timestamp":"2018-01-30T13:47:27.792Z"}
{"level":"info","message":"Start normalize EPUB content","timestamp":"2018-01-30T13:47:27.792Z"}
{"level":"info","message":"\u001b[34mIndexing local path\u001b[39m","timestamp":"2018-01-30T13:47:27.796Z"}
{"level":"info","message":"EPUB data local: \u001b[31mC:\MyData\epub_search\searchEngine\search-nodejs-command\node_modules\epub3-samples\linear-algebra\u001b[39m","timestamp":"2018-01-30T13:47:27.796Z"}
{"level":"info","message":"Analyse local:","timestamp":"2018-01-30T13:47:27.943Z"}
{"level":"info","message":"\t--> EPUB title \u001b[34m\u001b[1mA First Course in Linear Algebra\u001b[22m\u001b[39m will be added to index","timestamp":"2018-01-30T13:47:27.994Z"}
{"level":"info","message":"Ready with normalize EPUB","timestamp":"2018-01-30T13:47:32.735Z"}
{"level":"info","message":"Start normalize EPUB-data.","timestamp":"2018-01-30T13:47:32.757Z"}
{"level":"info","message":"DONE! EPUB is indexed.","timestamp":"2018-01-30T13:47:37.622Z"}
{"level":"info","message":"client request search","timestamp":"2018-01-30T13:48:17.287Z"}
{"level":"info","message":"REQ: [object Object]","timestamp":"2018-01-30T13:48:17.290Z"}
{"level":"info","message":"RES: [object Object]","timestamp":"2018-01-30T13:48:17.292Z"}
{"level":"info","message":"\u001b[34mrequest search\u001b[39m -> query: math -> bookTitle: * -> uuid: -1","timestamp":"2018-01-30T13:48:17.294Z"}

@larsvoigt
Copy link
Owner

larsvoigt commented Feb 12, 2018

Hello, i tested epub-search within a docker container and it works for me.

# Pull base image.
FROM ubuntu:16.04

# Install Node.js
RUN apt-get update && apt-get install -y curl 
RUN apt-get install -y sudo && rm -rf /var/lib/apt/lists/*
RUN curl --silent --location https://deb.nodesource.com/setup_8.x | sudo bash -
RUN apt-get install -y nodejs
RUN apt-get install -y build-essential

RUN mkdir epub-full-text-search 
WORKDIR epub-full-text-search 

# RUN npm i epub-full-text-search -g --unsafe
RUN npm i epub-full-text-search 

EXPOSE  8085
# CMD ["epub-search", "start"]
CMD ["node", "node_modules/epub-full-text-search/dist/lib/WebService.js"]

Build & Run image

$ docker build -t test .
$ docker run -p 8085:8085 -d test

Indexing via browser

http://localhost:8085/addToIndex?url=https://readium.firebaseapp.com/epub_content/accessible_epub_3/&uuid=10a51e32-cc8d-4ba2-b1e4-a567580db4fe

Search via browser

http://localhost:8085/search?q=epub&uuid=10a51e32-cc8d-4ba2-b1e4-a567580db4fe

Hope it helps a little bit :-)...

@ghost
Copy link
Author

ghost commented Feb 13, 2018

Hello,
thanks for your help. Just like I said, the problem with AWS lambda function that it works in Docker but the docker containter will be deleted after function timeout so I need another place to store levelDB files.

I looked deeper in the code and I noticed that you are creating instance from SearchEngine with no option "searchEngine({})" and this emtpy option will be assigned to

return searchIndex(options)
.then(si => {
SearchEngine.si = si;
SearchEngine._search = Q.nbind(si.search, si);
SearchEngine._close = Q.nbind(si.close, si);
SearchEngine._match = Q.nbind(si.match, si);
SearchEngine._add = Q.nbind(si.add, si);
SearchEngine._del = Q.nbind(si.del, si);
SearchEngine._get = Q.nbind(si.get, si);
SearchEngine._flush = Q.nbind(si.flush, si);
return SearchEngine;
});

and from searchIndex, I noticed that I can you my own implementation of levelDB.
https://github.com/fergiemcdowall/search-index/blob/master/docs/API.md#db

so I will use another instance of levelDB that support any AWS service for storing/saving data and I hope it works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant