Can we resurrect node-packer to get it working with latest Node.js versions? #145

saurabh-deep · 2020-05-04T09:13:06Z

Hey @pmq20! Hope you are doing well.

Is there a possibility of resurrecting this project? It has been a very popular project with some forks even managing to get this working till Node10. But it seems we are at the end of the road as I am not aware if anyone has had any success with Node12.

There are not many other options available to create Node.js binary and all of them have issues with the OS X signing/notarization and the binaries are too big.

I understand you being busy with your current job and projects. Still wanted to check if it was a possibility to bring this project back to life.

Many thanks! Cheers!

saurabh-deep · 2020-05-06T10:16:29Z

@pmq20 - I thought more about it and I feel that it can also be a viable option to pass on the knowledge to the community so that someone from the community can carry forward this great tool. Thoughts?

saurabh-deep · 2020-06-03T06:19:14Z

Hello @pmq20! I remember you mentioning on the Node 10 PR about working on upgrade to Node 12. Just wanted to follow up on the same. Do you have an approx time frame in mind for that? Would very much appreciate your response on this. :-)

saurabh-deep · 2020-08-18T12:11:50Z

Hey @pmq20! I had been looking forward to getting the Node12 support after your comment on the Node10 PR. But I just noticed in the about section that this repository has stopped maintenance.

Does that mean this project is officially dead? I hope NOT. If yes, it is very unfortunate. There are no active forks also.

There are a few options available to generate Node.js binary like pkg, nexe etc. But none of them close to the quality of the binary created by node-packer, without any custom coding being required.

Will it be possible for you to create a how to doc for me to try and take this forward? This will be a massive help.

Eagerly looking forward to your response.

pmq20 · 2020-08-18T18:07:16Z

@saurabh-deep Thank you for the feedback! Your feedback brings a lot of encouragement to me. I really appreciate it. I previously put up the stop-maintenance notice because there seems to be no time to maintain it for me personally due to my weekday work. But it does appear that the project is still relevant and many people want to use it, so I should continue maintaining it. Hence I have deleted that notice.

I also do agree that community is the key to keep the project alive, because one person's time is too limited. Please feel free to ask me any technical questions about its implementation. Everybody's participation is greatly welcome and appreciated!

Let me start resurrecting it this weekend (because of work I can only work on it in the weekends). I think the first thing to do is to upgrade it to node.js v14. I'll let you know what I accomplished after this weekend.

saurabh-deep · 2020-08-18T18:50:00Z

@pmq20 - Best news I have had in a while. 😄

v14 is great! We will also need v12 - but surely, that can come later. Please do let me know if I can help in any way.

These are some the node-packer positives which aren't to be found in any other packaging solution -

No custom coding and adjustments needed. All the node code and npm dependencies work as they are (at least for me).
Fairly small size. Actually less than half of what pkg and nexe create.
Best of the lot - Finally binary can be successfully signed for Mac OS and Windows both. I haven't seen any other tool which creates correctly formatted Mac OS binaries.

pmq20 · 2020-08-23T04:16:48Z

@saurabh-deep Thanks for the compliments. I am really glad that people actually love this tool. I tried upgrading to 14.8.0 this weekend and it was easier than I thought. I have reviewed all the hacks, and only kept the essential ones. They were gathered in this commit: a4ab86d

You can see that there's not too many essential hacks, so it is promising that we can maintain the project in the long run without too much burden. Of course the heavy lifting still happens in libsquash, but that part is easier to maintain (just copy and paste). But I have not actually tested if the old hacks still work for the new Node.js. Let me do that next week with the goal of setting up a CI to check the quality of the tool. Also I plan to make CI build and release nodec itself automatically, just like what I did in Ruby Packer.

saurabh-deep · 2020-08-23T09:11:49Z

This is wonderful @pmq20. I will give it a spin once the latest nodec with Node v14.8.0. executable is ready. Or do you think I can try directly with the source like we have been doing for a while since nodec was still at Node v8.x? Something like the below. Or do you think I should wait a little since your current changes are a WIP?

ruby ~/build/node-packer/bin/nodec --tmpdir=~/build/compile --output=~/build/a.out bin/index.js --skip-npm-install

Second, lets say I want to build nodec for the current Node.js LTE version (v12.18.3). What should be my steps?

Download Node.js v12.18.3 from https://github.com/nodejs/node/tree/v12.18.3
Put the Node.js downloaded code in the node directory of node-packer
Apply Enclose.IO hacks (As per commit a4ab86d)
Any other steps?
Compile all the new stuff into a new version of nodec - how to do this?

Also, I wanted to point out that Python 2 is End of Life now. Can we upgrade to Python 3 now somehow or do you think it will cause an issue?

I also noticed that you added cross-compilation to the README.md. That is a very welcome change. I haven't tried it yet but will certainly do once you think it is ready for a test drive.

Once again, many thanks for hard work and bringing this project to life! 🙇‍♂️

I will let others in the community know about this progress so that more people can banefit from your hard work and stop compromising on features (like Mac OS code-signing) which do not work correctly using any other tool.

Please let me know about my questions on building a new nodec for Node.js v12.18.3 and I will try and contribute on that as soon as my schedule allows.

pmq20 · 2020-08-24T21:51:07Z

@saurabh-deep Yeah, as a user one should probably wait a little since the current changes are WIP; unless one wish to try out the bleeding edge and contribute to the WIP development, which is greatly welcome.

Yeah that is pretty much the steps. To compile all the new stuff into a new version of nodec (standalone binary), you need to use Ruby Packer. Otherwise you can just use the ruby command you mentioned above on the checked-out code.

Thanks for the info on Python 3. I imagine it might already works, as I cannot recall any special usage of python 2 specifically.

To support both Node 12 and Node 14, I plan to maintain two node folders (https://github.com/pmq20/node-packer/tree/master/node). Perhaps we can name them lts and current.

I wish I had more time to work on it. Let me keep working on it the next weekend.

saurabh-deep · 2020-09-08T07:50:57Z

@pmq20 - I will be compiling my binary with Node 12 this week. For that, I will try to use the lts folder and use ruby command for the compilation since nodec is not packed yet. I will let you know how that goes. Keeping the fingers crossed. Thank you very much for bringing it this far.

pmq20 · 2020-09-08T18:21:15Z

@saurabh-deep Last time I tried, There was some errors when running that needs to be fixed. I plan to use some coming weekends to fix them. Also, if possible, could you share the project that you were trying to pack, so that I can use it as a reference to test the packing process? Thank you again for your feedback!

saurabh-deep · 2020-09-08T18:57:43Z

@pmq20 - My project is in a private (company-owned) GitLab repository, which I am not authorised to share, unfortunately. Apologies for this! But I will setup a similar public project on GitHub that you can use as a reference. I should be able to set this up over the weekend.

For the errors that you mentioned, please let me know whenever you get a chance to work on them. I will be more than happy to test and share feedback with you. One request though. Can we please start with lts version (12.x) first and look at the current version (14.x) after lte is finalised? Though I think, if both of them have similar problems, fixing one should help with the other as well.

To tell you the truth, we are currently running Node 10.15.3 using one of the node-packer forks. But we have already started to face issues where some npm dependencies require a higher Node version. So we are very much looking forward to this lts version build.

And yes, just so you know, once we have finalised this first new build, I will be sharing this important update with the rest of the community who have to use other inferior packing tools today because there is no option. I know for a fact that those people will love to use node-packer, as I do. 🥰

pmq20 · 2020-09-08T23:18:20Z

@saurabh-deep Thanks for the reply. No problems. I'll work on lts first this weekend. Looking forward to your public repo.

YYago · 2020-09-11T00:58:09Z

Windows 10

Branch: master;
Env: Windows 10 Pro 64; Ruby-2.6.3

Bat:

call bin/nodec.bat D:/devWorkSpace/AdRanking/win/index.js --lts --tmpdir=D:/devWorkSpace/nodec/temp --output=D:/devWorkSpace/nodec/a.exe --dest-os=win --clean-tmpdir > log.txt

Log:

Node.js Compiler (nodec) v140800.121803.dev
- entrance: D:/devWorkSpace/AdRanking/win/index.js
- options: {
    :lts=>true, 
    :tmpdir=>"D:/devWorkSpace/nodec/temp", 
    :output=>"D:/devWorkSpace/nodec/a.exe", 
    :os=>"win", 
    :clean_tmpdir=>true, 
    :npm=>"npm", 
    :make_args=>"-j4", 
    :vcbuild_args=>"x64"}

-> rm -rf D:/devWorkSpace/nodec/temp
-> mkdir -p D:/devWorkSpace/nodec/temp
-> cp -r "D:/devWorkSpace/nodec/node-packer/lts" "D:/devWorkSpace/nodec/temp/node-140800.121803.dev-lts"
-> rm -rf D:/devWorkSpace/nodec/temp/__work_dir__
-> mkdir -p D:/devWorkSpace/nodec/temp/__work_dir__
-> cp -r "D:/devWorkSpace/AdRanking/win" "D:/devWorkSpace/nodec/temp/__work_dir__/__enclose_io_memfs__"
........
........
6.14.5
audited 162 packages in 2.63s

8 packages are looking for funding
  run `npm fund` for details

found 2 vulnerabilities (1 low, 1 high)
  run `npm audit fix` to fix them, or `npm audit` for details
mksquashfs version 4.3 (2014/05/12)
copyright (C) 2014 Phillip Lougher <[email protected]>

[----------------------------------------------------------]
Compiled by Sebastiaan Ebeltjes @ 2015-06-08 (Domoticx.nl)
Compiled with Cygwin on Windows 8.1, compiler: GCC v4.9.2

This version comes with added support for:
- LZMA v4.65
- LZMA2 (XZ) v5.5.0
- LZO v2.09
- LZ4 r130

Enjoy!

[----------------------------------------------------------]

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2,
or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on deps/libsquash/sample/enclose_io_memfs.squashfs, block size 131072.

[=============================================================/] 7470/7470 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 6283.46 Kbytes (6.14 Mbytes)
	18.99% of uncompressed filesystem size (33083.26 Kbytes)
Inode table size 67746 bytes (66.16 Kbytes)
	26.82% of uncompressed inode table size (252581 bytes)
Directory table size 71139 bytes (69.47 Kbytes)
	42.44% of uncompressed directory table size (167603 bytes)
Number of duplicate files found 808
Number of inodes 7873
Number of files 7383
Number of fragments 152
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 490
Number of ids (unique uids + gids) 2
Number of uids 1
	omg (197609)
Number of gids 1
	None (197121)
Looking for Python
Python 2 found in C:\Python27\\python.exe
Looking for NASM
Looking for Visual Studio 2017
calling: "D:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\\Auxiliary\Build\vcvarsall.bat" amd64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.22
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
Found MSVS version 15.0
configure  --dest-cpu=x64
Warning: Missing input files:
tools\msvs\genfiles\node_etw_provider.rc
tools\msvs\genfiles\node_etw_provider.h
tools\v8_gypfiles\..\..\deps\v8\src\regexp\regexp-special-case.h
INFO: configure completed successfully
Project files generated.
......
......
    正在创建库 out\Release\node_mksnapshot.lib 和对象 out\Release\node_mksnapshot.exp
    正在创建库 out\Release\mkcodecache.lib 和对象 out\Release\mkcodecache.exp
enclose_io_libsquash.lib(enclose_io_win32.obj) : error LNK2019: 
    无法解析的外部符号 __imp_PathMatchSpecA，该符号在函数 EncloseIOFindFirstFileHelper 中被引用 [D:\devWorkSpace\nodec\temp\node-140800.121803.dev-lts\mkcodecache.vcxproj]

enclose_io_libsquash.lib(extract.obj) : error LNK2019: 
    无法解析的外部符号 __imp_PathFileExistsW，该符号在函数 squash_tmpf 中被引用 [D:\devWorkSpace\nodec\temp\node-140800.121803.dev-lts\mkcodecache.vcxproj]

enclose_io_libsquash.lib(enclose_io_win32.obj) : error LNK2019: 
    无法解析的外部符号 __imp_PathMatchSpecA，该符号在函数 EncloseIOFindFirstFileHelper 中被引用 [D:\devWorkSpace\nodec\temp\node-140800.121803.dev-lts\node_mksnapshot.vcxproj]

enclose_io_libsquash.lib(extract.obj) : error LNK2019: 
    无法解析的外部符号 __imp_PathFileExistsW，该符号在函数 squash_tmpf 中被引用 [D:\devWorkSpace\nodec\temp\node-140800.121803.dev-lts\node_mksnapshot.vcxproj]

out\Release\mkcodecache.exe : fatal error LNK1120: 
    2 个无法解析的外部命令 [D:\devWorkSpace\nodec\temp\node-140800.121803.dev-lts\mkcodecache.vcxproj]

out\Release\node_mksnapshot.exe : fatal error LNK1120: 
    2 个无法解析的外部命令 [D:\devWorkSpace\nodec\temp\node-140800.121803.dev-lts\node_mksnapshot.vcxproj]

saurabh-deep · 2020-09-27T22:55:19Z

Hi @pmq20 - Apologies for the delay due to some health issues in the past couple of weeks. I have now set up this test CLI project which you can use to test binary compilation. I haven't been able to set up GitHub Action for this as of yet, but I tried the latest node-packer code to build the binary on my local Mac and Windows, and in an ubuntu docker container for Linux - all 3 using the Ruby command. Till now, all 3 compilations are failing. Linux and Mac have the same error, while Windows shows a different one.

Please note that the project is fully compatible with Node12 (lts) and may not be fully compatible with Node14(current), but it should still compile. Also, Yarn is used with the project instead of NPM. I hope it doesn't cause any issues for you. I will try to set up GH Actions for the test Node project ASAP, but since I don't have much experience on GH Actions - so it will take time. If you want to give it a go, please let me know. I will very glad to share the access for that project with you.

Now the errors that I have been facing so far -

First of all, the compilation doesn't start and fail if the current working directory is set to the test Node project (node-cli-test). To be able to start the compilation, I am forced to execute the Ruby command from the node-packer directory, which is going to be a problem in CI. Earlier version of node-packer allowed us to execute the ruby command from the node project itself. It is important for the Ruby command to be executable from the node project itself. As of now, I receive the following error when I execute the following command from the test Node project -

Working directory: ~/Code/DX/node-cli-test
Command: ruby ~/Code/DX/node-packer/bin/nodec --tmpdir=~/adx-build/compile12 --output=~/adx-build/a.out --npm=yarn --lts bin/cli.js
Error: /Users/sdeep3/Code/DX/node-packer/bin/nodec:11:in <main>': private method require' called for Bundler:Module (NoMethodError)

Alternatively, when I execute the Ruby command from the node-packer directory, it starts the compilation but fails with the following error, both for Mac (OSX 10.14.6) and Linux (Ubuntu). Same errors are encountered if the pre-compiled nodec binary is used for compilation. Can I also take this opportunity to request you to downgrade the MacOS version from macos-10.15 to macos-10.14 in the GH Action workflow? The nodec binary created from macos-10.15 is not compatible with 10.14 and I am not in a position to upgrade as of this moment. Please let me know if you have any concerns related to this.

Working directory: ~/Code/DX/node-packer
Command: ruby bin/nodec --tmpdir=~/adx-build/compile12 --output=~/adx-build/a.out --npm=yarn --lts ~/Code/DX/node-cli-test/bin/cli.js
Error: cc -o /Users/sdeep3/adx-build/compile12/node-140800.121803-lts/out/Release/obj.target/uvwasi/deps/uvwasi/src/fd_table.o ../deps/uvwasi/src/fd_table.c '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_DARWIN_USE_64_BIT_INODE=1' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' -I../deps/uvwasi/include -I../deps/uv/include -O3 -gdwarf-2 -fvisibility=hidden -mmacosx-version-min=10.10 -arch x86_64 -Wall -Wendif-labels -W -Wno-unused-parameter -fno-strict-aliasing -MMD -MF /Users/sdeep3/adx-build/compile12/node-140800.121803-lts/out/Release/.deps//Users/sdeep3/adx-build/compile12/node-140800.121803-lts/out/Release/obj.target/uvwasi/deps/uvwasi/src/fd_table.o.d.raw -c
In file included from ../deps/uvwasi/src/clocks.c:8:
../deps/uv/include/uv.h:37:10: fatal error: 'enclose_io.h' file not found
In file included from ../deps/uvwasi/src/fd_table.c:9:
../deps/uv/include/uv.h:37:10: fatal error: #include "enclose_io.h"
^~~~~~~~~~~~~~
'enclose_io.h' file not found
#include "enclose_io.h"
^~~~~~~~~~~~~~
1 error generated.
1 error generated.
make[1]: *** [/Users/sdeep3/adx-build/compile12/node-140800.121803-lts/out/Release/obj.target/uvwasi/deps/uvwasi/src/fd_table.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [/Users/sdeep3/adx-build/compile12/node-140800.121803-lts/out/Release/obj.target/uvwasi/deps/uvwasi/src/clocks.o] Error 1
rm ae1ad5e88e93fc855faf392957725a5689c3fc98.intermediate
make: *** [node] Error 2

saurabh-deep · 2020-09-28T07:29:40Z

Hello @pmq20 - I forgot to include the Windows error that I am receiving when building from node-packer source using Ruby command. For Linux and Mac OS, I shared the same in my previous message.

Windows error:

Creating library out\Release\mkcodecache.lib and object out\Release\mkcodecache.exp
enclose_io_libsquash.lib(enclose_io_win32.obj) : error LNK2019: unresolved external symbol __imp_PathMatchSpecA referen
ced in function EncloseIOFindFirstFileHelper [C:\adx-build\compile\node-140800.121803-lts\node_mksnapshot.vcxproj]
enclose_io_libsquash.lib(extract.obj) : error LNK2019: unresolved external symbol __imp_PathFileExistsW referenced in f
unction squash_tmpf [C:\adx-build\compile\node-140800.121803-lts\node_mksnapshot.vcxproj]
out\Release\node_mksnapshot.exe : fatal error LNK1120: 2 unresolved externals [C:\adx-build\compile\node-140800.121803-
lts\node_mksnapshot.vcxproj]
enclose_io_libsquash.lib(enclose_io_win32.obj) : error LNK2019: unresolved external symbol __imp_PathMatchSpecA referen
ced in function EncloseIOFindFirstFileHelper [C:\adx-build\compile\node-140800.121803-lts\mkcodecache.vcxproj]
enclose_io_libsquash.lib(extract.obj) : error LNK2019: unresolved external symbol __imp_PathFileExistsW referenced in f
unction squash_tmpf [C:\adx-build\compile\node-140800.121803-lts\mkcodecache.vcxproj]
out\Release\mkcodecache.exe : fatal error LNK1120: 2 unresolved externals [C:\adx-build\compile\node-140800.121803-lts
mkcodecache.vcxproj]

pmq20 · 2020-10-09T00:53:30Z

@YYago @saurabh-deep Thanks for reporting! The master is currently an unfinished work in progress. Let me find time to finish it soon.

pmq20 · 2020-10-09T00:55:31Z

Is it possible to downgrade macos-10.15 to macos-10.14 in the GH Action workflow? I read GH's document and it appears that only 10.15 was provided. Maybe 10.14 would be there if I just declare os: [macos-10.14]. Let me try it.

EDITED:
I tried. No luck. It would still use Environment: macos-10.15 as shown in the log.

pmq20 · 2020-10-09T00:56:50Z

@saurabh-deep Also thank you for setting up the repository! I'll use it soon. I was too busy with some other responsibilities in my life recently.

saurabh-deep · 2020-10-09T07:57:42Z

Is it possible to downgrade macos-10.15 to macos-10.14 in the GH Action workflow? I read GH's document and it appears that only 10.15 was provided. Maybe 10.14 would be there if I just declare os: [macos-10.14]. Let me try it.

EDITED:
I tried. No luck. It would still use Environment: macos-10.15 as shown in the log.

Thanks for checking this @pmq20. I also verified and it seems they have removed macos-10.14 virtual environment, which was available earlier. We will anyways rely on compiling the binary using the Ruby command with node-packer code, which we have been doing in the past. Thank you anyways for trying. 👍

@saurabh-deep Also thank you for setting up the repository! I'll use it soon. I was too busy with some other responsibilities in my life recently.

Please let me know if you need write access to this repository or the two dependency package repositories. These have created specifically for testing binary creation, so there are at your disposal - whichever way you want to use them. And we all have other responsibilities in our lives, so don't even worry about node-packer at all. Personal life comes first. I hope everything is fine. 🤞

@YYago @saurabh-deep Thanks for reporting! The master is currently an unfinished work in progress. Let me find time to finish it soon.

That's fine and expected @pmq20. Please take your time. In your absence we tried to look into the macOS/Linux issue and also noticed a few lines of code missing in the Node.js hacks when compared with the old working versions. We made a few small changes and the macOS compilation actually worked after that. But the execution is very buggy. The executable doesn't seem to be able to read the standard input when using inquirer and also not able to read from the file system it seems. Please let me know if you want more details about this. I will also try to add a couple more scenarios to the Test repository.

saurabh-deep · 2020-10-16T11:53:40Z

Hi @pmq20 ! Hope you are doing well. Just wanted to let you know that I will be available to help you over this weekend, if you need.

We have been trying to make changes in the node-packer hacks in lts to make this work for us. But have been largely unsuccessful.

saurabh-deep · 2020-11-18T09:47:44Z

Hi @pmq20 - I wanted to ping here to check if everything is fine with you and if it is going to be possible for you to look into the issues reported above with the latest node-packer (lts and current)?

NexZhu · 2020-11-18T15:01:28Z

Hi @pmq20 - I wanted to ping here to check if everything is fine with you and if it is going to be possible for you to look into the issues reported above with the latest node-packer (lts and current)?

Just want to share that with commit a13e024771beaf86dd2b283987c8dad6ef55e15f of this repo, I've successfully build my project with Node.js v12.13.0. Using the latest commit of this repo has failed.

On the other hand, I haven't found a rev of this repo which works for Node.js v14 so I'm also waiting for that.

saurabh-deep · 2020-11-19T06:51:50Z

Just want to share that with commit a13e024771beaf86dd2b283987c8dad6ef55e15f of this repo, I've successfully build my project with Node.js v12.13.0. Using the latest commit of this repo has failed.

On the other hand, I haven't found a rev of this repo which works for Node.js v14 so I'm also waiting for that.

Thanks for sharing @NexZhu. We are in need of Node v12 only at the moment. We can wait for Node v14 for a bit longer. I will it a go using the commit id that you mentioned and share the updates here.

PureSpider · 2020-12-04T21:52:56Z

I'm also getting the "unresolved external symbol __imp_PathMatchSpecA referenced in function EncloseIOFindFirstFileHelper" error.

Any news on this yet?
Is anyone able to provide a working build as mentioned in the previous comment?

saurabh-deep · 2021-01-04T15:01:27Z

Hello @pmq20 - Happy New Year! I hope you have had time to sort out the stuff you were going through previously. I am very excited as I saw a commit from you in this repo over the weekend (after a few months) -- Read me only 😃 . Do you think you will be able to spend some time on the node-packer now to resolve the issues we are facing with Node 12 and 14? If only, any of us knew enough C, we would have been able to help you. 😸

pmq20 · 2021-01-04T23:20:02Z

@saurabh-deep Sorry but I do not think I have the time to maintain the project. I am willing to add anybody who wishes to maintain the project as a collaborator of the repo.

saurabh-deep · 2021-01-05T07:10:34Z

@pmq20 This is sad and heart-breaking, but understandable. There was no other tool that came anywhere close to node-packer. It is such a shame that I don't have what it takes to maintain this project, as it requires comprehensive C expertise.

Do you think we can get on a call to discuss the issues and the options?

Asp3ctus · 2021-01-06T17:47:54Z

this project is realy needs a second or a third life...
@roytan883 and @slee047 were almost succesfull with updating to 12 series ... it compiles and runs, but network sockets close after 1-3 seconds.... on ubuntu 18

I decided to try it again on ubuntu 20.04 and a test http script was succesful on https://github.com/slee047/node-packer/tree/TestSeries12 branch which it based on 12.13.0 nodejs ... i will try to compile my main project with this version and will post results, before i was stuck to 10.15.3 version

Asp3ctus · 2021-01-10T04:30:47Z

Well i tried to mentioned above fork/branch with my project and it is not realy working.
The connection to postgres is droped, and files which are outsite the enclose_io_memfs can not be accessed ...

i hope node-packer can be resurrected, so far i can not find any alternative which would work like this, but it has it down sides (unpredictible Segmentation Faults)

saurabh-deep · 2021-01-10T15:25:44Z

Thank you @Asp3ctus for sharing your findings. Like I said earlier, it is heart-breaking to see node-packer come to a close. I don't know anyone with the kind of C knowledge that seems to be required here, or I would ask them for help.

The way things stand right now, it is completely dependent on @pmq20. I wish he had some time to look into node-packer, only once in a while. If the current issues could be resolved, it would only need some work again only when Node 16 is out there. Till then we could easily live with these versions.

roytan883 · 2021-01-18T04:24:49Z

@Asp3ctus Actually i think i resolved the sockets and file problem on Ubuntu 16.04 and 18.04. The 20.04 may work which i have not tested. My modified version is nodejs v12.18.2. The way of fix it i think can be apply to all new v12.x of nodejs.
Because my project only support Ubuntu server now, so i can only guarantee it worked on Ubuntu platform.

@pmq20 The method to solve sockets and file problem is the virtual fd conflict with system fd, here is code hack it, it may help you get a clue

//node/deps/libsquash/include/squash.h
//roytan hack start -> vfd should start from 500000 avoid conflict with normal socket fd
#define VFD_START 500000
#define SQUASH_VALID_VFD(vfd) (((vfd) - 500000) >= 0 && ((vfd) - 500000) < squash_global_fdtable.nr && NULL != squash_global_fdtable.fds[((vfd) - 500000)])
#define SQUASH_VFD_FILE(vfd) (squash_global_fdtable.fds[((vfd) - 500000)])

//some modify on create vfd, then it should fix most case
//buf some code need use SQUASH_VFD_FILE to get virtual fd manually, like in node/deps/libsquash/src/dirent.c
handle = (int *)(squash_global_fdtable.fds[dir->fd]->payload);
//roytan hack
  handle = (int *)(SQUASH_VFD_FILE(dir->fd)->payload);
squash_global_fdtable.fds[dir->fd]->payload = (void *)dir;
//roytan hack
  SQUASH_VFD_FILE(dir->fd)->payload = (void *)dir;

saurabh-deep · 2021-01-18T05:32:39Z

@roytan883 - That's awesome to know that you have been able to identify the problem. Can you please share where the updated code is so that I can also give it a go?

Thanks much!
Saurabh

saurabh-deep · 2021-01-19T16:24:32Z

@roytan883 / @Asp3ctus - Can you please help me with the updated code, folks - a version with the above mentioned changes? Looking forward to test these on Mac OS and Windows too.

Asp3ctus · 2021-02-10T18:44:14Z

@saurabh-deep - Hi, i have not yet tried to folow @roytan883 instructions yet. I will actualy try it next week,
@roytan883 can you please, commit the changes you made to fix this issue ? I will be testing it on Linux

Asp3ctus · 2021-02-15T14:00:04Z

Well i tried to apply changes, mentioned above, but the build resulting test app did not start the execution of js code, so i was unsuccessful with this changes

Can we resurrect node-packer to get it working with latest Node.js versions? #145

Can we resurrect node-packer to get it working with latest Node.js versions? #145

Comments

saurabh-deep commented May 4, 2020

saurabh-deep commented May 6, 2020

saurabh-deep commented Jun 3, 2020

saurabh-deep commented Aug 18, 2020

pmq20 commented Aug 18, 2020 • edited Loading

saurabh-deep commented Aug 18, 2020

pmq20 commented Aug 23, 2020 • edited Loading

saurabh-deep commented Aug 23, 2020 • edited Loading

pmq20 commented Aug 24, 2020 • edited Loading

saurabh-deep commented Sep 8, 2020 • edited Loading

pmq20 commented Sep 8, 2020 • edited Loading

saurabh-deep commented Sep 8, 2020

pmq20 commented Sep 8, 2020 • edited Loading

YYago commented Sep 11, 2020

saurabh-deep commented Sep 27, 2020

saurabh-deep commented Sep 28, 2020

pmq20 commented Oct 9, 2020 • edited Loading

pmq20 commented Oct 9, 2020 • edited Loading

pmq20 commented Oct 9, 2020 • edited Loading

saurabh-deep commented Oct 9, 2020 • edited Loading

saurabh-deep commented Oct 16, 2020

saurabh-deep commented Nov 18, 2020

NexZhu commented Nov 18, 2020

saurabh-deep commented Nov 19, 2020

PureSpider commented Dec 4, 2020 • edited Loading

saurabh-deep commented Jan 4, 2021

pmq20 commented Jan 4, 2021

saurabh-deep commented Jan 5, 2021

Asp3ctus commented Jan 6, 2021

Asp3ctus commented Jan 10, 2021

saurabh-deep commented Jan 10, 2021

roytan883 commented Jan 18, 2021 • edited Loading

saurabh-deep commented Jan 18, 2021

saurabh-deep commented Jan 19, 2021

Asp3ctus commented Feb 10, 2021

Asp3ctus commented Feb 15, 2021

pmq20 commented Aug 18, 2020 •

edited

Loading

pmq20 commented Aug 23, 2020 •

edited

Loading

saurabh-deep commented Aug 23, 2020 •

edited

Loading

pmq20 commented Aug 24, 2020 •

edited

Loading

saurabh-deep commented Sep 8, 2020 •

edited

Loading

pmq20 commented Sep 8, 2020 •

edited

Loading

pmq20 commented Sep 8, 2020 •

edited

Loading

pmq20 commented Oct 9, 2020 •

edited

Loading

pmq20 commented Oct 9, 2020 •

edited

Loading

pmq20 commented Oct 9, 2020 •

edited

Loading

saurabh-deep commented Oct 9, 2020 •

edited

Loading

PureSpider commented Dec 4, 2020 •

edited

Loading

roytan883 commented Jan 18, 2021 •

edited

Loading