Possible to reduce node package size?

[dmb0058]

Describe the bug
AWS Lambda functions are restricted to 250Mb including layers. Packaging the aws-sdk, faiss-node, fs, langchain, openai and adm-zip blows the limit.

I've pulled out only the aws sub-packages I need (s3 and secrets-manager) and the fs and adm-zip packages are tiny, but faiss-node is pretty big. I see it has Readme files and stuff, so I guess it's not 'minimal' - any chance of a slimmed down npm package?

Environment:

• Operating system: AWS Lambda
• Nodejs Version: v18.x
• Package Version: latest

To Reproduce
Steps to reproduce the behavior:

1. Zip the nodejs/node_modules hierarchy
2. Attempt to create a Lambda layer
3. Error is "Layer max. size is 250Mb" (paraphrased)

Expected behavior
The layer is correctly created.

[ewfian]

@dmb0058 Now use MKL on linux and openblas on musllinux.
We can consider replacing it with openblas for reducing size.

[dmb0058]

Excellent news! This will make a lot of difference, thanks. I'll happily try this once the langchainjs issue is resolved.

…

On Wed, 12 Jul 2023, 08:27 Fan, ***@***.***> wrote: @dmb0058 <https://github.com/dmb0058> Now use MKL on linux and openblas on musllinux. We can consider replacing it with openblas for reducing size. [image: image] <https://user-images.githubusercontent.com/12423122/252887542-7ba8f9a2-08fd-4d14-96d2-172ca33346fc.png> — Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDXCQJXK2XUUY4N2SMHT73XPZGWTANCNFSM6AAAAAA2F7YBTY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dmb0058]

Just checking if the latest version is the one on https://www.npmjs.com/package/faiss-node, or need to install from git?

[ewfian]

@dmb0058 I will publish a new version to npm soon.

You can check pre-release version here: https://github.com/ewfian/faiss-node/releases/tag/v0.2.3-alpha.0

[ewfian]

@dmb0058 The new version v0.2.3 was released.

[dmb0058]

That's great news, thanks for all your help :)

…

On Mon, 7 Aug 2023, 02:14 Fan, ***@***.***> wrote: @dmb0058 <https://github.com/dmb0058> The new version v0.2.3 was released. — Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDXCQIZOJS6DHK2JXMGJP3XUA6QBANCNFSM6AAAAAA2F7YBTY> . You are receiving this because you were mentioned.Message ID: ***@***.***>