Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FAISS crashes NestJS app when index doesn't exist #1475

Closed
MarioUnlam opened this issue May 31, 2023 · 3 comments · Fixed by ewfian/faiss-node#13 or #1546
Closed

FAISS crashes NestJS app when index doesn't exist #1475

MarioUnlam opened this issue May 31, 2023 · 3 comments · Fixed by ewfian/faiss-node#13 or #1546
Assignees

Comments

@MarioUnlam
Copy link

I'm using langchain and the FAISS integration in a NestJS app, and I noticed that trying to do "FaissStore.load" with a path that doesn't exist results in a hard crash that cannot be catched. The app crashes with the following error:

Error: ENOENT: no such file or directory, open 'C:\...\docstore.json'
error Command failed with exit code 3221226505.

I'm not entirely sure why this happens, but the crash seems to occur when trying to run "FAISS.read" with a non-existent "faiss.index" file. Checking for the file existence before running that method seems to prevent the crash, although this is just a workaround, as the proper fix should be implemented on the faiss-node side.

In faiss.js, line 105

    static async load(directory, embeddings) {
        const fs = await import("node:fs/promises");
        const path = await import("node:path");
        const readStore = (directory) => fs
            .readFile(path.join(directory, "docstore.json"), "utf8")
            .then(JSON.parse);
        const readIndex = async (directory) => {
            const { IndexFlatL2 } = await this.importFaiss();
            // FIX
            const filePath = path.join(directory, "faiss.index");
            await fs.access(filePath, fs.F_OK);
            return IndexFlatL2.read(filePath);
            // ---
        };
        const [[docstoreFiles, mapping], index] = await Promise.all([
            readStore(directory),
            readIndex(directory),
        ]);
        const docstore = new InMemoryDocstore(new Map(docstoreFiles));
        return new this(embeddings, { docstore, index, mapping });
    }
@jacoblee93
Copy link
Collaborator

CC @ewfian?

@ewfian
Copy link
Contributor

ewfian commented Jun 1, 2023

@MarioUnlam @jacoblee93 The exception in the native code is not caught, I will fix it on the faiss-node side, Please assign this issue to me.

@jacoblee93
Copy link
Collaborator

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants