Skip to content
This repository has been archived by the owner on Jul 16, 2024. It is now read-only.

Commit

Permalink
Fix unique key missing problem with search (#232)
Browse files Browse the repository at this point in the history
When creating embeddings, we store the unique key attnum in pg_options. 
Previously, the unique key attnum was taken from the original table, so
the error occurs if the
embedding table and the original table have a different number or order
of columns.
This patch saves the unique key attnum from the embedding table instead.
  • Loading branch information
ruxuez authored Jan 31, 2024
1 parent a826c67 commit 9124f25
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion greenplumpython/experimental/embedding.py
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,9 @@ def create_index(
query_col_names = _serialize_to_expr(
list(self._dataframe.unique_key) + [column], self._dataframe._db
)
unique_key_col_names = _serialize_to_expr(
list(self._dataframe.unique_key), self._dataframe._db
)
sql_add_relationship = f"""
DO $$
BEGIN
Expand All @@ -168,13 +171,21 @@ def create_index(
SELECT FROM unnest({query_col_names}) AS query
WHERE attname = query
)
),emb_attnum_map AS (
SELECT attname, attnum FROM pg_attribute
WHERE
attrelid = '{embedding_df._qualified_table_name}'::regclass::oid AND
EXISTS (
SELECT FROM unnest({unique_key_col_names}) AS query
WHERE attname = query
)
), embedding_info AS (
SELECT
'{embedding_df._qualified_table_name}'::regclass::oid AS embedding_relid,
attnum AS content_attnum,
{len(self._dataframe._unique_key) + 1} AS embedding_attnum,
'{model_name}' AS model,
ARRAY(SELECT attnum FROM attnum_map WHERE attname != '{column}') AS unique_key
ARRAY(SELECT attnum FROM emb_attnum_map WHERE attname != '{column}') AS unique_key
FROM attnum_map
WHERE attname = '{column}'
)
Expand Down

0 comments on commit 9124f25

Please sign in to comment.