I have a products table that has a lot of columns, which from these, the following ones are important for our search:
- Title 1 to Title 6 (title in 6 different languages)
- Brand name
- Category name
- Product attributes like size, color, etc.
We are planning on using qdrant vector search to implement fast vector queries. But the problem is that all the data important for searching, are in different columns (6 languages + brand + category + attrs = 9 columns at least) and I do not think generating vector embeddings separately for all the columns is the best solution.
I came up with the idea of mixing the columns of one language together. By this I mean to concatenate title, brand, category, and attrs for language all together. Then, generate an embedding vector for this column.
Also I use the "BAAI/bge-m3" model which is a multilingual text embedding model that supports more than 100 langs.
So, in short, I have a vector column for each language containing the vector for the combined text of title, brand, attrs and category.
Now, the question is, is this a valid method? What are the pros and cons of this method?
It is important to note that currently we have more than 300.000 products and they will grow to more than 1.000.000 in the near future.
Источник: https://stackoverflow.com/questions/780 ... -on-qdrant
