I have a products table that has a lot of columns, which from these, the following ones are important for our search:
- Title 1 to Title 6 (title in 6 different languages)
- Brand name (in 6 different languages)
- Category name (in 6 different languages)
- Product attributes like size, color, etc. (in 6 different languages)
We are planning on using qdrant vector search to implement fast vector queries. But the problem is that all the data important for searching, are in different columns and I do not think generating vector embeddings separately for all the columns is the best solution.
I came up with the idea of mixing the columns together and generating separate collections; and I came up with this solution because the title, the category, brand and attrs columns are essentially the same just in different langs.
Also I use the "BAAI/bge-m3" model which is a multilingual text embedding model that supports more than 100 langs.
So, in short, for each collection I have a vector column containing the vector for the combined text of titles, brand, and category in each language and when searched, because we already know which language the website is, we will search in that specific language collection.
Now, the question is, is this a valid method? What are the pros and cons of this method?
It is important to note that currently we have more than 300.000 products and they will grow to more than 1.000.000 in the near future.
Источник: https://stackoverflow.com/questions/780 ... -on-qdrant
