I am using a system I designed on my own for the 'CRUD' style of thing for renames/deletions you are mentioning.
People just hook up a Vector DB and store all the embeddings once and call it a day.
I use three databases for the CRUD style system (idk its name, so i will just call it that lol).
I use QDrant as the main DB, the one every RAG Pipeline requires, then I use Redis as a Raw storage for all the files and docs that are embedded and stored in QDrant already. Its like a totally raw db with the raw files.
Then I used SQLite DB with timestamps and stuff to help with the "UPSERT" of new data (CRUD system) by looking at what changed against the DB as it already contains all the stuff from the last data sync.
So in short, a normal RAG DB, a DB for raw info (helps with comparison during UPSERT), and a DB purely to judge where new data goes (CRUD system)
I am using a system I designed on my own for the 'CRUD' style of thing for renames/deletions you are mentioning.
People just hook up a Vector DB and store all the embeddings once and call it a day.
I use three databases for the CRUD style system (idk its name, so i will just call it that lol).
I use QDrant as the main DB, the one every RAG Pipeline requires, then I use Redis as a Raw storage for all the files and docs that are embedded and stored in QDrant already. Its like a totally raw db with the raw files.
Then I used SQLite DB with timestamps and stuff to help with the "UPSERT" of new data (CRUD system) by looking at what changed against the DB as it already contains all the stuff from the last data sync.
So in short, a normal RAG DB, a DB for raw info (helps with comparison during UPSERT), and a DB purely to judge where new data goes (CRUD system)