Migrating from Pinecone to Qdrant: A Smooth Transition for My SaaS App

In the ever-evolving landscape of software development, it's imperative to stay updated with the most efficient and cost-effective solutions. Recently, I embarked on a journey of migrating the vector search engine of my SaaS application from Pinecone to Qdrant. This transition not only enhanced the functionality but also proved to be a prudent decision in terms of scalability and performance. Here’s a narrative of how I seamlessly migrated to Qdrant with minimal code changes.

Why Qdrant?

Before delving into the migration process, it's crucial to understand why Qdrant caught my attention. Qdrant is an open-source vector similarity search engine with a focus on production readiness and scalability. Its easy-to-use REST API, provided by @qdrant/js-client-rest, allowed me to interact with the Qdrant server effortlessly.

The imports

import { QdrantVectorStore } from "langchain/vectorstores/qdrant";
import {QdrantClient} from '@qdrant/js-client-rest';

The Migration

The migration required modifications in three critical files of my project. Below is a detailed explanation of the changes made:

1. Core Functionality Update:

In the core file, I integrated Qdrant by importing the necessary modules and replacing the vector store instantiation with QdrantVectorStore. This change is central to vectorizing and indexing the document upon upload completion. The QdrantVectorStore.fromDocuments method enabled the indexing of document data in the Qdrant collection.

Original code:

 const pinecone = await getPineconeClient()
    const pineconeIndex = pinecone.Index('quill')

    const embeddings = new OpenAIEmbeddings({
      openAIApiKey: process.env.OPENAI_API_KEY,
    })

    await PineconeStore.fromDocuments(
      pageLevelDocs,
      embeddings,
      {
        pineconeIndex,
        namespace: createdFile.id,
      }
    )

Modified code:

import { QdrantVectorStore } from "langchain/vectorstores/qdrant";
import {QdrantClient} from '@qdrant/js-client-rest';

...

const store = await QdrantVectorStore.fromDocuments(
  pageLevelDocs,
  embeddings,
  {
    collectionName: createdFile.id,
    url: process.env.QDRANT_URL
  }
);

2. Establishing Connection:

In the second file, I established a connection to the Qdrant server running locally by creating a function getQdrantClient. This function initializes a new QdrantClient instance with the provided URL and API key.

Original code:

import { PineconeClient } from '@pinecone-database/pinecone'

export const getPineconeClient = async () => {
  const client = new PineconeClient()

  await client.init({
    apiKey: process.env.PINECONE_API_KEY!,
    environment: 'us-east1-gcp',
  })

  return client
}

Modified code:

import {QdrantClient} from '@qdrant/js-client-rest';

export const getQdrantClient = async () => {
  const client = new QdrantClient({url: process.env.QDRANT_URL, apiKey: process.env.QDRANT_API_KEY});
  return client;
};

3. Vector Store Interaction:

Lastly, in the message route file, I replaced the existing vector store interaction logic with the QdrantVectorStore.fromExistingCollection method to retrieve the vector store of an existing collection. This method is employed to vectorize the message and perform a similarity search within the Qdrant collection.

Original code:

  const pinecone = await getPineconeClient()
  const pineconeIndex = pinecone.Index('quill')

  const vectorStore = await PineconeStore.fromExistingIndex(
    embeddings,
    {
      pineconeIndex,
      namespace: file.id,
    }
  )

  const results = await vectorStore.similaritySearch(
    message,
    4
  )

Modified code:

import { QdrantVectorStore } from "langchain/vectorstores/qdrant";

...

const vectorStore = await QdrantVectorStore.fromExistingCollection(
  embeddings,
  {
    collectionName: file.id,
    url: process.env.QDRANT_URL
  }
);
...
const results = await vectorStore.similaritySearch(message, 4);

Conclusion

The migration from Pinecone to Qdrant was a straightforward task with minor code adjustments. The transition has not only improved the search functionality but also has the potential to scale as the application grows. Qdrant’s simplicity and efficiency are indeed commendable, making it a worthy choice for any SaaS application looking to implement vector similarity search.

Thanks for reading! If you have any questions, feel free to reach out to me on Twitter or LinkedIn.

If you want to try the app (it's in finnish), you can try it out here.