{"id":97,"date":"2024-03-04T22:36:13","date_gmt":"2024-03-04T22:36:13","guid":{"rendered":"https:\/\/sanjayk7r.com\/?p=97"},"modified":"2024-03-07T16:40:02","modified_gmt":"2024-03-07T16:40:02","slug":"use-private-data-securely-with-an-llm","status":"publish","type":"post","link":"https:\/\/sanjayk7r.com\/index.php\/2024\/03\/04\/use-private-data-securely-with-an-llm\/","title":{"rendered":"Use private data securely with an LLM"},"content":{"rendered":"\n
\"\"\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n\n\n\n

Large language models (LLMs) are amazing but one shortcoming is that they wont be able to answer questions related to your private data. This is where RAG or retrieval-augmented generation comes in. RAG is a technique that lets you use LLMs with your private data.<\/p>\n\n\n\n

Dont roll your own RAG<\/strong><\/h6>\n\n\n\n

RAG architecture has quite a few moving parts. It involves a ingestion database, vector database, retrieval system, prompt, and finally a generative model. Moreover these need to be orchestrated so that they can ingest new or updated data. Managing these interconnected components adds challenges to the development and deployment process that can slow your team down. Open source libraries can simplify these tasks but risk introducing errors and require constant version updates. Using these libraries still demands substantial coding, determining data chunk sizes, and generating embeddings.<\/p>\n\n\n\n

\ud83d\udc49 Your team should focus on building your product and not on the undifferentiated heavy lifting behind a RAG process.<\/p>\n\n\n\n

Hello BEDROCK KNOWLEDGE BASE<\/h5>\n\n\n\n

Here is how simple it is to setup a fully managed RAG that is secure, simple and production grade.<\/p>\n\n\n\n

1. Create a Knowledge Base<\/h4>\n\n\n\n

Knowledge Bases for Amazon Bedrock is a serverless approach to building your RAG workflow. It automates synchronisation of your data with the vector database. <\/p>\n\n\n\n

Simply point to the location of your data on S3 and Knowledge Base takes care of the entire ingestion process into your vector database including embedding and synchronisation when your data changes. Your data could be in a multitude of formats e.g. txt, csv, xls, doc, pdf etc.<\/p>\n\n\n\n

You can simple use the default embedding models and vector database or choose your own.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

2. Query your knowledge base<\/h4>\n\n\n\n

Once your data is ingested. You can go ahead and start talking to it. There are different ways to query your data.<\/p>\n\n\n\n

USE the AWS Console<\/h5>\n\n\n\n

The console lets you quickly try both retrieval and generation.<\/p>\n\n\n\n

Retrieval<\/h6>\n\n\n\n

This means finding the correct set of documents based on the user query. Your can try different queries like hybrid search, semantic search or let bedrock choose one for you. More about that in this post<\/a>.<\/p>\n\n\n\n

GEneration<\/h6>\n\n\n\n

Next is “generation” i.e. taking the search results that you have pulled from the vector database and bundling it as a context together with the user prompt and sending it to an LLM. Choosing a model is just 1-click. e.g. Select Claude and have it generate answers for you. All the plumbing happens instantly behind the scenes and you can focus on getting the best answers from your data.<\/p>\n\n\n\n

Here is a simple screenshot but it demonstrates the entire RAG workflow. <\/p>\n\n\n\n

    \n
  1. Private data was uploaded to s3. <\/li>\n\n\n\n
  2. Data was ingested and embedded into a vector database. <\/li>\n\n\n\n
  3. The vector database prepared a context based on my query. <\/li>\n\n\n\n
  4. Then that context was sent to Claude together with my prompt. <\/li>\n\n\n\n
  5. Claude then generated a response that you see below, moreover it even attached links to my data that helps us trace the answer to the data.<\/li>\n<\/ol>\n\n\n\n

    The first step is manual, the rest is entirely managed by Bedrock for you!<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n
    Use the SDK<\/h5>\n\n\n\n

    If want to develop and app or talk to your data via a notebook, its easy as well. You just need the latest AWS SDK. e.g. Here are some api options if you use python.<\/p>\n\n\n\n

    – api: retrieve_and_generate<\/a> – Retrieve documents and obtains answers directly with the native Bedrock Knowledge Base.<\/p>\n\n\n\n

    – api: retrieve<\/a> – Retrieve documents based on your query and then pass it to an LLM of your choice.<\/p>\n\n\n\n

    build a fully customiSEd solution<\/h5>\n\n\n\n

    If you want to build you own fully customised Q&A application you can integrate bedrock with langchain. Have a look at Langchain integration<\/a>.<\/p>\n\n\n\n

    \ud83d\udc50 <\/h2>\n\n\n\n

    Thats it, it really is that straightforward to put together an LLM with RAG.<\/p>\n\n\n\n

    \ud83d\ude80 Get Started<\/h4>\n\n\n\n

    \ud83d\udc49 For a step by step walkthrough on setting on RAG on bedrock read this blog post<\/a>.<\/p>\n\n\n\n

    \ud83d\udc49 For a really good look at how to use RAG for drug discovery using Amazon Bedrock, read this blog post<\/a>.<\/p>\n\n\n\n

    <\/p>\n\n\n\n

    <\/p>\n","protected":false},"excerpt":{"rendered":"

    Large language models (LLMs) are amazing but one shortcoming is that they wont be able to answer questions related to your private data. This is where RAG or retrieval-augmented generation comes in. RAG is a technique that lets you use LLMs with your private data. Dont roll your own RAG RAG architecture has quite a […]<\/p>\n","protected":false},"author":1,"featured_media":184,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[6,13,11],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/posts\/97"}],"collection":[{"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/comments?post=97"}],"version-history":[{"count":37,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/posts\/97\/revisions"}],"predecessor-version":[{"id":210,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/posts\/97\/revisions\/210"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/media\/184"}],"wp:attachment":[{"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/media?parent=97"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/categories?post=97"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sanjayk7r.com\/index.php\/wp-json\/wp\/v2\/tags?post=97"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}