Azure RAGChat is a very popular application developed by Microsoft and made available for free for creating ChatGPT-like experiences with your own data.

This is the latest IProgrammer installment on RAG-based solutions, following previous articles looking at The Advanced + Agentic RAG Cookbooks, Getting Going With RAG and RAG from Scratch and has the starting point that RAG allows LLMs to amplify the user's query by connecting to external data in real time when generating their output.

This approach is lighter in resources, doesn't need constant updating since it consumes the data at run time and of course the big boon is that it retrieves up to date answers.

There are many tutorials and ready made solutions out there of implementing RAG mainly for chatting with various documents in various formats. Azure's answer to that is the RAGChat application. This app creates a ChatGPT-like frontend experience over your own documents using RAG. It is based on the Azure OpenAI Service to access GPT models and Azure AI Search for data indexing and retrieval. Its backend is written in Python, but there's also JavaScript, .NET, and Java samples.

The thing with the app is that it is massive and if you would like to understand more of its inner workings you'll find it difficult, despite the code being open source. As such, Microsoft has prepared a 10-part tutorial that goes through the application's components from start to finish.

The series ran through January and February and at the time required registration because they were being live streamed. But now, all the modules are available without registration for everyone to watch.

The 10 parts are:

The RAG solution for Azure-Intro
An end to end demo of the RAG solution from Azure AI Search to Azure OpenAI and from there to deployment on Azure. Discussing Azure's architecture.

Customizing our RAG solution
Shows how to customize the RAG solution for our own domain with our own data by modifying the default prompts and personalizing the UI.

Optimal retrieval with Azure AI Search
Using Azure AI Search for finding matching documents; looking into vector embeddings, hybrid search with RRF, and semantic ranking.

Multimedia data ingestion
How to handle multimedia documents, when your documents contain images or charts. Two approaches are shown. The first approach is purely during ingestion time, where it replaces media in the documents with LLM-generated descriptions. The second approach stores images of the media alongside vector embeddings of the images, and sends both text and images to a multimodal LLM for question answering.

User login and data access control
Controlling document visibility and who was permission on them. Using Azure AI Search with data access controls we can allow only logged in users
to search the documents.

Storing chat history
Using either IndexedDB for client-side storage or Azure Cosmos DB for persistent storage.

Adding speech input and output
Using Azure Speech API or VoiceRAG solution for those who want a real-time voice interface.

Private deployment
How to deploy the app to a virtual network that includes AI Search, OpenAI, Document Intelligence, and Blob storage,
keeping it inside our enterprise's walled gardens.

Evaluating RAG answer quality
How can you be sure that the RAG chat app answers are accurate, clear, and well formatted? By evaluating its responses. Evaluating RAG applications is important for understanding how well these systems work and effective evaluation helps optimize performance and builds confidence in RAG applications for real-world use. We've looked at the subject in "Advanced + Agentic RAG Cookbooks" which leveraged Athina's open source SDK to evaluate the performance of an LLM. This solution uses the azure-ai-evaluation SDK.

Finally, Monitoring and tracing LLM calls
Monitor your application for performance issues, runtime errors, and LLM-specific issues like Content Safety filter violations.

In conclusion, no matter that the solution is based entirely on Microsoft's ecosystem of Azure AI Search, Azure Cosmos DB, Azure OpenAI, Azure Speech API and is deployed on Azure Cloud, you'll learn a lot just by following the concepts and the techniques applied throughout the course. Even better, you can try it yourself using the free Azure tiers.

Perfect if you're looking to get into the world of RAG. You can adapt it to your own needs or even create your own solutions from scratch.

More Information

RAG Deep Dive: 10-part live stream series

Github-RAG chat app with Azure OpenAI and Azure AI Search (Python)

Related Articles

The Advanced + Agentic RAG Cookbooks

Getting Going With RAG

RAG from Scratch

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info