9  Ollama RAG App

Source: https://python.langchain.com/v0.2/docs/tutorials/local_rag/

9.1 Load Document

from langchain_community.document_loaders import WebBaseLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")
data = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

9.2 Vector Store

from langchain_chroma import Chroma
from langchain_ollama import OllamaEmbeddings

local_embeddings = OllamaEmbeddings(model="nomic-embed-text:v1.5")

vectorstore = Chroma.from_documents(documents=all_splits, embedding=local_embeddings)

Test search

question = "What are the approaches to Task Decomposition?"
docs = vectorstore.similarity_search(question)
len(docs)

4

docs[0]

Document(metadata={'description': 'Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.\nAgent System Overview In a LLM-powered autonomous agent system, LLM functions as the agent’s brain, complemented by several key components:', 'language': 'en', 'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/', 'title': "LLM Powered Autonomous Agents | Lil'Log"}, page_content='Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.')

9.3 LLM Execute

from langchain_ollama import ChatOllama

model = ChatOllama(
    model="llama3.1:8b",
)

Test it

response_message = model.invoke(
    "Hello"
)

print(response_message.content)

Hello! How can I help you today?

9.3.1 Summarization Chain

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template(
    "Summarize the main themes in these retrieved docs: {docs}"
)

# Convert loaded documents into strings by concatenating their content
# and ignoring metadata
def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)


chain = {"docs": format_docs} | prompt | model | StrOutputParser()

question = "What are the approaches to Task Decomposition?"

docs = vectorstore.similarity_search(question)

resp1 = chain.invoke(docs)
print(resp1)

The main themes in these documents are:

1. **Task Decomposition**: Breaking down complex tasks into smaller, manageable subgoals using techniques such as:
    * Simple prompting with LLMs (Large Language Models)
    * Task-specific instructions
    * Human inputs
2. **Autonomous Agent System**: Overview of a system that uses LLMs to power an autonomous agent, which involves:
    * Planning and task decomposition
    * Expert models for task execution
3. **Planning and Reflection**: Key aspects of the planning process, including:
    * Breaking down tasks into subgoals
    * Self-criticism and self-reflection
    * Learning from mistakes and refining future steps

These themes are closely related to artificial intelligence, machine learning, and autonomous systems.

9.3.2 Q & A

from langchain_core.runnables import RunnablePassthrough

RAG_TEMPLATE = """
You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.

<context>
{context}
</context>

Answer the following question:

{question}"""

rag_prompt = ChatPromptTemplate.from_template(RAG_TEMPLATE)

chain = (
    RunnablePassthrough.assign(context=lambda input: format_docs(input["context"]))
    | rag_prompt
    | model
    | StrOutputParser()
)

question = "What are the approaches to Task Decomposition?"

docs = vectorstore.similarity_search(question)

# Run
resp2 = chain.invoke({"context": docs, "question": question})
print(resp2)

There are three approaches to Task Decomposition. They include: (1) using Large Language Models (LLM) with simple prompting like "Steps for XYZ.", (2) employing task-specific instructions, and (3) incorporating human inputs. These approaches enable agents to break down large tasks into smaller, manageable subgoals.

9.3.3 Q & A with Retrival

retriever = vectorstore.as_retriever()

qa_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | rag_prompt
    | model
    | StrOutputParser()
)

question = "What are the approaches to Task Decomposition?"
qa_chain.invoke(question)

'Task decomposition can be done through three approaches: (1) using Large Language Models (LLM) with simple prompting, (2) employing task-specific instructions, and (3) utilizing human inputs. This process involves breaking down large tasks into smaller, manageable subgoals to enable efficient handling of complex tasks.'