John NnamaniAI engineering just got more interesting for me. My first project was docuquery — a local CLI tool that ingests your documents and lets you interact with them.
I had finished the LLM Zoomcamp course and reached the project stage where I had to build something demonstrating all the techniques we covered to get the certificate. I was stuck. The whole AI space was brand new to me, and even though I followed every step of the course, I didn't know how to start building on my own. There was just too much information to process at once.
To get unstuck, I had to start small. I decided to build smaller projects covering different sections of the course separately, working toward a final project that combined everything—Agentic RAG, embeddings and vector search, RRF and reranking, evaluation, and monitoring. The first project was docuquery. Here is how I built the CLI tool, the decisions I made, and what I learned along the way.
I had never built a CLI tool or published a Python package before, so I didn't know where to start. Researching how to build a Python package made it look simple enough—I just initialized a project using uv. That was the easy part. I also needed to collect data and create an interactive session on the CLI. After looking at a few packages, I chose Click because it was the easiest to work with.
Next, I mapped out what I wanted to build. I asked myself, "What do I want the outcome of this CLI tool to be?" I designed it on paper first, and that was exactly what I built. This post isn't about how to create a CLI tool, though—it's about how I utilized RAG to create docuquery.
How does Docuquery work?#
Docuquery ingests your documents by extracting the content, transforms it into vector embeddings using a sentence transformer, stores the content and the embeddings in a PostgreSQL database, and then queries the database using an embedded user query to answer questions.
The entry point#
Docuquery's entry point is a CLI tool. It includes several commands: init, ingest, ask, sessions, and resume.
init helps you set up your database, OpenAI API key, and optional settings for chunking, the OpenAI model, and the sentence transformer model.
After running init, you can ingest as many documents as you need using the ingest command. It takes one argument — the path to the PDF or text file. When you run it, the file is read, its content extracted and embedded, and everything is saved to the PostgreSQL database. From there, you can start interacting with the documents using the ask command.
The ask command#
The ask command uses a while loop to keep prompting you for questions about your ingested documents. Behind the scenes, your query is sent to the OpenAI model with a set of system instructions and tool definitions that the AI can call to gather context and return a tailored answer. The available tools are:
search(query, limit=5)— Search the user's ingested documents for passages relevant to a query.count_documents(query)— Search for documents and count the total number that match the query.list_documents(query, page=1, limit=5)— List documents, optionally filtering by a query string and supporting pagination.list_sessions(page=1, limit=5)— Get a list of conversation sessions.get_session_messages(session_id)— Retrieve the formatted conversation messages for a given session ID.
The LLM uses these tools to retrieve context based on your query. First, the query is embedded and used to run a vector search against the database. The retrieved data is then parsed and used to build the prompt passed to the LLM.
The ask command also remembers past conversations. If you asked questions in a session, you can ask follow-up questions without repeating yourself because the tool stores the conversation history and passes it as context to the LLM.
The sessions command#
To make this a complete CLI tool, I needed a way to view previous sessions and continue from them. That is what the sessions command does. When you run it, it calls the list_sessions function, formats the returned list of sessions with their titles, and prints them out.
The resume command#
If you can list sessions, you should be able to resume one — which is what the resume command does. It takes one argument: session-id. It fetches the session, retrieves the messages, and provides them as context to the LLM so it has full awareness of the conversation history. The ask command also supports this via a --session-id flag, and behind the scenes, the resume command simply invokes ask with that option.
Other tool functions#
Functions like list_documents, count_documents, and get_session_messages aren't tied directly to standalone top-level commands. Instead, they are passed as tools to the LLM, giving it the ability to query metadata and history as needed to answer questions accurately.
What I learned#
I learned that not everything needs to be done with the LLM while building a RAG system. I had an impressive idea to give the LLM the capability of loading conversation sessions and switching to them automatically without having to run the command separately. I did this and it was working perfectly, but that pushes it toward agentic behavior, and I wanted to stick to a straightforward RAG implementation first. So I left that idea for the next projects I will work on.
Next Steps#
If I continue working on this project, what I will do next is to add more features that will make the tool easier to use, and optimize it properly by adding the ability to compact messages when they get large to avoid high token costs. I also want to add monitoring and cost tracking, along with other ideas that will come naturally while working on it. For now, I will be focusing more on building the next project which will involve RRF, reranking, and hybrid search.
Conclusion#
That is the core of it. If you got to this point you must really love me so much to read all my ramblings. Either way, thanks for reading. If you have questions or thoughts, drop them below. A younger me would love to read about this, and this is a note to a younger me. Thank you!