Project · Data Science, Machine Learning and AI
Project: the retriever for a bank FAQ chatbot
The brief
A bank wants a chatbot that answers only from its help-centre pages. Build the retrieval half of RAG from scratch: tokens, TF-IDF vectors, cosine similarity, top-k search, a grounded prompt that refuses when nothing matches, and a hit-rate evaluation.
Why it matters
Retrieval-augmented generation (RAG) is how banks, telecoms, insurers and government help desks put LLMs in front of customers without the model making things up. AI engineers spend most of their time on the retriever and its evaluation, because a wrong chunk gives a confident wrong answer. Embeddings replace TF-IDF in production, but the pipeline and the evaluation are the same.
Skills you practise
- tokenising text
- TF-IDF
- cosine similarity
- top-k retrieval
- grounded prompts and refusal
- evaluating retrieval with hit rate
What you will have at the end
retriever.py and a short evaluation note: hit rate at k=1 and k=2, the question that failed and why, and what you would change (embeddings, more chunks, a higher min_score).
How it is marked (rubric, 100 points)
| Correct retrieverAll seven checks pass: tokens, IDF, vectors, cosine, top-k, the prompt and the hit rate. | 40 |
|---|---|
| Safe designThe prompt keeps the model to the context, cites chunk ids, and returns None when nothing relevant is found. | 25 |
| EvaluationYou report hit rate at two values of k and explain the failing question in plain words. | 20 |
| Clear codeEach function is small and reusable, ready to swap TF-IDF for embeddings later. | 15 |
0/7 steps done
Step 1 of 7
Tokenise the text
tokenize(text) returns lower-case words and numbers without stopwords: "How do I reset my UPI PIN?" gives ["reset", "upi", "pin"].
Search works on tokens, not raw text. Lower-case the text so "PIN" and "pin" match, and pull out runs of letters and digits with re.findall("[a-z0-9]+", ...), which also drops punctuation. Then remove stopwords: very common words like "how" and "my" that appear everywhere and carry little meaning. chunk_tokens() is given: it tokenises a chunk's title and text together.
Run shows what your code prints. Check runs the step's tests. Ctrl+Enter checks too.