Capstone · Data Science, Machine Learning and AI
Capstone: a scam SMS filter for a bank
The brief
Build a scam SMS filter from raw, messy data to a documented model: clean the messages, tokenise them, split them fairly, train a naive Bayes classifier from scratch, measure it with a confusion matrix, precision, recall and F1, choose the threshold on validation data, explain what the model learned and publish a model card.
Why it matters
Banks, telecom operators and payment apps fight KYC, lottery, loan and "electricity cut" scams every day, and scam filters like this one protect customers who are new to digital payments. The full workflow here (clean, split, train, evaluate, choose a threshold, explain, document) is what a data scientist or ML engineer is hired to do, and a finished capstone with a model card is a strong portfolio piece.
Skills you practise
- data cleaning
- tokenising text
- stratified train/test split
- naive Bayes with smoothing
- confusion matrix, precision, recall and F1
- choosing a threshold on validation data
- explaining a model
- writing a model card
What you will have at the end
filter.py and model_card.json: a working scam SMS filter written from scratch, its test results, the words it relies on and an honest list of limits. Add a short README and it is ready for your portfolio.
How it is marked (rubric, 100 points)
| Correct pipelineAll eleven checks pass, from cleaning to the model card. | 35 |
|---|---|
| Honest evaluationThe test set is used once, at the end; the threshold is chosen on validation data; duplicates cannot leak between train and test. | 25 |
| Explainability and documentationThe top words and the model card explain what the model does, how well, and where it fails. | 20 |
| Code qualitySmall, named functions that reuse each other, with no repeated logic. | 10 |
| Responsible useYou can explain the cost of a false positive and a false negative for bank customers, and why the filter warns rather than deletes. | 10 |
0/11 steps done
Step 1 of 11
Load and clean the messages
load_messages(SMS_CSV) returns 52 (label, text) pairs: labels cleaned, the label typo, the blank message and the repeated message left out.
Clean before anything else. Strip and lower-case each label and keep only "scam" and "ham", so a typo such as "spam?" does not become a third class. Strip each text and skip blanks. A message forwarded twice would sit in both train and test later and make the model look better than it is, so keep only the first copy of each text.
Run shows what your code prints. Check runs the step's tests. Ctrl+Enter checks too.