Data Science, Machine Learning and AI

Start the course: see how a data project runs from question to answer, then use plain Python to hold table rows as lists of dictionaries, read CSV and JSON text, count and group records, and write small functions you can trust. After this module you can load a small dataset with the standard library, filter, sort, count and total it, and check your own code with asserts.

1. What data science is, and how this course worksData science is the work of answering real questions with data: you ask a clear question, collect and clean the data, explore it, sometimes build a model, and then explain the answer to people. This course walks that path in 20 weeks at about 1 hour a day, using Python and its data tools.25 min
2. Python essentials for data: records, lists and dictionariesA table can live in plain Python as a list of dictionaries: each dictionary is one row, and its keys are the column names. List comprehensions filter and transform rows in one line, sorted() with key=lambda sorts by any column, and min, max, zip and enumerate finish the toolkit.30 min
3. Reading CSV and JSON with the standard libraryMost data arrives as CSV (rows of comma-separated text) or JSON (labelled, nested data from APIs). Python's csv module turns CSV text into lists or dicts, and json.loads turns JSON text into dicts and lists. Every CSV value arrives as text, so you must convert numbers yourself and decide what to do with blanks.30 min
4. Counting and grouping: collections and statisticsTwo questions come up in almost every analysis: "how many of each?" and "what is the total per group?". collections.Counter answers the first, and defaultdict makes grouping easy for the second. The statistics module gives mean, median and mode in one line.30 min
5. Writing data code that others can trustData results are only useful if people can trust and repeat them. Put each step in a small function with a clear name, check it with asserts on normal and edge cases, keep the raw data unchanged, fix random seeds, and record the libraries you used in a virtual environment.30 min

Descriptive statistics from zero, with every formula explained in words: the kinds of data, the centre (mean, median, mode), the spread (range, variance, standard deviation), percentiles and outliers, the normal curve and z-scores, and correlation. After this module you can describe any column of numbers honestly, compute each summary by hand in Python, spot outliers, compare scores from different exams, and explain why correlation does not prove causation.

1. Types of data, and the centre: mean, median, modeData is either numbers (counts or measurements) or categories (labels, sometimes with an order). To describe the "typical" value you use the mean (the average), the median (the middle value) or the mode (the most common value). When a few values are extremely large, like one huge salary, the median tells the truer story.30 min
2. Spread: range, variance and standard deviationTwo groups can have the same average but behave very differently. Spread measures how far values sit from the centre. The range is max minus min; the variance is the average squared distance from the mean; and the standard deviation is the square root of the variance, back in the original units. For a sample you divide by n − 1 instead of n.30 min
3. Percentiles, quartiles and outliersA percentile tells you where a value stands: the 90th percentile is the value that 90% of the data is at or below. Quartiles split sorted data into four parts, and the IQR (Q3 − Q1) is the spread of the middle half. Values more than 1.5 × IQR beyond the quartiles are flagged as outliers. Different tools compute quartiles in slightly different ways.30 min
4. The normal distribution and z-scoresMany measurements pile up in a symmetric bell shape called the normal distribution. In it, about 68% of values lie within 1 standard deviation of the mean, 95% within 2 and 99.7% within 3. A z-score says how many standard deviations a value is from the mean, which lets you compare scores from different tests fairly.30 min
5. Correlation: how two things move togetherCorrelation measures how strongly two numeric columns move together in a straight-line way. Pearson's r runs from −1 (perfectly opposite) through 0 (no straight-line link) to +1 (perfectly together). A strong correlation does not prove that one thing causes the other: a hidden third factor often drives both.30 min

Chance, conditional probability, Bayes' theorem, distributions, samples, confidence intervals and A/B tests. After this module you can say how sure you are about a number, read a p-value correctly and judge whether an app experiment really worked.

1. Probability basicsProbability is a number from 0 to 1 that says how likely something is: 0 means it never happens, 1 means it always happens. When all outcomes are equally likely, probability is the number of outcomes you want divided by the number of all outcomes. Simple rules let you combine chances with AND, OR and NOT.25 min
2. Conditional probability and Bayes' theoremConditional probability is the chance of A when you already know B happened, written P(A | B). Bayes' theorem turns P(B | A) around into P(A | B). It explains why a positive result from a good test can still be more likely wrong than right when the condition is rare.30 min
3. Random variables and distributionsA random variable is a number that comes from a chance process, like the number of customers in the next hour. Its distribution says how likely each value is. A few shapes appear again and again: uniform, binomial, Poisson and normal. The expected value is the long-run average.30 min
4. Samples, the central limit theorem and confidence intervalsWe usually study a sample, not everyone, so our numbers have some error. The standard error says how much a sample average would wobble from sample to sample. The central limit theorem says sample averages follow a bell curve, which lets us build a 95% confidence interval: a range that very likely contains the true value.30 min
5. Hypothesis tests, p-values and A/B testingA hypothesis test asks: could this difference be just luck? You assume there is no real difference (the null hypothesis) and compute a p-value, the chance of seeing a result at least this extreme if that were true. A small p-value is evidence of a real effect. An A/B test uses this to compare two versions of an app or page.35 min

NumPy arrays, pandas DataFrames, cleaning messy data, grouping and joining, exploratory data analysis and honest charts. After this module you can take a messy real-world table, clean it, summarise it by groups and show what it says in the right chart.

1. NumPy: fast arrays of numbersNumPy is the Python library for fast maths on large lists of numbers, called arrays. Instead of writing a loop, you write one expression and NumPy applies it to every element at once (vectorisation). Almost every data and ML library in Python is built on NumPy arrays.30 min
2. pandas basics: Series and DataFramespandas is the Python library for tables. A DataFrame is a table with named columns, and each column is a Series. You read a CSV with pd.read_csv, look at it with head, info and describe, pick rows and columns with filters, loc and iloc, and sort or add columns in one line.35 min
3. Cleaning messy dataReal data is messy: blank cells, repeated rows, numbers stored as text, extra spaces and the same city spelled three ways. Cleaning means finding and fixing these problems in a careful, repeatable way before any analysis. It often takes most of a data project's time.35 min
4. Grouping, joining and reshapingGrouping answers "per what?" questions: total sales per city, average marks per class. Joining combines two tables on a shared key, like orders and customers. Reshaping turns long tables into summary grids (pivot tables). pandas does these with groupby, merge and pivot_table.35 min
5. Exploratory data analysis (EDA)Exploratory data analysis means getting to know a dataset before you model it: its size, types, missing values, distributions, outliers and relationships. You ask questions, look at numbers and charts, and write down what you find. Good EDA prevents most later mistakes.30 min
6. Visualising data honestlyA good chart answers one question at a glance. Choose the chart by the question: bars to compare groups, lines for change over time, histograms for spread, scatter plots for relationships. Label everything, start bar axes at zero and use colours everyone can read.30 min

What machine learning is, the end-to-end workflow, linear regression, gradient descent and logistic regression. After this module you can frame a problem as regression or classification, build a baseline, fit a line by least squares, explain how gradient descent learns, and turn scores into yes/no decisions.

1. What machine learning isMachine learning (ML) is a way to make a computer learn a pattern from examples instead of writing every rule by hand. You give it features (the inputs) and, for supervised learning, labels (the right answers), and it learns to predict the label for new cases. Use ML when the pattern is real but too complex to write as rules.25 min
2. The ML workflow end to endEvery ML project follows the same loop: define the problem and metric, get and clean data, split it, build a simple baseline, train models, evaluate on held-out data, improve, deploy and monitor. scikit-learn gives every model the same simple interface: fit to learn, predict to use.30 min
3. Linear regression and least squaresLinear regression fits a straight line, y = w·x + b, through data to predict a number. The best line is the one with the smallest total squared error (least squares). For one feature, the slope is the covariance of x and y divided by the variance of x. The slope tells you how much y changes when x goes up by one.35 min
4. Gradient descent: learning step by stepMost models cannot be solved with a neat formula, so they learn by gradient descent: start with a guess, measure the error (the loss), work out which direction makes the loss smaller (the gradient), take a small step that way, and repeat. The learning rate sets the step size.35 min
5. Logistic regression and classificationLogistic regression predicts the probability of a yes/no outcome. It computes a score like linear regression, then squeezes it into the range 0 to 1 with the sigmoid function. A threshold, often 0.5, turns the probability into a decision. It is trained by minimising log loss.35 min

Train, validation and test sets, the confusion matrix, precision, recall and F1, thresholds and ROC-AUC, regression metrics, overfitting and regularisation, and cross-validation. After this module you can measure a model honestly, pick the metric that matches the business cost, and tell when a model is overfitting.

1. Train, validation and test setsA model must be judged on data it has never seen, so we split the data. The training set teaches the model, the validation set helps choose settings, and the test set is used once at the end for an honest score. Data leakage, when information from the answer sneaks into training, makes models look far better than they are.30 min
2. Confusion matrix, accuracy, precision, recall and F1A confusion matrix counts four outcomes of a yes/no model: true positives, false positives, false negatives and true negatives. Accuracy is the share right overall, which misleads when one class is rare. Precision asks "of the alarms, how many were real?"; recall asks "of the real cases, how many did we catch?"; F1 balances the two.35 min
3. Thresholds, ROC-AUC and imbalanced classesMost classifiers output a score, and the threshold decides what counts as positive. Moving the threshold trades precision against recall. The ROC curve shows this trade-off at every threshold, and AUC sums it up as one number: the chance that a random positive gets a higher score than a random negative.35 min
4. Regression metrics: MAE, MSE, RMSE and R²Regression models are judged by how far predictions are from actual numbers. MAE is the average size of the error, in the same units as the target. MSE averages squared errors and RMSE is its square root, both punishing big misses more. R² says what share of the variation the model explains compared with always predicting the mean.30 min
5. Overfitting, underfitting and regularisationUnderfitting means the model is too simple to capture the pattern, so it does badly on training and test data. Overfitting means it memorised the training data, including noise, so it does well on training but badly on new data. Regularisation, more data, simpler models and early stopping help a model generalise.30 min
6. Cross-validation and tuning hyperparametersCross-validation splits the training data into k parts (folds), trains k times, each time testing on a different fold, and averages the scores. It gives a steadier estimate than one split. Hyperparameters are settings you choose before training, like tree depth; you tune them with cross-validation and keep the test set for the very end.30 min

k-nearest neighbours, decision trees, random forests, gradient boosting and how to choose and explain a model. After this module you can explain how each of these models decides, code their core ideas by hand, and pick a sensible model for a tabular dataset.

1. k-nearest neighboursk-nearest neighbours (k-NN) predicts by looking at the k most similar training examples: for classification it takes a vote of their labels, for regression it averages their values. "Similar" means close by a distance measure, so features must be on similar scales. It needs no training, but predictions get slow on big data.30 min
2. Decision treesA decision tree makes a prediction by asking a series of yes/no questions about the features, like "income above ₹50,000?". It learns which question to ask at each step by choosing the split that makes the groups purest, measured by Gini impurity or entropy. Trees are easy to read but overfit if they grow too deep.35 min
3. Random forests and baggingA random forest trains many decision trees, each on a random resample of the data and with a random subset of features at each split, then combines their votes. Individual trees overfit in different ways, and averaging cancels much of that noise. Forests are strong, reliable defaults for tabular data.30 min
4. Gradient boostingGradient boosting builds trees one after another, where each new small tree tries to fix the mistakes (residuals) left by all the trees before it. A learning rate shrinks each tree's contribution so the model improves carefully. Libraries like XGBoost, LightGBM and CatBoost make it fast, and it is often the most accurate choice for tabular data.30 min
5. Choosing and explaining a modelNo single model is best for every problem. Choose by the data size, the need for explanations, speed and accuracy, and always compare with a baseline. Naive Bayes and SVMs are other classic options. Permutation importance and SHAP help explain which features drive a model's predictions.30 min

k-means and other clustering, PCA, encoding categories and dates, scaling, feature selection, leakage and pipelines. After this module you can segment customers without labels, turn raw columns into features a model can use, and build a leak-free preprocessing pipeline.

1. Clustering with k-meansClustering groups similar examples without any labels. k-means picks k centre points (centroids), assigns every example to its nearest centroid, moves each centroid to the average of its group, and repeats until nothing changes. You choose k, often with the elbow method or the silhouette score.35 min
2. Other clustering methods and dimensionality reductionHierarchical clustering builds a tree of groups you can cut at any level. DBSCAN finds dense regions of any shape and marks lonely points as noise. PCA reduces many columns to a few new ones that keep most of the variation, which helps visualisation and speed. t-SNE and UMAP make pictures of high-dimensional data, but only for looking.30 min
3. Encoding categories and datesModels need numbers, so text categories and dates must be turned into numeric features. One-hot encoding makes a 0/1 column per category. Ordinal encoding numbers categories that have a real order. Dates become useful parts such as month, day of week and "is it a festival week?". This is the heart of feature engineering.30 min
4. Scaling and transforming numbersScaling puts numeric features on similar ranges. Min-max scaling maps values to 0–1; standardisation subtracts the mean and divides by the standard deviation. Distance-based and gradient-based models need it; tree models do not. Fit the scaler on training data only. Log transforms tame skewed values like incomes.30 min
5. Feature selection, leakage and pipelinesFeature selection keeps the features that help and drops the ones that add noise, cost or risk. Leakage features must always go. A scikit-learn Pipeline chains preprocessing and the model so every step is fitted on training data only, which prevents leakage and makes deployment simple.35 min

The artificial neuron, layers and activations, training with backpropagation and optimisers, CNNs, RNNs and transformers, and the tools and hardware of deep learning. After this module you can explain how a neural network turns inputs into predictions and learns from its mistakes, compute a forward pass by hand, and plan a small deep learning project.

1. The artificial neuronAn artificial neuron multiplies each input by a weight, adds them up with a bias, and passes the total through an activation function to produce an output. The perceptron is the simplest neuron: it outputs 1 or 0 and learns by nudging its weights after every mistake. One neuron can only draw a straight dividing line, which is why we need networks of them.30 min
2. Layers, activations and the forward passA neural network stacks neurons into layers: an input layer, one or more hidden layers and an output layer. Each dense layer multiplies its inputs by a weight matrix, adds biases and applies an activation such as ReLU. Non-linear activations let networks learn curved patterns. Softmax turns the final scores into class probabilities.35 min
3. Training: loss, backpropagation and optimisersA network learns by repeating four steps: a forward pass to predict, a loss that measures the error, backpropagation to work out how each weight affected the loss (its gradient), and an optimiser step that nudges every weight to reduce the loss. Tools like dropout, weight decay and early stopping keep the network from overfitting.35 min
4. CNNs, RNNs and transformers at a glanceDifferent data shapes need different network designs. Convolutional neural networks (CNNs) slide small filters over images to find edges, textures and objects. Recurrent networks (RNNs, LSTMs) read sequences step by step. Transformers use attention to look at all parts of a sequence at once and now power most language and many vision models. Transfer learning reuses a pre-trained network for a new task.30 min
5. Deep learning tools and hardwarePyTorch is the most widely used deep learning framework today, with TensorFlow/Keras and JAX as other options. Training is fast on GPUs because neural networks are mostly matrix maths, which GPUs do in parallel. A model's memory need is roughly its number of parameters times the bytes per parameter. Free notebooks like Google Colab and Kaggle let you start without buying hardware.25 min

Cleaning and tokenising text, bag of words and TF-IDF, embeddings and cosine similarity, how large language models work, and how to prompt them well. After this module you can turn text into numbers, build a simple text classifier and semantic search, explain tokens, context windows and temperature, and write clear, safe prompts.

1. Working with text: tokens and cleaningNatural language processing (NLP) is about making computers work with human language. The first step is tokenising: splitting text into pieces such as words. Then we often lowercase, remove punctuation and very common "stop words", and count what is left. Indian languages, mixed scripts and Hinglish make this harder, so cleaning choices must fit the data.30 min
2. Bag of words, TF-IDF and text classificationA bag of words turns each document into counts of every word in a vocabulary, ignoring order. TF-IDF improves this by giving high weight to words that are frequent in one document but rare across all documents. These vectors feed simple, fast classifiers like Naive Bayes or logistic regression, which work well for spam, sentiment and ticket routing.35 min
3. Embeddings and cosine similarityAn embedding is a list of numbers (a vector) that represents the meaning of a word, sentence, image or product, so that similar things get similar vectors. Cosine similarity measures how closely two vectors point in the same direction, from −1 to 1. Embeddings power semantic search, recommendations and RAG.35 min
4. How large language models workA large language model (LLM) is a huge transformer network trained to predict the next token of text. It reads text as sub-word tokens, sees a limited context window, and generates answers one token at a time by sampling from its predicted probabilities. Temperature controls how random that sampling is. Because it predicts likely text, not checked facts, it can confidently make things up.35 min
5. Prompting wellA prompt is the instruction and context you give an LLM. Good prompts are clear and specific: say the task, give the needed context, show an example or two, state the output format, and allow the model to say "I don't know". Test prompts on real examples, treat any pasted text as untrusted, and never paste secrets or personal data you are not allowed to share.30 min

Retrieval-augmented generation, fine-tuning and when not to, evaluating LLM apps, AI safety, and MLOps from notebook to production. After this module you can design a RAG feature, decide between prompting, RAG and fine-tuning, measure an AI feature with a test set, add safety guardrails, and plan how a model is deployed and monitored.

1. Retrieval-augmented generation (RAG)RAG lets an LLM answer from your own documents. You split documents into chunks, turn each chunk into an embedding, and store them. For each question you retrieve the most similar chunks and put them in the prompt with an instruction to answer only from them and cite them. This gives fresher, private and checkable answers with fewer hallucinations.35 min
2. Fine-tuning, and when not toFine-tuning continues training a pre-trained model on your own examples so it learns a style, format or narrow task. It does not reliably teach new facts; RAG is usually better for knowledge. Try good prompting first, then RAG, and fine-tune when you have many high-quality examples and a stable task. LoRA makes fine-tuning cheaper by training small extra matrices.30 min
3. Evaluating LLM appsYou cannot improve what you do not measure. Build a test set of real questions with expected answers, and score every version of your prompt or system on it. Use exact match and token-overlap F1 where answers are short, retrieval recall for RAG, and human review or a carefully checked LLM judge for open-ended answers. Track cost and latency too.35 min
4. AI safety and responsible deploymentAI features can fail in harmful ways: confident wrong answers, leaking private data, following instructions hidden in documents (prompt injection), or producing biased or toxic text. Safe deployment layers defences: limit what the model can do, filter inputs and outputs, mask personal data, keep humans in the loop for risky actions, log everything and test with red-teaming before and after launch.30 min
5. MLOps: from notebook to productionMLOps is the practice of running ML models reliably in production: versioning code, data and models, tracking experiments, packaging and serving models, monitoring their inputs and accuracy, and retraining when the world changes. A model is not finished when it scores well in a notebook; that is where its working life begins.35 min

Bias and fairness, privacy and data protection, the four main career paths, building a portfolio and an end-to-end capstone project. After this module you can check a model for unfair outcomes, handle personal data responsibly under India's DPDP Act, choose a career path with a realistic plan, and present a complete project to employers.

1. Bias and fairnessA model trained on past data can repeat and even amplify past unfairness, for example approving loans or shortlisting candidates less often for some groups. Bias can come from history, from who is in the data, from labels and from proxy features like pin code. Measure outcomes per group with simple fairness metrics, then fix the data, the model or the decision process.35 min
2. Privacy and data protectionPersonal data is any information about an identifiable person. Use it only for a clear purpose people agreed to, collect as little as you need, protect it, and delete it when done. Removing names is often not enough to make data anonymous. In India, the Digital Personal Data Protection Act, 2023 (DPDP Act) sets the rules; the EU's GDPR applies to EU users.35 min
3. Career paths: analyst, scientist, ML engineer, AI engineerData careers share a base of Python, SQL and statistics but differ in focus. Data analysts answer business questions with SQL, spreadsheets and dashboards. Data scientists build models and run experiments. ML engineers put models into reliable production systems. AI engineers build products on top of LLMs and other pre-trained models. Pick a starting path, build proof of skill, and grow from there.30 min
4. Portfolio, projects and interviewsEmployers want proof that you can turn messy data into useful answers. Build two or three solid projects with real public data, each with a clear question, honest evaluation and a readable write-up on GitHub. Prepare for interviews that test SQL, Python, statistics, ML concepts, a case study and how you communicate.30 min
5. Capstone: an end-to-end projectThe capstone puts the whole course together: frame a question, get and clean data, explore it, build a baseline and better models, evaluate them honestly, check fairness and privacy, and present the result clearly, ideally with a small demo. Doing one project end to end teaches more than any single module.40 min