| Name | Student ID |
|---|---|
| Trần Hoàng Vỹ Khang | 2352502 |
| Đỗ Đàng Khoa | 2352554 |
| Dương Đàng Khoa | 2352551 |
| Milestone | Due | Minimum Required | State |
|---|---|---|---|
| M1 Draft (25%) | 23 Sep 2026 | EDA, Dataset/DataLoader, train/val loop, Linear + MLP runnable | Complete ✓ |
| M2 Final (75%) | 21 Oct 2026 | All five models, full comparison, report, slides, video, checkpoints | Not started |
The goal is to compare neural network architectures (Linear, MLP, CNN, LSTM/GRU, Transformer) on Fashion-MNIST classification and understand how each architectural choice affects performance. Some classes like Trouser or Bag are easy to separate by shape alone, while Shirt/Pullover/Coat/T-shirt share similar silhouettes and need the model to learn texture features — making this a useful benchmark for comparing model capacity.
Dataset: Fashion-MNIST (Xiao et al., 2017) — 70,000 grayscale 28×28 images, 10 classes.
Full EDA: notebooks/eda.ipynb (executed, outputs saved).
| Item | Value |
|---|---|
| Input shape | 1 × 28 × 28 (784 features when flattened) |
| Classes | 10 — T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot |
| Class balance | Perfectly balanced; imbalance ratio = 1.0000, normalised entropy = 1.0000 |
| Data split | 54,000 train / 6,000 val (stratified, seed 42) / 10,000 official test |
| Preprocessing | ToTensor + Normalize(mean=0.2860, std=0.3530); no augmentation |
| Trivial baseline | 10% (uniform random guess) |
Dataset is perfectly balanced, so accuracy ≈ macro-F1 throughout.
src/data/dataset.py) — stratified 90/10 split, seed 42, normalisation, DataLoaders.src/engine/trainer.py) — train/val loop, early stopping (patience 5), checkpoint on best val macro-F1.src/models/linear.py) — 784 → 10, 7,850 params.src/models/mlp.py) — 784 → 256 → 10, ReLU, dropout 0.2, 203,530 params.metrics.py (accuracy, macro-F1), plots.py (curves, confusion matrix, sample grids), seed.py.train.py and eval.py.CNN, LSTM/GRU, Transformer — in repo but empty, part of M2 Final.
| Setting | Value |
|---|---|
| Optimizer | Adam, lr = 0.001, weight decay = 0, batch size = 128 |
| Max epochs / early stop | 15 epochs, stop after 5 with no improvement |
| Checkpoint metric | Best validation macro-F1 |
| Seed | 42 (random, numpy, torch, torch.cuda) |
| Hardware | NVIDIA RTX 3050 Laptop (4 GB), Windows 11 |
| Software | Python 3.13.14, PyTorch 2.14.0+cu130, torchvision 0.29.0+cu130, NumPy 2.5.3, scikit-learn 1.9.1 |
Test split is never seen during training; evaluated once per model after training.
Test split is the official 10,000-image held-out set:
| Model | Params | Test Acc. | Test Macro-F1 | Best Epoch | Train Time |
|---|---|---|---|---|---|
| Linear / Softmax | 7,850 | 0.8440 | 0.8441 | 11 / 15 | 87 s |
| MLP (784→256→10, dropout 0.2) | 203,530 | 0.8884 | 0.8878 | 15 / 15 | 91 s |
| Class | Linear | MLP | Gain |
|---|---|---|---|
| Shirt | 0.5999 | 0.7035 | +0.104 |
| Pullover | 0.7323 | 0.8098 | +0.077 |
| Coat | 0.7442 | 0.8177 | +0.074 |
| T-shirt/top | 0.8073 | 0.8357 | +0.028 |
| Dress | 0.8492 | 0.8901 | +0.041 |
| Sneaker | 0.9238 | 0.9443 | +0.021 |
| Sandal | 0.9314 | 0.9624 | +0.031 |
| Bag | 0.9355 | 0.9744 | +0.039 |
| Ankle boot | 0.9499 | 0.9557 | +0.006 |
| Trouser | 0.9672 | 0.9840 | +0.017 |
MLP gains +4.4 macro-F1 points vs. linear, using 26× more parameters. Most of the gain is on the three hardest upper-body classes (Shirt, Pullover, Coat: avg. +0.085), vs. only +0.023 on the other seven — matches the EDA prediction. MLP stopped at epoch 15/15 with F1 still rising, so 0.8878 is a lower bound.
Train time is unreliable for speed comparison (num_workers=0, bottleneck is the data loader).