← Back to Course Landing Page

Assignment 2 – Deep Learning on Large-Scale Data for Specialized Tasks

Topic: Monocular Depth Estimation on DIODE Dataset (Indoor & Outdoor)

Course: CO3133 Deep Learning and Its Applications | Semester: 261 | Instructor: Lê Thành Sách

Group Members (Group CR7)

Attribution note. Every log entry in AI_USAGE.md records Đỗ Đăng Khoa as the member who carried out the M1 work, including the preprocessing, visualization and formatting listed above. This table is kept consistent with that record rather than describing an intended split: each member is credited only with work they performed, so the M1 workload is reported as uneven.

Milestone Status

1. Problem Statement

• Task Goal: Monocular Depth Estimation — predict physical distance (in metres) from camera per pixel from a single 2D RGB photo.
• Key Challenge: Scale ambiguity in 2D photos (a small nearby object vs. a giant distant object can take up identical pixel sizes). Model must learn visual perspective, texture gradients, and boundary cues.
• Data Scope: Evaluated on both Indoor (1–9m close-range) and Outdoor (up to 100–200m long-range) scenes. Ground truth is measured by LiDAR laser scanners. Missing laser reflections (sky, glass, sharp edges) use a validity mask (1 = valid depth, 0 = ignore).

2. Dataset Description & Preliminary EDA

2.1 Dataset Overview & Processing Workflow

• Source Dataset: DIODE — Dense Indoor and Outdoor DEpth (Vasiljevic et al., 2019), diode-dataset.org. Obtained through a Kaggle mirror of the official release. The upstream release totals 26,229 RGB images (train 8,574 indoor + 16,884 outdoor; val 325 indoor + 446 outdoor), which matches the mirror we downloaded.
• Version: the original 2019 DIODE release, obtained from the Kaggle mirror artemmmtry/diode-a-dense-indoor-and-outdoor-depth-dataset, Kaggle Version 8 (199.89 GB, last updated two years ago). Not DIODEv2: the mirror holds 26,229 images, exactly the 25,458 training plus 771 validation images of the 2019 release, and its data sources contain only train/ and val/.
• License: MIT, and consistent at both levels, which was checked rather than assumed: the upstream release states MIT on diode-dataset.org and in the official devkit repository diode-dataset/diode-devkit, and the Kaggle mirror we downloaded declares MIT in its own License field. The mirror therefore imposes no terms beyond the upstream licence, and redistribution of a derived subset is permitted provided the citation below is kept.
• Citation: I. Vasiljevic, N. Kolkin, S. Zhang, R. Luo, H. Wang, F. Z. Dai, A. F. Daniele, M. Mostajabi, S. Basart, M. R. Walter and G. Shakhnarovich. DIODE: A Dense Indoor and Outdoor DEpth Dataset. CoRR abs/1908.00463, 2019.
• Sample Volume: 7,195 RGB–Depth pairs (4,995 train, 1,100 val, 1,100 test across 15 scenes).
• Sample Modality: Each sample consists of an RGB image, a high-precision laser depth map in metres, and a binary validity mask indicating measured laser reflections.
• Processing Rule: Raw data is loaded dynamically during training via PyTorch DataLoaders; raw data files are kept local and excluded from repository version control.

2.2 Data Splitting Strategy (Scene-Level Split)

• Leakage Prevention: Split strictly by Scene (room / location), not by individual image. Photos from the same room cannot appear in both train and test.
• Split Ratio: ~70% Train / 15% Val / 15% Test (seed 42). Verified 0% scene overlap across splits (see scenes_used.csv).
• Sampling Rule: Max 700 images per indoor scene, max 400 per outdoor scene (seed 42). Cap spreads the subset across 15 scenes instead of exhausting one large scene.
• Note: All three splits come from the official DIODE training partition (no public test set is available), so our numbers are not directly comparable to published DIODE benchmarks.

2.3 Visual Inspection & Data Characteristics

• Visual Alignment: Nearby ceiling lamps and pipes display in cool blue; far background walls shift to dark red. Outdoor paths gradually transition from blue to red (~40–50m).
• Edge Noise (Edge Bleeding): Indoor invalid pixels appear mainly as thin boundary outlines around sharp object contours where laser reflection drops.
• Outdoor Sky Region: Sky, fine branches, and objects beyond sensor range are masked as invalid (e.g. crane photo is mostly sky, leaving 6.9% valid pixels).

2.4 Distribution Analysis & Noise Handling

• Outlier Threshold Cap: Pixels with depth > 50m (indoor) or > 200m (outdoor) are masked out (set mask = 0). Removes < 0.01% of pixels — only extreme sensor noise.
• Camera Tilt Finding: DIODE only has vertical tilt codes 000 to 050 (level to upward view, no downward-facing angles). This is a property of the original dataset, not our filtering.

3. Methodology

3.1 Preprocessing Pipeline

• Image Resizing: 1024×768 → 384×288 (RGB resized with bilinear; depth & mask with nearest-neighbour to avoid mixing in invalid zero values).
• RGB Normalization: Scaled to [0, 1] + standard ImageNet mean/std normalization.
• Depth Target: Kept in true physical metres; invalid pixels and depth > 50m/200m are masked out during loss computation.
• Augmentation: Synchronous Random Horizontal Flip (p = 0.5) applied simultaneously to RGB, depth, and mask.

3.2 Baseline & Pretrained Models

• Baseline Model: Lightweight U-Net (encoder–decoder with skip connections) trained from scratch.
• Pretrained Model: MiDaS Small (EfficientNet-Lite3 backbone, 21.3M parameters), predicting inverse depth (`1 / output`) trained with masked SILog loss.
• Loss Function: Masked Scale-Invariant Logarithmic (SILog) loss (Eigen et al., 2014) computed over valid pixels.

3.3 Evaluation Metrics

• Abs Rel (Absolute Relative Error): `(1/N) * sum(|d - d*| / d*)` → Lower is better (↓).
• RMSE (Root Mean Squared Error): `sqrt((1/N) * sum((d - d*)^2))` → Lower is better (↓).
• Threshold Accuracy (δ < 1.25^k): % of valid pixels with `max(d/d*, d*/d) < 1.25^k` for k = 1, 2, 3 → Higher is better (↑).

4. Experimental Setup

4.1 Controlled Experiment Design

• Research Question: Does full fine-tuning (Option B) outperform encoder-frozen fine-tuning (Option A)?
• Variable Factor: Option A: Frozen encoder (decoder only) vs. Option B: Full fine-tuning (all layers).
• Controlled Factors: Same split, preprocessing, SILog loss, AdamW, batch size 8, 20 epochs.
• Runs & Seeds: 3 seeds (42, 43, 44), reporting Mean ± Std Dev.
• Hypothesis: Full fine-tuning adapts the MiDaS encoder to DIODE's depth scale, so Option B achieves lower Abs Rel than Option A.

4.2 Hardware & Speed Benchmark

• Hardware: NVIDIA GeForce RTX 4060 Laptop GPU (8 GB VRAM).
• Inference Speed (Batch 8): 4.9 ms / image (~205 FPS), using 0.42 GB VRAM.
• Training Speed (FP32, Batch 8): ~1.5 min / epoch (3.2 GB VRAM).
• Training Speed (AMP Mixed Precision, Batch 8): ~1.0 min / epoch (2.0 GB VRAM).
• Total Compute Budget: 6 runs × 20 epochs = ~3 GPU hours. Total estimate including baseline & 2× thermal throttling buffer: ~6–8 GPU hours (runs locally on laptop).

5. Results

6. Comparison and Discussion

7. Error Analysis

8. Limitations and Conclusion

8.1 Dataset & Pipeline Limitations

1. Outdoor val has higher sky/invalid pixel rate (~44% vs ~30% in train).
2. Laser ground truth has minor noise at sharp object edges (edge bleeding).
3. DIODE only has upward/level camera angles (000–050); no downward floor views.
4. Downsampling to 384×288 smooths fine details like wires and thin branches.
5. All three splits come from the DIODE training set (no public test set available), so results are not directly comparable to published DIODE benchmarks.

8.2 M1 Conclusion

Our dataset subset is verified clean and free of scene leakage. Local GPU benchmarks confirm fast fine-tuning (~1 min/epoch with AMP), demonstrating that our planned controlled experiment is completely feasible on local hardware within 6–8 GPU hours.

9. Links & Resources

• Source Code Repository: GitHub Repository (assignment2/)
• Checkpoint Instructions & Pretrained Weights: Instructions to Reproduce Checkpoints
• Report & Presentation Slides: (To be added for M3 Final – Due 11 Nov 2026)
• YouTube Presentation Video: (To be added for M3 Final – Due 11 Nov 2026)
• AI Usage Disclosure: AI_USAGE.md