Handwriting ML

Overview

Handwriting ML is a compact deep learning project focused on recognising handwritten mathematical symbols across a large class set. Built during a PyTorch-focused hackathon organised by IETE at Amrita, the project reflects a fast-paced, collaborative effort under constraints. The final model handled 369 symbol classes and achieved a runner-up position in the Torch It Up Kaggle competition.

This was developed in collaboration with Sanjai, and the project reflects a shared workflow where responsibilities were split dynamically based on what needed to move forward at any given moment.

---

Problem

Handwritten mathematical notation is inconsistent, ambiguous, and highly variable across individuals. Standard OCR systems struggle with this domain because symbols can look similar, overlap in meaning, or vary significantly in shape.

The challenge was to build a model that could reliably distinguish between hundreds of such symbols from grayscale images, despite limited time, compute resources, and the usual unpredictability of real-world handwritten data.

---

Solution

The approach was deliberately simple and pragmatic. Instead of overengineering, the focus was on getting a clean end-to-end pipeline working quickly, then iterating.

A convolutional neural network was implemented using PyTorch, trained on resized grayscale images. The architecture used a couple of convolutional layers followed by fully connected layers with dropout. Nothing exotic, but stable and effective under time pressure.

Work was split fluidly between Sanjai and me. I initially focused on setting up the data pipeline and preprocessing steps, while Sanjai worked on stabilising the training loop and refining the model behaviour. As the project progressed, we frequently switched roles, especially when debugging issues like tensor mismatches, inconsistent predictions, or training instability.

Google Colab was used as the execution environment, which introduced its own quirks. Sessions would reset, forcing re-runs and making checkpointing essential. At one point, a near-complete training run was lost due to a runtime timeout, which led to quickly adding model saving mid-training.

Despite the constraints, the pipeline came together:

• Data ingestion via CSV mappings
• Custom dataset class for preprocessing
• CNN model for classification
• Training loop with live metrics tracking
• Inference pipeline producing submission-ready outputs

The final model weights and predictions were exported for evaluation, completing the full cycle from raw data to competition submission.

---

Developer Notes

This project is straightforward to reproduce, but a few practical notes matter more than the setup steps.

Running the notebook in Google Colab is the intended path. The dataset zip can be placed in the root directory and extracted automatically through the notebook. Most of the workflow is self-contained inside main.ipynb.

During development, iteration speed mattered more than perfection. Batch sizes and epochs were adjusted based on runtime limits rather than ideal tuning. Debugging often involved printing tensor shapes and visually inspecting samples rather than relying on tooling.

Testing was informal but effective. Predictions were sanity-checked by manually inspecting outputs against input images. This caught several issues early, including incorrect label mappings and preprocessing inconsistencies.

If extending the project:

• Data augmentation is the easiest improvement path and was intentionally skipped due to time limits
• Switching to a deeper architecture like ResNet would likely improve performance but requires careful handling of input size and training time
• Hyperparameter tuning was minimal and can yield gains with structured experimentation

The collaboration itself was a key part of the experience. Working with Sanjai meant constantly syncing progress, quickly dividing tasks, and adapting when something broke. Many decisions were made based on what could be implemented and tested fastest rather than what was theoretically optimal.

Overall, this project represents a practical snapshot of applied machine learning under constraints, where execution speed, teamwork, and adaptability mattered as much as the model itself.