Visual Recognition Pipeline for Industrial Parts & Fasteners

Objectives

What the project needed to achieve

• Hit ≥ 95% top-1 accuracy across 6,200 SKUs from warehouse photographs
• Keep training cost under ₹2 lakh end-to-end
• Ship an adapter small enough to deploy on commodity inference GPUs (T4, L4, RTX 4090)
• Make the fine-tuning pipeline reproducible so the client can extend it as the catalog grows

Challenge

Engineering constraint

The client needed visual SKU recognition for receiving, kitting, and stock-counting workflows but couldn’t justify the GPU budget for training a custom vision model from scratch (estimated ₹45 lakh on cloud compute alone for the dataset size). Off-the-shelf CLIP got them to 78.1% zero-shot accuracy across the 6,200 SKU catalog, which was useful but not deployable. They needed to close the gap to >95% while keeping training cost in the ₹1–2 lakh range.

Approach

How YantriX approached the work

1. 01

   Curated a labeled dataset of 18,000 warehouse photographs spanning the SKU catalog — mix of receiving-dock photos and existing product-listing images, with active-learning to focus labeling effort on SKUs CLIP zero-shot struggled with.

2. 02

   Applied LoRA adapters (r=16, alpha=32) to the OpenCLIP ViT-L/14 visual encoder, with the text encoder frozen — standard PEFT pattern for vision-language adaptation.

3. 03

   Trained on a single RTX 4090 with mixed-precision BF16 for 11 hours over 18 epochs, with cosine learning-rate schedule and a held-out test set per SKU class.

4. 04

   Compared against full fine-tuning (1.7 GB checkpoint, 4× the training time, 2× the GPU memory) and DoRA (slightly better accuracy at the same parameter count); shipped LoRA for its operational simplicity.

5. 05

   Built a serving setup using Predibase’s multi-adapter pattern so the same base CLIP model can serve multiple LoRA adapters (e.g. one per warehouse) on a single GPU without per-adapter deployment cost.