This project was completed during my time at Grabango — a checkout-free tech startup leveraging computer vision technology and machine learning. The goal was to improve the user experience of the internal data annotation tool, a key part of the company's core technology that provides the ground truth for training its machine learning models.
Throughout this project, I collaborated closely with a product manager and developer, designing an improved annotation flow for behind-the-counter (BTC) items that require ID verification, such as cigarettes and other nicotine products. Integrating automation to streamline the workflow and improve annotator efficiency.
The existing annotation workflow for behind-the-counter items highly manual and error-prone, resulting in frequent inaccuracies and increased annotation time.
To capture BTC items, annotators previously had to manually create a window of time to capture relevant scans and then timestamp pick up actions when clerks hand off the items to shoppers using the “clerk pick up” action. The system would then try to match pick up action to the corresponding scans and estimate the quantity of items based on the time alignment and the number of scans.
Eliminated the manual time windowing task using data-driven rules to automatically capture scans likely tied to the visit. Introduced a dynamic scan table displaying all relevant scans ranked by relevance based on scan timestamp.
Replaced "Clerk Pick Up" with a unified "Pick Up" action. Designed a three-tab interface within the product ID screen. Quantity is inferred from the number of scans and supporting video footage.
Add a built-in escalation feature in which annotators can flag an issue directly without having to switch contexts and disrupting the annotation workflow.
Throughout this project, I uncovered key pain points and focused on driving improvements that not only enhanced the annotation experience but also optimized workflows for greater efficiency. In close collaboration with product and engineering teams, I gained valuable insights into aligning design decisions with business objectives while advocating for usability.
Looking ahead, future improvements could include features like expanding the scan window when no matches are found and visualizing scans on a timeline relative to the pick-up event, providing greater context and clarity in the annotation process.