The Client

A Municipal Government Agency

Our client is a municipal department responsible for ensuring the cleanliness, safety, and functionality of urban spaces. They oversee street cleaning, waste management, and infrastructure maintenance to address issues like litter, potholes, and damaged public amenities.

PROJECT REQUIREMENTS

Large-Scale Image Annotation for Training a Street Maintenance System

The client sought our expertise in image annotation to get training datasets for an AI-powered system capable of identifying and categorizing certain objects on the street. We were tasked with annotating the following objects accurately in a large dataset of over 3000 images:

  • Public waste bins
  • Potholes and road damage
  • Manhole covers
  • Litter and debris

The goal was to ensure the street maintenance system could efficiently recognize these elements under varying conditions. Additionally, the client was concerned about the security of their data.

PROJECT CHALLENGES

Tackling Object Visibility, Obstruction, and Diversity Issues for Precise Image Labeling

The provided dataset presented several challenges for our team, primarily because of:

  • Image Quality Variations: The street images were captured under diverse conditions, including rain, fog, and different times of the day, which impacted the visibility of objects.
  • Urban Obstructions: Many images contained pedestrians, vehicles, and other urban elements that partially obscured the target objects, making precise labeling difficult.
  • Object Diversity: The appearance of target objects varied greatly across different urban environments. For instance, public bins and manhole covers had different designs in various city districts, while litter and potholes came in countless shapes and sizes. This diversity made it challenging to apply a one-size-fits-all annotation approach.
OUR SOLUTION

Balancing Accuracy and Efficiency in Large-Scale Image Annotation with a Multifaceted Approach

We employed a team of 10 experienced annotators who specialized in working on the client-specified image annotation tool, CVAT. For precise object detection and labeling, our annotators used bounding box annotation and image segmentation techniques.

A Collaborative Approach to High-Quality AI Training Data

To address the project challenges and maintain consistency across all annotations, we implemented a strategic approach that involved:

Annotator Training

Given the diversity of target objects, we developed labeling protocols and provided 3-days training to our annotators, focusing on the specific urban elements and potential variations they might encounter. We also trained them to handle diverse image conditions and correctly identify litter amidst various urban clutter.

Integrated Client Feedback Cycle

We conducted weekly feedback sessions with the client to refine our annotation process. Additionally, we developed a dynamic workflow that allowed for quick adjustments based on client feedback.

Data Security and Quality Assurance

As data security was one of the client's primary concerns, we ensured intellectual property (IP) protection throughout the project. We adhered to strict non-disclosure agreements and employed robust security measures, including role-based access control and secure file sharing. To uphold accuracy and consistency across annotations, we implemented a human-in-the-loop approach. From defining attributes in the CVAT for automated labeling to verifying the accuracy of images labeled by the tool, we involved our subject matter experts at every stage. Each image underwent a three-tier review process: first by peer annotators, then by senior annotators, and finally by project managers who scrutinized the labeled data, ensuring it met the client's quality standards.

How did we handle complex scenarios?

  • We implemented a "group annotation" approach for edge cases, where multiple annotators collaborated in real-time to reach a consensus.
  • We developed a decision tree to guide annotators through ambiguous scenarios, ensuring consistent decision-making across the team.

Project Outcomes

Despite the challenges of variable image quality and urban obstructions, we maintained a 98% annotation accuracy rate, as verified by the client's quality control team. Additionally, enhanced object recognition capabilities of the AI-powered tracking system allowed the client to respond more quickly to urban maintenance needs. The delivered training datasets helped the client achieve the following:

45% Improvement in Object Detection Accuracy of the AI Street Maintenance System

30% Reduction in Operational Costs as the Need for Manual Inspections Reduced with Improved AI System Accuracy

CONTACT US

Expand the Capabilities of AI Models with Accurately Annotated Image Datasets

Do you also want to improve the object detection accuracy and efficiency of your AI models? We can provide reliable training datasets that align with your project needs and quality standards. Our data annotation services comprise text, image, and video labeling to cater to diverse requirements. Take your AI model to production faster by partnering with our data labeling company.

Take AI to Production

Get high-quality training data. Request a FREE sample.

emailFree Sample
WhatsApp us