What is the true impact of using synthetic data when developing computer vision algorithms? Let’s cut through the noise and dive directly into a real-world end-to-end computer vision workflow with synthetic data to see the results.
A team of experts used the Rendered.ai Platform as a Service (PaaS), open-sourced real datasets, and commonly used computer vision models to test the efficacy of using synthetic data merged with real-world data to train a classification model. Here are the steps they took and how using different real and synthetic training data ratios affected model performance.
Test Case Design
Computer Vision Use Case
Training a classification model to identify new ship types in variable marine oblique scenes.
Objectives
To demonstrate how the Rendered.ai PaaS is used to generate customized synthetic imagery data for training and testing computer vision models:
- In significantly less time
- For a more diverse set of objects, classes, and scenarios
- Resulting in improved model performance
- Using smaller amounts of high-cost, real-world data
Test Dataset
Real images for four classes of maritime vessels from the “Ships Image Dataset” annotated in YOLO v5 PyTorch format, open-sourced from Kaggle.
Samples of real images for tugboat, sailboat, aircraft carrier, and submarines from the Kaggle-sourced “Ships Image Dataset”.
Model
NVIDIA TAO v5.5 with backbone pre-trained weights from the tiny Fully Attentional Network (FAN) model.
NVIDIA TAO model loaded into the Rendered.ai Platform with a configurable hyperparameter menu.
Methodology
Engineers used the following steps to expand the training dataset with synthetic data, train the model on varying real and synthetic data ratios, and optimize model performance.
- Establish a baseline of model performance using only real training data.
- Use the Rendered.ai Platform to generate synthetic data against the baseline and assess model performance training only on synthetic data.
- Adjust configurations for data generation workflows to balance the model performance of the synthetic dataset for all classes.
- Test different combinations of real and synthetic data on model performance.
To see additional detail on each step conducted, visit our README file in GitHub.
You can also watch an interview with the lead engineer on this project. He shares how he used Microsoft-developed Trellis 3d’s text-to-3D capabilities to expand imagery sample sets for an underperforming class.
The Results
Engineers found that only 20% of the real dataset was needed when combined with synthetic data to achieve the same, if not better, performance as the real dataset. Performance topped out at 91% accuracy in this example using a merged dataset of 80% of the real dataset from Kaggle and 20% of customized synthetic data generated in the Rendered.ai platform.
A sample of the real & synthetic training images used to achieve a 91% accuracy score in the classification model.
The Impact of Using Synthetic Data & the Rendered.ai PaaS
With an open-sourced real test dataset, a standard computer vision model for this use case, and the Rendered.ai platform, engineers demonstrated that they can effectively train a classification model to perform:
- In a fraction of the time
- With a lower reliance on real data
- At a higher accuracy rate for a broader set of classes
The additional expertise of the Rendered.ai team and the ability for engineers to collaborate, quickly iterate on synthetic data generation workflows, train the model, and run inference in the Rendered.ai platform made it possible to arrive at a performant classification model in days instead of months.
A Rendered.ai PaaS workspace where users can share 3D assets, track dataset creation, and collaborate on configuring synthetic data generation workflows and model training.
The Rendered.ai graph configuration used in the first iteration of synthetic data generation, easily copied and modified for later iterations.
Faster model training and experimentation with computer vision models uploaded to the Rendered.ai Platform and loss plot analytics.
Inference services in the Rendered.ai Platform to quickly make predictions against synthetic datasets generated or real datasets uploaded to the platform.
Access the “Marine Oblique – Classification” Workspace preloaded with the datasets, 3D assets, and model used here by signing up for a free trial of the Rendered.ai PaaS with content code: MARINECLASS.
Already a Rendered.ai User? Simply log in, create a new workspace, and enter the same content code when prompted.
Chat with the Rendered.ai team about the efficacy of synthetic data for a different use case and get a personalized demo of the Rendered.ai Platform at any time!
