One of the biggest challenges in getting autonomous vehicles on the road is proper testing. Traditional ADAS systems perform one function and can be tested in isolation. Autonomous systems need to be tested holistically and in virtually every scenario they could possibly encounter on the road. Some manufacturers estimate it will take 8.8 billion miles of testing to actually ensure the safety of autonomous vehicles. This is a task of epic proportions.
At Civil Maps, our approach is to test autonomous systems in a state-of-the-art simulation framework and at design time. Various sensor models, placements, and configurations can be easily entered into a web interface or SDK and run with instant results. This approach empowers vehicle designers to understand the full effect design changes have on the actual localization performance of the vehicle in different scenarios. The simulation results are then used to tune our deep neural networks and optimize them for each vehicle.
Key Benefits of Synthetics:
- Procedurally generate diverse environments
- Varied traffic, terrain, environmental noise, & weather conditions
- Multiple sensor types, models, placements, configurations & combinations
- Ray cast laser firings & capture stereoscopic camera imagery
- Predictive modeling for anticipatory driving behaviors
- Highly trained deep neural networks
Advantages of Synthetics:
- Speed – Instantly process point cloud data and extract semantic features
- Scale – Run multiple vehicles simulations in parallel
- Control – 100% control over sensor configurations and environment
Synthetics Procedural 3D Modeling
Procedural modeling allows us to create and manipulate large-scale scenes to test autonomous vehicles in a limitless set of scenarios. Algorithms are used to generate high-fidelity, 3D scenes of different terrains, 2D textures, variable building densities, road configurations, and weather conditions to mimic real-world driving environments. A physics engine is included to apply all context of the physical world, weight, force, gravity, friction, etc., to our simulated environment. Additionally, noise models are included to simulate real-world interference. This allows us to accurately gauge the results and interactions of various sensors in synthetics.
The use of algorithms for scene generation yields a variety of benefits:
- The scenarios require minimal overhead to initialize. In just a few short configuration steps, a new scene can be created.
- Algorithms are easily recreatable. Civil Maps stores a history of previous simulation runs so users can benchmark against previous results and/or tweak for minor variations.
- Algorithms use a set of controlled input factors, This allows users to isolate variables and test for specific results.
- They are easily scalable and highly optimized for uniqueness. Introducing new parameters introduces new scenarios to the system.
- They are fast. Scenes are generated on the fly, reducing load time and computing power.
Synthetics for Localization
In addition to creating 3D simulated environments, Synthetics can be used to test Localization performance by reverse-engineering point clouds and the sensor data. Using multiple vehicle trips in the same simulation environment, the precision of localization can be measured with a ground truth that cannot be replicated with real world data. Localization’s robustness can also be measured by changing parameters of the simulation and measuring how this affects the performance. Some examples include varying the number of lidar lasers, the complexity of the environment or vehicle speeds. Synthetics allows the isolation of different variables to ensure that the system is robust in that dimension.
Synthetics DNN Training
One of the benefits of Synthetics is the ability to finely train our deep neural networks. Autonomous vehicles need to be able to correctly identify important objects in the real world, such as stop signs, lane markings, and other cars. Millions of different images of those objects need to be fed into a DNN in order to properly train the DNN. One method for gathering those images and training the DNN is to physically collect images of objects in the real world. This is an expensive, tedious, and time consuming task, and ultimately, there tends to be a 20%-25% margin of error with this collected data. A stop sign with an element of noise, such as graffiti, damage, or an obstruction like a tree branch, will not register to the DNN properly.
The solution to this real-world noise is to use synthetic data to first train and highly tune the DNN. By training the DNN on simulated and flawless data, the DNN learns to accurately identify an object. Noise can then be gradually introduced to the DNN and not cause as much confusion to the system. This approach can improve DNN performance greatly.
Synthetics Map Exchange Modeling
With Synthetics, we are building a crowdsourcing model that allows for experimentation with multiple variables respective to fundamental traffic flow.