Engineering Updates

Scalability and Efficiency – A Glance at IV2017 – Deep Driving

By Ian Rios-Sialer, Engineering

Last month, I had the opportunity to attend a portion of the 2017 IEEE Intelligent Vehicles Symposium (IV2017) in Redondo Beach, CA. The event is a forum that calls for everyone involved in the future of transportation to come together and discuss their latest work. Themes discussed at the conference were very diverse, with a range of topics including traffic flow, eco-driving, autonomous vehicles, human-machine interaction (HMI), and more, reflecting the multidisciplinary and evolving nature of the transportation industry.

With limited time and multiple talks happening simultaneously, I focused my attention on one event: Deep Driving for Vehicle Perception (also known as Deep-Driving). There is a lot to say about the exciting advancements presented during the event that Sunday morning, but for the purposes of this blog I would like to view everything through the lens of an important overarching theme: Scalability and Efficiency.

Once you develop an appreciation for the notable advancements in deep learning that are powering the autonomous vehicle industry, it naturally leads you to the question: How does deep learning research efficiently scale into production for the autonomous vehicle space?

As Fisher Yu from UC Berkeley remarked in his talk, “There is a trade off between [deep learning models’] parameters, computation and memory.” The problem with efficient allocation of resources for deep learning in self-driving cars gets another compounded layer of complexity when we consider the plentiful idiosyncrasies of the automotive industry itself – I.E. bespoke hardware/software configurations, closed-source models, and deficiencies of cross-OEM standardization initiatives.

To further facilitate discussions surrounding the topics of the Deep Driving event, specifically within the context of scalability and efficiency, I use the following categories:

1. Computation and Memory Scalability

Deep learning methods usually require GPUs to train and deploy. In the past, we have seen efforts to make these models less expensive, both in computational and economic terms. This conference was no exception:

Motivated by the fact that GPUs are scarce resources, Bert De Brabandere presented a novel implementation that can multitask (Semantic Segmentation, Instance Segmentation, Monocular SLAM) in real-time, thus reducing the cycles of computation per individual task, allowing for greater efficiency, lower costs and greater potential innovation with existing and future GPU hardware.

Google’s Andrew Howard showed how MobileNets were able to succeed in computer vision tasks, using only CPU methods. He reminded us, however, of the familiar adage that there is no such thing as a free lunch; lighter, more efficient models would likely not attain the state-of-the-art accuracy associated with more advanced, deep models, but that these new lightweight methods were good enough in most applications.

Civil Maps can provide vehicular perception and cognition using a single in-car, consumer grade CPU. We’ve documented some of that capability here and will continue to report on our developments in future blog posts.

2. Sensor Scalability

Referring to the ability to integrate more sensors into autonomous vehicles, easily and inexpensively, sensor scalability is key to the advancement of robotics and future vehicle technology. As discussed during our recent webinar series, many of our readers may already be familiar with the variety of sensors that come into play to provide perception and cognition for autonomous vehicles. As the adoption of these advanced sensors continues, we see the cost go down every day. However, for the purposes of redundancy and a robust and resilient sensor stack, being able to reproduce fully capable perception solely from cameras or LiDAR data is necessary in the event of the total or partial failure of one or more vehicular senses. Exploring this topic, Professor Rudolf Mester presented to the IEEE symposium audience a way to enhance visual odometry with deep learning. Here, Professor Mester is presenting truly ground-breaking work, as deep learning is expected to shake up the field of visual odometry.

At Civil Maps, we are opening and charting roads into the future by making our hardware sensor-agnostic and loosely-coupled to sensor configuration.

Learn more about it here.

3. Infrastructure Scalability

Besides economic and computational efficiency, architecting dynamic vehicle-embedded smart infrastructure requires a challenging degree of flexibility. Infrastructure scalability relates to sensor scalability, but it also includes the bigger system and architectural decisions that allow for fast growth. We have previously seen some of this kind of work presented at IV2017, but not in the Deep Driving event, itself. However, while speaking with Ph.D. candidate and Machine Learning enthusiast Andrew Howard after the IEEE gathering, he did emphasize that I should keep an eye out for Tensorflow Lite. (Thank you, Andrew!)

For further detail on Civil Maps’ approach to providing dynamic scalability with crowdsourcing initiatives, you can read more here.

4. Data Scalability

Finally, we arrive at a topic central to deep learning: How do we train models when we lack sufficient data/sample size? Adrien Gaidon, from Toyota Research Institute, gave an amazing presentation on the need for realistic simulations while training deep learning algorithms. At the event, Adrien mentioned that the trend of training with simulation data has interested researchers for a while now. After presenting a method to create a virtual data set generated from real-world data, he also showed a new way to model and simulate realistic human actions. Adrien finished his talk by outlining the need for a global-scale realistic simulation engine.

Civil Maps is well aware of the challenges involved with data scalability and the need for dynamic iterative training models, as those issues were part of the initial an impetus for developing our in-house Synthetics application.

I would like to thank everyone who made the IV2017 symposium possible. Congratulations to all the other fantastic speakers at Deep Driving 2017 (Eike Rehder, Anelia Angelova, Xue Mei) and many others in attendance, with whom I met and discussed during the event. Even though I could only be there for a short portion of the conference, I had a great time and learned a lot about the challenges in this nascent industry.

Hopefully, you will see more of Civil Maps at future conferences! We’ve got a busy schedule ahead for the rest of year and would love to connect at any of these events. To setup a meeting, please email

P.S. We’re hiring! Don’t see your role there yet? Email:

Also published on Medium.