Imagine yourself driving down the road, when all of a sudden a big SUV cuts you off without a warning, putting you at risk to a major accident. You, however, are busy reading your news-feed to notice it and instead of giving those bad drivers a honk and a middle finger, your automated car sends a strongly worded letter of 1s and 0s to the offending vehicle. Sounds crazy but after years of hard work and research the world is finally on the brink of a revolutionary new form of transportation: Self-driving cars. 

Self-driving cars consist of five core components –

Computer vision

Computer vision consists of using camera images to reckon what the world around us looks like. The computer looks for colours and edges and gradients to find the lane on the road. Deep neural networks or deep learning which is an exciting upcoming part of machine learning and artificial intelligence, enables computers to learn what cars and other objects look like, simply by sending them lots and lots of data; they view a  lot of automobile images and then learn what cars look like.

Sensor Fusion

The car’s computer then augments its understanding of the world using sensors, radar and lasers. These are used in obtaining measurements that are difficult for a camera alone to understand, such as the distance between other cars and itself, how fast other objects in the environment are moving etc.

Self-driving cars are equipped with state-of-the-art software to make sure they can function as best as possible without human intervention. After many years of testing, the cars use top-of-the-line laser-based radar systems called LIDAR. LIDAR, short for light detection and ranging, functions in tandem with state-of-the-art GPS and inertial measurement unit sensors or IMU to pinpoint the location of the car down to a quarter of an inch. An additional radar control will kick in whenever an obstacle comes within a foot of the vehicle.  In order to be convenient and efficient, the cars brain uses deep learning technology to review camera and GPS data and predict the fastest routes to your destination. This is achieved using special algorithms that are always running between the computerized parts in and around the car’s engine. The vehicle can also calculate the probability of actions taken by other drivers as you speed down the road.

Localization

The next step involves locating itself in the simulated environment. Localization refers to precisely determining the car’s position in that simulated world. Once it approximately figures out where it is in the world and what the world looks like, it uses the previously mentioned sophisticated mathematical algorithms as well as high-definition maps to localize the vehicle in its environment. This is achieved with single-digit centimetre-level accuracy. The vehicle evaluates its distance from these landmarks while on the move and compares the same with the simulated map to know precisely where it is in the world. These landmarks might be street lights or traffic signs or mailboxes or even manhole covers.

Path planning

The next step is charting a path through that simulated world i.e. figuring an optimal route to its destination also known as path planning. A path planner predicts where the other vehicles on the road are heading to and then determines a manoeuvre, for its own vehicle in response. When its vehicle comes up on other traffic, it evaluates the possibility of slowing down and staying in its lane or shifting to the right or to the left, all of which is made possible by deep learning technology.

Control

The final step in the pipeline is Control, which refers to actually turning the steering wheel, hitting the throttle and hitting the brakes in order to execute the trajectory that was built during path planning. This is of course pretty complex and human drivers are familiar with how particularly difficult it is on tight turns to follow the exact line through the middle of the lane. Nevertheless, computers are really good at this and the distance between the trajectory they’d like to follow and the trajectory they actually follow is really close.

Uber, Tesla and Google have already shown off their self-driving car technology. Car companies of the likes of Toyota, Nissan, Ford, BMW and Volvo aim to roll out production of vehicles with driver-less capabilities by 2020.

Driver-less cars will be here before you know it.