Automotive ADAS

The technology for Advanced Driver Assistance System (ADAS) continues to evolve, largely driven by automotive regulations and consumer demand. ADAS helps drivers stay on top of the driving surroundings for an easier, safer and more comfortable ride. ADAS is the essential step between initial Driver Assistance (DA) systems and fully autonomous cars, and typical solutions include various digital sensors such as RADAR, LIDAR and digital CMOS cameras to capture, fuse and process data from the vehicle driving environment. The popular ADAS camera-based functions are:

  • Multi-camera 360-degree surround-view
  • Rear-view camera parking assistance
  • Lane Departure Warning (LDW) – based on line detection algorithm, informs driver about unintentional road lane departures
  • Pedestrian Detector (PD) – based on object detection algorithm, configured to detect pedestrians in a front of a vehicle
  • Forward Collision Warning (FCW) – based on object detection algorithm, configured to detect multiple vehicles in a front of driving path
  • Traffic Sign Recognition (TSR) – based on object detection and classification algorithms to detect and recognize traffic signs from the vehicle environment

 

DOWNLOAD WHITE PAPER: Xilinx Zynq-based Development Platform for ADAS

 

Figure 1: TySOM-3-ZU7EV Figure 2: FMC-ADAS expansion card

 

As ADAS technology continues to evolve, the need for high-performance re-programmable platforms has never been greater. Aldec provides ADAS development platforms including reference designs and tutorials based on the TySOM Embedded Development boards and FMC-ADAS extension card.

Figure 3: FMC-ADAS functionalities

 

The ADAS application contains the following features:

Figure 4: ADAS solution processing detail

 

Main Features

  • One Embedded development board (TySOM-3-ZU7EV/TySOM-2-7Z100/TySOM-2-7Z045/TySOM-2A-7Z030) and FMC-ADAS Hardware Platform
    • Dual-core/Quad-core ARM Cortex-A9 APU & 1GB DDR3 memory
    • Programmable Logic (PL) for custom hardware accelerators and peripheral controllers
    • On-board connectors for Human Interface Devices (HID): USB3.0,USB2.0, Ethernet, HDMI, QSFP+, Wi-Fi/Bluetooth
    • FMC-ADAS with 5 camera connectors based on DS90UB914Q deserializers, ultrasonic and LIDAR sensor connectors and a buzzer device
    • X4 automotive wide lens 192-degree HDR Blue Eagle cameras and 1x 52-degree
  • Multi-Camera Surround View, Driver Drowsiness Detection and Smart-rear view reference designs including hardware, firmware and software parts
    • Embedded Linux OS with command line interface over HDMI screen
    • V4L2 compatible implementation of Video Input Port (VIP) camera interfaces
    • HW/SW support for ultrasonic sensor
    • HW/SW support for buzzer device
    • DRM compatible implementation of HDMI output interface
    • Linux user application for simultaneous 4 cameras streaming, edge detection and frames merging with critical parts accelerated using Xilinx SDSoC tool

 

Solution Contents

  • One Embedded development board (TySOM-3-ZU7EV/TySOM-2-7Z100/TySOM-2-7Z045/TySOM-2A-7Z030) with FMC-ADAS extension card and Riviera-PRO Advanced RTL Simulation/Debugging Platform (optional)
  • Technical documentation, tutorials and white papers
  • Multi-Camera Surround View, Driver Drowsiness Detection and Smart-rear view Reference designs Quick Start Guide
  • Multi-Camera Surround View, Driver Drowsiness Detection and Smart-rear view Reference design binaries
  • SDSoC hardware platform package (Board Support Package - BSP)
  • Blue Eagle camera Linux device drivers source code
  • Linux userspace application source code
  • 1-year voucher for Xilinx Vivado Design Suite (latest version)
  • 1-year voucher for Xilinx SDSoC tool (latest version)

 


Bird's Eye View

The bird’s eye view is a vision monitoring system used in automotive ADAS technology that provides the 360-degree, top-down view. The main benefits of this system is to assist the driver in parking the vehicle safely. However, it can be used for lane departure and obstacle detection. Aldec has designed and created a bird’s eye view application to help ADAS designers. This application is implemented using the TySOM-3-ZU7EV embedded development board, an FMC-ADAS daughter card and four blue eagle cameras, each with 192-degree wide lenses running at 30fps. The hardware setup and the output are shown in the following image.

 

The most computational intensive parts of the code are offloaded from ARM Cortex-A9 to FPGA part of Xilinx® Zynq Ultrascale+ MPSoC device using Xilinx SDSoC™ tool, achieving the goal for real-time processing performance. Design process of this demo consists of 6 stages which are done using FPGA and the ARM processor as follows:

  1. Frame capturing (ARM processor)
  2. Frame resizing (FPGA)
  3. Remove lens distortion (FPGA)
  4. Perspective transformation (FPGA)
  5. Create the final image (ARM)
  6. Display the results (ARM)

The following diagram shows the detail of this implementation.

 

 

 


Multi-Camera Surround View

ADAS Multi-Camera Surround View technology is a parking assistance system available in today’s mid- and high-cost vehicles. The key feature is a set of 4 HDR wide-lens cameras installed around the vehicle for a full 360 degree view of the surroundings in a single screen. The reference design grabs, processes and displays 4 simultaneous camera video streams in real-time. The most computational intensive parts of the code are offloaded from ARM Cortex-A9 to FPGA part of Xilinx® Zynq-7000 All-Programmable and Xilinx® Zynq Ultrascale+ MPSoC device using Xilinx SDSoC™ tool, achieving the goal for real-time processing performance. The accelerated part includes edge detection, colorspace conversion and frame merging tasks. The Edge detector is used to highlight the possible obstacles around the vehicle which cannot be easily noticed by the human eye.

Multi-Camera Surround View

Figure 5: Multi-camera surround view processing detail

 

 


Driver Drowsiness Detection

Most of the car accidents nowadays occur because of the drowsiness of the driver, specially truck drivers. Nowadays, to prevent such a collision, a specific camera is used in smart vehicles to realize driver drowsiness in the early stages. Aldec’s ADAS solution also contains this feature by using one HDR camera allocated for the driver. The Pixel Intensity Comparison-based Object detection (PICO) algorithm is used for this application because of high processing speed and ease of modification. In this application, the most computational intensive parts of the code are offloaded from ARM Cortex-A9 to FPGA part of Xilinx® Zynq-7000 All-Programmable and Xilinx® Zynq Ultrascale+ MPSoC device using Xilinx SDSoC™ tool, achieving the goal for real-time processing performance. The accelerated parts contain colorspace conversion and Histogram Equalization. In the next step, face detection, eye detection and blink detection are processed. In the decision making section, a buzzer comes to play if the driver is detected to be sleepy. The following diagram shows the steps for the Driver Drowsiness Detection Application.

Figure 6: Driver Drowsiness Detection processing detail

 


Smart Rear-View Camera

Today’s modern vehicles include a rear-view camera as a basic feature which allows drivers to monitor the rear of the vehicle during reverse driving. Automotive grade megapixel HDR camera, ultrasonic sensor and the Human Interface Devices (HID) are the essential components that turn the rear license plate camera into a smart solution. The reference design is running on the TySOM-2 board under the control of embedded Linux OS, providing an easy way to collect, fuse and process multiple simultaneous data from several digital sensors using the best from ARM Processing System (PS) and the Programmable Logic/FPGA (PL) sides located on Xilinx Zynq-7000 All-Programmable device. The processed data is used to overlay the rear-view video stream with the visual warnings as well as for sonic alert generation in case of a possible obstacle close to the vehicle.

Smart Rear-View Camera

Figure 7: Smart rear-view processing details

Ask Us a Question
x
Ask Us a Question
x
Captcha ImageReload Captcha
Incorrect data entered.
Thank you! Your question has been submitted. Please allow 1-3 business days for someone to respond to your question.
Internal error occurred. Your question was not submitted. Please contact us using Feedback form.
We use cookies to ensure we give you the best user experience and to provide you with content we believe will be of relevance to you. If you continue to use our site, you consent to our use of cookies. A detailed overview on the use of cookies and other website information is located in our Privacy Policy.