locust project logo

The locust project - visual collision detection

Project objectives


Collision threat detection and avoidance defines a major Research and Development challenge for the automotive industry. Adaptive cruise-control systems incorporating some collision-avoidance features are offered today as pricey options on luxury cars. However, the performance of these systems is not always sufficient, and their cost is too high for wide use. Significant improvements are still needed for these systems to perform satisfactory and to become popular.

Future cruise-control systems will probably fuse data from different types of sensors, with optical images, and therefore vision, playing a significant role. However, present conventional approaches to vision, consisting of a camera that acquires the data and a separate digital processor that process it, are too slow for the most demanding tasks. New solutions are needed. This project brings together a multi-disciplinary team including mixed-signal microelectronic designers (IMSE-CNM), one company in the automotive sector (VCC), neurobiologists (UNEW), and experts on opto-electronic information technology and hardware/software systems (ANCL-HAS) to target the development of these new solutions.

The project embraces advanced research and development activities focused on creating single-chip bio-inspired visual perception systems for automotive applications. These systems will be based on the outstanding performance of natural visual sensory-processing systems. As a central reference point, the project focuses on the integrated visual neuro-system for collision avoidance found in grasshoppers, which will be studied, modelled and emulated by means of sensory-processing chips in standard CMOS technologies. The resulting electronic systems will be able to operate correctly within the wide range of environmental conditions encountered in real-life automotive applications, and will meet the strict reliability standards of the car industry.

Thus, this Project’s global objective is the conception and development of architectures and general design techniques for bio-inspired Vision Systems on Chip (VSoC) integrating sensing, perception and action-control features. In order to demonstrate the reliability of the developed techniques under real conditions, and as a way of measuring the project achievements, a demonstrator intended for course-to-collision signalling and avoidance in automobiles will be designed, fabricated, tested and evaluated under real world operating conditions. In this sense, the successful achievement of this demonstrator can be considered as a central project objective. However, the fundamental objectives are far more ambitious, since VSoCs designed with the same basic techniques to be developed will find applications in many other tasks, within the automotive field and in many other fields. In this other sense, the success of the project should be measured against the validity of the devised architectures and circuit techniques for similar applications.

The aimed VSoCs differ from conventional image sensors in that they will merge the following concurrent functions on a common silicon substrate:

  • Sensory circuitry;

  • Processing circuitry for, on the one hand, adapting the sensors response to the changing environment and; on the other hand, analysing the spatial-temporal characteristics of the visual flow, extracting the relevant information, and transmitting it to a higher-level processor.

  • High-level processor and actuator-control for action-oriented decision-making and driving.

The partners are confident that the achievement of the project objectives will lead to significant advances in automobile active-security and other technically-similar applications in many diverse fields. The reasons behind this confidence are related to performance and cost. Present aproaches to the use of vision in active car security employ conventional digital cameras and subsequent digital processing of the frame sequence. This approach is generaly slow, bulky, and expensive. For these reasons, this type of device has not made its way into widely available commercial products, despite the increasing value of safety in the automobile industry. The use of specific VSoCs will reduce the cost and physical sizes of the devices, and will also improve performance exploiting the new capabilities of CMOS technologies for image sensing and for concurrent processing.

On a lower and more specific level, the project global objective can be decomposed into the following sub-objectives,

  1. At the biological level: extraction, comprehension, and modelling of the functional aspects of the locust looming-alert biological system, with emphasis on the underlying mechanisms of robustness. Analysis of similar biological models for other functions in other insects, generalisation, and guidelines for versatility and robustness of operation.

  2. At the information technology level: system characterization of the underlying biological model, embodiment for electronic implementation, identification of relevant parameters, and VSoC software and prototyping/testing hardware development.

  3. At the chip design level: identification of an optimum set of functional primitives and corresponding circuit implementations, with emphasis on adaptive sensing, optimal sensory/processing integration, and a smart synergy of analogue and digital techniques.

  4. At the application level: VSoC specification, including operating-ranges and conditions, and system integration and evaluation under real conditions.

Each of these sub-objectives defines a technical work package, led by a corresponding partner. Thus, activities related to the modelling of the vision system of the insect Locust (Locusta Migratoria) will be basically realized by neuro-biologists (UNEW); architectural issues will be the responsibility of information technologists (ANCL-HAS); the design of the custom chips following this architecture will be done by mixed-signal chip designers (IMSE-CNM); and finally, car builders (VCC) will be in charge of defining the system-level specifications and of evaluating the demonstrator. Because VSoCs constitute the nuclear component of the proposal, IMSE-CNM has been selected to coordinate the project.