Low-Latency Streaming

Continuously rising efficiency demands, increasingly powerful sensor hardware, and rebound effects cause the volume of data streams to be processed in modern industrial AI applications to grow steadily. During Dr. Schäfer's tenure as product manager for autonomous driving at the German Aerospace Center (DLR), the deployed vehicles had to perceive their surroundings in real time. This required latency in the micro- to millisecond range for various processing steps. In industrial production we have encountered processes that must operate with extremely short cycle times to avoid bottlenecks. Throughput requirements of several gigabytes per second can arise for optical quality inspection on a production line.

At the same time, a fundamental paradigm shift occurred in the development of computing hardware around the turn of the millennium. Moore's Law is now applied to the number of compute cores rather than, as before, to their clock frequency. Even edge devices such as mobile phones and industrial PCs now incorporate multi‑core systems to meet growing performance demands. This necessitates a fundamental shift in software development toward parallel processing. However, there is a problem: Amdahl’s Law describes the scalability of parallel software, stating that the non‑parallelizable portion of a program inversely limits its scalability. In particular, conventional, costly synchronization mechanisms restrict scalability. With processors that will no longer achieve significantly higher clock speeds and parallelization paradigms that are highly limited, performance bottlenecks can no longer be addressed solely by adding more or faster hardware beyond a certain point. An innovation at the software level is needed.

Exactly here our research and development work begins. We are evaluating the implementation of a highly scalable platform for industrial AI applications. Central to our innovation is a middleware for sensor data processing built on lock-free memory management.