In the past, much focus has been put into detecting objects from images, video footage and 3D-point clouds. Useful approaches for images include Yolov5 and DETR2 from Facebook research, and for 3D-point clouds Pointnet++ is a well cited example. These examples are all based on deep neural network architectures. 4D imaging radars is a more recent sensor, which provide point clouds at 20 frames-per-second containing thousands of 4-dimensional detections (range, range rate, azimuth, elevation). Plus, they work in environments and weather situations where other sensors fail. At Qamcom we are developing such a radar. Example applications can be found here: https://arberobotics.com
An annotated data set of high quality is important for training neural networks, and when using images, it is often obtained manually. For 4D radars, manual annotation is more difficult since it is not obvious from the point cloud which points belong to a certain object and what type of object it is. Therefore, we will in this project use an automatic, but less accurate, annotation procedure based on projecting the point cloud onto an image. Objects are detected and classified in the image using an existing neural network. The radar points are then annotated with type and object id based on their projections onto the image. An annotated data set based on this principle already exist and seems to result in accurate annotations when the objects are well separated, and not occluded by other objects.
Given the annotated data set we want to train a neural network for detecting, clustering, and classifying objects in the radar point cloud. Additionally, we want to explore if the annotations can be improved in more difficult scenarios where the objects are not resolved by the radar. Based on a sequence of point clouds and images, where the objects are resolved only in parts of the sequence, object-tracking and smoothing can be used to predict the position of the objects also when they are not resolved by the radar.
Radar hardware details:
Our radar: https://www.qamcom.com/radar/dynamic-applications/
Radar examples and more details on the chipset: https://arberobotics.com
Our radar is one of the most advanced in the world and we think it is possible to make significant contributions to the research community and our customers. We would be very happy if we can glue your achievements into a publication and make it part of our product.