Thesis proposals using 4D imaging radar
Object detection, clustering and classification using 4D-imaging radar
Location: Gothenburg
In the past, much focus has been put into detecting objects from images, video footage and 3D-point clouds. Useful approaches for images include Yolov5 and DETR2 from Facebook research, and for 3D-point clouds Pointnet++ is a well cited example. These examples are all based on deep neural network architectures. 4D imaging radars is a more recent sensor, which provide point clouds at 20 frames-per-second containing thousands of 4-dimensional detections (range, range rate, azimuth, elevation). Plus, they work in environments and weather situations where other sensors fail. At Qamcom we are developing such a radar.
An annotated data set of high quality is important for training neural networks, and when using images, it is often obtained manually. For 4D radars, manual annotation is more difficult since it is not obvious from the point cloud which points belong to a certain object and what type of object it is. Therefore, we will in this project use an automatic, but less accurate, annotation procedure based on projecting the point cloud onto an image. Objects are detected and classified in the image using an existing neural network. The radar points are then annotated with type and object id based on their projections onto the image.
Given the annotated data set we want to train a neural network for detecting, clustering, and classifying objects in the radar point cloud. Additionally, we want to explore if the annotations can be improved in more difficult scenarios where the objects are not resolved by the radar.
Contact: joakim.sorstedt@qamcom.se
Scene reconstruction from 4D radar data
Location: Stockholm
Radar 4D imaging is increasingly becoming a critical component in the automotive industry. This rejuvenation is mainly attributed to the advancement in beamforming technology and hardware capabilities. However, the contextual information reconstructed from the 4D radar is more of a functional construct and does not replace cameras' (classical) visual information. The recent development of deep (conditional) generative models in AI has shown great potential for generating realistic images. Namely, it is possible to mimic the discriminative feature of the training data to generate new realistic images straight from 4D radar data.
These generative models can generate plausible images, given that the training data is of primary quality. Diffusion models and generative adversary networks (GAN) can generate images with higher resolution relative to other generative models, such as variational autoencoders (VAE) or normalizing flows (NF). However, these latter models do estimate the likelihood as well. Therefore, a conditional version of the diffusion model, GAN, VAE, or NF, could be the starting point for image reconstruction from 4D radar data.
Last but not least, the evaluation of this method remains of primary importance, given that there is no annotation on the given data. The data acquisition will be performed using a 4D radar and a high-resolution camera mounted on a designed road intersection.
Contact: ole.martin.christensen@qamcom.se