Parallelisation of the wide-band wide-field spectral deconvolution framework DDFacet on distributed memory HPC system

The next generation of radio telescopes, such as the Square Kilometer Array (SKA), will need to process an incredible amount of data in real-time. In addition, the sensitivity of SKA will require a new generation of calibration and imaging software to exploit its full potential. The wide-field direction-dependent spectral deconvolution framework, called DDFacet, has already been successfully used in several existing SKA pathfinders and precursors like MeerKAT and LOFAR. However, DDFacet has been developed and optimized for single node HPC systems. DDFacet is a good candidate for being integrated into the SKA computing pipeline and should, therefore, have the possibility to be run on a large multi-node HPC system for real-time performance. The objective of this work is to study the potential parallelism of DDFacet on multi-node HPC systems. This paper presents a new parallelization strategy based on frequency domains. Experimental results with a real data set from LOFAR show an optimal parallelization of the calculations in the frequency domain, allowing to generate a sky image more than four times faster. This paper analyses the results and draws perspectives for the SKA use case.


Theme – Data Processing Pipelines and Science-Ready Data