The ASKAP/EMU Source Finding Data Challenge

The Evolutionary Map of the Universe (EMU) is a proposed radio continuum survey of the Southern Hemisphere up to declination +30 deg., with the Australian Square Kilometre Array Pathfinder (ASKAP). EMU will use an automated source identification and measurement approach that is demonstrably optimal, to maximise the reliability, utility and robustness of the resulting radio source catalogues. As part of the process of achieving this aim, a "Data Challenge" has been conducted, providing international teams the opportunity to test a variety of source finders on a set of simulated images. The aim is to quantify the accuracy of existing automated source finding and measurement approaches, and to identify potential limitations. The Challenge attracted nine independent teams, who tested eleven different source finding tools. In addition, the Challenge initiators also tested the current ASKAPsoft source-finding tool to establish how it could benefit from incorporating successful features of the other tools. Here we present the results of the Data Challenge, identifying the successes and limitations for this broad variety of the current generation of radio source finding tools. As expected, most finders demonstrate completeness levels close to 100% at 10sigma dropping to levels around 10% by 5sigma. The reliability is typically close to 100% at 10sigma, with performance to lower sensitivities varying greatly between finders. All finders demonstrate the usual trade-off between completeness and reliability, whereby maintaining a high completeness at low signal-to-noise comes at the expense of reduced reliability, and vice-versa. We conclude with a series of recommendations for improving the performance of the ASKAPsoft source-finding tool.