Not yet reviewed by Pith; the record is open.

With the upcoming large-scale surveys like LSST, we expect to find approximately $10^5$ strong gravitational lenses among data of many orders of magnitude larger. In this scenario, the usage of non-automated techniques is too time-consuming and hence impractical for science. For this reason, machine learning techniques started becoming an alternative to previous methods. In our previous work, we proposed a new machine learning architecture based on the principle of self-attention, trained to find strong gravitational lenses on simulated data from the Bologna Lens Challenge. Self-attention-based models have clear advantages compared to simpler CNNs and highly competing performance in comparison to the current state-of-art CNN models. We apply the proposed model to the Kilo Degree Survey, identifying some new strong lens candidates. However, these have been identified among a plethora of false positives, which made the application of this model not so advantageous. Therefore, throughout this paper, we investigate the pitfalls of this approach, and possible solutions, such as transfer learning, are proposed.