We observed inconsistencies in reported method performances across existing SAC benchmarks. To evaluate this systematically, we conducted a meta-analysis of 13 studies7,12,14,23,24,25,26,27,28,29,30,31,32, using the widely used Visium dataset of the human brain dorsolateral prefrontal cortex (LIBD DLPFC) region33. We focused on BayesSpace, a foundational SAC method, as a benchmark control. We expected relatively consistent BayesSpace results across studies; however, collected adjusted Rand indices (ARIs) revealed substantial variability (Fig. 1a). For instance, slice 151673 showed strong and consistent performance, whereas the results of slice 151669 fluctuated dramatically, highlighting the lack of robustness of reported method performances across previous studies. As expected, we observed that performances reported in method papers are frequently higher than those reported by subsequent competitors (Fig. 1b). This discrepancy could arise from original studies optimizing their methods to demonstrate superior performance or suboptimal parameter choices in subsequent comparisons. While neutral benchmarking studies have aimed to systematically address such biases, we still encountered inconsistencies. For example, BayesSpace was ranked as 4th of 14 methods by Yuan et al.3 but 13th of 16 methods by Hu et al.14 on the basis of the same LIBD DLPFC dataset. Together, this meta-analysis questions the utility of conventional SAC benchmarks. An extensible framework...

Recently developed spatial gene expression technologies s...