If you're looking for statistical correlation between a swarm of SNPs and some kind of outcome, it's worth remembering that each of those SNPs is probably pleiotropic (in the sense that it influences multiple things) - some of them antagonistic (in the sense of good under some circumstances, bad in others. For example, APOE4 brings better working memory when young, then increased dementia risk when old.
'Highly polygenic' means a massive swarm of SNPs. Gotta wonder how many of those are like APOE4.