- Crowdsourcing this: what are the best examples of scaling laws in foundation models for neuroscience? My top 3 examples: Ctrl+Labs, Sato et al., Mineault et al. @colehurwitz.bsky.social , @mehdiazabou.bsky.social , @nandahkrishna.bsky.social , @averyryoo.bsky.social, @tyrellturing.bsky.social ?Apr 17, 2025 20:47
- In Lurz et al., ICLR 2021 we did quite some analysis on scaling and generalization across animals in the context of visual response prediction (incl. behavioral modulation) with @sinzlab.bsky.social and @andreastolias.bsky.social: openreview.net/forum?id=Tp7...
- Thanks, we re-use some of this data in the third meta-plot above. Interestingly, scaling curves match qualitatively and relatively well across different groups. See section 1 where we derive a functional form for the scaling laws that work across sims and Lurz et al. arxiv.org/abs/2411.18526
- There are several other examples of evidence for the benefits of scale (e.g. the recent paper from @colehurwitz.bsky.social), but in terms of demonstration of power-laws in scaling, there's not a lot yet!
- But number go up good!