October 2024
Bluedot-related alignment readings:
- https://course.bluedot.org/certification?id=recuAgZK0QQY8Hihq My certificate!
- https://www.ai21.com/jamba New open-source hybrid models
- https://arxiv.org/abs/2404.08801 Megalodon, a transformer alternative from Meta
- https://www.philschmid.de/fsdp-qlora-llama3 A guide to QLora finetuning
- https://arxiv.org/pdf/2404.08819 A takedown of state-space models
- https://arxiv.org/pdf/2404.11018 Many-shot in-context learning
- https://www.microsoft.com/en-us/research/blog/sammo-a-general-purpose-framework-for-prompt-optimization/ a prompt optimisation framework
- https://vast.ai/ gpu rentals
- https://replicate.com/home open source model hosting
- https://www.lesswrong.com/posts/TYTEJxzeK3jBMq2TZ/your-posts-should-be-on-arxiv exhortations against linkrot
- https://transformer-circuits.pub/2024/july-update/index.html Anthropic interpretability work
- https://arena3-chapter1-transformer-interp.streamlit.app/%5B1.3.2%5D_Interpretability_with_SAEs ARENA SAE intro
- https://www.lesswrong.com/posts/rhEXTkDmssrHBNrfS/mats-ai-safety-strategy-curriculum-v2 the new MATS curriculum
- https://www.alignmentforum.org/posts/TMAmHh4DdMr4nCSr5/showing-sae-latents-are-not-atomic-using-meta-saes critique of current SAE thinking
The Nanda Zone:
(A bunch of interesting microblogs, mostly on self-improvement)
- https://www.neelnanda.io/blog/38-slack
- https://www.neelnanda.io/blog/32-macro-procrastination
- https://www.neelnanda.io/blog/41-helplessness
- https://www.neelnanda.io/blog/44-agency
- https://www.neelnanda.io/blog/27-retrospective
ML/DS/Python/Stats
- https://ia.samaltman.com/ Sama sets out his stall
- https://tensorcookbook.com/ Building intuition about tensor operations
- https://www.anthropic.com/news/contextual-retrieval Anthropic attempt to improve RAG
- https://www.youtube.com/watch?v=WacFkcNIKxc A guide to improving Python performance
- https://www.youtube.com/watch?v=hM_h0UA7upI Karpathy podcast episode
- https://hynek.me/articles/docker-uv/ Guide to using uv with Docker
- https://simonwillison.net/2024/Apr/8/files-to-prompt/ Making the llm CLI tool more useful
- https://github.com/anthropics/courses/tree/master/prompt_engineering_interactive_tutorial/Anthropic%201P Anthropic prompt-engineering guide.
- https://github.com/randyau/datawebring An attempt to bring back DS blogging
- https://docs.astral.sh/uv/guides/tools/ uv continuing to dominate
- https://github.com/DAGWorks-Inc/hamilton A feature engineering toolkit
- https://maharshi.bearblog.dev/tensors-from-scratch-part-1/ an intro to tensors
- https://andrich.me/2024/09/uv-i-am-somewhat-sold/ uv contuing to dominate, part 2
Misc:
- https://www.sandofsky.com/lambda-school an explanation of what went wrong at Lambda
- https://damagemag.com/2024/08/21/how-the-online-right-fell-apart/ exploring the online Right
- https://en.m.wikipedia.org/wiki/Silo_(series) Sci-fi short story series that was recommended to me
- https://www.mimuw.edu.pl/~bojan/ Very pretty website inspo
- https://www.latent.space/p/gpu-bubble Looking at market dynamics for GPUs
- https://andrewconner.com/things/ random item recommendations
- https://ludwigabap.bearblog.dev/on-becoming-competitive-when-joining-a-new-company/ mindset of a 10x dev
- https://www.youtube.com/watch?v=kiy05zewUpg floating points boiler room set
- https://www.science.org/content/blog-post/fraud-so-much-fraud yet more academic fraud
- https://blog.jacobtrefethen.com/nonprofits/ Jacob T continues fighting the good fight
- https://ukfoundations.co/ a UK jeremiad
- https://www.gutenberg.org/files/1719/1719-h/1719-h.htm Ballad of the White Horse by Chesterton
- https://medium.com/bull-market/a-cynic-s-guide-to-fintech-3cd0995e0da3 fintech scepticism
- https://www.owlposting.com/p/generative-ml-in-chemistry-is-bottlenecked thoughtful bio summary
- https://www.asimov.press/p/models-life biotech scifi from the above author
- https://mdickens.me/2024/09/26/outlive_a_critical_review/#people-should-take-omega-3-supplements longevity book review
- https://www.jonathancohenweb.com/america.html casting doubt on Amerigo -> America
- https://gwern.net/matt-levine trying to explain Matt Levine’s unique place among writers
- https://vitalik.eth.limo/general/2024/09/28/alignment.html Vitalik keeps trying
- https://pudding.cool/2024/09/lineup/ baseball stats
- https://www.scotthyoung.com/blog/2020/05/04/do-the-real-thing/ against spinning your wheels
- https://drmaciver.substack.com/p/creative-fury against mediocre writing