December
Papers
- Playing Atari with Deep Reinforcement Learning
- Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers
- Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges
- An Evolved Universal Transformer Memory
- Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
- Byte Latent Transformer: Patches Scale Better Than Tokens
- Automating the Search for Artificial Life with Foundation Models
Links
- https://press.asimov.com/articles/crick Francis Crick Was Misunderstood
- https://lilianweng.github.io/posts/2024-11-28-reward-hacking/ Reward Hacking in Reinforcement Learning
- https://mathematical-tours.github.io Mathematical Tours of Data Sciences
- https://arxiv.org/abs/1803.00567 Computational Optimal Transport
- https://gobook.eu Welcome to the Multilingual Go Book Project
- https://neuralmagic.com/blog/24-sparse-llama-smaller-models-for-efficient-gpu-inference/ 2:4 Sparse Llama: Smaller Models for Efficient GPU Inference
- https://anthropic.com/research/building-effective-agents Building effective agents
- https://openai.com/index/solving-rubiks-cube/ Solving Rubik’s Cube with a robot hand
- https://openai.com/index/emergent-tool-use/ Emergent tool use from multi-agent interaction
- https://openai.com/index/learning-montezumas-revenge-from-a-single-demonstration/ Learning Montezuma’s Revenge from a single demonstration
- https://openai.com/index/reinforcement-learning-with-prediction-based-rewards/ Reinforcement learning with prediction-based rewards
- https://sakana.ai/cycleqd/ Population-based Model Merging via Quality Diversity
- https://quality-diversity.github.io/papers Quality-Diversity optimisation algorithms
- https://sakana.ai/evolutionary-model-merge/ Evolving New Foundation Models: Unleashing the Power of Automating Model Development
- https://arxiv.org/abs/2410.14735 Agent Skill Acquisition for Large Language Models via CycleQD
- https://www.lesswrong.com/posts/Z7R6jFjce3J2Ryj44/exploring-the-lottery-ticket-hypothesis Exploring the Lottery Ticket Hypothesis
- https://neuralmagic.com/blog/sparsegpt-remove-100-billion-parameters-for-free/ SparseGPT: Remove 100 Billion Parameters for Free
- https://huggingface.co/docs/transformers/en/perplexity Perplexity of fixed-length models
- https://www.lesswrong.com/posts/BaEQoxHhWPrkinmxd/announcing-neuronpedia-as-a-platform-to-accelerate-research Announcing Neuronpedia: Platform for accelerating research into Sparse Autoencoders
- https://sakana.ai/namm/ An Evolved Universal Transformer Memory
- https://huggingface.co/blog/winning-aimo-progress-prize How NuminaMath Won the 1st AIMO Progress Prize
- https://aimoprize.com/ Artificial Intelligence Mathematical Olympiad
- https://ai.meta.com/research/publications/large-concept-models-language-modeling-in-a-sentence-representation-space/ Large Concept Models: Language Modeling in a Sentence Representation Space
- https://arxiv.org/abs/2412.06769 Training Large Language Models to Reason in a Continuous Latent Space
- https://ai.meta.com/research/publications/flow-matching-guide-and-code/ Flow Matching Guide and Code
- https://ai.meta.com/research/publications/explore-theory-of-mind-program-guided-adversarial-data-generation-for-theory-of-mind-reasoning/ Explore Theory-of-Mind: Program-Guided Adversarial Data Generation for Theory of Mind Reasoning
- https://ai.meta.com/research/publications/meta-clip-12/ Meta CLIP 1.2
- https://ai.meta.com/research/publications/discrete-flow-matching/ Discrete flow matching
- https://ai.meta.com/results/ Meta AI Research
- https://lesswrong.com/tag/solomonoff-induction Solomonoff Induction
- https://paulgraham.com/essay.html The Age of the Essay
- https://arcprize.org/2024-results The ARC Prize 2024
- https://blog.google/technology/research/google-willow-quantum-chip/ Meet Willow, our state-of-the-art quantum chip
- https://blog.samaltman.com/idea-generation Idea Generation
- https://blog.samaltman.com/hard-startups Hard Startups
- https://blog.samaltman.com/how-to-be-successful How To Be Successful
- https://quantum.country/qcvc Quantum computing for the very curious
- https://arxiv.org/abs/1905.09749 How to factor 2048 bit RSA integers in 8 hours using 20 million noisy qubits
- https://arxiv.org/abs/2009.05045 Forecasting timelines of quantum computing
- https://x.ai/blog/grok-image-generation-release Grok Image Generation Release
- https://r4j4n.github.io/blogs/ Rajan Ghimire Blog
- https://pubmed.ncbi.nlm.nih.gov/17446403/ Neuronal competition and selection during memory formation
- https://arxiv.org/abs/2412.06769 Training Large Language Models to Reason in a Continuous Latent Space
- https://nature.com/articles/ncomms13276 Random synaptic feedback weights support error backpropagation for deep learning
- https://dustintran.com/blog/muprop-unbiased-backpropagation-for-stochastic-neural-networks MuProp: Unbiased Backpropagation for Stochastic Neural Networks
- https://dl.heeere.com/conditional-flow-matching/blog/conditional-flow-matching/ A Visual Dive into Conditional Flow Matching
- https://xenaproject.wordpress.com/2024/12/11/fermats-last-theorem-how-its-going/ Fermat’s Last Theorem — how it’s going
- https://press.asimov.com/articles/mirror-life The Dangers of Mirrored Life
- https://nature.com/articles/s41587-019-0339-0 The accelerating pace of biotech democratization
- https://en.m.wikipedia.org/wiki/Rotating_locomotion_in_living_systems Rotating locomotion in living systems
- https://synthesis.cc/synthesis/2022/10/dna-synthesis-cost-data DNA Cost and Productivity Data, aka "Carlson Curves"
- https://ai.meta.com/research/publications/byte-latent-transformer-patches-scale-better-than-tokens/ Byte Latent Transformer: Patches Scale Better Than Tokens
- https://apoorvx.com/posts/ppo/ PPO in 180 lines of numpy
- https://andrewkchan.dev/posts/yalm.html Fast LLM Inference From Scratch
- https://rish-01.github.io/blog/posts/ml_estimation/ Maximum Likelihood Estimation and Loss Functions
- https://thetransmitter.org/neural-networks/what-are-recurrent-networks-doing-in-the-brain/ What are recurrent networks doing in the brain?
- https://main-horse.github.io/posts/visualizing-6d/ Visualizing 6D Mesh Parallelism
- http://jackterwilliger.com/attractor-networks/ Attractor Networks, (A bit of) Computational Neuroscience Part III
- https://adam.math.hhu.de/#/g/leanprover-community/NNG4 Welcome to the Natural Number Game
- https://lodev.org/cgtutor/xortexture.html The XOR Texture
- https://distill.pub/2021/gnn-intro/ A Gentle Introduction to Graph Neural Networks
- https://nature.com/articles/d41586-024-02504-4/ How neurons make a memory
- https://yitay.net/blog/model-architecture-blogpost-encoders-prefixlm-denoising What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives
- https://nature.com/articles/s41586-024-08325-9 Synaptic basis of feature selectivity in hippocampal neurons
- https://blog.computationalcomplexity.org/2024/09/natural-proofs-is-not-barrier-you-think.html Computational Complexity and other fun stuff in math and computer science from Lance Fortnow and Bill Gasarch
- https://theory.stanford.edu/~liyang/teaching/projects/natural-proofs-barrier-and-P-NP.pdf The Natural Proofs Barrier and P =?NP
- https://www.sciencedirect.com/science/article/pii/S002200009791494X?via%3Dihub Natural Proofs
- https://en.wikipedia.org/wiki/Circuit_complexity Circuit complexity
- https://medium.com/inspiredbrilliance/exploring-lora-part-1-the-idea-behind-parameter-efficient-fine-tuning-and-lora-ec469d176c26 Exploring LoRA — Part 1: The Idea Behind Parameter Efficient Fine-Tuning and LoRA
- https://www.karlsims.com/evolved-virtual-creatures.html Evolved Virtual Creatures - Karl Sims (1994)
- https://szhaovas.github.io/2022-09-15-me/ MAP-Elites Introduction
- https://eater.net/boids Boids algorithm demonstration
- https://blog.comma.ai/autonomy/ Autonomy - Comma AI
- https://jamchamb.net/2018/07/11/animal-crossing-nes-emulator-hacks.html Finding and exploiting hidden features of Animal Crossing's NES emulator
- https://chanzuckerberg.com/science/technology/virtual-cells/ Virtual Cells
- https://francisbach.com/my-book-is-out/ Learning Theory from First Principles
- https://arxiv.org/abs/2203.14465 STaR: Bootstrapping Reasoning With Reasoning
- https://arxiv.org/abs/2410.06205v1 Round and Round We Go! What makes Rotary Positional Encodings useful?
- https://xenaproject.wordpress.com/2024/12/22/can-ai-do-maths-yet-thoughts-from-a-mathematician/ Can AI do maths yet? Thoughts from a mathematician
- https://icml.cc/virtual/2021/11320 Some Thoughts on Generalization, Robustness, and their application with CLIP
- https://thetransmitter.org/books/future-watch-what-should-neuroscience-prioritize-during-the-next-10-to-20-years/ Future watch: What should neuroscience prioritize during the next 10 to 20 years?
- https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/ Genie 2: A large-scale foundation world model
- https://pub.sakana.ai/asal/ Automating the Search for Artificial Life with Foundation Models
- https://goodfire.ai/blog/announcing-goodfire-ember/ Goodfire Ember: Scaling Interpretability for Frontier Model Alignment
- https://nature.com/immersive/d41586-024-03425-y/index.html What's so special about the human brain?
- https://nature.com/immersive/d42859-023-00069-2/index.html BICCN: The first complete cell census and atlas of a mammalian brain
- https://nature.com/articles/d41586-023-03192-2 This is the largest map of the human brain ever made
- https://pubmed.ncbi.nlm.nih.gov/34707291/ A human-specific modifier of cortical connectivity and circuit function
- https://arxiv.org/abs/2211.00241 Adversarial Policies Beat Superhuman Go AIs
- https://kyunghyuncho.me/i-sensed-anxiety-and-frustration-at-neurips24/ I sensed anxiety and frustration at NeurIPS’24
- https://svbrain.xyz/2024/12/20/cerebrum Introducing Cerebrum: What if we could simulate your brain?
- https://params.com/@jeremy-berman/arc-agi How I came in first on ARC-AGI-Pub using Sonnet 3.5 with Evolutionary Test-time Compute
- https://cameronrwolfe.substack.com/p/model-merging Model Merging: A Survey
- https://scp-wiki.wikidot.com/introductory-antimemetics Introductory Antimemetics
- https://minds.md/zakirullin/cognitive Cognitive load is what matters
- https://aphyr.com/posts/378-seconds-since-the-epoch Seconds Since the Epoch
- https://github.com/deepseek-ai/DeepSeek-V3/blob/main/DeepSeek_V3.pdf DeepSeek-V3 Technical Report
- https://onnx.ai/onnx/technical/float8.html Float stored in 8 bits
- https://ams.org/notices/202501/rnoti-p6.pdf Machine-Assisted Proof
- https://nature.com/articles/nature12742 Context-dependent computation by recurrent dynamics in prefrontal cortex
- https://lesswrong.com/posts/fAW6RXLKTLHC3WXkS/shallow-review-of-technical-ai-safety-2024 Shallow review of technical AI safety, 2024
- https://arxiv.org/abs/2412.06769 Training Large Language Models to Reason in a Continuous Latent Space
- https://en.m.wikipedia.org/wiki/Von_Neumann_universal_constructor Von Neumann universal constructor
- https://darioamodei.com/machines-of-loving-grace Machines of Loving Grace
- https://blog.atomsonly.com/p/bionumbers-1 Order-of-Magnitude Biology
- https://www.biorxiv.org/content/10.1101/2024.06.24.600095v1 Controlling semiconductor growth with structured de novo protein interfaces
- https://www.asimov.press/p/levers Levers for Biological Progress
- https://www.asimov.press/p/tinker Tinker
- https://arxiv.org/abs/2409.11654 How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities