🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

# A new type of information theory this paper is not super well-known but has changed my opinion of how deep learning works more than almost anything else it says that we should measure the amount of information available in some representation based on how *extractable* it is,

weights || activations one day we will be able to perfectly decode either one back to plaintext

Really excellent article by jxmo! Esp. found this thought provoking, as someone who’s not been as deeply immersed in the frontier of Reasoning LLMs of late.

when i first learned Machine Learning, our professor ingrained into us how every ML problem starts by splitting data into train, test, and validation these days there is just train and test. in many cases there is just train and more train where’d all the validation sets go?

today’s LLMs have reduced the cost of mediocrity to next-to-nothing unfortunately, the cost of greatness remains high as it’s ever been

how’s this going Andrej Karpathy? the people want an update :)

pretty crazy that DeepMind got self-play working w AlphaZero in *2017* yet basically no one has been able to make it work since

New blog post about asymmetry of verification and "verifier's law": jasonwei.net/blog/asymmetry… Asymmetry of verification–the idea that some tasks are much easier to verify than to solve–is becoming an important idea as we have RL that finally works generally. Great examples of

s3 does vectors now vector databases are officially dead

i think about this a lot

i feel intense pang of nostalgia thinking about how i will never have to write an immense amount of repetitive code to eg build a website ever again. there was a quiet bluecollar satisfaction to this type of work, knowing that if you don’t stay up all night smacking keys the

again, the AI labs are obsessed with building reasoning-native language models when they need to be building *memory-native* language models - this is possible (the techniques exist) - no one has done it yet (no popular LLM has a built in memory module) - door = wide open

now AI can write novel proofs at the level of a world-class competitive mathematician but it still can’t reliably book me a weekend trip to boston so strange

I am just posting this for encouragement to be active! I started posting about 8 months ago, before that I was like 90% of people on Twitter, just viewing not contributing. But because of my posting I was indirectly able to meet with Jack Morris, a super awesome guy. jxmo

🧵 Academic job market season is almost here! There's so much rarely discussed—nutrition, mental and physical health, uncertainty, and more. I'm sharing my statements, essential blogs, and personal lessons here, with more to come in the upcoming weeks! ⬇️ (1/N)

i was on point with this one