OpenMined (@openminedorg) 's Twitter Profile
OpenMined

@openminedorg

We're building open-source tech that helps app builders & researchers get answers from data without direct access to it.
Join us on slack → slack.openmined.org

ID: 895328122924564480

linkhttps://openmined.org calendar_today09-08-2017 16:57:13

675 Tweet

10,10K Followers

0 Following

⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

A different take — when LLMs allow people to summarise (more or less) infinite amounts of content, attention will cease to be a bottleneck as it once was. The attention economy is an imbalance of two things: - broad-casting scale: 1 person can talk to 1 million - broad-listening

⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

IMO — this paper misses the core driver of hallucinations A LLM with a billion neurons is like a billion tiny databases — database per neuron When you prompt it, the LLM looks in all the databases (i.e. neurons) for patterns it recognizes For example, when you prompt "Kim

⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

Genuine breakthrough in hallucination detection UX, but the fine-tuning approach repeats the exact flaw that creates hallucinations. But that's fixable — which makes me optimistic the hallucination problem is solvable w/ 3 ingredients 1) take this UX breakthrough 2) combine it

⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

IMO — Decentralized AI is more than: - an AI model in the sky, with good external auditing - an AI model in the sky, which people vote on how to use - an AI model in the sky, which is free for anyone to use - open source AI - federated training None of these are truly an

⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

IMO — Ilya is wrong - Frontier LLMs are are trained on ~200 TBs of text - There's ~200 Zettabytes of data out there - That's about 1 billion times more data - It doubles every 2 years The problem is the data is private. Can't scrape it. The problem is not data scarcity, it's

Foresight Institute (@foresightinst) 's Twitter Profile Photo

We are very excited to announce our amazing speaker line-up for Vision Weekend! Join these field-leading researchers and builders as we explore the frontiers of neurotech, biotech, AI, security, space, and energy! Get tickets: foresight.org/events/vision-… Speakers: • Ed Boyden

⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

I've just drafted a new blogpost "GPU demand is (~1Mx) distorted by efficiency problems which are being solved" Mid-2024, Andrej Karpathy trained GPT-2 for $20. Six months later, Andreessen Horowitz reported LLM costs falling 10x annually. Two months after that, DeepSeek

I've just drafted a new blogpost

"GPU demand is (~1Mx) distorted by efficiency problems which are being solved"

Mid-2024, Andrej Karpathy trained GPT-2 for $20. Six months later, Andreessen Horowitz reported LLM costs falling 10x annually. Two months after that, DeepSeek
⿻ Andrew Trask (@iamtrask) 's Twitter Profile Photo

If writing technical blogs/tutorials on AI (decentralized / federated / privacy-preserving / etc.) that get on #HackerNews / #Reddit / X sounds like a fun day job... DM me. (I would mentor you.)