🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Worth a read: "How do I become a graphics programmer? - A small guide from the AMD Game Engineering team": gpuopen.com/learn/how_do_y…

I implemented an LLM end-to-end in hardware, and ran it on an FPGA. Zero Python. Zero CUDA. Just pure SysVerilog. All my progress + everything I learned from 200h of LLM chip design (demo at the end)👇

raylib running on emacs... on emacs lisp! 🤯

Please check out our Qwen3 Technical Report. 👇🏻 github.com/QwenLM/Qwen3/b…

DeepSeek just dropped the single best end-to-end paper on large model training. It covers — Software (MLA, training in FP8, DeepEP, LogFMT) — Hardware (Multi-Rail Fat Tree, Ethernet RoCE switches) — Mix (IBGDA, 3FS filesystem) DeepSeek's engineering depth is insane. Must read.

🚨 New preprint 🚨 We introduce Generative Distribution Embeddings (GDEs) — a framework for learning representations of distributions, not just datapoints. GDEs enable multiscale modeling and come with elegant statistical theory and some miraculous geometric results! 🧵

And, we are back in triangles for neural graphics :D First it was just a neural network (NeRF), then sparse octree (NeRF converted to octree), then gaussian splats (basically particle splatting) and now triangles :)

The methods we used to trace the thoughts of Claude are now open to the public! Today, we are releasing a library which lets anyone generate graphs which show the internal reasoning steps a model used to arrive at an answer.

chat pointed out that facebook forked jemalloc a few weeks ago so maybe that's where development is now going

Two years in the making, we finally have 8 TB of openly licensed data with document-level metadata for authorship attribution, licensing details, links to original copies, and more. Hugely proud of the entire team.

The laws of Physics are not what is limiting the performance of Microsoft Teams.

1/2) It's finally out on Arxiv: Feedback guidance of generative diffusion models! We derived an adaptive guidance methods from first principles that regulate the amount of guidance based on its current state. Complex prompts are highly guided while simplem ones are almost free

So it looks like today people are discovering Shaders

I'm working on a good heuristic to put would-be tech influencers on mute/block (depending on obnoxiousness). Current idea is >10K (5K?) followers without any discernable achievements, or some scaled version of same for people with *some* achievements but who have clearly ...

Take the LLaMA 3 paper for another example. I know (from personal experience and talking to others) that many authors of this paper endorse the above view. And yet, not a single model in their scaling laws plots is that large! (7B / 1T = 4.2e22 FLOP)

Still looking for my next gig. Open for contracting opportunities as well. Reach out in DMs if you have something.

This is THE laptop on which Sun Microsystems' kernel engineers developed Solaris 10.

This year’s summer blockbuster is coming soon to a computer screen near you… 👀 👀 👀

.Sam H Smith's talk is out! Come delve into AST-free compilation and optimizing with sea of nodes! youtu.be/NxiKlnUtyio

Ex-DeepSeek author of Native Sparse Attention won best paper award of ACL. I was lucky enough to attend a live lecture where he revealed: - scaling up context length to 1 million - this will be in the next frontier model There is good reason to believe DeepSeek V4 will use NSA.