writing hard things

I’ve worked on hard problems: implemented AI papers, optimized backend services, troubleshot networking issues in production, databases. Some for work, some for fun.

Building hard things gives you understanding and skills. Everyone talks too much about understanding. Nobody talks enough about skills. If you don’t build hard things, you won’t know these skills exist. You won’t learn them.

Understanding:

What I cannot create I don’t understand! - Richard Feynman

Skills:

Know how to solve every problem that has been solved. - Richard Feynman

Write hard things. Build a VM from scratch. Write a compiler. Implement a database. These projects teach you skills you won’t develop any other way.

Easy advice

Most programming advice targets simple and easy problems. This advice will fail.

Try applying “keep functions small” to a complex concurrent and parallel state machine. Try applying “use design patterns” when you’re optimizing for cache locality. Try applying “write tests first” when you don’t know how to solve the problem yet. It doesn’t work.

Traditional unit testing doesn’t work for distributed systems or databases. You need formal methods, deterministic simulation testing, OS-level fault injection, and deep analysis that validate correctness.

“Clean code” abstractions that help with simple problems become performance bottlenecks in systems programming. A memory allocator isn’t “clean.” It can be hard, dirty, and ugly. It has pointer arithmetic, bit manipulation, low-level operations. But it can be readable. The code shows what it does, why it does it, what invariants it maintains.

Simplifying a paper is difficult. Implement it first as you understand it. Write tests and benchmarks. Compare it to alternatives. Prove it. Iterate on the invariants and properties until you minimize them to a successful local minimum. Then simplify by removing state, improving efficiency or performance. If you reach this point, congratulations! You’ve achieved a new state of the art result for your field. Publish it.

And more: Workflows, manual memory management, concurrency runtimes, network protocols. Writing public APIs for widely adopted tools, debugging and performance tracing. These hard problems need different skills.

Hard advice

You can’t understand it without implementing it. You can’t simplify it without first making it work. You can’t test without knowing what correctness means and how to test reliably. You need to profile and debug. You may need to read assembly and target different CPU architectures. You may need to open a math book.

Write hard things.

../