Direct Atomics. A Post-Mortem

Reflection on my time working on Direct Atomics (deep tech startup) with Curtis (May-Sep 2024).

The Slow Science Problem

We started Direct Atomics thinking we could help companies screen materials faster. TSMC, BASF and Merck spend years and billions testing materials for chips, fuel catalysts and drugs, and the product can still flop.

Atomic simulations help researchers decide what's worth making and testing. Drug companies use them to design molecules, and NASA uses them for materials that turn spacecraft heat into electricity.

The simulations are expensive to run, since every electron interacts with every other one. Add more atoms, and accounting for those interactions takes a lot more compute.

Cheating Atomic Math

Density Functional Theory (DFT) works with electron density: how electrons are distributed through space. To find the density, DFT guesses a solution, refines it, and repeats until the answer stops changing. That's a lot of computation to do for every candidate material.

We trained a neural network on atomic structures and their DFT energies and forces, so it could estimate those for new structures. We wanted the estimates to be close to DFT, with much less compute.

At 3,000 atoms, the network ran over 17 million times faster than the simulation, 0.3 seconds on one GPU against two months on 36 CPU cores.

The appeal was being able to screen more materials and study larger systems, including proteins. We thought there was a business in making that easier. As we found out, finding a better material wasn't necessarily what customers were stuck on.

Click, Drop, Compute

The core technology is an Equivariant Graph Neural Network. The input is the atomic system (types, positions of atoms, lattice structure), and the output is the system property like total energy or forces per atom.

In June, we trained a small model with 13.8 million parameters (332 MB) using the eSCN architecture on data from the top 20 most common elements from the Alexandria dataset. Our goal was to validate the training process on a manageable scale before moving to the full datasets (≈20 million), running it on a single GPU for 8 hours. We found duplicate entries in the data, and force values a lot higher than we expected, so we cleaned the dataset. We also wrote tests to check the model's accuracy. Some of them transformed the input to see if the outputs stayed consistent.

We also worked on the user experience, starting from the workflow we wanted to replace.

Play1:38

What Went Wrong?

A lot of things. In short, we made something that people don’t want.

What the heck, there are literal playbooks not to fumble like we did- Unusual VC’s Discovery’s Playbook, The Mom Test. Now as with anything, it's nuanced. But in a few sentences, here is the problem:

We started with the idea that computational tools could help industries screen materials faster and better. We made this decision based on what we saw in academia and national lab research.

In industries like semiconductors, material screening isn’t the bottleneck. There are billions of dollars invested in optimizing materials and processes to achieve precise, physical outcomes. If a computationalist finds a material that’s 20% better for a use case, nobody else in the company cares. It’s not just about finding a better material-it’s about everything else already in place.

Oops.

Here are a few of the other influencing reasons we decided to pivot:

• Market size is too small: Only about 100 multi-million dollar contract customers = Long negotiated deals.
• Customers won’t use SOTA models: They rely on cheaper heuristics (element types, in-house models) to narrow search spaces, limiting revenue.
• Schrödinger's scaling problem: 30 years in, they generate $100M ARR, mostly from on-prem installs, with scaling hampered by heavy customization needs.
• Workflow pride: Computationalists see drag-and-drop interfaces as insulting; they prefer running specific simulations to understand material chemistry.
• In-house capabilities: Companies want to develop their models to avoid third-party reliance.
• Wealthy companies: Private companies that can pay are rich enough to run expensive physics calculations themselves.
• Open-source decentralization: Many functionalities are already decentralized due to open-source models.

You wouldn’t know these challenges unless you have tried selling to industry customers.

Mistakes We Made

One of our best decisions was taking a day for a pre-mortem-writing down every critical assumption that had to hold true for our startup to succeed. No founder wants to do this; it’s like a parent listing every flaw of their child. But facing those uncomfortable truths helped us gather the learnings below and make the pivot we needed.

There is The Pocket Guide of Essential YC Advice which we initially thought didn’t apply to us because we were doing Deep Tech- spoiler alert it did:

1. We spent a month trying to train a diffusion model to generate new materials-turns out, we didn’t have enough data. Then, we tried training the model again-didn’t have the money. Another month down the drain. Don’t build stuff that would be useful someday -build for now.

2. If you’re proud of the first product you launched, you probably waited too long to launch. There’s always a 90/10 solution. In our case, the janky, fragile, non-scalable version was the one that worked. From an engineering standpoint-holy shit-it was horrible. But it got us traction.

3. Don’t scale your team until you’ve built something people want. We reached out to 200 ML and infrastructure engineers-algorithm-heavy talent from quant firms and academia-and convinced eight to join once we secured funding. We did this to impress investors and outpace the competition. In hindsight, private industry didn’t want the product. Scaling before knowing that was a waste of time.

4. Ignore your competitors-you’ll die of suicide, not murder. But we couldn’t help ourselves. Stalking them filled some domain expertise gaps and prevented us from getting grilled by VCs, but it also caused unnecessary stress. How were we supposed to outcompete Microsoft Quantum with an army of MIT PhDs? Or Matlantis with most of Asia’s Kaggle grandmasters? Or CUSP with Geoffrey Hinton? In reality, the problem space was already kicking our collective asses, Nobel Prize winner or not. Worrying about things we couldn’t control was a waste of time.

5. Talk to users first, then write code. You can’t learn or build everything. Reading papers and writing code is not a substitute for real user conversations. You never want to be in the position of a technology searching for a problem. Assume you know nothing unless you’re the expert. Code should test specific, predefined hypotheses. We failed because of business issues, not technical ones, so focus on solving real-world problems by listening first.

Days to Pivot

Days before our second YC interview and investor calls, we realized Direct Atomics wasn’t venture-scale, though some investors remained interested. We needed a new idea. Here’s what we tried:

1. Scoured Our Experiences: Looked for gaps in our work and personal lives-but only came up with ordinary ideas.
2. Considered Recent World Changes: This led us to solutions searching for problems.
3. Targeted Low NPS Giants: Identified big companies with poor customer satisfaction to improve their user experience.
4. Talked to Uncommon Voices: Spoke with professionals outside tech-doctors, nurses-to gain fresh perspectives.
5. Explored Online Epiphanies: Read about life-changing experiences on forums.

We rated each idea from 1 to 10 on:

• Opportunity Size
• Founder-Market Fit
• Ease of Starting
• Early Market Feedback

No new insights came from this, however. Ideas best happen organically.

The Technical

Density Functional Theory

For a fixed arrangement of nuclei, the electron problem is:

E is the energy. The wavefunction $\psi$ depends on every electron's position: 3N spatial coordinates for N electrons.

The first term in the Hamiltonian is kinetic energy. The second covers the potential due to nuclei and any other external fields, and the third is the repulsion between each pair of electrons.

Hohenberg and Kohn proved in 1964 that the electron density n(r) is enough to determine the ground-state energy. The density is how many electrons you'd expect per unit volume at r. You get it by integrating the other N-1 positions out of the squared wavefunction and multiplying by N:

n(r) only has three spatial coordinates, however many electrons there are. Computing it from the integral above needs the full wavefunction, with all 3N coordinates. Kohn-Sham DFT builds the density from one-electron orbitals that all move in the same effective potential:

Square the occupied orbitals and add them up to get the density:

Part of the effective potential is the external potential and part is the repulsion from the average electron density. Everything else gets lumped into exchange-correlation, and we approximate that. Since the effective potential depends on the density we're solving for, we start with a guess and iterate:

1. Guess n(r).
2. Work out the effective potential from it.
3. Solve the Kohn-Sham equation for the orbitals.
4. Square and add them up for a new n(r).
5. Compare old and new. If they're further apart than your tolerance, go back to step 2.

You run this loop again for every new structure.

Equivariant GNNs

Turn a crystal around and you've still got the same crystal, so its energy stays the same and its forces turn with it. A regular network only learns that if you train it on every structure at a bunch of different angles. We went with an architecture that has it built in.

Where R is a rotation and f(x) is the model's output. Forces are vectors, so rotating the input by R should rotate them by R too (equivariance). Energy is a scalar and shouldn't change (invariance).

Equivariant Graph Convolutional Layer (EGCL)

The EGNN paper describes a compact equivariant network, and these are its layer equations. The eSCN model we trained uses a different architecture. Each node represents an atom, with a position and a learned set of features (its embedding).

1. Edge Message Calculation

   $$m_{ij} = \phi_e(h^l_i, h^l_j, ||x^l_i - x^l_j||^2, a_{ij})$$

   This equation computes a message $m_{ij}$ between two connected nodes $i$ and $j$, where:

   • $h^l_i$ and $h^l_j$ are node embeddings at layer $l$,
   • $||x^l_i - x^l_j||^2$ is the squared distance between their positions,
   • $a_{ij}$ represents edge attributes, such as the product of two particles' charges.

2. Position Update

   $$x^{l+1}_i = x^l_i + C \sum_{j \neq i} (x^l_i - x^l_j) \phi_x(m_{ij})$$

   This updates the position of node $i$ based on the weighted sum of the relative differences in position from every other atom. With $M$ atoms, $C = 1/(M-1)$ turns the sum into an average. The function $\phi_x$ provides the weights. The messages use positions only through distances, which rotation doesn't change, so the weights don't change either. Since the relative positions in the sum rotate with the structure, the new positions rotate too.

3. Message Aggregation

   $$m_i = \sum_{j \neq i} m_{ij}$$

   This aggregates all messages $m_{ij}$ for each node $i$, combining the information from its neighboring nodes.

4. Node Embedding Update

   $$h^{l+1}_i = \phi_h(h^l_i, m_i)$$

   This updates the node's features from its previous state and the combined messages from its neighbors.

Why it matters

DFT data is expensive to produce, so we wanted a model that learns from less of it. On water and ice, NequIP had lower force errors than DeepMD with 1,000x fewer training examples. Both models respect rotational symmetry, and NequIP's messages carry directions as well as distances. In the same paper, a NequIP limited to distances learned more slowly. The authors also ran NequIP on small organic molecules and periodic materials. The 1,000x is only for water and ice, where NequIP's energy errors were higher than DeepMD's.

Running the network takes compute too. eSCN made the equivariant convolutions cheaper, and its authors trained across multiple GPUs. We had to measure for ourselves how fast and accurately we could predict energy and forces for structures we'd never seen.

Multitask Learning

Will flesh out soon. TLDR: you can learn across multiple DFT datasets computed with different functionals and settings simultaneously, improving generalization for predicting properties like energies and forces with fewer data. Each dataset keeps its own output head, since the same structure gets a different energy in each one.