Direct Atomics. A Post-Mortem

Reflection on my time working on Direct Atomics (deep tech startup) with Curtis (May-Sep 2024).

The Slow Science Problem

We started Direct Atomics thinking we could help companies screen materials faster. TSMC tests materials for chips, BASF for fuel catalysts, Merck for drugs. Years of experiments, billions in R&D, and the product can still flop.

Atomic simulations help researchers decide what's worth making and testing. They're used for everything from drug molecules to materials that turn spacecraft heat into electricity (NASA).

The catch is that the simulations are also expensive. Electrons interact with each other. Add more atoms, and accounting for those interactions takes a lot more compute.

Cheating Atomic Math

Density Functional Theory (DFT) works with electron density: how electrons are distributed through space. DFT works by guessing a solution, refining it, and repeating until the answer stops changing. That's a lot of computation to do for every candidate material.

We trained a neural network on atomic structures with their DFT results. The input is the types and positions of atoms; the output is the energy and forces. New structure in, estimated properties out. We wanted those estimates to be close to DFT, with much less compute.

The 3,000-atom comparison below is two months on 36 CPU cores versus 0.3 seconds on one GPU, roughly 17 million times faster for that calculation.

The appeal was being able to screen more materials and study larger systems, including proteins. We thought there was a business in making that easier. As we found out, finding a better material wasn't necessarily what customers were stuck on.

Click, Drop, Compute

The core technology is an Equivariant Graph Neural Network. The input is the atomic system (types, positions of atoms, lattice structure), and the output is the system property like total energy or forces per atom.

In June, we trained a small model with 13.8 million parameters (332 MB) using the eSCN architecture on data from the top 20 most common elements from the Alexandria dataset. Our goal was to validate the training process on a manageable scale before moving to the full datasets (≈20 million), running it on a single GPU for 8 hours. There were duplicate entries and unexpectedly high force values in the data. We cleaned it up. For the model, we added tests: did the outputs stay consistent when we transformed the input?

We also worked on the user experience. Here is the workflow we wanted to replace, alongside Direct Atomics:

What Went Wrong?

A lot of things. In short, we made something that people don’t want.

What the heck, there are literal playbooks not to fumble like we did- Unusual VC’s Discovery’s Playbook, The Mom Test. Now as with anything, it's nuanced. But in a few sentences, here is the problem:

We started with the idea that computational tools could help industries screen materials faster and better. We made this decision based on what we saw in academia and national lab research.

In industries like semiconductors, material screening isn’t the bottleneck. There are billions of dollars invested in optimizing materials and processes to achieve precise, physical outcomes. If a computationalist finds a material that’s 20% better for a use case, nobody else in the company cares. It’s not just about finding a better material-it’s about everything else already in place.

Oops.

Here are a few of the other influencing reasons we decided to pivot:

• Market size is too small: Only about 100 multi-million dollar contract customers = Long negotiated deals.
• Customers won’t use SOTA models: They rely on cheaper heuristics (element types, in-house models) to narrow search spaces, limiting revenue.
• Schrödinger's scaling problem: 30 years in, they generate $100M ARR, mostly from on-prem installs, with scaling hampered by heavy customization needs.
• Workflow pride: Computationalists see drag-and-drop interfaces as insulting; they prefer running specific simulations to understand material chemistry.
• In-house capabilities: Companies want to develop their models to avoid third-party reliance.
• Wealthy companies: Private companies that can pay are rich enough to run expensive physics calculations themselves.
• Open-source decentralization: Many functionalities are already decentralized due to open-source models.

You wouldn’t know these challenges unless you have tried selling to industry customers.

Mistakes We Made

One of our best decisions was taking a day for a pre-mortem-writing down every critical assumption that had to hold true for our startup to succeed. No founder wants to do this; it’s like a parent listing every flaw of their child. But facing those uncomfortable truths helped us gather the learnings below and make the pivot we needed.

There is The Pocket Guide of Essential YC Advice which we initially thought didn’t apply to us because we were doing Deep Tech- spoiler alert it did:

1. We spent a month trying to train a diffusion model to generate new materials-turns out, we didn’t have enough data. Then, we tried training the model again-didn’t have the money. Another month down the drain. Don’t build stuff that would be useful someday -build for now.

2. If you’re proud of the first product you launched, you probably waited too long to launch. There’s always a 90/10 solution. In our case, the janky, fragile, non-scalable version was the one that worked. From an engineering standpoint-holy shit-it was horrible. But it got us traction.

3. Don’t scale your team until you’ve built something people want. We reached out to 200 ML and infrastructure engineers-algorithm-heavy talent from quant firms and academia-and convinced eight to join once we secured funding. We did this to impress investors and outpace the competition. In hindsight, private industry didn’t want the product. Scaling before knowing that was a waste of time.

4. Ignore your competitors-you’ll die of suicide, not murder. But we couldn’t help ourselves. Stalking them filled some domain expertise gaps and prevented us from getting grilled by VCs, but it also caused unnecessary stress. How were we supposed to outcompete Microsoft Quantum with an army of MIT PhDs? Or Matlantis with most of Asia’s Kaggle grandmasters? Or CUSP with Geoffrey Hinton? In reality, the problem space was already kicking our collective asses, Nobel Prize winner or not. Worrying about things we couldn’t control was a waste of time.

5. Talk to users first, then write code. You can’t learn or build everything. Reading papers and writing code is not a substitute for real user conversations. You never want to be in the position of a technology searching for a problem. Assume you know nothing unless you’re the expert. Code should test specific, predefined hypotheses. We failed because of business issues, not technical ones, so focus on solving real-world problems by listening first.

Days to Pivot

Days before our second YC interview and investor calls, we realized Direct Atomics wasn’t venture-scale, though some investors remained interested. We needed a new idea. Here’s what we tried:

1. Scoured Our Experiences: Looked for gaps in our work and personal lives-but only came up with ordinary ideas.
2. Considered Recent World Changes: This led us to solutions searching for problems.
3. Targeted Low NPS Giants: Identified big companies with poor customer satisfaction to improve their user experience.
4. Talked to Uncommon Voices: Spoke with professionals outside tech-doctors, nurses-to gain fresh perspectives.
5. Explored Online Epiphanies: Read about life-changing experiences on forums.

We rated each idea from 1 to 10 on:

• Opportunity Size
• Founder-Market Fit
• Ease of Starting
• Early Market Feedback

No new insights came from this, however. Ideas best happen organically.

The Technical

Density Functional Theory

For a fixed arrangement of nuclei, the electron problem is:

E is the energy. The wavefunction $\psi$ depends on every electron's position: 3N spatial coordinates for N electrons.

The first term in the Hamiltonian is kinetic energy. The second covers the potential due to nuclei and other external fields; the third is repulsion between electrons.

For ground-state DFT, we want the electron density n(r). Integrating over the other electron positions gives the expected number per unit volume at r:

n(r) needs only three spatial coordinates. Calculating the full wavefunction first would defeat the point, though. Kohn-Sham DFT gets the density through one-electron orbitals in an effective potential:

Sum over the occupied orbitals:

The effective potential depends on the density. It includes the external potential, electron repulsion and an approximation for exchange-correlation.

So the calculation loops:

1. Guess the density.
2. Construct the effective potential.
3. Solve for the orbitals.
4. Recalculate the density from them.
5. Repeat until the change is smaller than the chosen tolerance.

And we pay for that loop for each new atomic structure.

Equivariant GNNs

Rotate a molecule and its energy should stay the same. The forces on its atoms should rotate with it. We want the model to obey those rules without having to learn every orientation from the data.

R is a rotation. For a vector output like force, rotating the input should rotate the output by the same amount. That's equivariance. Energy is a scalar, so it stays unchanged: invariance.

Equivariant Graph Convolutional Layer (EGCL)

The EGNN paper gives a compact example of how this works. These are its layer equations; the eSCN model we trained uses a different architecture. Each node represents an atom, with a position and a learned set of features (its embedding).

1. Edge Message Calculation

   $$m_{ij} = \phi_e(h^l_i, h^l_j, ||x^l_i - x^l_j||^2, a_{ij})$$

   This equation computes a message $m_{ij}$ between two connected nodes $i$ and $j$, where:

   • $h^l_i$ and $h^l_j$ are node embeddings at layer $l$,
   • $||x^l_i - x^l_j||^2$ is the squared distance between their positions,
   • $a_{ij}$ represents edge attributes such as charges or bond types.

2. Position Update

   $$x^{l+1}_i = x^l_i + C \sum_{j \neq i} (x^l_i - x^l_j) \phi_x(m_{ij})$$

   This updates the position of node $i$ based on the weighted sum of the relative differences in position from its neighbors. The function $\phi_x$ provides the weights, making sure that the update remains equivariant.

3. Message Aggregation

   $$m_i = \sum_{j \neq i} m_{ij}$$

   This aggregates all messages $m_{ij}$ for each node $i$, combining the information from its neighboring nodes.

4. Node Embedding Update

   $$h^{l+1}_i = \phi_h(h^l_i, m_i)$$

   Update the node's features using its previous state and the combined messages from its neighbors. The position update happened in step 2.

Why it matters

DFT data is expensive to produce. NequIP got comparable accuracy with up to 1,000x fewer training examples than the methods it compared against. The tests included water, ice, small organic molecules and periodic materials. Useful evidence, though that number belongs to those experiments.

There is also the cost of running the network. eSCN made the equivariant convolutions cheaper; its authors trained across multiple GPUs. That still leaves the question we cared about: given a new structure, how quickly and accurately can we predict its energy and forces?

Multitask Learning

Will flesh out soon. TLDR: you can learn across multiple DFT datasets with different pseudopotentials simultaneously, improving generalization for predicting properties like energies and forces with fewer data, while preserving task-specific outputs for diverse applications.