How Lookup Tables Could Cut AI Energy Costs

Much of modern AI runs on multiplication. Neural networks behind everything from generated answers to photo organization and song recommendations perform millions or billions of operations that multiply inputs by learned weights. Lizy K. John thinks that’s more work than the job requires.

John, a professor of electrical and computer engineering at the University of Texas at Austin, has spent the past five years working on a class of models called weightless neural networks. Instead of repeatedly multiplying inputs by weights, these networks pass binary inputs through interconnected lookup tables—closer to consulting a collection of stored answers than solving the same arithmetic problem repeatedly. Depending on the task, she says, the networks can be less than a thousandth the size or 1,000 times as fast as conventional alternatives while maintaining comparable accuracy.

Her team’s work has so far focused on small, specific problems: medical sensors, activity tracking, keyword spotting. But she thinks the same approach could eventually reach much bigger targets, including the transformer models behind today’s chatbots.

What made you walk away from weights in the first place? Was there a specific moment that pushed you toward lookups instead?

Lizy K. John: A friend casually invited me to a weekly meeting a few years ago to talk about using lookups instead of weights, a technique that wasn’t new. Someone in the U.K. had built a commercial product around it in the ’80s for pattern recognition, and then it just disappeared. A couple of professors at the Federal University of Rio de Janeiro kept working on it quietly for years, just one research group, so it wasn’t a lot of work.

My friend knew I had hardware implementation experience, so he thought I could help make it real. I had a student who I thought was perfectly positioned to take this on. He’d been working on spiking neural networks, another alternative to standard neural networks. The common thread for me was energy efficiency. I was upset by how much power was consumed by AI, and my ears were open for other ways of doing it with less technology, less power.

In less than six months, we had it running on an FPGA, basically a ready-made chip. We were able to create some very small neural networks that used the lookup methodology that could fit on tiny chips and didn’t need a GPU to run them. We could get them 1,000 times smaller than what everyone else was getting. That was encouraging.

Energy-Efficient Weightless Neural Networks

Why is now the time to look into weightless neural networks?

John: When you think about current AI models, it’s amazing what they do. I’m impressed by ChatGPT every time I use it, even though it gets things wrong sometimes. But when it’s coming up with the next word in a sentence, it’s doing millions or billions of multiplications to get there. If you ask me a question, I’m not doing multiplications to answer you. I’m thinking, yes, no, I…