Investing.com -- Leading chip companies at this year’s Hot Chips conference identified a shared engineering priority — maximizing computing performance within fixed power constraints, according to Barclays.

Analyst Tom O’Malley said tokens per watt has become "one of the most critical yardsticks of success today," as data center space remains constrained and demand for computing tokens continues to outpace supply. Hardware providers, hyperscalers, and component suppliers all emphasized getting the most performance out of fixed power budgets.

AMD said it enhanced its transcendental engine and added a tensor data mover to boost compute and efficiency. Microsoft introduced a new architecture called Software Defined Local Access Dataflow, which uses explicit software orchestration to lower total cost of ownership through better energy efficiency.

O’Malley said the industry is also shifting focus from individual components to full-system performance, with racks now treated as "the unit of compute." He noted differing strategies for partitioning AI workloads between training and inference chips, with the current debate centered on disaggregated inference.

Nvidia discussed a speculative decode approach at the conference, while Cerebras is pursuing disaggregated inference through partnerships with AMD and AWS, which O’Malley said "can have an order of magnitude improvement" on tokens-per-second-per-kilowatt. OpenAI took a different approach with its Jalapeño chip, keeping prefill, draft, and decode functions on the same chip rather than splitting them across separate hardware.

On the custom chip front, O’Malley flagged growing dual-sourcing trends, noting Google is using MediaTek for one line of chips while also bringing in Marvell for an inference variant. The analyst said Broadcom and Marvell’s SerDes expertise and supply chain capabilities keep their positions "defensible," even as more attention turns to the "chip-on-tile" model.

Memory emerged as another key theme, with capacity, bandwidth, and power efficiency described as central considerations. O’Malley pointed to Nvidia’s recent NVHBM announcement, which the company says could improve memory bandwidth by up to 30%, cut HBM power consumption by 15%, and free up 25% more area on the compute die compared with standard HBM4E.

Looking further ahead, O’Malley said the next major memory architecture shift may involve 3D DRAM, with Samsung and Cerebras both exploring stacked configurations. He said these efforts remain "very early days" given thermal and power delivery challenges.

CXL memory expansion also drew continued industry interest, though the analyst said he remains "wary until we see evidence of deployments."