It’s incredible to see the concentration of hardware buying power that has occurred with today’s paradigm of “centralized processing” in the cloud/datacenter. This is obvious from the reporting, from huge customers buying lots of DRAM, and suppliers reaching a level of consolidation that makes it straightforward to see enterprise technology trends. Consumer and Client are more nebulous and so it’s much more difficult to paint a clear picture of the technologies that are gaining and keeping traction. I’ll split this into 2 sections since they are somewhat orthogonal to each other, but still both memory related.
The Enterprise - Theme: Capacity Is King #
DDR5 has been out for a full generation now, but you’d think it was ancient technology due to how much attention is being spent on HBM and other types of DRAM in the market, namely on superchips and AI accelerators, which are driving a large portion of the market’s attention right now. But when you look at the sheer numbers, how quickly and gracefully DDR5 has made it’s transition into existence (despite a drastic departure from the previous DDR implementations), DDR5 is an enormous success story and it’s just getting started. There is something new in the DDR5 landscape called MCR DIMM (Multiplexer Combined Ranks) which allow for better use of the physical space (assuming the CPU can leverage it). You can couple that with clever things like tall DIMM form factors to get memory density and performance, that is an enormous leap forward from previous DDR generations. There are of course plenty of kinks to work out in this new deployment style before it lands anywhere outside of specialized use cases, but it’s still something to keep an eye on. DIMM’s have also changed in capacity due to the availability of non-binary half steps in capacity, and we are seeing 24Gbit based DIMM and could even see higher densities in the near future. Memory as a part of the BOM cost of a server has been going up, not because it’s getting more expensive, but because applications with real time needs are growing in the enterprise, putting pressure on the need to expand memory capacity. This includes the exceptional strain that inferencing at scale puts on the entire memory ecosystem.
There is also clever use of LPDDR5x happening in the market to augment fully integrated systems and usecases that are okay with embedded DRAM. This should be monitored as we see fabric attached accelerators becoming more the norm in the next 2-3 years.
The Consumer - Theme: Efficiency is King #
When you look at the consumer market the trend is “I want a device that lasts days without powering down and I need it to do more than ever”. Today’s mobile devices and even SFF computers give SERVER infrastructure less than a decade old a run for their money, with some consumer devices on sub 10W power budgets. When we think about what is happening and evolving in the mobile space especially, we have seen the benefits of leveraging LPDDR and LPDDRx technologies in phones. What if we could take those benefits and improvements in power consumption and raw performance and package them in a way that didn’t mandate soldering and removing the servicability from the device? Enter LPCAMM, a new device form factor that many manufacterers are working on bringing to market. It has a 60% smaller footprint and consumes significantly less power than traditional SO-DIMM.
As with enterprise, we are entering a new era of compute and inference at the endpoint and edge. Low power extremely efficient DRAM is going to play a huge role in the real time nature of the customer experience, interacting with on-device processing and inference.