[JM]: Jumping straight into follow-up this morning, on a recent episode, we asked this rhetorical question: Has Apple given up on producing their own servers?
[JM]: Because it seems that there was this rumor a while back that Apple was using the M2 Ultra processor and producing their own servers for Apple Intelligence, Private Cloud Compute, and other generative software stuff.
[JM]: But we never got any confirmation about those rumors -- still haven't.
[JM]: And there's a lot of speculation as to whether or not that's something that Apple still has an interest in doing.
[JM]: When we talked about it, my take on it is that if you follow the Tim Cook doctrine of wanting to control your own destiny and control things that are core to your business, and if Apple Intelligence and related functionality
[JM]: is something Apple deems core to its business, then you would think this is something that Apple would continue to work on so that they aren't fully dependent on Nvidia and other suppliers for that functionality.
[JM]: And according to rumors, it sounds like Apple is indeed continuing to work on this.
[JM]: It sounds like Apple is targeting the M7 Ultra chip to form the basis of a new Apple server product.
[JM]: And it's not really clear because again, this is all just rumor mongering.
[JM]: So you never know what's actually happening, but it doesn't seem clear to me whether this is something Apple is working on for their own internal consumption
[JM]: to power Apple Intelligence and other needs that Apple has from a server perspective, or whether this is something that they intend to sell as a product outside of Apple, in addition to perhaps consuming and using themselves.
[JM]: But it's an intriguing idea because Nvidia already sells not just their GPUs, but an integrated computer that you can use to run large language models on.
[JM]: And it would be really intriguing for Apple to produce a similar product that's targeted at people who want to do high-end large language model inference.
[JM]: I think particularly because just like Amazon, when they created Amazon Web Services or AWS, basically told the people tasked with creating it, "This is something that has to stand on its own.
[JM]: Other companies have to buy it.
[JM]: This is not going to just be something that we use internally."
[JM]: That was the primary reason for building it, was for Amazon to use it.
[JM]: But the team was essentially told like, "Hey,
[JM]: This is not just a blank check.
[JM]: If it's not good enough for other companies to pay for it, then it's not good enough for us to use it."
[JM]: And I hope that Apple is doing something similar and producing a product that is good enough for lots of people to want to buy and use for the same reasons, because it means that's something that will be good enough for Apple to use for their own purposes as well.
[JM]: I also hope that this M7 Ultra that is supposed to be the processor powering this server also ends up in a product that we mere mortals can someday afford, whether that's in the form of a Mac Studio or some other thing that won't have to be traded for a kidney, because the memory that
[JM]: this server product is supposed to support apparently doubles the maximum amount of memory from a previous maximum.
[JM]: I actually don't even know if that previous maximum effort shipped.
[JM]: I know that they were thinking, they were talking, there's rumors of an M5 Ultra that was supposed to ship with 768 gigabytes maximum RAM.
[JM]: But that never happened because of the RAM crunch.
[JM]: But in any case, the rumor mill says that this M7 Ultra is going to ship with support for 1.5 terabytes of RAM, which I think unless your income is the same as, I don't know, a small to medium-sized country's gross domestic product, I think you're going to have a hard time affording anything Apple sells that contains 1.5 terabytes of RAM.
[DJ]: Given recent events like price hikes, I think you could have just ended that sentence at "anything Apple sells".
[DJ]: I'm a little skeptical at the notion that Apple is going to start shipping a server product.
[DJ]: What, like a rack mount or something like that?
[DJ]: Because you mentioned AWS, and as of the era of AWS and other cloud providers, very few companies have a closet full of server hardware anymore anymore.
[DJ]: That's kind of the bargain, for better or worse, that we've all signed up for, is that Amazon builds the data centers and fills them full of computer hardware and we just pay them to run our infrastructure for us.
[DJ]: So who's going to buy a server from Apple, especially if it costs, I don't know, $100,000?
[DJ]: And then in terms of the consumer market, again, I don't want to be narrow-minded, but...
[DJ]: and maybe I am being...
[DJ]: But when you mentioned like it would be awesome if Apple shipped this computer that's great for like LLM inference, I was thinking like, yeah, the Mac Studio already exists.
[DJ]: You could just go buy one, although they cost $12,000 now or whatever.
[DJ]: So what is this product other than just the M7 version of the Mac Studio, which is what I assume they're going to ship in like 2028?
[DJ]: By the way, what happened to the poor M6?
[DJ]: Like, doesn't it deserve to have its time in the sun?
[DJ]: Why are we skipping right past it to the M7?
[JM]: My understanding is that the rumors are saying that the M6 is going to exist, but it's not going to exist in a Pro, Max, or Ultra version.
[JM]: That it's just going to be the entry-level, step-up version from the M5.
[JM]: Again, non-Pro, non-Max, non-Ultra.
[JM]: Well, there is no Ultra version of the M5.
[JM]: But the idea is that the M6 is just going to be the base model.
[JM]: of that line, and there isn't going to be a non-base model.
[JM]: That's just it.
[JM]: The M6: no M6 Pro, no Max, no Ultra.
[JM]: And then the M7 will carry those additional levels, much in the same way that the M3 had an Ultra, but the M4 did not, the M5 also did not.
[JM]: I think that's the idea.
[JM]: So it's not that the M6 isn't going to exist, it's just that it's only going to be out in its base configuration, not with any of the souped-up versions.
[DJ]: Well, I don't understand why that would be the case, but rumors are never wrong, so I guess we'll just have to wait for the explanation.
[JM]: I remember hearing the explanation, but I don't remember it, so I'm not gonna try to piece it together.
[JM]: And also I don't care that much.
[JM]: Ultimately, it sounds like the M7 is where these higher-powered processors are going to be focused.
[JM]: And I agree with you that if they produced an M7 Ultra machine that's great for inference, I agree with you.
[JM]: Like why not just a Mac Studio?
[JM]: Like why a server product?
[JM]: Presumably the answer to that is
[JM]: density in a data center environment, like whether it's rack mounted or not.
[JM]: But again, it's such a weird, like Apple hasn't produced one of those since the XServe.
[JM]: I mean, it's been... pick a number... 15, 20 years?
[JM]: I don't even know how long it's been, but it's been a long time.
[JM]: So I agree with you that it seems really unlikely that they would do that.
[JM]: And the product that I mentioned as this would be competition, right?
[JM]: This product that Nvidia sells as an all-in-one inference box
[JM]: is itself a tiny Mac Studio sized server called the DGX Spark.
[JM]: So I'm with you.
[JM]: I don't see why Apple would produce something, unless you can't fit 1.5 terabytes of RAM into a Mac Studio.
[JM]: That could be the case.
[JM]: I'm not a hardware engineer.
[JM]: I don't know what the inside of this thing on my desk looks like.
[JM]: But the thing that Nvidia sells is $5,000 with 128 gigabytes of RAM.
[JM]: So you figure something with more than 10 times that amount of RAM and probably 10 times the cost
[JM]: or more, is probably going to take the form of something maybe a little bit larger than a Mac Studio.
[JM]: But who knows?
[DJ]: Well, the Apple rumor mill has always been a fun thing to participate in.
[DJ]: And to some degree, it would actually be nice to hear something wild come out of it.
[DJ]: Because for the last very many years, with the possible exception of the Vision Pro a few years ago,
[DJ]: I mean, Apple has mostly been making has just been iterating.
[DJ]: They've been like, well, here's the new iPhone.
[DJ]: Here's the new Mac.
[DJ]: Here's the new Apple Watch.
[DJ]: And once in a while, here's a monitor, which is fine.
[DJ]: Like, don't get me wrong, but it's fun to come across a rumor that's like, hey, Apple's making some super weird thing.
[DJ]: And then we can all go, what?
[DJ]: Why would they do that?
[DJ]: Maybe they are.
[DJ]: Maybe they aren't.
[DJ]: In this case, the notion of Apple making a server product and trying to sell it to either consumers or businesses doesn't make any sense to me for a variety of reasons.
[DJ]: Maybe it will happen, and that would be fascinating.
[DJ]: But I think the aspect of this I'm more interested in is the framing of the original conversation we had about this, which was,
[DJ]: Is Apple building out their own infrastructure for themselves for whatever variety of reasons so that they're not dependent on other companies' data centers?
[DJ]: I think that's an interesting question, especially in light of, though I don't think these things are directly related, but especially in light of the ongoing lawsuit now between Apple and OpenAI, right?
[DJ]: Where these two companies are now kind of at odds with each other.
[DJ]: Well, there's only so many big players in this game, where if you're not going to run your own large language model inference and you're a giant company like Apple, I mean, you're going to have to go to one of only a very small number of companies.
[DJ]: And we already know that Apple's working with Google for the actual infrastructure behind a lot of Apple Intelligence.
[DJ]: I think that was the last version of that that we heard about.
[DJ]: And so I am very interested in are they doing that while they build their secret bunker somewhere full of M7s so that they can eventually, even if they don't make a big marketing deal out of this, so they can eventually move more and more of the back end of Apple Intelligence into their own infrastructure.
[DJ]: That does feel like something that Apple would do.
[JM]: Yeah, and that is precisely what I argued for when we talked about this last time.
[JM]: To me, it makes a lot of sense that that's something that Apple would do.
[JM]: And when we talked about Apple using Google's Gemini model for certain aspects of Apple Intelligence, I said much the same thing, which is that I feel like this is just a stepping stone.
[JM]: This is just them saying, okay, right now we are behind until we can catch up on our own
[JM]: Google, we're going to utilize some of your Gemini infrastructure.
[JM]: And similarly, I think Apple feels this way about not just the model technology, but also the infrastructure itself.
[JM]: I think they feel like they're overly reliant on other companies, and them shipping their own server, whether it's entirely for internal use, or whether they actually ship it to other customers.
[JM]: I think it makes a lot of sense for Apple.
[JM]: in terms of controlling their own destiny.
[JM]: You posed the question like why, why would Apple sell this as a product?
[JM]: And I think my answer is because it would be really profitable.
[JM]: Apple would make, I think, a lot of money doing it.
[JM]: The DGX Spark that I mentioned before that goes for $5,000 -- you can't buy one.
[JM]: They're sold out.
[JM]: So there's a lot of demand for this product.
[JM]: The Mac Studios that have this older generation M3 Ultra chip, but have lots of RAM back when RAM was affordable, those are now going for...
[JM]: tens of thousands of dollars, like huge multiples over what they cost when people bought them.
[JM]: So there's a lot of demand for devices that can run language models locally, but there's not a lot of great solutions for doing that.
[JM]: And Apple is very well positioned for shipping products, they can meet that demand.
[JM]: So I think it's actually really smart if that's what they're gonna do.
[JM]: When I think of all the different ways that Apple could make money selling new product categories or product line extensions, this seems like a very promising, profitable way for them to do that.
[JM]: And I would much rather
[JM]: that they try it, then say, I don't know, to waste a bunch of money trying to build a self-driving car or an Apple branded TV or any of the other things that they've done in-house and never shipped.
[DJ]: I think you've sold me on the idea, actually, because that's true that there's so much demand for large language model inference right now.
[DJ]: Like if we talk about what's really in demand, it's that it's, it's that people want the output of large language models and they want it now.
[DJ]: And you can either get that from the so-called frontier companies like OpenAI or Anthropic, or you can get it in a variety of other ways.
[DJ]: And one of those is running inference on your own hardware instead of someone else's.
[DJ]: You can't use their models, but as we've occasionally discussed and will continue to, there are
[DJ]: pretty good models that you can run yourself these days for a lot of workloads that you would want a large language model for.
[DJ]: So I was saying a moment ago, like we've moved past the era where companies have a closet full of hardware, but maybe to some extent, at least for some use cases, that era will come back.
[DJ]: And for some companies, possibly many, because as you said, there's so much demand for the outcome of having this sort of hardware...
[DJ]: that yeah, maybe Apple could sell a bunch of really highly powered machines that are even more attuned to running inference.
[DJ]: Like maybe they're easier to hook together in parallel, for example, than their current hardware.
[DJ]: I have thought before that, like, I don't know to what degree they could have foreseen this when they were designing Apple Silicon, but they...
[DJ]: one way or another, they ended up in a really good place where they designed this system that has an inherently combined memory architecture where the memory is shared between the CPU and the GPU, which it turns out is, as far as I'm aware, like the ideal hardware environment for running large language model inference.
[DJ]: I bought a non-Apple machine, the Framework Desktop, which is kind of also designed for that.
[DJ]: So there are other parties that are building these.
[DJ]: In that case, it's hardware from AMD, and they call it an APU, where it's a whole system on a chip that combines the graphics processor and the central processor.
[DJ]: That's how Apple Silicon works also, essentially.
[DJ]: So it is very interesting that one way or another, and for all the things we can say about Apple being behind in so-called AI, they have also built themselves like a hardware platform that seems particularly good at it.
[DJ]: So yeah, maybe they should capitalize, or I'm sure they will capitalize on that one way or another.
[JM]: Yeah, and as we've talked about in the past and we'll talk about again momentarily, most of the money in this whole race to produce generative software solutions is being made by the hardware producers.
[JM]: It's being made by the companies that are making the processors and the RAM and the solid state drives and all the other components that go into the computers that train and use the large language models that that training yields.
[JM]: And Apple is one of the few companies that is very well positioned to make a lot of money on hardware.
[JM]: So the more that we talk about it, the more sense it makes to me and the more I hope they do this, both from the perspective of, okay, I own Apple stock and sure, that would be cool.
[JM]: I'm sure that would be very profitable and for them and thus for me.
[JM]: But I also want to buy one of these things, assuming that I could actually afford it.
[DJ]: Maybe if you sell some of your Apple stock, you'll be able to afford it.
[JM]: Right.
[JM]: And even if I decide that it's more money than I have or want to spend, I think it's a cool product in and of itself.
[JM]: And I think it would be cool for it to exist.
[JM]: So here's hoping.
[JM]: All right, moving on to another bit of follow-up.
[JM]: I wanted to do a quick little check in on the share price of everyone's favorite company, SpaceX.
[DJ]: Are we sure it's not called SpaceXAI now?
[JM]: I've actually seen it referred to that way.
[JM]: I've seen people write out SpaceXAI, all one blob of a word.
[DJ]: Yeah, I have also.
[DJ]: And I have to admit, I saw it and just assumed, yeah, they probably did rename the company that.
[JM]: They're just going to merge it into, I think I was about to say they're just going to merge it into X, but maybe they already did that.
[DJ]: I can't remember.
[DJ]: Well, it's hard to tell because there's already an X in the name.
[DJ]: So is SpaceX and X the same thing?
[DJ]: Isn't X called XAI now?
[DJ]: I think so.
[DJ]: Anyway, given all this uncertainty about what this company is called, their stock must be doing great.
[JM]: Absolutely.
[JM]: The slurry that is the companies that have been smashed together into one is now trading at $120 a share,
[JM]: compared to its IPO price of $160.
[JM]: So in other words, debuted at $160, went up to like $220 something or other, and then has now, like Icarus, come back to earth.
[JM]: And if you were one of those people that wanted to get in early on the hype train and didn't sell on day one or two, I'm sorry, you are underwater and that's not a fun place to be, but...
[JM]: For those of us who don't own the stock and are sitting here on the sidelines with buckets of popcorn, it is entertaining.
[JM]: I'm not going to pretend otherwise.
[DJ]: Well, or I guess if you believe that someday SpaceX will be worth $100 billion a share...
[DJ]: For sure.
[DJ]: Then, you know, this is just a temporary blip.
[DJ]: I mean, it did come down awfully fast.
[DJ]: One thing I'm curious about, though, Justin, you said the IPO price was $160.
[DJ]: And I thought it had launched at like $135 and then gone up and then come back down.
[DJ]: I mean, it's below that either way.
[DJ]: But what's the $160?
[JM]: My understanding is that that price refers to if you tried to place an order on the first day of trading, that is effectively what it opened at.
[JM]: Now the IPO price of $135 is presumably like if you're an institutional investor
[JM]: or whatever, you arrange to buy the stock at the issue price on that day.
[JM]: And so you're buying it at that institutional IPO price.
[JM]: But if you're buying it off the public market on the first day of trading, all of those people that had it at $135 and then sold it-
[DJ]: They pushed the price up, yeah.
[JM]: Exactly.
[JM]: Well, the demand is what pushed the price up, not the supply, but yeah.
[DJ]: I know, but no, that's a good point.
[DJ]: And that had slipped my mind, the notion that like, well, it's issued at a given price, but as soon as people start buying it, the price goes up.
[DJ]: So you're saying for the average, at least like retail investor who could have bought it in the first day of trading, they would have been paying about $160 per share and that's now worth $120 per share.
[JM]: Right.
[JM]: And like you said, if you bought it for the long haul because you think it's a great investment, it's a long-term investment, and that is by and large how people should be buying stocks in the stock market, then there's no inherent problem.
[DJ]: Well, other than that, it would have been better for you to get on the train now and not when it first opened.
[DJ]: But of course, that's a bet some people would make.
[DJ]: When I talk to people who don't really get stocks in the stock market,
[DJ]: It's kind of funny to try to explain things and people are like, well, this sounds a lot like gambling.
[DJ]: And I'm like, well, yeah, I mean, it is.
[DJ]: You're making bets on an uncertain outcome.
[DJ]: It's just sort of decorated with a lot of other things, including in fairness, like financial regulation to try to prevent you from losing all your money quite as easily as other forms of gambling.
[DJ]: But yeah, so there are people who said this thing is going to start here and then only skyrocket forever.
[DJ]: Right.
[DJ]: So I'm going to buy it at $160.
[DJ]: And then there are probably other people who've been waiting for this dip, but still think that in the long run, SpaceX is going to be a valuable company.
[DJ]: So they'll buy it at like $120 or whatever and hold it.
[DJ]: And then, yeah, there's the other people.
[DJ]: I don't know that we're hoping they could.
[DJ]: I mean, there are the other people inevitably who will end up having bought it at $160 and selling it at $120 and be down money because that's human nature.
[JM]: Yeah.
[JM]: It's also, unfortunately, the fate of most retail investors.
[JM]: The adage is, sadly, that retail investors are more or less livestock meant for slaughter.
[JM]: That is how they are referred to, by and large.
[JM]: They're the ones that get creamed by the Wall Street traders.
[JM]: That's at least how it is portrayed in...
[JM]: the media and in everyday discussion of how the stock market operates, whether that's true or not, I will leave to the listener.
[JM]: But I think that if you were someone who thought you were getting in on the ground floor and weren't necessarily in it for the long haul, then I think you have proven that adage true.
[DJ]: Yeah, well, the conventional wisdom is "buy low and sell high", and typically retail investors, which for anyone who's unaware refers to like you and me and normal people who do not work for gigantic investment companies, which are called institutional investors.
[DJ]: That retail investors usually do the opposite,
[DJ]: that we buy stuff when it's overpriced and end up selling it when it goes down because we freak out.
[DJ]: And unfortunately, that's exactly the wrong thing to do.
[DJ]: And double unfortunately, the people who are the beneficiaries of that are the institutional investors.
[DJ]: Because who do you think is buying those stocks when you're selling them at $120?
[JM]: Indeed.
[JM]: Speaking of companies who have been in the news, in part because their financial performance hasn't been so great, I saw the other day that the debt of Oracle...
[DJ]: Everyone's favorite company.
[JM]: Everyone's favorite producer of databases that only make sense if you are a Fortune 10 company, by and large.
[DJ]: And even then, I'm pretty sure the only reason they make sense is graft and not actual like technical underpinnings.
[JM]: I would probably say the same, but I'm not a Fortune 10 company.
[JM]: I was trying to give those customers the benefit of the doubt.
[JM]: But Oracle's debt has been downgraded from BBB to BBB-.
[JM]: And that may not sound like a big deal if you aren't familiar with how corporate debt is rated.
[DJ]: Here's a hint.
[DJ]: The rating system makes a lot less sense than your grades in high school.
[JM]: a lot less.
[JM]: Triple B minus means that Oracle's debt is now rated one rung above junk bonds.
[JM]: And that's not good.
[JM]: In case that's not clear.
[JM]: You generally don't want your debt to be classified as junk.
[JM]: And what does it mean when people say junk?
[JM]: Well, what they mean
[JM]: is that there is a significant risk of default.
[JM]: People refer to them as junk bonds because the price of that bond is generally lower than other higher quality bonds.
[JM]: And that price is lower because of a reduced confidence that that company will pay that debt back to you.
[JM]: And for a company who's been around since the 1980s,
[JM]: And for better or worse, has been one of the pillars of the tech infrastructure sector...
[JM]: For worse.
[JM]: ... since that time ...
[DJ]: Yeah, no, sorry.
[DJ]: I'm willing to make a call on this.
[DJ]: Both due to their technology and the aims of the people who own and run that company.
[DJ]: It's for worse.
[DJ]: But anyway, they've been around for a long time.
[DJ]: It is surprising that they've destroyed this much of their value, I guess.
[JM]: Yeah, that there is so little confidence in the credit worthiness of their liabilities that the bonds they're issuing are seen essentially as just a hair above junk bonds.
[JM]: And if you look at some of the dynamics that are causing this, like why is this happening?
[JM]: Well, before we get to the why, let's look at one other indicator.
[JM]: Oracle's stock on June 1st, which was not that long ago, what, six weeks, maybe seven, was trading at $248 a share.
[JM]: Yesterday, it closed at 121 or less than half.
[DJ]: Whoa.
[DJ]: Okay.
[DJ]: That is a much larger drop than SpaceX.
[JM]: Yeah, to lose half of your company's market capitalization in the span of seven weeks, that is an impressive feat.
[JM]: And I think what we're seeing here is one of the first few dominoes to fall when it comes to this huge investment in generative software infrastructure, because Oracle is among a few of these companies that has just plowed money into data centers and
[JM]: all things related to large language model infrastructure without a corresponding indication of future revenue.
[JM]: It's not like there's lots of evidence that they're going to not only make that money back, but that they will achieve a
[JM]: significant return on that investment.
[JM]: And so that's why we're seeing their debt get downgraded.
[JM]: That's why we're seeing their stock price cut in half.
[JM]: And I think it is perhaps an omen for what we've been talking about coming down the road, which is a significant correction in the tech sector because of this, what I consider to be overinvestment in the generative software area.
[JM]: And I've kept my eye on CoreWeave as well.
[JM]: And they're also continuing their slide.
[JM]: So it'll be interesting.
[DJ]: One thing that's interesting in this is that there's like different tiers of the zeitgeist in tech where like you've definitely heard of.
[DJ]: In fact, you can't stop hearing about.
[DJ]: In fact, you wish you would just not hear about them anymore.
[DJ]: Companies like OpenAI and Anthropic and Microsoft and Apple and Google and Meta.
[DJ]: Oh, sorry.
[DJ]: I mean, Facebook.
[DJ]: And etc.
[DJ]: Like you hear about these companies are always in the news.
[DJ]: A lot of what they're in the news about is plowing all this money into generative AI infrastructure, data centers, etc, etc.
[DJ]: And there's all of this like, is this going to pay off?
[DJ]: Is it not?
[DJ]: Is it going to ruin the world?
[DJ]: Is it going to etc.?
[DJ]: But then there's this other tier of companies like Oracle and CoreWeave, where I've definitely heard of both of those companies, and I've definitely picked up the vibe that they've both been spending lots and lots of money on infrastructure for generative AI.
[DJ]: But it's never been clear to me, like, what are they doing?
[DJ]: Who are they selling it to?
[DJ]: And it's not like that stuff necessarily makes the news.
[DJ]: But I guess I'm less surprised to hear...
[DJ]: hey, these companies that have been getting way out over their skis in terms of spending money, the market is saying, it's really not clear to me how you're going to make back the revenue.
[DJ]: Because that ends up being what this is about, right?
[DJ]: Especially things like stock price and a company's market cap...
[DJ]: is essentially people's belief about the future and the future is always unknowable.
[DJ]: But like the reason that a company's stock is worth $248 on June 1st and $121 on July 20th doesn't necessarily mean that that company's fundamentals have changed, but it's that investors no longer think that it's going to be able to grow or make more money than it spends or et cetera.
[DJ]: That may or may not be true, of course.
[DJ]: If we could predict the future, stocks would, by very definition, not have any value because you'd already know what they would be worth.
[DJ]: And so it's very interesting to me, does this slide in those, let's call them like tier two companies, at least tier two in terms of like sentiment, like the degree to which people are thinking about them, does a slide in their value suggest that we're going to see a slide in the value of all of these companies
[DJ]: as a whole, like the entire tech sector?
[DJ]: Or are the big popular companies going to stay big and popular kind of no matter what they do?
[DJ]: Well, a bunch of these other companies that don't necessarily have the same faith of investors.
[DJ]: Investors might be going, well, Facebook's going to be fine.
[DJ]: Apple's going to be fine, even if they do spend a trillion dollars on data centers.
[DJ]: Whereas these other companies, it's like, yeah, but Oracle, though, what are they doing?
[DJ]: It's not like they themselves are competitive in generative large language models.
[DJ]: So what are they going to do with all this stuff?
[DJ]: That's what I'm curious about.
[DJ]: I'm really interested in...
[DJ]: Because some people will say, well, now we're seeing the beginning of the bubble bursting.
[DJ]: But is that really true?
[DJ]: Or are there like these multiple layers and one of them is starting to slide, but that doesn't mean the other ones will or will yet or will as much?
[JM]: I think this is indeed indication that the bubble is starting to give way.
[DJ]: I think "pop".
[DJ]: "Pop" is the verb that you're looking for.
[DJ]: Bubbles don't really give way.
[DJ]: They just pop.
[DJ]: Which is kind of why the word bubble is not a great metaphor for stuff like this.
[DJ]: I mean, sometimes that does happen where it's like, everything seems mostly fine.
[DJ]: Oh, the economy has cratered.
[JM]: Fair enough.
[JM]: We'll say deflate then.
[JM]: And I think that the...
[JM]: I agree with you that the deflation will be uneven, that companies like Apple will be less affected, even if they see a drop in their share price, they'll be less affected than other companies will that have made what I consider to be riskier investments.
[JM]: And it's funny that you said, like you had said, Apple and Facebook, and I wouldn't put those in the same bucket.
[JM]: I would say Apple and Google, sure.
[JM]: Maybe even Apple and Google and Microsoft, because those are companies that have thriving businesses.
[JM]: But I think if you look at Facebook, I would put them more in the category of Oracle in terms of, sure, they make tons of ad revenue, but they've also invested a lot of money into the "metaverse" and lost all of that, wrote it all down...
[JM]: and are now doing what I think is something very similar in terms of investing a bunch of money into data centers.
[JM]: And I don't really know that Facebook is necessarily good at data centers and large language models.
[JM]: They feel a lot more like Oracle than they do like Apple to me in terms of what their potential exposure is when this whole thing deflates.
[DJ]: Yeah, that is an interesting one.
[DJ]: I guess when I was talking about the big companies, I was thinking about companies where there seems to be some kind of moat around their value, at least from the perspective of the market, and at least for the immediate future.
[DJ]: Whereas I'm imagining, like, it's interesting to look at a company like SpaceX, which has been around for a while, but not that long.
[DJ]: And it's not super clear what their proposition, well, maybe it is clear, but like when they IPO'd, their proposition for, hey, we should be worth a billion trillion dollars or whatever, is we're going to take humankind into space.
[DJ]: And before we do that, we're going to build lots of AI data centers.
[DJ]: So we're worth a lot of money.
[DJ]: That's my understanding of essentially SpaceX's IPO.
[DJ]: It's not really clear if what I just said doesn't work out, how that company is worth very much.
[DJ]: Because basically they launched some rockets and they've got some data centers.
[DJ]: And I guess they own Twitter also, by the way.
[DJ]: So I can see why when there is a shift in investor confidence, their share price crashes.
[DJ]: And with Oracle...
[DJ]: I don't really know how well Oracle's traditional business of selling overpriced crappy database software to businesses has been doing, might be editorializing a little there, but has been doing for the last 10 years.
[DJ]: But again, I can see that like whatever it is they're doing now doesn't seem to have a clear connection to the established business, right?
[DJ]: Like they're buying all this infrastructure speculatively, like, well, we're gonna somehow be able to turn this infrastructure into a trillion dollars of revenue.
[DJ]: The market goes, I don't think so.
[DJ]: And so their share price crashes.
[DJ]: But I wonder if for companies like Apple and even Facebook, there's enough of an established business there that appears like it'll still be successful, at least into the near future, that even if they're making this kind of speculative bet on building data centers or whatever, and I realize that's more something Facebook is doing, not something Apple is doing, but I'm just kind of lumping all the giant tech companies together to some degree...
[DJ]: include Microsoft in this as well.
[DJ]: If there's enough confidence that like, well, even if the thing they're doing right now burns a lot of money, the other assets they have are worth enough that it's going to at least prop their share price up, that they're not going to see the giant crash.
[DJ]: Like their price might go down, but it's not necessarily going to get cut in half.
[DJ]: Again, I have no idea and I'm not even making a prediction.
[DJ]: I'm just trying to think through like in what ways are some of these companies different from each other or at least perceived to be different from each other, which is really what matters.
[JM]: Indeed.
[JM]: And speaking of things that potentially could be the canary in the coal mine as it relates to where our stock market might go.
[JM]: It doesn't feel like it was that long ago when an open weights large language model called DeepSeek R1 came out and the market responded from a perspective of fear instead of greed.
[JM]: And there was a rather significant sell-off at the time because people were worried that this meant that the American-driven generative software sector powered by OpenAI, Anthropic, etc.,
[JM]: could be at risk in terms of their ability to stay out ahead of upstart competitors and create a real defensible moat.
[JM]: But since that time, it seems like the frontier models have increased their lead.
[JM]: We've talked about the inflection point of November, 2025, where these models started to become really useful to a wider group of people for a wider range of applications.
[JM]: But in recent weeks, we've seen the release of at least four new open weights models, including DeepSeek 4, GLM 5.2, Kimi K3, and Qwen 3.8.
[JM]: And all of these have significantly narrowed this lead that frontier model producers like OpenAI and Anthropic have opened up until very recently.
[JM]: And if you look at some of the benchmarks, and who knows how valuable the benchmarks are, right?
[JM]: But this isn't just one benchmark.
[JM]: So it does lend some credence to the idea that these models really are producing at a high...
[JM]: technical level relative to ChatGPT and Claude, but you look at their scores on benchmarks and I was really surprised to see that on a lot of benchmarks, they are outperforming Opus 4.8 and are just behind Anthropic's latest and greatest Fable model.
[JM]: And this to me really does feel like a sea change of sorts to see just how competitive these models have become just in the span of the last month relative to what's available from the top tier frontier model producers.
[JM]: Now, who knows what could happen a week, a month, six months from now, possibly OpenAI and Anthropic produce some unexpected jump and are able to widen the gap once again, like they did last year.
[JM]: But if we're just looking at the trend line, that feels unlikely.
[JM]: And I think that this narrowing of the gap might be here to stay.
[JM]: And I've been trying to think about what this means.
[JM]: For me personally, it's gratifying to know that this might end up being a very competitive market.
[JM]: And we've been talking about that as a likelihood for a long time.
[JM]: This idea that maybe there is no moat.
[JM]: Maybe this is going to be a commodity business,
[JM]: with lots of competition, lots of end user choice, and the release of these four models, all of which from Chinese companies, by the way, does give me some hope that this will be a vibrant, competitive market with lots of innovation and not a...
[JM]: winner-takes-all type of market where, unlike say mobile phones, where you've got two platforms, you have Apple and Google, and there is no third...
[JM]: I don't think that's going to happen here.
[JM]: I don't think we're going to just have OpenAI and Anthropic and nobody else.
[JM]: All indications from what I'm seeing is that we're going to have this very competitive market.
[JM]: And I saw people trying to analyze what the prices of these new models mean, right?
[JM]: Because we're seeing prices that are way below on average
[JM]: what Anthropic and OpenAI charge.
[JM]: And one quote that I saw that I thought was interesting was by Ben Thompson.
[JM]: He said, "I highly doubt that Chinese models are cheaper to serve on a marginal cost basis.
[JM]: They just seem cheaper because Anthropic and OpenAI are so supply constrained that they're charging far more
[JM]: than they would if there were sufficient supply to meet the demand for intelligence."
[JM]: And my first thought when I saw this quote by Ben was, "more"?
[JM]: They're charging "far more"?
[JM]: Because my thought was, this is the opposite of what I keep seeing everyone else talk about, that they are charging prices that are *below* their costs,
[JM]: that they are *losing* money every time someone does a search on these models.
[JM]: It can't be both.
[JM]: And then I remembered that I think when I see people saying that they're losing money, that they're probably referring to the subscription prices.
[JM]: So if you're paying $20 or so a month for Claude Pro, it's quite possible that Anthropic is losing money on that customer.
[JM]: And I think what Ben is talking about is the API price, the cost per million tokens when you are paying a metered price, not a capped X dollars per month price for a certain amount of usage.
[JM]: It does seem like there's a real possibility that the profit margins are going to go from, say, somewhere around 60 to 80% on these price per token API calls to probably something close to zero over time.
[JM]: Because I do think that to some degree, at least in terms of API calls and how people pay for them, that this will start to resemble a commodity.
[JM]: And at that point, I think the winners, as we've talked about, are clearly going to be hardware companies...
[JM]: and users like us who get to have access to cheap large language model usage.
[JM]: I think the losers could end up being infrastructure providers, like Oracle, Facebook... companies that are investing a lot into a business that could have the profit margins just totally drop out underneath them.
[JM]: And then there's kind of the "who knows".
[JM]: Like Anthropic might do well in the B2B market that they seem to be focused on, but they might not as people realize they can just replace Claude with another model using the same API.
[JM]: And then you have OpenAI that might decide to just fully focus on the consumer market.
[JM]: They're already making money serving ads to consumers. Maybe that's the route they go, so they could find a foothold.
[DJ]: I am optimistic about this idea that the essence of what we call generative software, which is essentially like, "I have some money and I want a token. Who can give me that token for the least amount of money?"
[DJ]: ... will end up being a good outcome for consumers, for people like us.
[DJ]: This kind of splits in two directions.
[DJ]: The having high quality, open weight, open source, etc. models.
[DJ]: Non-proprietary, is that the right way to put it?
[DJ]: Like models that you could actually run yourself, for starters, is good for two reasons.
[DJ]: There's the large scale one where there are competitors to OpenAI and Anthropic.
[DJ]: for the same kind of use case.
[DJ]: We're hosting your thing in a data center.
[DJ]: We're hosting the model in a data center.
[DJ]: You come to us and ask us for tokens and give us money in return.
[DJ]: But there's also the use case that I know you and I have been enthusiastic about, which is running large language models on our own hardware.
[DJ]: And while it takes a while for the cutting edge open weight models to be processed into a form that you could run on your own hardware, like I think most of these models I've been hearing about, like Kimi K3 and Quen 3.8s,
[DJ]: are gigantic.
[DJ]: They're open weights, but unless you have your hypothetical one and a half terabyte M7 server in your closet, you can't run inference on this thing yourself.
[DJ]: You're going to have to use someone's data center for that.
[DJ]: These things end up being distilled.
[DJ]: I'm not sure if that's quite the right term, quantized, whatever it is.
[DJ]: They get turned into a smaller version that's a little less effective, but you could actually run on, say, a 128 gig system or something like that.
[DJ]: And I'm really enthusiastic about that because it lets us no longer be beholden to these other companies.
[DJ]: Maybe we'll talk in a future episode about some of the latest experiments you and I have been doing with local models.
[DJ]: Like I've been playing around with what is essentially like an open source alternative to Claude Code, where I've got the model on my own machine and I'm running this open source model
[DJ]: harness around it.
[DJ]: And I can get a quite similar experience when it comes to generating code, which is exciting.
[DJ]: So I'm hopeful that whatever else happens and whatever happens in the larger macroeconomic sense with these companies, who knows?
[DJ]: But at the very least, to the extent that ordinary people want the benefits of running large language model powered software, that it'll get easier and cheaper to do that.
[DJ]: And we'll have more control and be less beholden to these few giant monopolistic companies.
[JM]: Absolutely.
[JM]: Totally agree.
[JM]: All right, everyone, that's all for this episode.
[JM]: Thanks for listening.
[JM]: You can find me on the web at justinmayer.com and you can find Dan at danj.ca.
[JM]: Reach out with your thoughts about this episode via the Fediverse at justin.ramble.space.