ReportWire

Tag: john herrman

  • Inside the AI Bubble

    Photo: Intelligencer; Photo: Getty Images

    On Wednesday evening, Nvidia, the chip firm at the center of the world, reported its quarterly earnings. It was by any measure a blowout for the world’s largest company: the company made 65 percent more profits than in the same quarter last year, sales were even higher than analysts expected, and leadership is forecasting at least $500 billion in AI chip sales by the end of 2026. Permanently pumped CEO Jensen Huang bragged that the company was “sold out” before going oracular: “We’ve entered the virtuous cycle of AI. The AI ecosystem is scaling fast — with more new foundation model makers, more AI startups, across more industries, and in more countries. AI is going everywhere, doing everything, all at once.”

    Things couldn’t be going much better for Nvidia, which is one of the few large companies making serious profits that are primarily and unambiguously attributable to AI. The response from investors, though, was strange. The next morning, the stock popped a few percent but remained below recent highs, and ended the day slightly down. For many analysts and industry watchers, this wasn’t a story about the greatest quarter for the greatest company of all time. It was merely a relief. The “AI trade” was still alive, and the party could continue; more broadly, the anomalous sector propping up economic indicators would, for at least another quarter, and maybe even a bunch of quarters, continue to do so. It was, above all, an assurance and occasion to talk about it. You know. The bubble.

    In late 2025, AI bubble talk isn’t just for outsiders, skeptics, and short-sellers. Increasingly, it’s the frame through which the industry’s most important figures, and biggest boosters, talk about their technology, their companies, and the industry around them. “When bubbles happen, smart people get overexcited about a kernel of truth,” OpenAI’s Sam Altman told a group of reporters in August. “Are we in a phase where investors as a whole are overexcited about AI? My opinion is yes.” Mark Zuckerberg, while suggesting there were “compelling arguments” that AI could be an “outlier,” drew parallels to bubbles past. “I do think there’s definitely a possibility, at least empirically, based on past large infrastructure buildouts and how they led to bubbles, that something like that would happen here,” he said on the ACCESS podcast in September.

    There’s some transparent positioning here, of course — both Altman and Zuckerberg were implying that their companies were unique and would be fine either way — but inside-the-bubble bubble talk has since morphed into an odd strain of conventional wisdom, a premise from which high-level conversations about AI now proceed, or at least a possibility that has to be acknowledged. Google CEO Sundar Pichai invoked the dotcom crash. “I expect AI to be the same. So I think it’s both rational and there are elements of irrationality through a moment like this,” he said this month. In the event of a major correction, he said, “I think no company is going to be immune, including us.” The CEO of Google DeepMind, Demis Hassabis, emphasized Google’s particular strength but conceded on Hard Fork that there are “some parts of the AI industry that are probably in a bubble.” Jeff Bezos has said that while AI is “real,” and “is going to change every industry,” it’s also showing signs of an “industrial bubble.”

    Against the backdrop of all this hedging and narrower speculation about markets, the remaining practitioners of wide-open AI CEO futurism — that is, tech leaders still speaking the way most of them did as recently as last year – suddenly sound like outliers. At the Saudi Investment Forum, onstage with Huang, Elon Musk confidently stated that AI, with humanoid robots, will “eliminate poverty” and “make everyone wealthy.” In the future, he added on X, the “most likely outcome is that AI and robots make everyone wealthy. In fact, far wealthier than the richest person on Earth.” For the last few years, the public has been left to interpret competitively extreme visions of the future floated by strangely cavalier tech executives, who agreed on little but the inevitability of total change: mass unemployment; luxurious post-scarcity; human obsolescence; hyper-accelerated scientific progress; and, perhaps, total annihilation. Now, markets are concerned with narrower questions, with more specific answers, and more immediate consequences: How many GPUs has Nvidia sold? How many can it make? (Or, rather, how many can Taiwan Semiconductor Manufacturing Company manufacture for it?) There are plenty of theories about how generative AI might diffuse into the economy and change the world, and as more people use it, and companies start to deploy it, a few of them are snapping into focus (buy a drink for any young programmers in your life). But after years of boosterish warnings about the extraordinary and esoteric risks posed by mysterious and profound technology — we’re creating software so powerful even we can’t control it — tech executives are instead trying to get out in front of a profound non-technological risk that may be manifesting much sooner: that if they lose even a little bit of momentum, they might end up tanking the American economy.

    If Huang’s everything, everywhere, “all at once” line was a reference to the 2022 absurdist multiverse movie, it’s a funny one: the film opens with its protagonist shuffling through a pile of papers, anxiously preparing for a financial audit (and features a villain who “got bored one day” and decided to collapse the entirety of creation into a bagel-shaped singularity). As the AI boom has sprawled into a larger and more complicated financial story, scrutiny of the businesses behind the models has become as intense as scrutiny of the models themselves. To raise money and finance data center deals, OpenAI, which is both the leading consumer AI company and one of the industry’s most aggressive and, let’s say, inventive dealmakers, has manifested some truly dizzying arrangements, many of which involve Nvidia, a circular deal innovator in its own right. Take CoreWeave, a crypto-mining company that pivoted to AI data centers in 2022. CoreWeave rents access to Nvidia chips to firms that need them for AI inference and training. OpenAI is a CoreWeave customer, but also a Coreweave investor. Nvidia is a CoreWeave vendor — it supplies the GPUs – but also an investor and, somehow, a customer. Coreweave also loses a lot of money, and its stock price has, after peaking in July, collapsed.

    Lately, the deals are getting more brazen and less convoluted. In September, Nvidia announced it would invest $100 billion in OpenAI, which OpenAI said it would use to build data centers full of Nvidia hardware. This month, alongside Microsoft — OpenAI’s biggest early investor and primary partner — Nvidia announced the companies would invest up to $15 billion in OpenAI competitor Anthropic in exchange for a $30 billion commitment from the company to buy computing capacity from Microsoft, powered, naturally, by Nvidia hardware. Altman’s moments of candor about a possible bubble have been scattered between more defensive messaging from the company, which may be losing as much as $12 billion per quarter. In a recent podcast interview, investor Brad Gerstner asked Altman, “How can a company with $13 billion in revenues make $1.4 trillion of spend commitments?” Altman shot back: “If you want to sell your shares, I’ll find you a buyer. Enough.”

    That insiders seem to agree that we could be in a massive bubble is, counterintuitively, not very useful: whether or not they mean it, and whether or not they’re right, their incentives as leaders of mega-scale startups and public tech companies are such that raising, spending, and committing as much money as possible for as long as possible is probably the rational, self-interested choice either way. Anxious, skeptical, or merely satisfied investors looking for excuses to pull back or harvest gains don’t have to look hard, and there’s evidence some are; before its earnings report, Peter Thiel’s investment firm unloaded its position in Nvidia, and Softbank cashed out of the chipmaker at around the same time. Similarly, OpenAI’s ability to send public companies’ stocks soaring by announcing massive “commitments” seems to be fading — Oracle’s recent $300 billion valuation bump, based on some shockingly optimistic guidance it offered investors in September, has since gone negative.

    But focusing on the flagrant circularity of AI financing can feed the impression that the risks are contained within Silicon Valley. The bigger problem is the ways in which they’re already not. If it exists, you might call it a load-bearing bubble. In the first half of 2025, “investment in information processing equipment and software” — a sort of informal, private stimulus package — accounted for 92 percent of GDP growth for the United States, while AI-related tech stocks account for nearly all recent growth in the S&P 500. Early funding for companies like OpenAI came from venture capitalists and incumbent tech giants, while Google and Meta pushed into AI with their own massive revenue and cash, but multi-hundred-billion-dollar commitments mean they’re getting more creative, both in how they raise money and how they distribute risk. Companies like Meta are funding data centers with “special purpose vehicles,” which may sound familiar if you were reading the financial news in 2008, and with massive corporate bond sales. As the investor Paul Kedrosky has argued, the AI boom has traits, at least, of every major financial bubble in modern history: a narrative-driven tech bubble, a credit bubble, a real estate bubble, and an infrastructure bubble. To tie it all together, you’ve got OpenAI’s CFO floating, then frantically backtracking on, the idea of a government backstop for financing AI expansion, almost instantly elevating the prospect of an AI bailout into fodder for conservative and progressive lawmakers.

    Huang has two typical responses to all this. One speaks for itself: look at all those GPUs we’re selling. The other is more direct. “There’s been a lot of talk about an AI bubble. From our vantage point,” he said after earnings, “we see something very different.” In other words: No it’s not. The “virtuous cycle” is just beginning, and the accelerating potential of the most versatile technology the world has ever seen will one day expose complaints about incremental model updates and hand-wringing about data center deals as short-sighted and insignificant. Huang is still able to speak with authority and tell a story that, for investors, still has juice.

    For everyone else, though, neither side of this wildly polarized, high-stakes bet sounds ideal. If this really is a bubble, and it deflates even a little, it could send the American economy into a serious slump, with consequences for almost everyone, getting rid of plenty of jobs the old-fashioned way. If it doesn’t — and Huang’s sanitized visions of mass automation rapidly start to spread across the economy, justifying all that CapEx, and all those strange deals, and then some — well, aren’t we getting laid off anyway?

    John Herrman

    Source link

  • The Global Internet Is Coming Apart

    Photo-Illustration: Intelligencer; Graphic: MAX/Apple

    The early rise of the internet is usually told as an extension of globalization. New networking technology made instantaneous communication possible, complementing and accelerating international commerce and cultural exchange. As in the rest of the world economy, the U.S. was unusually influential online, exporting not just technology but culture and political norms with it.

    The alternative story of the rise of the internet was exemplified by China, which limited the reach of western tech companies, maintained strict control over its domestic networks, and started building a parallel internet-centric economy of its own. And contra western reporting suggesting that this was purely an exercise in isolationism and control, in 2025, the international influence of the Chinese internet and tech companies — even here, as evidenced by the growth and semi-seizure of TikTok — is enormous.

    In that context — and the context of America’s renewed trade war — it shouldn’t be surprising that more countries are taking a second look at digital sovereignty and that the global internet as we knew it is pulling apart. Russia, which has a long history of internet censorship and state-aligned tech companies, has taken the extraordinary recent step of interfering with access not just to WhatsApp but also Telegram, the messaging app founded by Pavel Durov, a creator of VK, Russia’s Facebook alternative, who left the country more than a decade ago. The throttling coincided with the launch of MAX, a new government-controlled everything app — basically a messaging app with other features layered on top, modeled on China’s Weixin — and an all-out marketing campaign to get people to switch. “Billboards are trumpeting it. Schools are recommending it. Celebrities are being paid to push it. Cellphones are sold with it preloaded,” the Times reports.

    Russia’s obviously in an … unusual diplomatic position these days, but you can hear a version of its stated position — We should have our own big internet platforms as well as greater control over and access to what people do on them — coming from all over the world. (Indeed, the American government’s rationale for the TikTok deal can be understood as a defensive version of the same argument.) In India, the government is talking more openly about favoring homegrown apps for economic and security reasons and highlighting its own domestic “super-app.” From the Financial Times:

    A chorus of top Indian officials in recent weeks have publicly backed a domestically developed messaging platform, as the country tries to project its ability to create a homegrown rival to US-developed apps. “Nothing beats the feeling of using a Swadeshi [locally made] product,” [Minister of Commerce] Piyush Goyal wrote on X, adding: “So proud to be on Arattai, a Made in India messaging platform.”

    In the aughts, fights over digital globalization were about search engines and popular websites; in the 2010s, they were largely about social networks. Now, they’re about messaging apps, which are different in a number of ways. A lot of messaging traffic is private communication on services like Meta-owned WhatsApp — one of the most popular apps in India, which is WhatsApp’s largest market. Messages on the platform are encrypted by default, meaning that even governments with extensive surveillance capabilities can’t easily see what people are using them for.

    China’s Weixin, which operates internationally as WeChat, demonstrates two tantalizing possibilities for other governments: It’s aligned with the state and surveillable; also, as it grew popular and expanded its ambitions, it became the default interface for shopping, banking, media consumption, and interacting with other businesses. This sort of everything app — which American tech executives have openly lusted after, most recently and explicitly Elon Musk — is appealing to tech companies and governments alike for its total centralization. MAX’s goals are clear, with messaging, calls, ID functionality, and plans to allow users to “connect with government services, make doctors’ appointments, find homework assignments, and talk to local authorities.”

    The looming segmentation of what we colloquially call the “internet” into various national, nationalist, and perhaps compromised messaging apps leaves governments without such ambitions in an awkward position. The European Union, citing some of the same concerns as the Russian and Indian governments — although mostly focusing on child protection — is considering, against widespread opposition from its citizenry and foreign tech companies alike, “chat control” legislation, which would require tech firms to allow messages to be scanned by authorities for offending content. The EU has some leverage here, of course — nobody wants to lose access to such a large and wealthy market — but tech companies based elsewhere insist that such a requirement is impossible to implement without fundamentally breaking their services or violating user privacy. Under the narrower auspices of stopping online sexual abuse, in other words, the EU is asking — or wishing — for a limited version of the same power China wanted when it made onerous demands of American tech companies in the 2010s, preventing them from entering its market: to regulate and control influential applications that have, up until this point, mostly come from somewhere else.

    Taken together, this looks an awful lot like a global shift in how most governments — and their citizens — approach the internet: not as an intrinsically and necessarily global project but as a source of domestic power to be cultivated, protected, and protected against.

    John Herrman

    Source link

  • Why Is Amazon Watching Us?

    Photo-Illustration: Intelligencer; Photo: Amazon

    What kind of company is Amazon? It’s a question with a lot of reasonable answers: It’s a peerless e-commerce giant; it’s a massive shipping and logistics operation; it’s a devices and digital-services company that sells millions of gadgets; it’s a cloud provider so widely used that a regional outage can take out a good chunk of the entire internet. With the help of Whole Foods, it’s become a credible competitor in groceries, and with Prime Video, a major streamer and producer of TV and movies. In the course of becoming America’s quintessential neo-conglomerate, though, Amazon has also become something else: a serious — and diverse — surveillance firm.

    Two pieces of news from this month help map the depth and scope of Amazon’s investment in surveillance technology. This week, following smartglasses and headset updates from Meta and Apple, Amazon previewed a new pair of glasses that it says will add to its “system of technology to support [delivery] drivers.” The pitch:

    Designed specifically for [Delivery Associates], these glasses help them scan packages, follow turn-by-turn walking directions, and capture proof of delivery—all without the use of their phone. The glasses create a hands-free experience, reducing the need to look between the phone, the package, and the surrounding area.

    The company emphasizes potential safety benefits for delivery workers, whose jobs are already substantially dictated (and tracked by) mobile apps with a lot of the same functionality. Amazon also says it’s “leveraging the latest advancements in AI to create an end-to-end system” that runs from “inside our delivery stations, to over the road, to the last hundred yards to a customer’s doorstep.” Then it shifts to the future tense, imagining future versions of the glasses that might detect various hazards or, perhaps, “help notify drivers if they’ve mistakenly dropped a package at a customer doorstep that does not correspond with the house or apartment number on the package.”

    Photo: Amazon

    For workers as for the packages they help deliver, Amazon’s “end-to-end” system is already nearly complete, as documented in more than a decade of reporting: In warehouses, products and people are tracked extensively and precisely in ways that improve throughput by, in part, applying metrics-driven pressure to employees, measuring not evidence of productivity but “time off task.” In delivery vehicles, drivers are surveilled and measured in numerous different dimensions by default, with technology that was also marketed, at launch, with a lot of language about driver safety:

    In various nearby industries, Amazon is regarded as a leader in employee surveillance and “algorithmic management” and is likewise held up as an example of its possible effects on workers (more productive, less content, and less likely to organize). In a brief rendering demonstrating how its delivery glasses will work, Amazon shows how worker surveillance will soon extend outside of the vehicle, right up to the doorstep. Which brings us to the second bit of news, from TechCrunch:

    Amazon’s surveillance camera maker Ring announced a partnership on Thursday with Flock, a maker of AI-powered surveillance cameras that share footage with law enforcement.

    Now agencies that use Flock can request that Ring doorbell users share footage to help with “evidence collection and investigative work.”

    Ring cameras are an underrated Amazon success story: With an appealing pitch — see who or what is on your doorstep, even if you’re not home — the company sold millions of units and constructed a massive surveillance network with numerous benefits for the company itself. They’re a way to counteract package theft and to make it easier for customers to receive deliveries.

    They also provide an additional way for the company and its customers to surveil its workers, meaning that Amazon’s glasses aren’t just extending its “end-to-end” delivery apparatus — they’re closing the loop in its employee-monitoring ecosystem. Ring cameras were, for obvious reasons, always interesting to law-enforcement agencies, with which Amazon has had a generally cooperative but heavily scrutinized and limited relationship, at least until recently. Now, by partnering with companies like Axon and Flock, which operate nationwide fleets of license-plate scanners and work with local, state, and federal law-enforcement agencies, including ICE, the company is making its surveillance network widely and comprehensively available to the state.

    Like most of Amazon’s internal surveillance systems, there’s a clear managerial logic at work in Ring’s expansion, and it shares a lot of DNA with the company’s internal monitoring tools. From the perspective of a large organization, more surveillance is always tempting: It means more control, more data, more chances for optimization, and more security. (Of course, all of that comes at the cost of the privacy, which the people within those systems might not appreciate.)

    In Ring’s case, customers largely saw themselves as on the same side as Amazon in a fight against porch piracy and for convenience, if they thought of their purchase in such terms at all. The privacy-based case against buying a Ring camera wasn’t terribly persuasive from the perspective of potential customers, because they were the ones doing the surveillance, from their property, protecting their stuff and homes. Ring cameras invited users to adopt an Amazonian logic of their own, in which total stoop awareness is, clearly, a good and desirable thing.

    Barely a decade after Ring’s doorbell-camera pitch first aired on Shark Tank, though, Amazon and millions of its customers have haphazardly and perhaps not entirely consciously teamed up to build something strange, unprecedented, and very Amazon: a crowd-sourced, nationwide, neighborhood surveillance network to which the government now has a set of keys.

    John Herrman

    Source link

  • Wikipedia Is Getting Pretty Worried About AI

    Over at the official blog of the Wikipedia community, Marshall Miller untangled a recent mystery. “Around May 2025, we began observing unusually high amounts of apparently human traffic,” he wrote. Higher traffic would generally be good news for a volunteer-sourced platform that aspires to reach as many people as possible, but it would also be surprising: The rise of chatbots and the AI-ification of Google Search have left many big websites with fewer visitors. Maybe Wikipedia, like Reddit, is an exception?

    Nope! It was just bots:

    This [rise] led us to investigate and update our bot detection systems. We then used the new logic to reclassify our traffic data for March–August 2025, and found that much of the unusually high traffic for the period of May and June was coming from bots that were built to evade detection … after making this revision, we are seeing declines in human pageviews on Wikipedia over the past few months, amounting to a decrease of roughly 8% as compared to the same months in 2024.

    To be clearer about what this means, these bots aren’t just vaguely inauthentic users or some incidental side effect of the general spamminess of the internet. In many cases, they’re bots working on behalf of AI firms, going undercover as humans to scrape Wikipedia for training or summarization. Miller got right to the point. “We welcome new ways for people to gain knowledge,” he wrote. “However, LLMs, AI chatbots, search engines, and social platforms that use Wikipedia content must encourage more visitors to Wikipedia.” Fewer real visits means fewer contributors and donors, and it’s easy to see how such a situation could send one of the great experiments of the web into a death spiral.

    Arguments like this are intuitive and easy to make, and you’ll hear them beyond the ecosystem of the web: AI models ingest a lot of material, often without clear permission, and then offer it back to consumers in a form that’s often directly competitive with the people or companies that provided it in the first place. Wikipedia’s authority here is bolstered by how it isn’t trying to make money — it’s run by a foundation, not an established commercial entity that feels threatened by a new one — but also by its unique position. It was founded as a stand-alone reference resource before settling ambivalently into a new role: A site that people mostly just found through Google but in greater numbers than ever. With the rise of LLMs, Wikipedia became important in a new way as a uniquely large, diverse, well-curated data set about the world; in return, AI platforms are now effectively keeping users away from Wikipedia even as they explicitly use and reference its materials.

    Here’s an example: Let’s say you’re reading this article and become curious about Wikipedia itself — its early history, the wildly divergent opinions of its original founders, its funding, etc. Unless you’ve been paying attention to this stuff for decades, it may feel as if it’s always been there. Surely, there’s more to it than that, right? So you ask Google, perhaps as a shortcut for getting to a Wikipedia page, and Google uses AI to generate a blurb that looks like this:

    This is an AI Overview that summarizes, among other things, Wikipedia. Formally, it’s pretty close to an encyclopedia article. With a few formatting differences — notice the bullet-point AI-ese — it hits a lot of the same points as Wikipedia’s article about itself. It’s a bit shorter than the top section of the official article and contains far fewer details. It’s fine! But it’s a summary of a summary.

    The next option you encounter still isn’t Wikipedia’s article — that shows up further down. It’s a prompt to “Dive deeper in AI Mode.” If you do that, you see this:

    It’s another summary, this time with a bit of commentary. (Also: If Wikipedia is “generally not considered a reliable source itself because it is a tertiary source that synthesizes information from other places,” then what does that make a chatbot?) There are links in the form of footnotes, but as Miller’s post suggests, people aren’t really clicking them.

    Google’s treatment of Wikipedia’s autobiography is about as pure an example as you’ll see of AI companies’ effective relationship to the web (and maybe much of the world) around them as they build strange, complicated, but often compelling products and deploy them to hundreds of millions of people. To these companies, it’s a resource to be consumed, processed, and then turned into a product that attempts to render everything before it is obsolete — or at least to bury it under a heaping pile of its own output.

    John Herrman

    Source link

  • Get Ready to See Yourself in Ads

    Photo-Illustration: Intelligencer; Photo: Glance

    One of the main promises of online advertising is targeting. Tracking browsing habits helped early digital advertisers slice up audiences and sell them to marketers. Search engines like Google let brands buy ads around specific requests and revealed clear user intent. Social networks like Facebook used detailed profiles and voluminous social interactions to promise ads that were both precisely targeted and displayed in the context of highly personalized feeds, making billions of dollars in the process. Generative AI throws a few more possibilities into the mix. Companies can use automated copywriting and image generation to customize ads per user or draw on chatbot interactions, which Meta is already ingesting into its advertising platform.

    New AI tech also makes possible stuff like this, as reported by the Verge:

    DirecTV wants to use AI to put you, your family, and your pets inside a custom TV screensaver. If that’s not uncanny enough, you’ll find items you can shop for within that AI environment, whether it’s a piece of clothing similar to the one your AI likeness is wearing or a piece of furniture that pops up alongside it.

    That feature comes courtesy of a company called Glance, which built a similar app for smartphones that claimed to use “a single selfie or an image from the image gallery to generate hyper-real images of consumers in outfits best suited for them,” after which users could “make real-time purchase decisions with just a tap.”

    These are opt-in features tucked into a marginal product. But in digital advertising, uncomfortable ideas have a way of working their way in from the fringes of the industry, and much bigger firms are already playing with technology that’s at least conceptually similar to what Glance is offering here. Meta will generate AI avatars from a photo and then show you variations in your feed, which some users have mistaken for using their likenesses in targeted ads. Snap, which has been experimenting with automatically generated avatar content as well, started telling users last year that its “My Selfie” feature could be “used to power Generative AI, Cameos and other experiences on Snapchat that feature you, including ads.” Companies like HeyGen are rolling out AI spokespeople, making influencers’ likenesses available for ad campaigns. OpenAI just released Sora, which allows people to realistically deep-fake themselves and their friends, creating, among other things, ad-placement possibilities marketing executives couldn’t have fathomed just a few years ago — enabled by a company that needs to start making money and has expressed an interest in advertising. Google has been rolling out virtual try-ons in Google Shopping and has released a standalone app called Doppl for dressing yourself up in virtual outfits.

    Photo: Google Shopping

    Virtual try-ons are actually showing up all over the place in retail — you’ll find them on both luxury-brand apps and Shein — and they tend to suffer from the same issues, given that they’re essentially mashing images together: The clothes don’t fit quite right; the lighting is always strange; generations can take a little while; body types are approximated but rarely nailed. But they might be good enough to help rule something out or consider something new, and they’re certainly good enough to be compelling and occasionally fun to use. The temptation to merge virtual try-ons with ads — to follow users around the internet not just with evidence that you know a lot about them but with actual photos of themselves using new products — is surely massive but is currently constrained by two things. First, generating such images or videos is computationally and literally expensive. And second, putting users’ faces in, say, a branded Instagram Reel would probably feel to many like a violation. (For now, Google’s explainer about its virtual try-on feature marks a boundary far more conservative than that, emphasizing that the feature won’t currently show up on sponsored products.)

    Photo: Google Shopping

    One of those constraints will probably go away as AI models become more efficient. The other is a matter of expectations and norms around privacy, which, if the last two decades are any guide, have a tendency to either erode or get bulldozed. Given the hunger for return on AI investment and the close relationship between the advertising industry and AI — Google and Meta, for example, are advertising and AI giants, and it’s hard to imagine the latter holding out too long on a new technology that might improve ad performance — it’s probably a matter of time before someone breaks the seal and starts using customers to market to themselves at scale. Small companies are already trying something weird and uncomfortable. The bigger ones probably aren’t far behind.

    John Herrman

    Source link

  • Users Have One Big Problem with Sora

    Photo-Illustration: Intelligencer

    OpenAI’s Sora, the TikTok-like video generator app, was an instant hit, shooting to the top of the download charts and hitting a million installs within a few days of launch. Its early popularity makes sense: It’s a free chance to try recent video generation technology, a fun way to joke around with friends who consent to being included in one another’s clips, and an early look at the weird dynamics of celebrity-endorsed AI.

    Judging by App Store reviews, though, it seems like people aren’t entirely happy with Sora. Some of the complaints dragging on its rating — it’s at 2.8 stars out of 5, compared to 4.9 for ChatGPT or 4.6 for Google — are straightforward and easily addressed: it’s sort of glitchy; the voices aren’t as good as they could be; some generations are uncanny or unconvincing; the app, despite being free, is invite-only, which can make for a disappointing download. But the most popular complaint by far is one that OpenAI can’t easily deal with. It’s that the app has rules:

    Photo: Apple App Store

    Running into Sora’s “guardrails,” as the company explicitly calls them, is a frequent experience if you’re using the app to, say, goof off with your friends. (The app’s public feed provides plenty of evidence that people are using this app to tease and embarrass their friends and the few public figures who have given permission for their likenesses to be used, like Sam Altman.) Given that the main early use case seems to be jokes — many of which are just variations on puppeteering others into saying or doing unexpected or odd things — I can imagine that encountering these guardrails is a pretty common experience.

    I can also imagine why a company like OpenAI would set a huge number of limits on a social media app built around deepfakes, which are elsewhere associated with porn, harassment, and misinformation. Obviously, OpenAI is going to try to prevent people from using Sora as nudifying app, or from generating violent imagery of children. But such a company also has pretty clear reasons to be careful with celebrity likenesses, copyrighted material, especially as it lobbies for a more lenient legal environment and deals with public and legal backlash. Sora’s “guardrails” are indeed high and conservatively placed, but plenty of users, upon seeing their own faces and bodies reproduced as realistic, playable digital avatars, can probably figure out why.

    Still, plenty of others experience such limits as a simple deficiency: Clearly, this model is capable of generating material that it won’t, which amounts to OpenAI telling users what they can and can’t do or see. ChatGPT, which has plenty of guardrails of its own, frequently draws similar complaints from users, as does the nominally more permissive Grok, but the rate of outright refusals from Sora is a sore point among reviewers. One factor that might be contributing to this, as reported by Katie Notopoulos at Business Insider, is the emerging culture and demographics of the young app:

    As more and more people join the app, I’m starting to see them making cameos of what appear to be their real-life friends. (There are lots of teenage boys, it appears.) On one hand, teenage boys and young men are a great demographic for your nascent AI social app because they like to spend a lot of time online. On the other hand, let’s think about platforms that were popular with teenage boys: 4chan, early Reddit, Twitch streaming …. these are not necessarily role models for Sam Altman’s OpenAI.

    And then there’s the Jake Paul of it all. Jake Paul is pretty much the only recognizable celebrity who lets anyone create cameos on Sora with his face — and people have gone wild… Right now, a big meme on Sora is making Jake Paul say he’s gay — with him wearing a pride flag or dressed in drag. Har har har. (I’ve started to see this bleed out into non-celebrity users, too: teen boys making cameos of their friends saying they’re gay.) Paul has responded with a TikTok video about the meme, acknowledging it and laughing about it.

    As Notopoulos points out, the app’s feeds feature few women while its emerging sensibilities skew young and male. (This could obviously change or broaden out if Sora keeps growing, but for now it seems to be intensifying; beyond the already skewed demographics of AI early adopters, women have plenty of other reasons to be wary of an industrial-scale deepfaking app.) If Sora’s ratings are anything to go by, the people driving its early growth don’t have much sympathy for the complicated situation OpenAI has created for itself — fair enough! — or concern for how an unmoderated TikTok full of deepfakes might go wrong for others. They open a video generator, enter the first prompts that the app brings to mind, and get told no, imprecisely, for a variety of reasons: Sorry, that one’s copyrighted; of course we can’t render a video of your friend firing a gun into a crowd; our apologies, but that request is pretty clearly sexual. Try again!

    OpenAI has released another popular app, in other words, but finds itself immediately at odds with its most eager early users: young men who want to make gay jokes about Jake Paul, tease each other relentlessly, or simply make the machine do something it’s not supposed to, and feel slightly persecuted when a corporation gets in their way. To broaden this beyond edgelord teens, OpenAI has created an app in which, for many users, locating and trying to circumvent guardrails is the core experience, in a world where more permissive open source models with similar capabilities are probably just a few months behind. Best of luck with this dynamic!

    John Herrman

    Source link

  • Sam Altman Is Trying to Manifest a New Nvidia

    Photo: Intelligencer; Photo: Getty Images

    Nvidia has only attained household-name status recently, but it’s the opposite of an overnight success. For decades, the company was mostly known for designing graphics processor units, or GPUs, for gaming, a market it eventually dominated. More recently, Nvidia’s longtime investment in applications beyond gaming started to bear fruit. First, crypto miners started hoarding Nvidia hardware, giving the company a surprise boost. Then, the company’s early alignment with GPU-hungry AI firms really paid off, turning it into arguably the biggest beneficiary of the recent boom. The global AI-driven data center buildout has pushed Nvidia’s quarterly revenue from around $13 billion when ChatGPT was released to more than $165 billion; currently, by market cap, it’s the most valuable company in the world.

    This is all wonderful news for Nvidia founder and CEO Jensen Huang, as well as the company’s shareholders. Things are a bit more complicated for Nvidia’s major customers, whose all-in bets on AI depend on, among many other things, getting sufficient allocations of hardware from a single company with massive pricing power. Google, Amazon, and even Meta have tried to deal with Nvidia’s dominance by investing heavily in their own chips to deploy alongside Nvidia’s while preparing their infrastructure to handle more alternative hardware in the future. The Chinese government sees things similarly, responding to American export limits intended to limit AI progress with a broader ban on Nvidia hardware intended to spur domestic competition. Across the board, these efforts are starting to pay off.

    OpenAI is in a particularly weird position. The AI giant is a huge Nvidia customer, spending much of the money it has raised on Nvidia products, or with other firms that have access to Nvidia hardware. As of last month, Nvidia is also a major investor in OpenAI, which will spent at least some of this capital with … Nvidia. The two companies are close and co-dependent but in ways that are far riskier for OpenAI. OpenAI has made some noise about building its own chips and is currently partnering with Broadcom on some custom hardware. But then, this week, it did this:

    OpenAI and Advanced Micro Devices announced on Monday a new partnership that would see OpenAI deploy six gigawatts of AMD’s chips over multiple years, starting in the second half of next year. That deal is larger than the $300 billion, 4.5 gigawatt cloud deal OpenAI struck with Oracle earlier this year.

    As part of the deal, OpenAI will receive the option to purchase up to 10% of AMD stock if it hits certain milestones.

    AMD, Nvidia’s old rival from the graphics-card wars, has been trying for years to get in on the AI boom, and its recent chips have started gaining traction in limited applications. Still, according to insiders, it’s a comparatively small and specialized firm with a lot of catching up to do.

    Plenty of people have started noting, with some nervousness, the circularity of recent AI deals, but they’re also becoming more speculative. In this case, OpenAI gets access to its stake in AMD if it follows through on spending, and if AMD’s stock price reaches certain milestones (it was up nearly 25 percent on Monday, and Nvidia’s was slightly down). OpenAI’s commitment also gets way out ahead of what AMD can competitively provide now, which means the whole thing is contingent on the company hitting its goals for developing much better hardware as well — something that Nvidia can attest often takes longer than expected. OpenAI isn’t just looking around for an Nvidia competitor, in other words. In AMD, it’s hoping it can help manifest one.

    John Herrman

    Source link

  • Sora and the Sloppy Future of AI

    Photo-Illustration: Intelligencer; Photo: Sora

    Last week, Meta’s newly appointed chief AI officer, Alexandr Wang, who co-founded the AI data-collection and annotation firm Scale.ai, announced Vibes, “a new feed in the Meta AI app for short-form, AI-generated videos.” Early responses were basically unanimous: What? Why? In testing, it felt, at most, like a chance to try out recent clip-generation tech for free. Otherwise, the feed was unfocused and confusing, full of overcooked generations by strangers unified only by the drab meta-aesthetics that have come to define AI imagery over the last couple of years: CGI-ish scenery and characters; real places and people rendered as uncanny stock photography; the work of Thomas Kinkade if he got into microdosing mushrooms. Even the more positive reviews couldn’t help but call it a feed of slop.

    Photo: Vibes

    In hindsight, though, the release of Vibes now makes a little more sense. Meta, which has been spending massive amounts of money to poach talent from other AI firms, probably knew a great deal about what OpenAI was about to release: a new version of its video-generation product, Sora, this time packaged as a TikTok-style app, dropped on Tuesday. In contrast with Vibes, Sora — an invite-only app with limited access — was an instant viral hit.

    What’s the difference? Underlying models matter a little bit here. In an apparent rush to get the app out, and lacking better tech of its own, Meta ended up leaning on an outside image-and-video-generation company. Meanwhile, the latest OpenAI model is obviously more capable of producing what you ask it for, whether that’s a fairly realistic clip of a real person doing something normal, a jokey visual mash-up — or something stranger or more illicit, despite OpenAI’s attempts to include a wide range of guardrails. Early examples going around on social media included macabre or gross videos, the nonconsensual likenesses of public figures, and copyrighted characters. The Sora videos tended to feel either shockingly good, sort of grimy, or both (its own easily circumventable rules are notable, but as much as anything make it impossible to ignore the vastly more dire possibilities that already exist beyond the platform, courtesy of open and increasingly capable video-generation models).

    What really made the app work, however, were two features. One encouraged users to create avatars of themselves, called Cameos, which could then be included in videos. The other was the ability to, with users’ permission, include their Cameos in your videos. Some of the first videos to get momentum on Sora were people making jokes about Sam Altman, commanding his avatar into various absurd, embarrassing, or simply unexpected situations.

    Far more compelling, though, was to try Sora with people you know. If the sanctioned (and ultimately flattering) ritual of teasing OpenAI’s CEO with his own product as it harvested users’ likenesses defined early popular content, the experience of actually using the app was defined by stranger and more personal experiences. Oh, there’s a version of me, looking and sounding slightly wrong but uncomfortably familiar, operating at my command. Oh, I can include a friend in this one, and I can make him say or do whatever we want.

    Sora, the model, is a tech demo. Sora, the feed, is an experiment tucked into a TikTok clone, only somewhat more interesting than Vibes. Sora, the consensual deep-fake automator, makes the most sense in the context of a group chat, or within the app in little-noticed interactions between friends. Messing around with your own avatar is unsettling, interesting, and entertaining, an in the queasy tradition of social-media face filters or those old apps that used “AI” to make you look old in exchange for your privacy. Messing around with the avatars of people you know, together, is startling and fun. Using Sora is a bit like dressing up as one another for Halloween: It’s easy to get a laugh and easy to go a little bit too far.

    Photo: Sora

    Sora is presumably extremely expensive to run, hence OpenAI’s use of a chained invite program to roll it out. In its early form, it brings to mind the early days of image generators like Midjourney, the video model for which Meta is now borrowing for Vibes. Like Sora, Midjourney in 2022, was a fascinating demo that was, for a few days, really fun to mess with for a lot of the same reasons:

    A vast majority of the images I’ve generated have been jokes — most for friends, others between me and the bot. It’s fun, for a while, to interrupt a chat about which mousetrap to buy by asking a supercomputer for a horrific rendering of a man stuck in a bed of glue or to respond to a shared Zillow link with a rendering of a “McMansion Pyramid of Giza…”

    …I still use Midjourney this way, but the novelty has worn off, in no small part because the renderings have just gotten better — less “strange and beautiful” than “competent and plausible.” The bit has also gotten stale, and I’ve mapped the narrow boundaries of my artistic imagination.

    Playing with Sora is a similar experience: a destabilizing encounter with a strange and uncomfortable technology that will soon become ubiquitous but also rapidly and surprisingly banal. It also produces similar results: a bunch of generations that are interesting to you and your friends but look like slop to anyone else. The abundant glitches, like my avatar’s tendency to include counting in all dialogue, help make the generations interesting. Many of the videos that are compelling beyond the context of their creation are interesting largely as specimens or artifacts — that is, as examples of how a prompt (“sam altman mounted to the wall like a big mouth Billy bass, full body”) gets translated into … something. (As one friend noted, many of these videos become unwatchable if you can’t see what the prompt was.)

    This makes Sora interesting to compare to more straightforwardly “social” networks, where most content is likewise produced for small audiences and lacks appeal beyond that context. Here, too, celebrities and brands are what people want to see, but with the expectation that they can be commanded and manipulated, not just consumed. Sora: It’s pretty fucking weird!

    It’s also clearly compelling, and it (or imitations of it) are going to confront way more people — soon. OpenAI’s willingness to release strange, glitchy preview products (and to gleefully violate norms in the process) is at this point an established strategy and one that has reliably netted it users (in the case of ChatGPT, which now has hundreds of millions of users) or at least renewed attention (as when OpenAI demonstrated new image-generation tools through a viral aesthetic rip-off campaign). In terms of the AI “alignment” debate, for whatever it’s worth, the leading AI firm optimizing its product output for engagement is somewhere between “bad” and “apocalyptic.” It’s also something that Meta, despite its reputation for engagement-at-all-cost product design, can’t seem to pull off no matter how hard it tries.

    In the broader discourse around what modern AI is for and where it might be going, it’s also useful to try to reconcile with some of the rhetoric from companies like OpenAI about why they need to raise so much money to build so many data centers. (For a glimpse of the view from within Google, see the above posts from people who work on AI there.) Are we trying to cure cancer? Obsolete knowledge workers? Build robots? Compete with Instagram? Buy some more runway? For now, the answer seems to be, “Hey, check out Sora, the app where you can make Sam Altman dance.”

    John Herrman

    Source link

  • AI’s Go-for-Broke Regulation Strategy

    Photo-Illustration: Intelligencer; Photo: Getty Images

    In the AI world, everyone always seems to be going for broke. It’s AGI or bust — or as the gloomier title of a recent book has it, If Anyone Builds It, Everyone Dies. This rhetorical severity is backed up by big bets and bigger asks, hundreds of billions of dollars invested by companies that now say they’ll need trillions to build, essentially, the only companies that matter. To put it another way: They’re really going for it.

    This is as clear in the scope of the infrastructure as it is in stories about the post-human singularity, but it’s happening somewhere else, too: In the quite human realm of law and regulation, where AI firms are making bids and demands that are, in their way, no less extreme. From The Wall Street Journal:

    OpenAI is planning to release a new version of its Sora video generator that creates videos featuring copyright material unless copyright holders opt out of having their work appear, according to people familiar with the matter …

    The opt-out process for the new version of Sora means that movie studios and other intellectual property owners would have to explicitly ask OpenAI not to include their copyright material in videos the tool creates.

    This is pretty close to the maximum possible bid OpenAI can make here, in terms of its relationship to copyright — a world in which rights holders must opt out of inclusion in OpenAI’s model is one in which OpenAI is all but asking to opt out of copyright as a concept. To arrive at such a proposal also seems to take for granted that a slew of extremely contentious legal and regulatory questions will be settled in OpenAI’s favor, particularly around the concept of “fair use.” AI firms are arguing in court — and via lobbyists, who are pointing to national-security concerns and the AI race with China — that they should be permitted not just to train on copyrighted data but to reproduce similar and competitive outputs. By default, according to this report, OpenAI’s future models will be able to produce images of a character like Nintendo’s Mario unless Nintendo takes action to opt out. Questions one might think would precede such a conversation — how did OpenAI’s model know about Mario in the first place? What sorts of media did it scrape and train on? — are here considered resolved or irrelevant.

    As many experts have already noted, various rights holders and their lawyers might not agree, and there are plenty of legal battles ahead (hence the simultaneous lobbying effort, to which the Trump administration seems at least somewhat sympathetic). But copyright isn’t the only area where OpenAI is making startlingly ambitious bids to alter the legal and regulatory landscape. In a deeply strange recent interview with Tucker Carlson, Sam Altman forced the conversation back around to an idea he and his company have been floating for a while now: AI “privilege.”

    If I could get one piece of policy passed right now relative to AI the thing I would most like, and this is intentional with some of the other things that we’ve talked about, is I’d like there to be a concept of AI privilege.

    When you talk to a doctor about your health or a lawyer about your legal problems, the government cannot get that information …

    We have decided that society has an interest in that being privileged and that we don’t, and that a subpoena can’t get, that the government can’t come asking your doctor for it or whatever. I think we should have the same concept for AI. I think when you talk to an AI about your medical history or your legal problems or asking for legal advice or any of these other things, I think the government owes a level of protection to its citizens there that is the same as you’d get if you’re talking to the human version of this.

    Coming from anyone else, this could be construed as an interesting philosophical detour through questions of theoretical machine personhood, the effect of AI anthropomorphism on users’ expectations of privacy, and how to manage incriminating or embarrassing information revealed in the course of intimate interactions with novel new sort of software. People already use chatbots for medical advice and legal consultation, and it’s interesting to think about how a company might offer or limit such services responsibly and without creating existential legal peril.

    Coming from Altman, though, it assumes an additional meaning: He would very much prefer that his company not be liable for potentially risky or damaging conversations that its software has with users. In other words, he’d like to operate a product that dispenses medical and legal advice while assuming as little liability for its outputs, or its users’ inputs, as possible — a mass-market product with the legal protections of a doctor, therapist, or lawyer but with as little responsibility as possible. There are genuinely interesting issues to work out here. But against the backdrop of numerous reports and lawsuits accusing chatbot makers of goading users into self-harm or triggering psychosis, it’s not hard to imagine why getting blanket protections might feel rather urgent right now.

    On both copyright and privacy, his vision is maximalist: not just total freedom for his company to operate as it pleases, but additional regulatory protections for it as well. It’s also probably aspirational — we don’t get to a copyright free-for-all without a lot of big fights, and a chatbot version of attorney-client privilege is the sort of thing that will likely arrive with a lot of qualifications and caveats. Still, each bid is characteristic of the industry and the moment it’s in. So long as they’re building something, they believe they might as well ask for everything.

    John Herrman

    Source link

  • Sam Altman’s Plan to Turn ChatGPT Into a Feed

    This week, Sam Altman announced his “favorite feature of ChatGPT so far.” It’s called Pulse, and according Altman, it “works for you overnight” by “thinking about your interests, your connected data, your recent chats, and more.” In the morning, you get a “custom-generated set of stuff you might be interested in,” akin to something a “super-competent personal assistant” might prepare. More broadly, he says, it represents “a shift from being all reactive to being significantly proactive, and extremely personalized.” And then, a recommendation: “It performs super well if you tell ChatGPT more about what’s important to you.”

    These are the words of a CEO, of course, so we should expect him to be in sales mode. They’re also the words of a person who has not just adopted the language and jargon of generative AI but done so to the exclusion of everything else. In the narrow context of ChatGPT, and through the personified language of generative AI, Pulse can be given agency, ascribed new talents and qualities, and imbued with novelty. Most other people, however, will look at Pulse and see something less futuristic than familiar: a recommendation feed.

    A recent study of ChatGPT use helped clarify what the service’s users are most commonly getting from the chatbot, outlining strong consultative habits: a lot of Google replacement, plenty of quick questions and advice, and some task completion. These interactions all depend on the user initiating in the first place, which, if your goal is to maximize engagement and/or draw people into a more comprehensive platform — to make your product the beginning and end of a user’s computing experience — is limiting. People spend lots of time searching, chatting, and working on their devices, sure. But they also spend a lot of time scrolling. Pulse looks like an attempt to secure at least some of the massive amount of attention captured by feeds and to turn ChatGPT into something more than a tool you can consult — specifically, into a source of content you can consume.

    To back up a little bit: Before the post-ChatGPT AI boom, which has been defined by large language models and chatbot interfaces, the tech industry’s conversations about AI and machine learning centered on recommendations. That was the case for good reason. Platforms that deployed surveillant recommendation engines were taking over the world. Through the 2010s, social platforms drifted from chronological feeds to algorithmic recommendations, drawing on users’ data and behaviors to show them personalized material. TikTok took this model a step further, treating social connections as firmly secondary to AI-driven learning and recommendation (or, put another way, embracing the model of digital ad targeting for the entire social-media experience).

    You can hear, in Altman’s announcement, the description of something akin to a TikTok feed: a “custom-generated set of stuff you might be interested in.” For the logical endpoint of compounded “generation” looks like, Meta helpfully announced a cautionary tale in the form of a new AI-feed product called Vibes:

    Anyway, an even closer cousin to Pulse, given the use of ChatGPT as a Google replacement, is the algorithmic homepage popularized by products like Google Now, introduced in 2012 with the following description:

    It tells you today’s weather before you start your day, how much traffic to expect before you leave for work, when the next train will arrive as you’re standing on the platform, or your favorite team’s score while they’re playing. And the best part? All of this happens automatically. Cards appear throughout the day at the moment you need them.

    By 2016, after Google had abandoned the Now branding but incorporated the features across its product lineup, the company said that it was using “machine learning algorithms to better anticipate what’s interesting and important to you.” The aim was to show Google users “sports highlights, top news, engaging videos, new music, stories to read and more” based not only on their interactions with Google but also “what’s trending in your area and around the world. The more you use Google, the better your feed will be.” By then, it had become obvious that personalized recommendation engines were ascendant and that they’d be incorporated into basically any software product that could accommodate them. And why not? At their best, they were useful and therefore sticky; at worst, they produced low-value engagement that could still be monetized.

    Early reviews from heavy ChatGPT users suggest the concept makes sense for them: Pulse is like “a newsfeed tailored to recent conversations,” one writes, saying that he wants to “dump even more information and context and app connections into ChatGPT so I can get an even better daily feed.” It’s easy enough to see how populating ChatGPT with recommendations could increase time spent on the app by casual users, too.

    In tech product terms, in other words, this is OpenAI doing an obvious and precedented thing with the growing piles of data it’s accumulating on its users: feeding it back to them in the form of content. Pulse also has specific business uses beyond encouraging more ChatGPT use. Despite (and in part because of) its popularity, ChatGPT is still a money furnace, and a large majority of its users don’t pay for subscriptions. OpenAI has been planning to expand advertising into the platform for a while but hasn’t yet settled on its solutions. Inserting too many ads into chatbot interactions risks shattering the illusions that help make them compelling in the first place (not that companies won’t try). In contrast, feeds full of recommendations — collections of algorithmically recommended content — are exactly where people expect to encounter marketing. They’re also where some of OpenAI’s biggest competitors, all now racing for AI supremacy and chatbot users, made their money in the first place.

    John Herrman

    Source link

  • The AI-Scraping Free-for-All Is Coming to an End

    Photo-Illustration: Intelligencer; Photo: Getty Images

    You can divide the recent history of LLM data scraping into a few phases. There was for years an experimental period, when ethical and legal considerations about where and how to acquire training data for hungry experimental models were treated as afterthoughts. Once apps like ChatGPT became popular and companies started commercializing models, the matter of training data became instantly and extremely contentious.

    Authors, filmmakers, musicians, and major publishers and internet companies started calling out AI firms and filing lawsuits. OpenAI started making individual deals with publishers and platforms — including Reddit and New York’s owner company, Vox Media — to ensure ongoing access to data for training and up-to-date chat content, while other companies, including Google and Amazon, entered into licensing deals of their own. Despite these deals and legal battles, however, AI scraping became only more widespread and brazen, leaving the rest of the web to wonder what, exactly, is supposed to happen next.

    They’re up against sophisticated actors. Lavishly funded start-ups and tech megafirms are looking for high-quality data wherever they can find it, offline and on, and web scraping has turned into an arms race. There are scrapers masquerading as search engines or regular users, and blocked companies are building undercover crawlers. Website operators, accustomed to having at least nominal control over whether search engines index their content, are seeing the same thing in their data: swarms of voracious machines making constant attempts to harvest their content, spamming them with billions of requests. Internet infrastructure providers are saying the same thing: AI crawlers are going for broke. A leaked list of sites allegedly scraped by Meta, obtained by Drop Site News, includes “copyrighted content, pirated content, and adult videos, some of whose content is potentially illegally obtained or recorded, as well as news and original content from prominent outlets and content publishers.” This is neither surprising nor unique to one company. It’s closer to industry-standard practice.

    For decades, the most obvious reason to crawl the web was to build a useful index or, later, a search engine like Google. A Google crawl meant you had a chance to show up in search results and actual people might visit your website. AI crawlers offer a different proposition. They come, they crawl, and they copy. Then they use that copied data to build products that in many cases compete with their sources (see: Wikipedia or any news site) and at most offer in return footnoted links few people will follow (see: ChatGPT Search and Google’s AI Mode). For an online-publishing ecosystem already teetering on the edge of collapse, such an arrangement looks profoundly grim. AI firms scraped the web to build models that will continue to scrape the web until there’s nothing left.

    In June, Cloudflare, an internet infrastructure firm that handles a significant portion of online traffic, announced a set of tools for tracking AI scraping and plans to build a “marketplace” that would allow sites to set prices for “accessing and taking their content to ingest into these systems.” This week, a group of online organizations and websites — including Reddit, Medium, Quora, and Cloudflare competitor Fastly — announced the RSL standard, short for Really Simply Licensing (a reference to RSS, or Really Simple Syndication, some co-creators of which are involved in the effort). The idea is simple: With search engines, publishers could indicate whether they wanted to be indexed, and major search engines usually obliged; now, under more antagonistic circumstances, anyone who hosts content will be able to indicate not just whether the content can be scraped but how it should be attributed and, crucially, how much they want to charge for its use, either individually or as part of a coordinated group.

    As far as getting major AI firms to pay up, not to mention the hundreds of smaller firms that are also scraping, RSL is clearly an aspirational effort, and I doubt the first step here is for Meta or OpenAI to instantly cave and start paying royalties to WebMD. Combined with the ability to use services like Cloudflare and Fastly to more effectively block AI firms, though, it does mark the beginning of a potentially major change. For most websites, AI crawling has so far been a net negative, and there isn’t much to lose by shutting it down (with the exception of Google, which crawls for its Search and AI products using the same tools). Now, with the backing of internet infrastructure firms that can actually keep pace with big tech’s scraping tactics, they can. (Tech giants haven’t been above scraping one another’s content, but they’re far better equipped to stop it if they want to.)

    A world in which a majority of public websites become invisible to AI firms by default is a world in which firms that have depended on relatively unfettered access to the web could start hurting for up-to-date information, be it breaking news, fresh research, new products, or just ambient culture and memes. They may not be inclined to pay everyone, but they may eventually be forced to pay someone, through RSL or otherwise.

    John Herrman

    Source link

  • ChatGPT Users Want Help With Homework. They’re Also Very Horny.

    ChatGPT Users Want Help With Homework. They’re Also Very Horny.

    Photo-Illustration: Intelligencer; Photo: WildChat/The Allen Institute for AI

    A recent research paper sounded an alarm for AI developers: Training data is drying up. Concerns about theft, copyright, and commercial competition are leading public and semi-public resources to tighten protections against AI scraping. The result, the paper’s authors argue, “will impact not only commercial AI, but also non-commercial AI and academic research” by “biasing the diversity, freshness, and scaling laws for general-purpose AI systems.” It’s an interesting problem that’s already starting to bite. It was also probably inevitable. A bunch of companies are raising and spending money to steal data, and the people and companies they’re stealing from are not pleased.

    Deeper in the report, researchers for the Data Provenance Institute identified another problem: Not only is the data drying up, but what remains available is out of step with what AI companies need most, at least according to their users. They included this set of charts:

    Photo-Illustration: Intelligencer; Photo: The Data Provenance Institute

    AI companies are training on a ton of news and encyclopedia content, in large part because that’s what’s available to scrapers in great quantities. (In the top chart, “tokens” can be understood as units of training data in sampled sources.) Meanwhile, actual ChatGPT users are barely engaging with news at all. In reality, they’re asking ChatGPT to write stories, often of a sexual nature. They’re asking it for ideas, for assistance with research and code, and for help with homework. But, again, they’re very horny. This is, as the paper notes, an issue for model training, accuracy, and bias: People aren’t using these things in ways that match the data on which they’re trained, and AI model performance is very much determined by the quality and quantity of training data. It’s also out of step with a lot of the discourse around AI, in which concerns about the news, disinformation, and the media in general have played — for reasons both novel and obvious — an outsize role. ChatGPT users are asking a newsbot to write erotic fiction. Not ideal!

    But wait — since when do we know how people are actually using ChatGPT? OpenAI doesn’t share data like this, which would be extremely valuable to people trying to figure out what’s going on with the company and with AI in general. On this, the Data Provenance Institute cites WildChat, a project from the Allen Institute for AI, a nonprofit funded by Microsoft co-founder Paul Allen. Researchers “offered free access to ChatGPT for online users in exchange for their affirmative, consensual opt-in to anonymously collect their chat transcripts,” resulting in a data set of “1 million user-ChatGPT conversations.” These conversations aren’t perfectly representative of ChatGPT use — researchers warn that because of where it was provided, and the fact that it allowed for anonymous use, it probably overselected for tech-inclined users and people who “prefer to engage in discourse they would avoid on platforms that require registration.” In any case, these conversations are searchable, and they’re some of the most illuminating things I’ve ever seen on the question of what people actually expect from their chatbots.

    To get this one out of the way, the horniness is relentless — search any explicit term and you’ll get hundreds of conversations in which persistent users are trying (and usually failing) to get ChatGPT to write erotic stories about video-game characters, celebrities, or themselves. There’s a huge amount of “explanation” that’s very clearly just help with schoolwork — a fascinating Washington Post analysis of the data found that about one in six conversations was basically about homework:

    Photo-Illustration: Intelligencer; Photo: WildChat/Allen Institute forAI

    There’s also a great deal of assistance with interpersonal issues and communication: help writing messages for work and school, but also dating apps:

    Photo-Illustration: Intelligencer; Photo: WildChat/Allen Institute forAI

    Again, if you’re not sure what people are getting from services like ChatGPT, and trying them yourself hasn’t helped, poke around here for a while. It’s probably not ideal for OpenAI (and others) that users spend so much time trying to coax chatbots into doing things they’re not supposed to, or into helping them do things that they’re not supposed to, but the broader sense you get from these interactions is that, generally, a lot of ChatGPT users expect the chatbot to be capable of a really wide range of things — they treat it like a more comprehensive resource than it probably is and more like a person — which indicates belief, trust, and plausible demand.

    Glimpses into real user habits for new technologies are pretty rare — the last time I remember being able to eavesdrop such strange, rich, and occasionally bracing material like this was when AOL released a massive cache of search logs back in 2006, revealing that its users were talking to the search engine, revealing incredibly poignant and sometimes dark secrets in the course of something like — but also clearly unlike — conversation. (It doesn’t take long to find similarly moving material in the anonymized WildChat records; similarly, while the data has been cleaned somewhat, it’s easy to find intensely distressing sexual and violent requests.)

    The main takeaway then was that people were ready to place a great deal of trust in open text boxes, and that “search,” for a lot of users, was something more like an all-purpose companion, a box into which they could put anything and frequently get at least something back — in the broadest possible sense, a bullish outlook for then-rising companies like Google. The new text boxes actually pretend to have conversations with you, and users are responding with similarly extreme candor. They’re behind on their work. And they’d like to read some porn.

    John Herrman

    Source link