Live from Spotify — the future of podcasting

What is that training data? There the company is less loquacious.

Meta said the 15 trillion tokens on which its trained came from “publicly available sources.” Which sources? Meta told The Verge’s Alex Heath that it didn’t include Meta user data, but didn’t give much more in the way of specifics.

It did mention that it includes AI-generated data, or synthetic data: “we used Llama 2 to generate the training data for the text-quality classifiers that are powering Llama 3.” There are plenty of known issues with synthetic or AI-created data, foremost of which is that it can exacerbate existing issues with AI, because it’s liable to spit out a more concentrated version of any garbage it is ingesting.

AI companies are turning to such data because there’s not enough good, public data on the entire internet to train their increasingly greedy AI models. (Meta had reportedly floated buying a publisher like Simon & Schuster to satisfy its insatiable data needs.)

Meta, of course, isn’t the only company that’s tight-lipped about where its AI data is coming from. In a now infamous interview with WSJ’s Johanna Stern, OpenAI’s chief technology officer Mira Murati was unable to answer questions about what Sora, OpenAI’s video generating app, was trained on. YouTube? Facebook? Instagram — she said she wasn’t sure.

Working as a seller called Big River, a secret group of Amazon employees gained access to rival platforms, including Walmart, FedEx, and Alibaba. They used Big River email addresses and went to seller conferences as Big River employees. They even stayed hidden within Amazon itself. These employees would take screenshots of competitors’ systems that they would then show others at Amazon in person to avoid an email paper trail.

Perhaps most strange of all, the company created a fake Japanese streetwear brand called “Not So Ape” (clearly a play on A Bathing Ape) and continues to sell products from the brand on a Shopify store, presumably as an attempt to learn the inner workings of the shopping platform. Of course, copying is old hat for Amazon.

In meetings where they’d use this clandestine information to inform Amazon’s own business practices, the group resorted to literal paper. “[T]he team avoided distributing presentations electronically to Amazon executives. Instead, they printed the presentations and numbered the documents. Executives could look at the reports and take notes, but at the end of the meeting, team members collected the papers to ensure that they had all copies."

But it’s run into trouble: the orbs have been banned across Europe and Africa, and the associated WLD crypto token has plunged 50% over the past month.

For project insiders, who reportedly received a token allocation of 25% of supply, that could equal significant losses. 

Which is what may make World Chain attractive. Crypto exchange Coinbase launched its own L2, Base, last year. Base has since seen rapid user growth — activity that’s generated the exchange millions of dollars in weekly fees. 

Worldcoin could benefit from similar revenue if its L2 is adopted around the world.