One concern with genAI systems, according to Rogoyski, is that we’re entering a realm where nobody knows how they work — even when they benefit people. As AI gets more capable, “new products appear, new materials, new medicines, we cure cancer. But actually, we won’t have any idea how it’s done,” he said.
“One of the challenges is these decisions are being made by a few companies and a few individuals within those companies,” he said. Decisions made by a few people “will have enormous impact on…global society as a whole. And that doesn’t feel right.” He pointed out that companies like Amazon, OpenAI, and Google have far more money to devote to AI than entire governments.
Rogoyski pointed out the conundrum exposed by solutions like the one California is trying to arrive at. At the core of the California Policy Working Group’s proposal is transparency, treating AI functionality as a kind of open-source project. On the one hand, outside experts can help flag dangers. On the other, transparency opens the technology to malicious actors. He gave the example of AI designed for biotech, something designed to engineer life-saving drugs. In the wrong hands, that same tool might be used to engineer a catastrophic bio-weapon.
