AI Could Threaten Humanity. How Should We Respond?

The BBC article, “Why are there concerns AI could threaten humanity, and how real are they?”, explores a debate that is becoming increasingly difficult to ignore.

Some researchers believe advanced AI could eventually pose an existential threat. Others argue that these predictions exaggerate what the technology can do, distract from existing harms or serve the interests of the companies making them.

The article leaves readers with an uncomfortable tension: the organizations racing to build more capable AI are also among those calling for stronger safeguards.

My view is that this deserves serious scrutiny. We should examine both the warnings and the assurances of safety.

For business leaders, it also raises an immediate question:

As we give AI more responsibility, are we building the capability to remain in control?

What the article tells us

The BBC reports that Anthropic alignment researcher Evan Hubinger believes there is a greater than 10% chance that AI could kill all humans within the next decade.

That is a striking claim. It is also an individual assessment, not an established probability or a scientific consensus. The article notes that he considers the risk from current systems low.

The broader concern is about where the technology could go next.

Researchers describe scenarios in which increasingly capable systems act independently, help develop more powerful AI or enable severe misuse. The BBC also reports that major developers have called for measures such as additional safety work and external evaluation.

Alongside those warnings, it presents criticism of the industry’s motives and of the attention given to speculative future threats while harms such as scams and abusive deepfakes already exist. Source: BBC News, “Why are there concerns AI could threaten humanity, and how real are they?”

How real is the threat?

My reading is that the article establishes the seriousness of the disagreement more clearly than it establishes the likelihood of catastrophe.

That distinction matters.

A researcher can identify a plausible mechanism for harm without being able to reliably calculate its probability. Equally, uncertainty about a prediction does not demonstrate that the underlying concern is imaginary.

We should therefore be careful with percentages that sound more precise than the available evidence allows.

I would apply the same scrutiny to confident promises of safety. If someone says a powerful system will remain controllable, what evidence supports that assurance? What was tested? Under which conditions? What remains unknown?

For me, a credible discussion needs to distinguish between observed behavior, a plausible future scenario, and speculation. Putting all three in the same category makes the debate louder and less useful.

The point where capability becomes authority

From an enterprise perspective, the most consequential shift is the decision to let AI act.

Consider the difference between a system that drafts a supplier payment recommendation and one that can approve and execute the payment.

The underlying task may look similar. The authority is different.

The second system needs clear limits, approval rules, monitoring and a way to contain errors. Someone must also remain responsible for the consequences.

This is the lens through which I approach the wider debate. As the scope of action grows, our ability to evaluate and supervise the system must grow with it.

A successful demonstration is useful evidence that something can work. I would not treat it as sufficient evidence that it will remain reliable across every situation it may encounter.

The questions leaders should ask before expanding autonomy

For organizations deploying AI, I would turn the debate into five practical questions.

1. What is the system allowed to do?

Define the actions it can take, the information it can access, and the boundaries it must respect. Broad instructions such as “improve efficiency” need to become specific permissions and constraints.

2. Which decisions require human approval?

Review should reflect the consequences of an action. Drafting an internal summary and changing a customer’s contractual terms should not carry the same approval requirements.

3. How will we detect unexpected behavior?

Decide what needs to be recorded, which signals require attention and who will investigate them. Oversight needs to be part of the operating model from the start.

4. Can we stop it and recover?

The ability to suspend a system matters. So does understanding what can be reversed and what damage may require a different response. A shutdown button alone does not answer the recovery question.

5. Who owns the outcome?

Assign responsibility clearly. “The AI did it” cannot be the organization’s explanation to a customer, an employee, or a business partner.

These questions do not settle the existential risk debate. They help leaders make more accountable decisions today.

Existing harms deserve attention too

One of the article’s strongest points is its attention to the risks already affecting people.

My view is that organizations should manage present harms while preparing for more uncertain future ones. The evidence, time horizons, and responses will differ.

For a business, that means allocating effort according to the risks it actually faces. A company deploying a customer service assistant has immediate responsibilities around accuracy, data access and escalation. It should address those responsibilities directly, even while the wider industry debates the limits of advanced AI.

Distant scenarios should not become an excuse to neglect foreseeable problems.

What responsible progress should look like

I support continued AI development and adoption. I also believe that greater autonomy should require stronger evidence that the system can operate within its intended limits.

For me, a sensible approach is to expand authority gradually, evaluate performance under realistic conditions, and make the criteria for further deployment explicit.

Independent evaluation can contribute to that process. It should complement the accountability of the organization building or deploying the system.

The standard should become more demanding as the potential consequences grow.

That is how I interpret responsible progress: ambition supported by evidence, with clear ownership of the outcome.

My takeaway

The BBC article is valuable because it brings together the warnings, uncertainties and competing interests behind this debate.

It does not resolve whether advanced AI will threaten humanity. It gives us reasons to examine how decisions about increasingly powerful technology are being made.

For leaders, that examination should begin inside their own organizations.

We should pursue useful AI with conviction, ask difficult questions about its limits and require evidence before extending its authority.

Ship AI. Deliver outcomes. Keep control.

If your organization is moving from AI pilots to autonomous workflows, reach out. Let’s discuss the permissions, oversight and accountability needed to turn that ambition into business value.

---

Source: BBC News, “Why are there concerns AI could threaten humanity, and how real are they?”