Before We Leave the Room.

Humanity has been given a marvelous new toy, unfortunately, the toy is an excavator and by God, like a child, we're having fun with it. wE pull one lever and produce code in seconds. Another lever screens a thousand candidates before lunch. A third writes, classifies, scores, recommends, monitors or decides. The machine responds immediately, the numbers move in the desired direction and the child operating it feels extraordinarily clever.

Playing is not malicious. That matters. We are curious, impatient and completely fascinated by what the new toy can do. The broken lamp was not the objective. Neither was the hole in the floor. They are simply consequences the child has not yet learned to connect to the game. AI is not digging our grave. It is the excavator. We are operating it, rewarding ourselves for the volume removed and preparing to blame the machine when we finally notice the shape of the hole. The problem is that we are learning how to make it more powerful much faster than we are learning how to use it responsibly. And the bedroom is already a mess.

The marks already on the wall

Over the past few weeks, I examined four apparently separate problems: a hiring market that rewards performance over truth; recruitment pipelines made opaque by layers of intermediaries and automation; processes that preserve every formal stage while quietly ceasing to recruit; and a software industry capable of producing more code while weakening the human pipeline that produces engineers.

Those arguments are already developed in “The Hiring Game: Who Lies Better”, “Recruitment by Obfuscation”, “You Can Automate the Recruitment Process Until It Stops Recruiting” and “The Code Will Keep Coming. The Engineers Might Not.” Their links are at the end of this article. They are the incident reports. There is no reason to reproduce them here.

Placed next to one another, however, they reveal something larger than hiring or software: A new capability increases volume. More applications can be generated, more code can be produced, more content can be published and more decisions can be attempted. The surrounding process cannot absorb that volume, so we automate the next stage. Automated production produces automated filtering; automated filtering removes context; the loss of context creates uncertainty and distrust; uncertainty is answered with more monitoring, more scoring and more automation.

And while every individual step can be defended as efficient, the complete system becomes steadily less capable of understanding what it is doing.

This is the vicious circle hiding beneath the surface. Companies receive more applications and trust fewer of them. Candidates encounter more filters and optimize harder for the filters. Employers become more suspicious and add more automated screening. Genuine expertise becomes harder to recognize because recognition itself has been delegated to mechanisms designed to process volume, not understand people. The failure to hire strong humans creates more pressure to automate the work they would have done. The circle closes neatly.

Apparently, this is what passes for optimization when every participant is rewarded for their own five meters of pipeline.

Search engines, recommendation systems, workflow automation, performance scoring and applicant tracking systems were moving us in this direction long before generative AI entered the room. AI is simply the most capable toy we have placed in a system already trained to confuse speed with progress and measurement with understanding.

What the child learns from the game

Children repeat what produces an immediate reward. So do organizations. If we count applications screened, the system learns to screen applications. If we count code produced, it learns to produce code. If we count tickets closed, content published, assignments submitted or cases processed, the tool will help us move those numbers with astonishing efficiency, but what it cannot tell us is whether we chose the right numbers. We rarely count the senior engineer rejected before speaking to a human. We do not see the exception silently converted into an error code, the architectural knowledge that was never transferred, the junior who learned to obtain an answer without learning how to challenge it, or the citizen who abandoned a legitimate request because the process offered no path outside its predefined categories. The dashboard remains green because the dashboard was never taught the colour of those failures.

In software, as in many other disciplines, expertise is not transmitted by downloading the final answers from a senior person. It is transmitted through review, disagreement, partial responsibility, recoverable mistakes and exposure to the reasoning between the problem and the solution. The middle layer is where knowledge is tested, corrected and prepared for transfer. Remove that layer and the organization may temporarily look wonderfully efficient. Juniors can produce with powerful tools. A shrinking group of seniors can supervise at greater distance. The people in between, expensive enough to attract scrutiny and insufficiently rare to trigger panic, become an inviting target for optimization.

This is why the disappearance of the professional middle concerns me more than anything, because as the time passes, the seniors leave. The juniors have generated output but have not accumulated the experiences from which senior judgment is formed. When the system encounters a failure outside its training, policy or happy path, the organization discovers that it preserved human responsibility while dismantling the process that creates responsible humans.

This is not a new engineering surprise. Lisanne Bainbridge described the “ironies of automation” in 1983: the more reliable an automated system becomes, the less opportunity human operators have to practice, even though the remaining abnormal situations demand greater expertise. Automation removes routine involvement and leaves the human waiting for the rare moment when routine knowledge is least useful and deep understanding is most necessary. We have known this for more than four decades. Yet we looked for much larger excavators.

The bedroom is not the house

Software and recruitment are where I first saw the signs though it does not end there. Education is already becoming a contest between generated work, automated detection and increasingly elaborate surveillance. When one tool performs the cognitive task and another estimates whether the first tool was used, the institution can become extremely efficient at processing assignments while losing sight of whether a person learned anything. UNESCO’s guidance keeps returning to human agency and human capacity because education is not a content-generation pipeline. Its product is a person capable of thinking.

The workplace is another part of the house. The European Commission’s Joint Research Centre reports that one third of workers in its 2024–2025 EU survey used AI for work-related purposes, while digital monitoring and algorithmic management had already become significant. Algorithms allocate work, determine pace, evaluate performance and influence rewards or penalties. The JRC also identifies the information asymmetry this can create between institutions and workers: one side can inspect the person while the person cannot inspect the decision.

Public administration and customer service show the same fingerprints. Routine cases become faster and cheaper, which is genuinely useful. Exceptions become stranded because the human who once applied judgment has been removed, deprived of authority or hidden behind several automated layers. The process works beautifully, provided reality has the decency to behave like the dropdown menu.

Media adds generated supply to algorithmic distribution and automated moderation. More content requires more filtering. Filtering changes what receives attention. Producers then optimize for the filter, which creates more of the material the next filter must manage. Each actor behaves rationally toward the mechanism directly in front of them. Together, they build an environment nobody consciously chose.

The bedroom, is not a metaphor for the technology sector. It is the collection of places where the damage has become visible early enough for us to learn from it. The rest of society is the house and the door is already cracked open.

The moment the child understands

In 1996, during a technical seminar at the high school where I was a substitute teacher, I presented a projection of how computing might evolve over the following decades. In the pessimistic model, the curve took an alarming upward turn around 2030. I revisited that projection in “The Code Will Keep Coming. The Engineers Might Not.” and explained why the steepening appears to have arrived roughly a decade earlier than I expected.

The mathematics belongs in that article. The consequence belongs here: the interval between new capability, mass adoption and systemic consequence is shrinking.

And somehow my mind keeps picturing an uncomfortable historical image: Fifty years from now, Carnegie Mellon University will award a “Schmidhuber Prize” for work that ensures advanced intelligence serves humanity. The parallel with Nobel is entirely intentional. It is also a thought experiment, not an accusation against Jürgen Schmidhuber, Carnegie Mellon, Alfred Nobel or a machine that does not yet exist.

Nobel did not cause Trinity. Dynamite did not mechanically produce the atomic bomb. The connection is moral and institutional, not technical: human beings create extraordinary capabilities, scale them through systems of power and understand their full meaning only after those capabilities appear in a form that can no longer be treated as theory. Oppenheimer had Trinity — a visible instant after which the implications of the work could no longer remain abstract. We may not receive such a mercifully obvious moment. Our breaking point may arrive without an explosion, a sentient machine or one spectacular failure. It may be distributed across millions of individually reasonable decisions: one more human checkpoint removed, one more skill allowed to decay, one more exception denied because no category exists for it, one more consequential judgment transferred to a system whose result nobody can explain and nobody is authorized to challenge.

Everything may continue to operate. The applications will be processed. The code will keep coming. The scores will be calculated. The reports will remain green. Until we discover that the people who understood the room have gone, the people still inside cannot repair it, and the child has reached a load-bearing wall. That would be our Oppenheimer realization point: not the moment the technology becomes powerful, but the moment we finally understand what our use of it has made us unable to do. Recognition after that point would not be governance. It would be an autopsy.

Growing up without throwing away the toy

The answer is not to ban AI, retreat to paper forms or require a committee meeting before somebody automates a spreadsheet. The toy is marvelous. Used well, it can remove drudgery, widen access to expertise, reveal patterns, improve consistency and allow people to attempt work that was previously impossible. Growing up does not mean losing curiosity. It means accepting that capability and permission are not the same thing.

First, we must stop rewarding the inner child only for how quickly the excavator moves material. Throughput matters, but it is not outcome. Applications processed, code generated, cases closed and costs reduced must be measured beside the expertise missed, decisions reversed, exceptions mishandled, knowledge lost and human competence preserved. Automation must prove that it improves the system, not merely that it reduces the cost of one step.

Second, we must keep the lights on. Consequential systems need traceability: what tool was used, for what purpose, what evidence and general criteria influenced the result, what limitations were known, and which person or institution remains responsible. This does not require publishing model weights, security controls or legitimate trade secrets. A company may keep its recipe secret. It may not conceal that the product poisoned the customer.

Third, the people affected need a real door handle. Contestability is not administrative kindness and an appeal is not an annoying edge case. It is the return path through which the system receives evidence that its model of reality is wrong. A decision that cannot be meaningfully challenged is a system deliberately designed without error reporting.

Fourth, the adult in the room must have actual authority. A nominal human reviewer who cannot understand the recommendation, inspect the relevant information, override the output or stop the process is not human oversight. It is liability furniture. The person needs competence, time and the power to say no.

Fifth, we must preserve the ability to repair what we automate. That means maintaining professional pathways, supervised practice, mentorship, bounded responsibility and recoverable failure. In engineering, education, medicine, public administration and management, the inconvenient work through which judgment develops cannot all be classified as inefficiency. Some friction is waste. Some friction is how humans learn where the walls are.

Finally, powerful deployments must be reversible. We need limited introductions, comparison points, audit windows, rollback procedures and safe ways to stop. If removing a tool would cause operational collapse because nobody remembers how the work functions without it, the organization has not adopted a tool. It has acquired a dependency and renamed it innovation.

These are not merely European regulatory instincts or American risk-management preferences. The EU AI Act classifies specified uses in recruitment, worker management, education and access to essential services as high-risk, and establishes requirements including record-keeping, transparency to deployers and effective human oversight; in qualifying circumstances, it also provides routes for complaints and explanations. The US NIST AI Risk Management Framework organizes voluntary practice around governing, mapping, measuring and managing risk. The OECD principles connect transparency with the ability of affected people to understand and challenge an outcome.

Different institutions, different legal force, same engineering discovery: At scale, feedback is safety.

Before we leave the room

The child in this story is not a "somebody". It is not an irresponsible CEO, an overenthusiastic developer, a careless recruiter, an ambitious vendor or a regulator arriving late to the meeting. Those people exist, naturally, but blaming them individually would allow the rest of us to stand outside the metaphor and nod wisely at the damaged furniture. We are the child. We are also the only adult who is coming.

That is the human part of the problem, but it is also the hopeful part. The same species capable of turning a tool into an opaque machine for avoiding responsibility is capable of designing transparency, preserving knowledge, creating meaningful review and deciding that not everything technically possible must immediately become normal.

AI is not asking us to surrender judgment. It is making the cost of pretending we can operate without judgment impossible to hide; we can preserve the benefits of automation, or we can preserve the comfort of decisions nobody must explain but never both.

We also do not need to throw away the toy. We need to grow up before we leave the room and start playing with the house.

---

Previous articles used as the incident log

• "The code will keep coming. The engineers might not"
• “You Can Automate the Recruitment Process Until It Stops Recruiting”
• “Recruitment by Obfuscation”
• “The Hiring Game: Who Lies Better”

---

Sources and governance references

Lisanne Bainbridge, “Ironies of Automation”, Automatica, Volume 19, Issue 6, 1983.

https://doi.org/10.1016/0005-1098(83)90046-8 (https://www.linkedin.com/redir/redirect?url=https%3A%2F%2Fdoi%2Eorg%2F10%2E1016%2F0005-1098%2883%2990046-8&urlhash=JsQb&trk=article-ssr-frontend-pulse_little-text-block)

European Commission Joint Research Centre, “Algorithmic management and digital monitoring of work”, including the 2024–2025 AIM-WORK survey.

https://joint-research-centre.ec.europa.eu/projects-and-activities/employment/algorithmic-management-and-digital-monitoring-work_en

UNESCO, “Guidance for generative AI in education and research”.

https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research

Regulation (EU) 2024/1689, Artificial Intelligence Act, consolidated text of 27 July 2026. Relevant provisions include Articles 12–14, 85–86 and Annex III.

https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:02024R1689-20260727

European Data Protection Board, “Guidelines on Automated individual decision-making and Profiling”.

https://www.edpb.europa.eu/documents/guideline/automated-decision-making-and-profiling_en

US National Institute of Standards and Technology, AI Risk Management Framework and AI RMF Playbook.

https://www.nist.gov/itl/ai-risk-management-framework

https://airc.nist.gov/airmf-resources/playbook/

US Government Accountability Office, “Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities”.

https://www.gao.gov/artificial-intelligence

US Equal Employment Opportunity Commission, resources on artificial intelligence and algorithmic fairness in employment decisions.

https://www.eeoc.gov/eeoc-publications

OECD AI Principle 1.3, “Transparency and explainability”.

https://oecd.ai/en/dashboards/ai-principles/P7

The Nobel Prize, “Alfred Nobel — his life and work”; US Department of Energy, “Trinity Site — World’s First Nuclear Explosion”. The Nobel–Oppenheimer comparison in this article is a moral analogy about capability, institutions and consequence, not a claim of direct technical causation.

https://www.nobelprize.org/alfred-nobel/alfred-nobel-his-life-and-work/

https://www.energy.gov/lm/trinity-site-worlds-first-nuclear-explosion