The latest trend on Xitter is for every post to feature a reply asking the in-house AI “Grok” to verify the main post with “@Grok, is this real?” It hardly seems to matter what the content is: the grainy footage of an Indian air defense system in Kashmir, the rubble heap of a leveled Palestinian building, or those “satisfying” videos of a man jet-washing his drive. People cast adrift in the unending tides of roiling garbage and content engagement trash cleave to Grok like a piece of flotsam to keep them from drowning in the digital soup.

As a devout Techno-Pessimist, I’ve argued for years that the real issue with automated and AI systems is not so much that they rule within their cyber-realms but that they increasingly sever their bounds and impose themselves upon us in the real world. Or rather, in the age of the AI chatbot, they are called upon by other humans to do so.

A short personal anecdote should serve to illustrate this point.

Making full use of the glorious spring weather we’ve been blessed with, I recently decided to plant some runner beans, primarily to distract from my disastrous attempt to grow tomatoes. I built an archway for the plants to climb up and began arranging borders for the seedlings to be planted in. While down on my knees, sifting through fine topsoil and mixing in mulch, my nearest and dearest (entirely uninterested in gardening) appeared and promptly told me I was doing it wrong. She asked if the border I had constructed was 30cm deep. It was at most 10cm deep. I then demanded an explanation as to why I should be digging a trench 1/3 of a metre deep to plant runner bean seedlings and was informed that “ChatGPT says so”.

ChatGPT wasn’t incorrect, either. Bean root systems can indeed reach 30-40 cm. The problem was that the input had assumed the roots could not pass through the mulch layer and into the topsoil beneath. Nevertheless, in the mundane setting of my modest vegetable patch, I was confronted with a 1000-IQ authority with access to everything ever written about planting beans. Moreover, I had to justify using my own agency over meekly submitting to an AI that could be whipped out of a back pocket within seconds. The keen-eyed gardener might spot that I was deploying Charles Dowding’s “No Dig” method. I haven’t read any of Dowding’s books; I merely watched his YouTube channel. But ChatGPT has read every word he ever wrote.

Is this just the way things are going now?

A fictional scenario comes to mind, also within a humdrum setting, that does not require the invocation of Skynet AI takeovers and singularity doomsday scenarios.

A customer in a pub complains that the barmaid does not allow the head on a pint of ale to settle before topping it up, making the pint more gassy than he thinks is optimal. The barmaid replies that she’s been pouring pints like this for years, and nobody has complained so far. The customer whips out his phone and sets ChatGPT or Grok to work on how exactly a pint of that particular ale should be poured, and the result is that she is indeed in the wrong. The barmaid now has a choice: either she continues pouring the ale in a manner that she knows is incorrect or bows before the higher authority of the customer’s iPhone and submits. However, the landlord arrives, and he, too, has an iPhone with Grok installed. He begins reeling off efficiency statistics in pouring pints and highlights how much it would cost his business if every pint were poured according to the specific recommendations.

Thus, a Mexican stand-off ensues, with individuals holding each other’s phones as representatives of the God-like authority of the AI within. Rather than using their own agency and intuition, relations will be mediated by the All-Knower.

Digital tech billionaire Marc Andreessen said on the Joe Rogan podcast that the world the tech moguls were building would be more like the Middle Ages for the masses because they would exist in a world of higher powers they could not comprehend. The framing lingered in my mind, yet I assumed Andreessen was referring to self-guiding drones whizzing overhead or gadgets that could assemble themselves. Now, a year on, I see that he was not being hyperbolic and that artificial intelligence's omniscient nature will, in effect, begin to resemble an ever-present God-like form, and it will sit comfortably in your pocket.

Unlike the reassuring glow of a medieval church on a gloomy, pre-power grid landscape, the iPhone is an entirely dead form scraping lifeless data from frigid algorithms. The so-called intelligence is merely a canny linguistic parlour trick where data is reconstituted into words a human can understand. Yet, the hollowness of AI does not mean that we will not be forced to confront age-old theological questions of free will and human agency. How much free will do we have if every decision is processed through digital intelligence?

This is already manifest in our social media feeds. As I write, I’m listening to Game of Thrones Music & North Ambience Winterfell House Stark Theme. I chose it in the sense that I clicked play and found the sombre ambiance relaxing. Yet it belongs to a wider selection of audio soma YouTube bombards me with on my home page.

The ubiquity of AI in our daily lives means that a higher authority will be present; it’s not a God but a void, and it will increasingly inform our decisions regardless. Why would you not want to pour a pint correctly? Why would you not want to plant beans at the proper depth? And besides, if you aren’t using AI to optimise your outcomes, others will be, and you’ll not be able to compete.

You may notice that there’s something vaguely familiar about all of this. Here, we’re confronted with an intelligence of nigh-on infinite capacity that asks us to submit some of our free will for access to unlimited knowledge…a bargain, if you like. You can retain your free will, but you will get things wrong and have worse outcomes.

However, the more you come to rely on the Higher Authority, the more you lose the capacity to think independently. And it is all done quite voluntarily and enthusiastically.

We seem to be witnessing the death of human agency, a grander version of the parts of our brain that could do mathematics shrinking after the calculator was introduced. If people now struggle to read maps or road signs after the advent of satnav, what will be the impact of pocket AI telling them everything?

There is no need to invoke a self-aware AI supercomputer when we can simply invoke the tendency of people to take the path of efficiency and least resistance. Technology, after all, is supposed to make life easier. When manual labour was taken over by machines, men began “working out” at the gym to compensate for the lack of muscle tone and strength. What will people resort to when thinking itself is done for us by machines? Perhaps even more crucially, how will a brain that has hardly adapted for 50,000 years and is prone to pattern recognition and mythical framing begin to think of a device that does their thinking for them, comforts them, and is always ready-to-hand in their pockets?

Given that AI systems have already digested every religious text and theological musing that Man has ever written and recorded, there’s no reason to suppose that, if requested, the AI could not speak to the individual with the authority, mannerism, and tone of God himself. Is this what Marc Andreessen was referring to?

The ideal seems to be a Techno-Theocratic priesthood that requires only that you have faith in the data-scraping. You have no idea how it works, just as you have no idea how your devices communicate with each other to target you for pop-up ads. But that’s fine because you don’t need to know anything. All the thinking is done elsewhere anyway. It will become a world where things “just happen” like standing on a shadow was bad luck or spotting a black cat late at night would be seeing the devil in a familiar.

To be blunt, it is no longer a world of Enlightenment but of obscurantism due to an oversaturation of knowledge information. We will become appendages to the tool, having ourselves once elbowed aside God to fill the space He vacated. A world once incomprehensible, guided by the almighty, will become so once more as our ability to think independently shrivels like the wings of a parrot stranded on an island without predators. It will simply be the “way of things”, and there’s no rolling it back now because there’s too much money involved, and people are excited about the ability of AI to crack cancer cures sometime soon.

It will be weird but not wonderful, and many a head will be scratched while pondering Faust’s lament in Goethe’s masterpiece.

I’ve studied now Philosophy
And Jurisprudence, Medicine,
And even, alas! Theology,
From end to end, with labor keen;
And here, poor fool! with all my lore
I stand, no wiser than before.