Diary of an AI skeptic: All reap, no sow?

I find myself in a difficult position, as a technology professional whose career is inextricably linked to AI. I am deeply skeptical of AI’s impact on humanity.

As a disclaimer: I use AI. I explore new cloud architectures with AI. I develop lesson plans with AI. I write code with AI tools. I do not use AI for published written or spoken word of any kind (outside of a codebase). My hope is to be a trusted human agent, and that these thoughts spark some curiosity, conversation, or action that nudge us toward a better tomorrow.

I love gardening and have thought much about sustainability in systems that are extractive in nature. I cannot help but draw parallels between the rapid adoption of AI and the adoption of modern agricultural practices. Both seem to value the Fruit above all else. We arrive at fertile soil, grow until we have depleted the soil of all life and nutrients, then move on to the next fertile field. We see the ease with which we can produce Fruit this way, and regenerating the resources that enriched the soil in the first place becomes an afterthought, or perhaps a problem for a future generation. The few who care to lag behind and tend to the soil and biodiversity are cast aside by the economy-driven society; seen as luddites, losers, or outcasts.

How is AI like this?

I can hear the retort, “But agriculture deals with limited resources, constrained by the laws of ecology. AI’s fertility comes from information, which is not bounded by such laws; and AI only stands to create more information, creating a self-sustaining, self-fertilizing ecosystem.”

Let me try to convince you otherwise. For the sake of brevity and accessibility, my arguments will center around LLM-powered AI applications (think “ChatGPT”), and LLM-powered AI agents which are gaining popularity in the corporate world. But many of these points are transferrable to other “GenAI” technologies.

My first concern is for the health of primary sources of information that AI uses to train, reference, and deliver relevant information to human or agentic users. Perhaps the most popular form of AI use in the world at the time of this writing is Google search AI summary. It takes the key details relevant to a Google search from the top two or three search results and compiles them in a tidy search summary for the user. Google does not own or create the information that is reaped for these means, but it controls the output and advertising revenue that may result from it. It is clear that this functionality is convenient for the end user, but the underlying information provider is often cut out from revenue generation. Also, a rapidly expanding percentage of primary resources are actually secondary, or tertiary resources in disguise; they themselves having been AI-generated.

In the end, primary sources of information which take on the task of translating information - reality, current events, local news, data, and statistics - to a form consumable by the internet and the dispersed network of AI consumers, may simply be pushed out of business. Systems will do their best to grok reality through data from social media users and massive data ingestion pipelines, but I fear the signal may be lost in the noise of AI-generated information. No matter how good a translator or summarizer an AI system is, enough translations will degrade the quality of the underlying information.

My next concern, perhaps more urgent, is for the minds of those whose critical thinking is being supplanted by AI. Some against their will, some with unrestrained glee, and many as a simple path of least resistance. Society depends on a network of humans acting for their own self-interest in diverse and unpredictable ways, and cooperation and (relative) stability emerging as a result.

Let me be clear: I do not think humans are or will (or should) be better critical thinkers than AI. As much fun as AI-haters have poking fun at “How many Rs are there in ‘strawberry?’”-type-hallucinations, I would rely on AI output over a random person’s output 999/1000 times. I break out in a sweat thinking about how my own bloviations and mis-speak would be ridiculed if it were received as a reply from ChatGPT.

So no, I don’t think that people having better reasoning skills than AI is realistic, nor do I think it’s particularly important. I do think that human people having strong and independently developed critical thinking skills is crucial to a fair and just society. Outsourcing our critical thinking will create a “flattening of the curve”, reducing the weight of diverse and marginalized voices, and creating an ecosystem where controlling input data will have an outsized effect on how we move forward as a civilization. We have already seen how effective misinformation and attention-monopolizing campaigns can be. The erosion of our critical thinking skills has and will create fertile ground for the extraction of our money, attention, and time. This is a recipe for systemic oppression, designed by our own hand.

Perhaps that’s a bit too pessimistic, but the point remains that critical thinking is increasingly becoming optional. As someone who values democracy and human liberty, I fear the erosion of our collective intellect, morals, and ethics (which are all downstream from critical thinking, in my estimation). I don’t believe you can remove the responsibility of humans in the shepherding of humanity, no matter how many layers of LLMs you put on top of it.

On a more practical note, I believe AI tools are also leading to an erosion of context and problem framing within business organizations. Humans excel at patching together disparate systems, seeing patterns, and “making it work”. The more we rely on AI to externalize critical thought at work, the less we understand about the interconnectedness of the information ecosystem we are interacting with. I think complicated codebases are a perfect example. Historically, developers spend hours navigating, orienteering, and debugging obscure errors in huge codebases. Given enough time and pressure, these developers become experts in your organization. Not only can they translate requirements into logical code (something LLMs are quite good at), but they also understand the politics, motivations, and shortfalls of the requirements themselves. You can prompt an LLM to do this, but they simply do not have access to the data, context, or subtle communication skills to address every edge case.

Leveraging AI to do all of the exploration, identification, and contextualization of business problems on our behalf keeps us from becoming experts. And when a novel problem that requires expertise arrives, I’m not sure we can always trust AI to handle it with grace.

Conclusion

If you’ve made it this far, you’re probably either resonating deeply with my writing, or you’re raging. In either case, I hope to throw you a bone here.

I do believe we can arrive at a point of tech-optimism with AI technology, though there will likely be some pain in the interim. AI can be a powerful learning tool, dynamically creating a learning environment ideally suited to your situation and interests. You must simply make the choice to engage with it critically. AI is an extremely powerful business tool, exceling at making sense of unstructured data, or data from disparate sources. I am genuinely excited to see who the “winners” are in the AI-powered economy, and I think AI is uniquely positioned to enable entrepreneurship to address niche, localized problems. My hope is that we can realize this future with our critical thinking and communities intact.

My advice to any who will heed it:

• Engage with the world outside of the AI data loop (take care of yourself, make some music)
• Use AI for good (think local enrichment, not global extraction)
• Feed the soil without expecting immediate return (think, write, make, plant, grow)
• Don’t be the hype guy or the doomer, let’s find some nuance and work together on a way forward