Harms of the Generative AI

Recently, an email was posted to the emacs-devel mailing list about the Human Emacs project and its concerns about LLMs being used to develop Emacs in the future. It got mixed responses, but one caught my eye and eyes of many in the Human Emacs community:

Excuse me, but which harms are you talking about? You mention them, and claim they will mortally wound our solidarity and trust. You also threaten to fork Emacs unless the project accepts your views. But you haven’t said a word about the harms themselves, let alone explained how and why they are relevant to writing code for Emacs.

This is a response from Eli Zaretskii, a Emacs co-maintainer, who does a lot for Emacs, possibly the most. The response is interesting to me, because I was wondering what harms LLMs did to me specifically in the past year. So I decided to write some thoughts on this.

But, in order to start this, I want to establish two key points that I know about myself, and assume it applies to some other people, maybe even you.

• My brain is Lazy.

  If there’s a chance of not doing something, or a shortcut is available, my brain (not me) will do everything to avoid doing a lot of work.

• My brain is an Addict.

  Give it something addictive, like, say, World of Warcraft¹, and my brain (not me) will want to participate in that as long as possible.

I believe these two points apply to a lot of people. Not all of them, of course, some have exceptional willpower and discipline. The brain can be trained, I believe, it’s just harder to do with age.

The fact that brain is an addict should not be news to most people. Introduce it to harmful things that provide momentary sense of pleasure, like cigarettes, alchohol, drugs, even hormones, and it will be very hard to stop. Because the brain sends, let’s call them, painful signals, when it doesn’t get what it wants. Be it another nicotine dose or a dose of dophamine ^{(even though you stopped receiving it long ago but still play League of Legends, because maybe in the next game you’ll dominate so hard that it will give you another)}, your brain wants that, even if you know that it is bad for you.

Brain being lazy is a bit harder to describe, but I’ll try.

If you haven’t learned something in a while - try. Like, try learning a second language, or a third one, if you already speak two. Or try to pick up mathematics, if you thought that it might be interesting to you but putted it aside every time you had time to spare. Tell me, if you have a weird sensation in the top of your back, near the neck. I certainly do.

I’m not a biologist, and defenitely not a doctor, so take the above with a grain of salt. Salt makes food taistier, and it is increadibly hard to stop adding salt to your meals by the way. Years ago I stopped adding salt to my meals - it was hard. What does it have to do with the post? Nothing, I just wanted to brag about that I don’t add salt.

Now, back to the point - harms of LLMs and other generative AI. I wanted to establish those two keypoints about my brain because I believe LLMs are designed to exploit both of them². Maybe it wasn’t the goal from the start, but it sure feels like these companies have taken that route as the technology got more popular.

Before, every time you had a problem and were stuck, you had to either research, learn and do it, or ask for help. Asking for help was the easy way - a shortcut, because learning is hard, and I still have a habit of asking someone the moment I find myself stuck. Act of asking the question (writing it in a messenger without even sending) is often enough to get me unstuck, but not always.

Today, we have LLM-based services like Claude or Codex, and other products like these. Depends on a person, but for many, the first time you try them, you’re likely amazed, and it develops a positive response in your brain. And if you find yourself stuck, you now have a third option - instead of doing research and learning, or asking someone for help, you can ask an LLM.

For a lot of people, especially today, bothering someone with questions may invoke feelings of unease, which is a negative sygnal from the brain. Asking an LLM a question doesn’t invoke that, because despite their human-like conversation skills, you still know that this is a machine, even if you don’t actively think of it.

But what’s the problem with that, I hear you ask? You learn either way.

Here’s where they get you. Before, when you asked for help, you needed to process that - nobody gave you a solution right away. It was an advice, a direction you could follow, but you still had to do the work. Of course, if you were given a solution, there’s a high chance that you didn’t learn from that, but it depends on a person. And it would happen rarely, because nobody got time solving other people’s problems in a work environment, at least.

Now? Now even if you ask an LLM for some advice, it will often include a solution to your problem. Full, ready to be slapped on, solution, and that’s bad.

To understand how so, I need to remind you, that brain tries to avoid as much work as it can, unless you trained it not to. And thinking is work. We think all the time, you’d say, but focused thinking, that is needed for problem solving is not our default thinking mode, if you will. And now you have an LLM that gives you a solution, that may or may not be optimal, but it’s there. The very fact of it being presented to you is a positive signal to your brain, because it doesn’t need to do the work. It’s a dangerous trap.

Now, a common reply to that is “We’re not that stupid - we’ll read the proposed solution, evaluate it, see if it fits - we’ll do code review, everything will be fine”. That’s what a disciplined person will do, but there’s one fact about discipline is that you need to maintain it. There’s a third mechanism at play people not ofthen actively think of, but we rely on it in our work quite often: trust.

Trust is a shortcut in itself too, in my opinion. When we build trust towards someone, we dismiss the need to think about malicious intentions that person may have. A lot of people take advantage of it, mind you, so you shouldn’t blindly trust someone, even if they seem like a very trustworthy person.

And guess what, LLMs try very hard to seem like a trustworthy person. The way they formulate their messages is tuned, tuned to sound engaging in your head, like they’re always here for you, always ready to help. If you ever had an unfortunate opportunity to know someone who used your trust against you, I’m sure you’ll find some parallels here.

Except, the goal of these servises is not to build your trust only to betray you later, that wouldn’t make sense. Or not right now, at least. Their goal is to make you dependent on them. Make your brain an addict.

Advice here, solution there, and before you know it, LLM thinks for you, writes code for you, and then reviews it, while you desperately try to look like you’re still in control. All that work done, for a small price “of 69.99” per month for the rest of your life! “If you order now, you’ll get a bonus 20% context windo”… I’m not going there.

I noticed this happening to me. A mere thought of doing something myself, like writing code, or researching something, learning a new library or a programming language, I have a thought sitting at the back of my head: I can ask an LLM about that. I had a few projects that I wanted to make, but the momen I got stuck, I had an urge to open a chatbot, and let it think for me. And I gave in. I’ve written about it several months ago.

Since then, I had some time to reflect on this. What I want to do, how I want to be, what I want to feel when I’m doing something. And LLMs² have no place in any of this.

Doesn’t matter if models are fully open source, trained on ethically sourced assets, on hardware that doesn’t do too much harm to the environment, and running locally without subscription. I can’t accept that still, because it is a brain shortcut that only leads to even more laziness.

“Ok, but what about calculators?” I hear you ask. “Calculators did not replace our ability to count, likewise LLMs won’t replace our ability to think”, or something along the lines is often thrown in the conversation about LLMs.

I don’t like analogues when discussing AI, and in general, because it’s really hard to find one that really fits without contradictions. Yes, people can still count, and calculators did not replace mathematicians, but calculators don’t solve an open-ended problem for you. You can’t ask a calculator to find proof of a theorem, it merely helps you count big numbers faster - its purpose is focused on a single task, it’s a calculator.

Yes, if someone asked you to subtract 3957 from 7846 right now, you would probably reach for your calculator, or felt a sense of unease if you were asked to do it specifically in your head. It does fall in to the same laziness argument I’m making above, I wouldn’t argue with that, however I see it more as a “gotcha!” rather than a valid argument here. Some calculations are almost impossible to do without a paper, or highly impractical, so a calculator is a handy tool, not a replacement for understanding math or counting in your head. So I’m not buying into the “calculator argument”, no.

I don’t think we can put LLMs and generative AI in the same space as tools like a Calculator, or others for that matter. It is a whole other thing that lives in its own space. However, I do agree that you can find other examples where ordinary tools may make people more lazy, and calculators are certainly one of the examples - kids in schools nowadays hate to do calculations manually, but the scope is different. You still need to understand some of the math in order to use a calculator, and you can’t suddenly solve any math problem by just using a calculator. LLMs can make you feel that you don’t need to understand anything, only have a vision to be able to do anything.

So, to put it shortly:

• LLMs make it harder to think for yourself.
• They make your brain addicted to laziness.
• Laziness makes it hard to learn anything new.
• You become less viable in the long run.
• You stop understanding what you’re working on, and you’re already too lazy to do so even if you try.
• It’s extremely hard to work your way out of it.

And no amount of promised productivity gains will be enough to cover that.