The dark side of AI in the workplace
AI slop, data leaks, and a workforce that struggle to function without an LLM. An opinion piece from inside a startup.
AI slop is everywhere
AI has made it so much easier that now all our internal folks, tech and non-tech alike, are using Claude, Gemini, ChatGPT and similar tools to design their own “software” or “tools” to help with their work. This is great, up to a certain point.
The problem I am facing now is that none of us can agree on a final design anymore because everyone wants to showcase their version and why it’s better than mine or someone else’s, and sometimes purely for cosmetic reasons. I have seen examples of a Lovable “app” they want us, the devs, to build for them based on their design. Finance has a view on this. The operations team now has a version of their own. Another team is doing something similar too. For me, this is quite discouraging and I will tell you why.
First, I don’t actually mind this approach at all. Really. As long as it stays within the boundary of a person’s internal team and it is helping them, then let’s do it. It’s like building an Excel sheet with many columns and formulae that only you understand and can interpret. That’s fine. But what’s really annoying is when people then start using such tools to pass them on to the rest of the business, or sometimes even asking the tech team to start building similar tools as if they are production-ready specs. Nobody seems to understand that while AI has made it easier to build new things rapidly, there will always be someone that needs to maintain it and there is a real cost to that.
Every month there is a new app idea
Honestly, this is a continuation of the above. The new idea is again, some kind of HTML page or artifact produced by Claude, Lovable, or another LLM-based app. I have no idea where they expect to get that kind of data from, or if the person has done any actual research on whether that data even exists. Where do you even host such a site?
The other aspect of this is that developers spend a lot of time on a design, only for someone to come in and kill it because they now have a “better” version that took them fifteen minutes to generate. That is genuinely demoralising and I don’t think people realise the effect it has on the people doing the actual work.
The danger of data leaks
There is no way we would be able to tell if users have shared work, and sometimes sensitive data, with third party LLM tools. Scary stuff. Most people are not thinking about this at all when they paste a document or a spreadsheet into whatever chat interface is open in their browser. GDPR, confidentiality agreements, client data. All of it potentially sitting in a training pipeline somewhere and we would have no idea.
Too reliant on AI now, and the cost is spiking
This is one of my biggest worries for any small to medium sized business. Before, users used to learn how to use their tools and find the data they wanted themselves. Now everything is driven by LLM outputs. Even Google search is pushing this direction. Users are now reaching for some of the most expensive LLM products out there to ask questions they could easily answer by looking at their own Excel sheet or BI dashboard. And then they just accept the number the LLM gives them at face value, without questioning how it came to that conclusion or where it got the data from.
I have a simple rule of thumb for this. If tomorrow all AI tools disappeared, could you still do your job? If the answer is “yes” or “yes, but it will take a bit longer”, then you are fine. If the answer is “no”, then there is a massive blindspot there and a very real reason to be worried.
Assuming AI will just fix things
People don’t want to change their process. For example, there are 100s of Excel files that people are using internally. Whenever there is an opportunity to digitalise the tech, there is always friction to not do it. Or there is a user that wants to somehow digitalise their all 100s of Excel sheets with fragmented data. The thing is, this is the area where AI is encouraging users to be lazy and far too lenient on the data, just trying to make the LLM make sense of it. And the LLM will hallucinate confidently. Imo, being unwilling to disrupt your own process to augment your work is the main driver of more AI slop, and of people hating AI.
This is an opinion piece based on my own experience working in a startup environment. I am not anti-AI. I am anti-thoughtlessness.