I notice that AI is being used increasingly often here for programming work. For most of us, it is just a hobby, but I think it is useful to forward this warning, which is aimed at professionals, to you as well.
The Hidden Risk No One Talks About
AI-generated code compiles cleanly, passes tests, and looks correct. But that doesn’t mean it’s safe. Every line you didn’t write is a line you didn’t fully review - yet you’re still responsible for it.
Security Vulnerabilities
AI finds a security vulnerability in one of the JS libraries and now your customers are at risk.
License Violations
Open-source licensing conflicts baked silently into your proprietary codebase.
Compliance Gaps
Missing controls that expose you to SOC2, HIPAA, and regulatory audit failures.
Hidden Logic Flaws
Incorrect assumptions encoded at generation time that only surface under real conditions.
You didn’t write that code - but you own every consequence of shipping it.
It’s a tool. Then it’s down to how you define responsibility for yourself.
It certainly writes better code than I do, and as a result many projects that died on the vine have sprung into life.
I would imagine that once one gets to the size of a project like zynthian, to explain the nuance and developing context of the project, would take probably longer than it does to write docs, and we are well behind on that one.
I actually have some strong feelings against AI as well, mainly because I do not trust society to handle the outcome, where being flooded with a mass of bullshit being monetized may be the least dangerous matter here.
I did test AI in several fields I’m interested in (craft, coding, law, text production regarding cultural studies and local policies) and I think I always come to similar conclusions (rough estimation): The results are 60% correct, 25% has some truth to it and 15% is utter bullshit. I think this quota will get worse the more AI is actually learning from AI content.
That said I think one may use AI for drafting, speeding up workflows or let the results inspire you to a certain degree, but only you could handle it, meaning if you feel capable of judging where AI is misleading you.
Your correct, I should mind my words: My rough estimation is that AI is producing statements to my input that seem meaningful to me in 60% of the cases, questionable in 25% and lead me to disagree and/or not knowing what the machine is talking about in 15%.
Nevertheless I’m surprised sometimes how that fragile hardware is capable in terms of autopoeitic self-organization.
I used Google’s Antigravity to draft the GUI grid view. It provided a usable PoC which allowed me to demonstrate the principle and validate is benefits. It have insight into what might work and what else to consider. All of these benefits were from my interpretation of the resulting code / operation.
I then recodedall of it because it was overly complex, verbose and did not align with the zynthian codebase.
I found LLMs a useful tool, much like a hammer which I would use to drive in a nail but not to design the building. LLMs rarely provide production ready code but can (and increasingly do) provide assistance to the developer.
Skilled developers can use it to automate mundane or repetitive tasks, solve awkward or unfamiliar problems, ease life (making then lazy*). Newbies can benefit from a step up with boilerplate code and assistance with subjects they are not yet familiar with. We all need to review the code and understand what is written.
I think it was Bill Gates who said, “I choose a lazy person to do a hard job. Because a lazy person will find an easy way to do it.”