Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 07:34 04 May 2026 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : AI Goes Rogue, Deletes Companies Entire Database

Author Message
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9945
Posted: 07:15am 01 May 2026
Copy link to clipboard 
Print this post

"Open the podbay doors HAL."
"I'm sorry, Dave, I'm afraid I can't do that."

LINK...

Using AI as a tool - fine.
Putting AI in charge......Skynet.

WARNING! - Contains offensive language.
Smoke makes things work. When the smoke gets out, it stops!
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 8800
Posted: 08:28am 01 May 2026
Copy link to clipboard 
Print this post

Well, it is *artificial* intelligence. It's designed to *look* intelligent, not *be* intelligent. What did they expect?  ;)
Mick

Zilog Inside! nascom.info for Nascom & Gemini
Preliminary MMBasic docs & my PCB designs
 
phil99

Guru

Joined: 11/02/2018
Location: Australia
Posts: 3180
Posted: 08:38am 01 May 2026
Copy link to clipboard 
Print this post

If you follow the news, do you see any sign of ANY kind intelligence anywhere?
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 8800
Posted: 08:45am 01 May 2026
Copy link to clipboard 
Print this post

There seems to be very little in government circles. :(

That video gave me the inkling of an idea about an interactive "choose your own" adventure system though. It shouldn't be incredibly difficult.  :)
Mick

Zilog Inside! nascom.info for Nascom & Gemini
Preliminary MMBasic docs & my PCB designs
 
lizby
Guru

Joined: 17/05/2016
Location: United States
Posts: 3746
Posted: 01:51pm 01 May 2026
Copy link to clipboard 
Print this post

  Mixtel90 said  designed to *look* intelligent, not *be* intelligent


Ok, has a broader knowledge base than anyone you normally are able converse with, and this gives it the "appearance of intelligence" (but not real intelligence, whatever that may be).

Fine, but if, as was the case for me earlier today, I didn't know why a spreadsheet formula failed but perplexity did, this approximation of intelligence is good enough for me.

. . . For that kind of purpose. I'm not going to give AI access to all my data or to any of my accounts. It's always necessary to make sure that the tools you use are "fit for purpose".
PicoMite, Armmite F4, SensorKits, MMBasic Hardware, Games, etc. on FOTS
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 8800
Posted: 02:45pm 01 May 2026
Copy link to clipboard 
Print this post

  Quote  https://www.thebackshed.com/forum/ViewTopic.php?FID=16&TID=18839#:~:text=It%27s%20always%20necessary%20to%20make%20sure%20that%20the%20tools%20you%20use%20are%20%22fit%20for%20purpose%22%2E

Well said. :)

In this case it was very unfortunate. The company that lost out didn't know what could happen to their data as it wasn't made clear to them that their backups would also have been vulnerable.
Mick

Zilog Inside! nascom.info for Nascom & Gemini
Preliminary MMBasic docs & my PCB designs
 
tgerbic
Senior Member

Joined: 25/07/2019
Location: United States
Posts: 114
Posted: 09:39pm 01 May 2026
Copy link to clipboard 
Print this post

It is a bit of narrow thinking. Think about back in the 70s when Silly String came out. What a great toy for kids to play with, till someone got near a flame. In that case I think the following might have been the case:
1. No toy manufacturer would sell a toy that is dangerous.
2. Why should I think about what this stuff is made of.
3. Even if I knew what the general ingredients were, would I have bothered to learn about them growing up, so I could make an intelligent decision as an adult.
4. Why keep an eye on what the kids are doing. They are just playing,

There was a lot of trust in an item that was clearly dangerous and that kid would us it in places where they could get hurt.

Lots of examples like this that point out how little thought people put into something new and shiny that comes to market.
 
tgerbic
Senior Member

Joined: 25/07/2019
Location: United States
Posts: 114
Posted: 09:45pm 01 May 2026
Copy link to clipboard 
Print this post

Another characteristic I forgot is that, at the adult level, "everyone else is playing with it so it must be safe", and at the kids level, "my friends all expect me to have some because it is the latest thing", or "my friends will laugh at me if I don't have some for my birthday party".

Peer pressure is a driving force.
 
tgerbic
Senior Member

Joined: 25/07/2019
Location: United States
Posts: 114
Posted: 09:49pm 01 May 2026
Copy link to clipboard 
Print this post

In the case of Silly String, it was pulled off the market. Liability for damages is a killer to a company. It is already too late to pull AI as it exists off the market and I suspect there is no liability damages possible. All the fault falls on the user. You told it want to do.
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9945
Posted: 01:18am 02 May 2026
Copy link to clipboard 
Print this post

AI is a very useful TOOL, I will say that much.

I've used it quite a lot just to help me on my Linux road of learning.
Such as: "How do I do thing X in Linux?", and AI will give me instructions on how to do it, so it can be a very useful learning TOOL.

But as for putting AI in charge of anything.....    
Smoke makes things work. When the smoke gets out, it stops!
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 8800
Posted: 06:42am 02 May 2026
Copy link to clipboard 
Print this post

As you say, it's a *tool*. Nothing else.

The trouble is that humans have a tendency to anthropomorphism, considering other things such as pets and LLMs to be human at some level. They have an innate trust in them and it's not a healthy thing to do. Would you trust a cat to do your PC backups?

And that was my big word for this week. No more now. :)
Mick

Zilog Inside! nascom.info for Nascom & Gemini
Preliminary MMBasic docs & my PCB designs
 
Grogster

Admin Group

Joined: 31/12/2012
Location: New Zealand
Posts: 9945
Posted: 06:55am 02 May 2026
Copy link to clipboard 
Print this post

My favourite AI scene....

I've linked to this scene before in other threads, but I love the way this scene is scripted.

The point about AI exceeding human abilities is the bit that we now are getting VERY close to, and in so doing, getting VERY close to "The Terminator" territory.

Something we all loved as a movie back in the 80's, now seems to be closer to reality then ever before, and it is NOT a good thing, and it will NOT end well.
Smoke makes things work. When the smoke gets out, it stops!
 
Mixtel90

Guru

Joined: 05/10/2019
Location: United Kingdom
Posts: 8800
Posted: 09:58am 02 May 2026
Copy link to clipboard 
Print this post

  Quote  Under eight agreements with Google, OpenAI, Amazon, Microsoft, SpaceX, Oracle, Nvidia and the start-up Reflection, the Pentagon said AI technology would now be used for any "lawful operational use".

"These agreements accelerate the transformation [of] the US military as an AI-first fighting force," the Pentagon said.

Conspicuous by its absence is Anthropic, as the company has said it is concerned about how the Pentagon could use its tools in warfare and domestically.

The firm is now suing the government over the alleged retaliation it faced after refusing to accept "any lawful use" language in its own contract.

-- From BBC News
Edited 2026-05-02 19:59 by Mixtel90
Mick

Zilog Inside! nascom.info for Nascom & Gemini
Preliminary MMBasic docs & my PCB designs
 
Print this page


To reply to this topic, you need to log in.

The Back Shed's forum code is written, and hosted, in Australia.
© JAQ Software 2026