Software and Computer Reliability

Any other tech-related topics
Post Reply
Message
Author
User avatar
juverax
Posts: 355
Joined: Mon Jun 11, 2018 5:19 am

Software and Computer Reliability

#1 Post by juverax »

Software and Computer Reliability
....
"The presumption that computer evidence is correct is based on a naïve and simplistic understanding of software systems. Large systems are complex and lay people cannot discern whether these systems are reliable or be confident that they can spot errors as they happen. It is difficult even for experts to judge the reliability of systems or detect any but the simplest errors."
in
https://en.wikipedia.org/wiki/British_P ... ce_scandal
and
https://www.theguardian.com/commentisfr ... al-inquiry

In brief ...people were accused of theft and their lives ruined, when actually the culprit was a defective computer/software accounting application !

.... and what about Artificial Intelligence ???

User avatar
Andrew Lee
Posts: 3048
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Software and Computer Reliability

#2 Post by Andrew Lee »

Frankly, AI scares me. Not that I am a luddite, I think I have a pretty good idea of how LLM (Large Language Models) work, and their shortcomings. But the problem is, your layperson does not, and that's a big problem. When more and more people start to delegate to AI without checking its output, you are going to have unforeseen problems.

Right now, LLMs are just outputting sentences. But as they start to control APIs and robotics autonomously without checks, bad things will happen.

I get the "it's just a tool" argument. I think it's a great tool to help scientists, doctors or engineers improve productivity. But when you get laypersons to start eg. talking to AI doctors and take in their recommendations without expert verification... am I the only one who thinks that's a bad idea? :?

User avatar
SYSTEM
Posts: 2041
Joined: Sat Jul 31, 2010 1:19 am
Location: Helsinki, Finland

Re: Software and Computer Reliability

#3 Post by SYSTEM »

Andrew Lee wrote: Tue May 02, 2023 9:06 pm I get the "it's just a tool" argument. I think it's a great tool to help scientists, doctors or engineers improve productivity. But when you get laypersons to start eg. talking to AI doctors and take in their recommendations without expert verification... am I the only one who thinks that's a bad idea? :?
You're not. An AI doctor could indeed be useful, both for convenience and because some people may feel more comfortable not disclosing their condition to a real human being, but an AI can hallucinate completely wrong answers and, which could outright endanger people's lives.
My YouTube channel | Release date of my 13th playlist: August 24, 2020

User avatar
Midas
Posts: 6705
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: Software and Computer Reliability

#4 Post by Midas »

Like the joke goes, it's not so much Artificial Intelligence that scares me but natural stupidity... :shock:

It has been shown time and time again and beyond any reasonable doubt that automated systems can be outright biased, maliciously manipulated or just plain faulty. And yet everyone and their cousin seems to be intent on blindly transferring highly critical missions & functions to systems that are basically unfathomable. SkyNet, anyone?


User avatar
Midas
Posts: 6705
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: Software and Computer Reliability

#5 Post by Midas »


User avatar
Andrew Lee
Posts: 3048
Joined: Sat Feb 04, 2006 9:19 am
Contact:

Re: Software and Computer Reliability

#6 Post by Andrew Lee »

Midas wrote: Wed May 03, 2023 5:07 am Like the joke goes, it's not so much Artificial Intelligence that scares me but natural stupidity... :shock:
Haha! I dig that :lol:

AI versus natural stupidity, it's hard for me to decide which is scarier..

User avatar
Midas
Posts: 6705
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Re: Software and Computer Reliability

#7 Post by Midas »

Often times, reliability issues can be caused by malicious 3rd parties vying for unlawful entry so the following may be somewhat relevant:
Using stealth techniques, the attacker –- referred to as "Volt Typhoon" –- exploited existing resources in compromised networks in a technique called "living off the land". Microsoft made a concurrent announcement, stating the attackers' targeting of Guam was telling of China's plans to potentially disrupt critical communications infrastructure between the US and Asia region in the future.

Here's the original source with an in-depth analysis of the attacker's modus operandi (PDF):
And another similar but unrelated event affecting WebArchive.org:

Post Reply