Page 1 of 1

Software and Computer Reliability

Posted: Tue May 02, 2023 8:04 pm
by juverax
Software and Computer Reliability
....
"The presumption that computer evidence is correct is based on a naïve and simplistic understanding of software systems. Large systems are complex and lay people cannot discern whether these systems are reliable or be confident that they can spot errors as they happen. It is difficult even for experts to judge the reliability of systems or detect any but the simplest errors."
in
https://en.wikipedia.org/wiki/British_P ... ce_scandal
and
https://www.theguardian.com/commentisfr ... al-inquiry

In brief ...people were accused of theft and their lives ruined, when actually the culprit was a defective computer/software accounting application !

.... and what about Artificial Intelligence ???

Re: Software and Computer Reliability

Posted: Tue May 02, 2023 9:06 pm
by Andrew Lee
Frankly, AI scares me. Not that I am a luddite, I think I have a pretty good idea of how LLM (Large Language Models) work, and their shortcomings. But the problem is, your layperson does not, and that's a big problem. When more and more people start to delegate to AI without checking its output, you are going to have unforeseen problems.

Right now, LLMs are just outputting sentences. But as they start to control APIs and robotics autonomously without checks, bad things will happen.

I get the "it's just a tool" argument. I think it's a great tool to help scientists, doctors or engineers improve productivity. But when you get laypersons to start eg. talking to AI doctors and take in their recommendations without expert verification... am I the only one who thinks that's a bad idea? :?

Re: Software and Computer Reliability

Posted: Tue May 02, 2023 11:24 pm
by SYSTEM
Andrew Lee wrote: Tue May 02, 2023 9:06 pm I get the "it's just a tool" argument. I think it's a great tool to help scientists, doctors or engineers improve productivity. But when you get laypersons to start eg. talking to AI doctors and take in their recommendations without expert verification... am I the only one who thinks that's a bad idea? :?
You're not. An AI doctor could indeed be useful, both for convenience and because some people may feel more comfortable not disclosing their condition to a real human being, but an AI can hallucinate completely wrong answers and, which could outright endanger people's lives.

Re: Software and Computer Reliability

Posted: Wed May 03, 2023 5:07 am
by Midas
Like the joke goes, it's not so much Artificial Intelligence that scares me but natural stupidity... :shock:

It has been shown time and time again and beyond any reasonable doubt that automated systems can be outright biased, maliciously manipulated or just plain faulty. And yet everyone and their cousin seems to be intent on blindly transferring highly critical missions & functions to systems that are basically unfathomable. SkyNet, anyone?


Re: Software and Computer Reliability

Posted: Wed May 03, 2023 5:29 am
by Midas

Re: Software and Computer Reliability

Posted: Wed May 03, 2023 8:19 pm
by Andrew Lee
Midas wrote: Wed May 03, 2023 5:07 am Like the joke goes, it's not so much Artificial Intelligence that scares me but natural stupidity... :shock:
Haha! I dig that :lol:

AI versus natural stupidity, it's hard for me to decide which is scarier..

Re: Software and Computer Reliability

Posted: Mon May 29, 2023 9:09 am
by Midas
Often times, reliability issues can be caused by malicious 3rd parties vying for unlawful entry so the following may be somewhat relevant:
Using stealth techniques, the attacker –- referred to as "Volt Typhoon" –- exploited existing resources in compromised networks in a technique called "living off the land". Microsoft made a concurrent announcement, stating the attackers' targeting of Guam was telling of China's plans to potentially disrupt critical communications infrastructure between the US and Asia region in the future.
theconversation.com/the-highly-secretive-five-eyes-alliance-has-disrupted-a-china-backed-hacker-group-in-an-unusually-public-manner-206403

Here's the original source with an in-depth analysis of the attacker's modus operandi (PDF):
And another similar but unrelated event affecting WebArchive.org: