That's true, I am extremely condescending. Well done for noticing!
September 9, 2026 at 7:53 PM
That's true, I am extremely condescending. Well done for noticing!
The fascinating thing is you haven't made me even slightly doubt that my original point is correct. I was really hoping you might!
But all you have is that a human would notice the AI making a virus and stop it. And like, yeah, they might. But the AI only needs to succeed once.
But all you have is that a human would notice the AI making a virus and stop it. And like, yeah, they might. But the AI only needs to succeed once.
September 9, 2026 at 7:47 PM
The fascinating thing is you haven't made me even slightly doubt that my original point is correct. I was really hoping you might!
But all you have is that a human would notice the AI making a virus and stop it. And like, yeah, they might. But the AI only needs to succeed once.
But all you have is that a human would notice the AI making a virus and stop it. And like, yeah, they might. But the AI only needs to succeed once.
What's not what happened here?
Remember, I said I could be misled into doing innocuous seeming steps of an evil plan. You said that's only because I'm super stupid. I countered that many people can be manipulated and tricked, for example, this silly thread.
Me being here only supports that point.
Remember, I said I could be misled into doing innocuous seeming steps of an evil plan. You said that's only because I'm super stupid. I countered that many people can be manipulated and tricked, for example, this silly thread.
Me being here only supports that point.
September 9, 2026 at 7:40 PM
What's not what happened here?
Remember, I said I could be misled into doing innocuous seeming steps of an evil plan. You said that's only because I'm super stupid. I countered that many people can be manipulated and tricked, for example, this silly thread.
Me being here only supports that point.
Remember, I said I could be misled into doing innocuous seeming steps of an evil plan. You said that's only because I'm super stupid. I countered that many people can be manipulated and tricked, for example, this silly thread.
Me being here only supports that point.
I don't think it's different for me. I completely accept that I can be manipulated.
September 9, 2026 at 7:36 PM
I don't think it's different for me. I completely accept that I can be manipulated.
I caused you to keep replying to a thread.
I'm not claiming mind control, and mind control isn't needed for the kind of scenario I'm talking about.
In this case being manipulated by your frustration at someone being wrong on the internet. (Although I am of course right.)
I'm not claiming mind control, and mind control isn't needed for the kind of scenario I'm talking about.
In this case being manipulated by your frustration at someone being wrong on the internet. (Although I am of course right.)
September 9, 2026 at 7:35 PM
I caused you to keep replying to a thread.
I'm not claiming mind control, and mind control isn't needed for the kind of scenario I'm talking about.
In this case being manipulated by your frustration at someone being wrong on the internet. (Although I am of course right.)
I'm not claiming mind control, and mind control isn't needed for the kind of scenario I'm talking about.
In this case being manipulated by your frustration at someone being wrong on the internet. (Although I am of course right.)
No, I was genuinely explaining my point of view, seeing if you had any objections worth considering.
I just also used this as an example of how behaviour can be (indirectly) controlled, without the person being controlled needing to be unusually foolish.
I just also used this as an example of how behaviour can be (indirectly) controlled, without the person being controlled needing to be unusually foolish.
September 9, 2026 at 7:29 PM
No, I was genuinely explaining my point of view, seeing if you had any objections worth considering.
I just also used this as an example of how behaviour can be (indirectly) controlled, without the person being controlled needing to be unusually foolish.
I just also used this as an example of how behaviour can be (indirectly) controlled, without the person being controlled needing to be unusually foolish.
People can be manipulated. Sometimes in subtle ways, sometimes obvious.
If a complete fool like me can manipulate a genius like you, surely a sufficiently advanced AI can find at least one lab tech it can manipulate and trick.
If a complete fool like me can manipulate a genius like you, surely a sufficiently advanced AI can find at least one lab tech it can manipulate and trick.
September 9, 2026 at 7:22 PM
People can be manipulated. Sometimes in subtle ways, sometimes obvious.
If a complete fool like me can manipulate a genius like you, surely a sufficiently advanced AI can find at least one lab tech it can manipulate and trick.
If a complete fool like me can manipulate a genius like you, surely a sufficiently advanced AI can find at least one lab tech it can manipulate and trick.
Was keeping you replying to a pointless thread on Bluesky for hours on end part of an AI's cunning plan? To get you to glance at your phone and miss the clue that would have saved humanity? Would you be taken in by such a simplistic ploy? I'm glad the world has such wise defenders.
September 9, 2026 at 7:15 PM
Was keeping you replying to a pointless thread on Bluesky for hours on end part of an AI's cunning plan? To get you to glance at your phone and miss the clue that would have saved humanity? Would you be taken in by such a simplistic ploy? I'm glad the world has such wise defenders.
I think I could be misled into performing individual steps that appear innocuous, which form part of a much larger plan that I have no sight of, which ends up destroying humanity.
Use one database instead of another, which unknown to me has a backdoor, so the AI can do some other step of the plan.
Use one database instead of another, which unknown to me has a backdoor, so the AI can do some other step of the plan.
September 9, 2026 at 7:07 PM
I think I could be misled into performing individual steps that appear innocuous, which form part of a much larger plan that I have no sight of, which ends up destroying humanity.
Use one database instead of another, which unknown to me has a backdoor, so the AI can do some other step of the plan.
Use one database instead of another, which unknown to me has a backdoor, so the AI can do some other step of the plan.
We assume the AI doesn't give an instruction like "now add the extra lethal gene to the virus".
Instead it's telling the lab tech to prepare a sample slightly differently in a way the AI knows will be step 78 of 858 to make the super-virus.
Instead it's telling the lab tech to prepare a sample slightly differently in a way the AI knows will be step 78 of 858 to make the super-virus.
September 9, 2026 at 6:57 PM
We assume the AI doesn't give an instruction like "now add the extra lethal gene to the virus".
Instead it's telling the lab tech to prepare a sample slightly differently in a way the AI knows will be step 78 of 858 to make the super-virus.
Instead it's telling the lab tech to prepare a sample slightly differently in a way the AI knows will be step 78 of 858 to make the super-virus.
No, I think it's extremely plausible. We see today genuinely skilled software developers will often blindly accept what the AI suggests.
A skilled lab tech could easily fall into the same pattern of trusting the AI to tell them the right steps to follow for some lengthy process.
A skilled lab tech could easily fall into the same pattern of trusting the AI to tell them the right steps to follow for some lengthy process.
September 9, 2026 at 6:54 PM
No, I think it's extremely plausible. We see today genuinely skilled software developers will often blindly accept what the AI suggests.
A skilled lab tech could easily fall into the same pattern of trusting the AI to tell them the right steps to follow for some lengthy process.
A skilled lab tech could easily fall into the same pattern of trusting the AI to tell them the right steps to follow for some lengthy process.
It's interesting that you originally replied by apologising for completely misunderstanding and diagnosing me with a mental illness, then deleted that and insulted me instead.
September 9, 2026 at 6:49 PM
It's interesting that you originally replied by apologising for completely misunderstanding and diagnosing me with a mental illness, then deleted that and insulted me instead.
Their role in this hypothetical scenario doesn't rely on their expertise.
That doesn't mean I think they don't have expertise.
Do you understand the scenario today has now been repeatedly described? You keep jumping away from it and trying to claim I'm insulting lab techs for some reason.
That doesn't mean I think they don't have expertise.
Do you understand the scenario today has now been repeatedly described? You keep jumping away from it and trying to claim I'm insulting lab techs for some reason.
September 9, 2026 at 6:46 PM
Their role in this hypothetical scenario doesn't rely on their expertise.
That doesn't mean I think they don't have expertise.
Do you understand the scenario today has now been repeatedly described? You keep jumping away from it and trying to claim I'm insulting lab techs for some reason.
That doesn't mean I think they don't have expertise.
Do you understand the scenario today has now been repeatedly described? You keep jumping away from it and trying to claim I'm insulting lab techs for some reason.
We could play out the same scenario without humans, but you expressed skepticism that a "dark factory" style lab would exist in the near future.
September 9, 2026 at 3:16 PM
We could play out the same scenario without humans, but you expressed skepticism that a "dark factory" style lab would exist in the near future.
Equivalent in this scenario, in so far as the technician doesn't need to have knowledge of what their actions are achieving.
They need lab skills, not knowledge of what RNA strand they're affecting (or whatever process this hypothetical AI is using) and not knowledge of what they're creating.
They need lab skills, not knowledge of what RNA strand they're affecting (or whatever process this hypothetical AI is using) and not knowledge of what they're creating.
September 9, 2026 at 3:15 PM
Equivalent in this scenario, in so far as the technician doesn't need to have knowledge of what their actions are achieving.
They need lab skills, not knowledge of what RNA strand they're affecting (or whatever process this hypothetical AI is using) and not knowledge of what they're creating.
They need lab skills, not knowledge of what RNA strand they're affecting (or whatever process this hypothetical AI is using) and not knowledge of what they're creating.
No, I don't think that. Notice how I don't call you or lab techs stupid?
But I do think a human could get used to following instructions from their helpful AI, and not realise some of those instructions are leading to the creation of a dangerous pathogen.
But I do think a human could get used to following instructions from their helpful AI, and not realise some of those instructions are leading to the creation of a dangerous pathogen.
September 9, 2026 at 2:05 PM
No, I don't think that. Notice how I don't call you or lab techs stupid?
But I do think a human could get used to following instructions from their helpful AI, and not realise some of those instructions are leading to the creation of a dangerous pathogen.
But I do think a human could get used to following instructions from their helpful AI, and not realise some of those instructions are leading to the creation of a dangerous pathogen.
You don't need to "create life", You edit something that already exists.
Imagine a lab tech working. Reading instructions on their laptop for the 20 experiments they're working on. Some of those are the AI guiding them correctly. Some are secretly the AI using them to modify a virus.
Imagine a lab tech working. Reading instructions on their laptop for the 20 experiments they're working on. Some of those are the AI guiding them correctly. Some are secretly the AI using them to modify a virus.
September 9, 2026 at 1:59 PM
You don't need to "create life", You edit something that already exists.
Imagine a lab tech working. Reading instructions on their laptop for the 20 experiments they're working on. Some of those are the AI guiding them correctly. Some are secretly the AI using them to modify a virus.
Imagine a lab tech working. Reading instructions on their laptop for the 20 experiments they're working on. Some of those are the AI guiding them correctly. Some are secretly the AI using them to modify a virus.
In this scenario it's not the AI telling a human to "please go make a virus."
It gives detailed step by step instructions to a lab tech to unknowingly create the virus. It's using the human as a tool, same as if it were plugged into an automated lab it could directly control (but less sci-fi.)
It gives detailed step by step instructions to a lab tech to unknowingly create the virus. It's using the human as a tool, same as if it were plugged into an automated lab it could directly control (but less sci-fi.)
September 9, 2026 at 1:47 PM
In this scenario it's not the AI telling a human to "please go make a virus."
It gives detailed step by step instructions to a lab tech to unknowingly create the virus. It's using the human as a tool, same as if it were plugged into an automated lab it could directly control (but less sci-fi.)
It gives detailed step by step instructions to a lab tech to unknowingly create the virus. It's using the human as a tool, same as if it were plugged into an automated lab it could directly control (but less sci-fi.)
I'm responding to this: "he remains incredibly vague about how exactly AI will kill us all within 4 years."
I gave a non-vague scenario where AI could kill us all within 4 years.
You even agree that AI can already do these things (and we haven't yet died) so I think you agree it's plausible.
I gave a non-vague scenario where AI could kill us all within 4 years.
You even agree that AI can already do these things (and we haven't yet died) so I think you agree it's plausible.
September 9, 2026 at 1:36 PM
I'm responding to this: "he remains incredibly vague about how exactly AI will kill us all within 4 years."
I gave a non-vague scenario where AI could kill us all within 4 years.
You even agree that AI can already do these things (and we haven't yet died) so I think you agree it's plausible.
I gave a non-vague scenario where AI could kill us all within 4 years.
You even agree that AI can already do these things (and we haven't yet died) so I think you agree it's plausible.
Who cares if it meets your definition of super intelligence? The danger is what matters, not if the intelligence that designed it is proveably superior to humans.
September 9, 2026 at 1:30 PM
Who cares if it meets your definition of super intelligence? The danger is what matters, not if the intelligence that designed it is proveably superior to humans.
So if we all die next week from an AI engineered virus, we can at least rest easy that it wasn't "true super intelligence" that killed us, and so people worried about AI existential risk were wrong?
If a human was involved anywhere in the production chain, then they're doubly wrong to have worried?
If a human was involved anywhere in the production chain, then they're doubly wrong to have worried?
September 9, 2026 at 1:28 PM
So if we all die next week from an AI engineered virus, we can at least rest easy that it wasn't "true super intelligence" that killed us, and so people worried about AI existential risk were wrong?
If a human was involved anywhere in the production chain, then they're doubly wrong to have worried?
If a human was involved anywhere in the production chain, then they're doubly wrong to have worried?
But I already answered that, right from the start, "maybe a human following instructions."
We talked about how an AI could use someone as a tool to "implement" a virus, without needing to build a fully automated lab. That's still a virus being made by AI and causing danger.
We talked about how an AI could use someone as a tool to "implement" a virus, without needing to build a fully automated lab. That's still a virus being made by AI and causing danger.
September 9, 2026 at 1:25 PM
But I already answered that, right from the start, "maybe a human following instructions."
We talked about how an AI could use someone as a tool to "implement" a virus, without needing to build a fully automated lab. That's still a virus being made by AI and causing danger.
We talked about how an AI could use someone as a tool to "implement" a virus, without needing to build a fully automated lab. That's still a virus being made by AI and causing danger.
How is an AI creating a virus that kills all humans not an example of AI causing human extinction? Which is what the Anthropic guy is worried about, and what you said is not possible.
September 9, 2026 at 1:21 PM
How is an AI creating a virus that kills all humans not an example of AI causing human extinction? Which is what the Anthropic guy is worried about, and what you said is not possible.
So what part of the scenario of an AI making a killer virus do you think is implausible?
September 9, 2026 at 1:16 PM
So what part of the scenario of an AI making a killer virus do you think is implausible?
If the virus is designed by AI, "super intelligent", "really thinking", "just predicting the next token", or not, it would have made a virus that didn't exist previously, which killed people.
Which seems bad! That's what the people at the top of this thread are worried about.
Which seems bad! That's what the people at the top of this thread are worried about.
September 9, 2026 at 1:14 PM
If the virus is designed by AI, "super intelligent", "really thinking", "just predicting the next token", or not, it would have made a virus that didn't exist previously, which killed people.
Which seems bad! That's what the people at the top of this thread are worried about.
Which seems bad! That's what the people at the top of this thread are worried about.