Nostr notes by Benjamin Paassen

Yes, happy to. It's this one: ...

2026-10-07T19:43:36Z

In reply to nevent1q…5xw6
_________________________

Yes, happy to. It's this one: https://doi.org/10.1037/tmb0000210

But our core argument still stands and has survived peer review. ...

2026-10-06T18:30:21Z

In reply to nevent1q…3km8
_________________________

But our core argument still stands and has survived peer review. So to cite our conclusion: For research questions that aim to characterize human judgments, experiences, or behavior, however, human participants cannot be replaced.

2. We have strongly revised our theory section, incorporating ...

2026-10-06T18:30:17Z

In reply to nevent1q…2s8q
_________________________

2. We have strongly revised our theory section, incorporating more arguments now and referencing cognitive architecture work.
3. We have a lot more supplementary analysis to look deeper into the patterns in our data.

And, sadly, our title had to become much less concise.

1. We have now evaluated more LLMs, including more recent ones. ...

2026-10-06T18:30:01Z

In reply to nevent1q…yyly
_________________________

1. We have now evaluated more LLMs, including more recent ones. Our core effect is still maintained: under rewording, the correlation between human responses and LLM responses significantly reduces. But we also find differences between LLMs: Older and smaller LLMs tend to just ignore the rewordings, whereas newer LLMs do react to them - but not in the same way humans do.

I want to thank all authors for their tireless work on this - but ...

2026-10-06T18:28:25Z

In reply to nevent1q…dcc0
_________________________

I want to thank all authors for their tireless work on this - but also our reviewers for some helpful discussion and feedback. Some highlights of what we changed since the preprint:

Our core argument is that LLMs should not be regarded as reliable ...

2026-10-06T18:28:08Z

In reply to nevent1q…epek
_________________________

Our core argument is that LLMs should not be regarded as reliable simulators of human participants because subtle changes to the input text can lead to very different reactions from humans and LLMs. This is also shown in our key figure below. The x axis corresponds to different moral scenarios, the y axis to the reaction if we subtly change the text of the scenario. You can see that the reactions of LLMs and humans are clearly very different for most scenarios (exact correlations in the paper).

Well, it turns out, it has. The journal "Technology, Mind, ...

2026-10-06T18:27:40Z

In reply to nevent1q…xfd4
_________________________

Well, it turns out, it has. The journal "Technology, Mind, and Behavior" just published our paper under the new title "Exploring the Limits of Large Language Models as a Reflection of Human Cognition: An Illustration in the Context of Moral Judgment".

https://doi.org/10.1037/tmb0000210

Over one year ago, we got a lot of response for our preprint ...

2026-10-06T18:27:36Z

Over one year ago, we got a lot of response for our preprint "Large language models do not simulate human psychology". While psychologists seemed to be somewhat puzzled that such claims are made in the first place, some colleagues from computer science or application reacted quite skeptically to our claim - we even got some comments that our paper would never make it through peer review.

I agree with that. One sometimes also can hear/read/see how ...

2026-03-05T09:59:31Z

In reply to nevent1q…77tu
_________________________

I agree with that. One sometimes also can hear/read/see how scientists try to make the role they are currently taking more explicit - but it is still imperfect, I feel, and lines can be blurry.

For myself, I find it at least somewhat helpful to think of ...

2026-03-05T09:53:24Z

In reply to nevent1q…gf7u
_________________________

For myself, I find it at least somewhat helpful to think of science as informing us about the consequences (including uncertainty estimates, ranges, etc.) of political decisions but politics has to make the decision itself. In that sense, science is "apolitical" or "neutral". But a lot of political actors try to occlude/mislead about the consequences of their political program. And in such cases, I think, science has to take a clear stance in opposition.

But there are also plenty of criticisms. Empirically, we know ...

2025-08-12T15:09:16Z

In reply to nevent1q…dmt2
_________________________

But there are also plenty of criticisms. Empirically, we know that LLMs are biased (in different ways than humans), their answers vary much less compared to human answers, and they make mistakes that humans wouldn't make. (5/12)

Some papers claim that. Dillion et al. (2023) let LLMs respond to ...

2025-08-12T15:09:11Z

In reply to nevent1q…5l6d
_________________________

Some papers claim that. Dillion et al. (2023) let LLMs respond to morality questions and they did indeed respond like average humans. Binz et al. (2025) fine-tuned an LLM to respond like humans to psych questions. (4/12)

https://doi.org/10.1016/j.tics.2023.04.008

https://doi.org/10.1038/s41586-025-09215-4

LLMs as participants can only work if LLMs really respond like ...

2025-08-12T15:08:14Z

In reply to nevent1q…57m2
_________________________

LLMs as participants can only work if LLMs really respond like humans to psychological questions. Do they? (3/12)

Our starting point is a trend in psychology: Recruiting human ...

2025-08-12T15:08:10Z

In reply to nevent1q…l6fr
_________________________

Our starting point is a trend in psychology: Recruiting human participants for psychological studies is a lot of effort and expensive. Wouldn't it be great if we could just use LLMs as participants, instead? N = 1000 for just a few dollars. (2/12)

Large Language models (LLMs) do not simulate human psychology. ...

2025-08-12T15:08:06Z

Large Language models (LLMs) do not simulate human psychology. That's the title of our new paper, available as preprint today (1/12):

https://arxiv.org/abs/2508.06950