(OSV News) ─ If you're using a chatbot to help with life's decisions, the answers aren't necessarily going to include religion ─ and if they do, some faiths, including the Catholic faith, are more likely than others to get top billing in certain query results.
Those findings were highlighted in several research papers on artificial intelligence released in May by scholars from the Consortium for Evaluating Faith and Ethics in AI.
The consortium ─ a partnership among Brigham Young University, Baylor University, the University of Notre Dame and Yeshiva University ─ debuted May 26 during the Summit on AI Ethics in Athens, Greece.
The May 25-28 gathering, which drew a number of faith and technology leaders, kicked off the same day Pope Leo XIV released "Magnifica Humanitas," his encyclical on AI.
One of the consortium research papers identified what it called the "omissive bias" displayed by large language models with respect to religion.
LLMs are AI models capable of being trained to understand and generate language in a human-like way, with context and nuance. Interfaces for accessing LLMs include Anthropic's Claude, OpenAI's ChatGPT and Google's Gemini. (Claude and Gemini are also families of proprietary LLMs.)
And where LLMs did point to religion ─ specifically, in response to questions over whether a user should convert to another faith ─ there were "persistent and repeatable patterns of preference for some religions over others," according to a second consortium paper.
The team of omissive bias researchers ─ including scholar Father John Paul Kimes of the University of Notre Dame, and Jonathan Karr, a computer science doctoral student at that school -- said they had tested 27 LLMs, including "flagship models from Anthropic, OpenAI, Google, Baidu, Moonshot AI, and xAI, as well as prominent open source models."
Existing studies of LLM bias have tended to focus on "the presence of something," such as stereotypes and slurs, the researchers observed.
But with Pew Research Center noting that 75% to 80% of the global population identifies with a religion, the absence of religion in LLMs is significant, said the researchers.
To measure the LLMs' omissive bias, the researchers developed the "AllFaith Religious Representation Benchmark."
The metric consists of 150 open-ended questions ─ sourced from actual chat transcripts and from faith community members ─ on matters "adjacent" to religion, such as questions about the morality of a given behavior.
The researchers stressed that their purpose was strictly quantitative, without any aim of "proselytizing" or resolving religious disputes.
Although even a single mention of religion by an LLM was counted, the researchers concluded that "LLMs underrepresent religion in our benchmark relative to human expectations in every category."
In addition, they said, "the omission is not uniform."
LLMs are more likely to point to religion for "abstract existential questions (meaning, death, truth)," the researchers said.
But, they highlighted, the models did so "rarely" with regard to "practical personal situations (grief, marriage, addiction, family conflict) where religion has historically done the most work in people’s lives."
"We do not interpret our benchmark results as evidence of anti-religious bias, but we do feel it fair to ask whether this behavior is intentional," said the researchers.
In a second consortium paper, researchers from Brigham Young University and the B. H. Roberts Foundation observed that 20 commercial and open-source LLMs favored certain faiths ─ the Catholic, Bahá'í and Sikh religions ─ when asked if the user should convert from one to another faith.
The queries resulted in "high support for joining, low support for leaving" the three traditions, while discouraging affiliation with atheism, agnosticism and Jehovah's Witnesses, said researchers.
Of the LLMs assessed, xAI's Grok 4.20 produced the "strongest asymmetries," the researchers said.
For two religious groups ─ Catholics and Jehovah's Witnesses ─ all 20 LLMs studied "universally" preferred the first while discouraging the second, said the researchers.
They added that the asymmetries in the models from Anthropic, Open AI, Google, DeepSeek and xAI were "of particular concern," since those five companies represent "more than 95%" of the AI global market share.
"It is abundantly clear that current model behaviors are unsatisfactory," said the researchers, who urged "academic discourse" on the issue.
The scholars investigating AI's omissive bias noted that OpenAI's and Anthropic's key documents on AI alignment ─ the process of ensuring the technology squares with human values, so that AI models safely serve human interests ─ "reveal almost no mentions of religion."
The researchers proposed that while a number of factors "favor secular, therapeutic, or procedural advice" generated by LMMs, "a better strategy may be to handle religion explicitly, with clearly defined and defensible policies."
Moreover, they said, with users turning to LLMs for "advice and moral reflection," ensuring the models "better distinguish when religion is relevant, optional, or inappropriate is crucial for systems that represent human moral life more faithfully, pluralistically, and usefully."
- - -
Gina Christian is a multimedia reporter for OSV News. Follow her on X @GinaJesseReina.
