In our conversations with market research professionals, the issue of AI bias and misinformation comes up frequently. At Bellomy, we’ve created controls around our AI to limit both potential threats. In this article, first we’ll explore how bias and misinformation occur in AI, and then we’ll outline Bellomy’s process to keep them in check.
How bias happens
The data an AI model has been trained on contributes to the potential of biased output in several ways, including the origin data’s framing and word choice, and over/under-representation, or complete omission, of demographics in origin data.
It’s important to note that humans are also prone to bias based on a variety of factors (including past experiences, social beliefs, upbringing and socioeconomic status), and our own bias can affect AI-output as well. For instance, prompts that seek affirmation of specific viewpoints might produce biased output.
Yes, AI hallucinates
Many AI tools are prone to what are known as “hallucinations,” where AI produces incorrect or nonsensical output. Hallucinations can range from small matters (perhaps the date a company formed) to quite extensive — sometimes entire paragraphs of misinformation. What’s more, AI does not separate fact from fiction, meaning that a researcher cannot detect hallucinations without manually fact-checking the AI output. In essence, AI is a very good liar in some situations.
The following factors typically contribute to hallucinations:
- An older large language model (LLM). ChatGPT3.5, for example, is better than GPT3, but GPT4 is much better than 3.5 and less likely to hallucinate.
- Truncated conversations. LLMs have limits to how much information they can process and produce. Sometimes during a long conversation, parts of the conversation are dropped so the AI model can continue to function.
- Poor prompts. AI models are as capable of producing creative fantasy and fiction as factual output. Because AI output is ultimately directed by user prompts, unclear requests can cause the AI to hallucinate. If a user asks the AI to perform a function without giving it adequate information or direction, it can assume the user wants false information as part of the response.
Throttling for bias and misinformation
There are several ways Bellomy exerts some control over AI output. Primary among them is the throttle in place to adjust the Bellomy AI Analytics for Text’s “creative freedom.” Along with the option of giving specific prompts around the data to be analyzed, you can apply one of five levels of creative freedom to the tool.
- None/Limited
- Very Limited
- Mid-Range
- High
- Max
Consider these levels as different “temperature settings” for the tool. At one end, “None/Limited” freedom instructs the AI to stick very closely to the information provided within a given dataset when performing its analysis. It is necessary for the AI to have some freedom to associate the data with the related concepts, but this option constrains the AI to a more focused boundary. For most use cases, combined with the prompts we have engineered, this eliminates the possibility of hallucinations and bias extrapolated from external information. You’d want this setting for things like examining customer feedback data or surveys that seek to examine sentiment or opinions about a topic, tool, or brand.
At the other end, “High” and “Max” levels of creative freedom give the AI permission to make “educated guesses” or extrapolate based on previously learned information or training. These settings essentially encourage the AI to use its imagination. This might be useful when doing exploratory research, seeking solutions to a problem, or looking for new product innovations and ideas.
Additional measures to ensure quality output
Other ways Bellomy reduces the risk of bias and misinformation:
- Always using the latest, most up-to-date LLM possible.
- Building a visual indicator into our AI Workflows that alerts users when a conversation has been truncated.
- Engineering prompts into our system that leverage the AI to address common weaknesses in human input. We extensively test and review these prompts to save our users time and ensure high-quality outputs.
- Finally, we train users on all aspects of the tool, including when to use the tool’s various temperature settings and best practices for prompts that produce the cleanest output. when to use the tool’s various temperature settings.
While bias and hallucinations will likely be a factor in AI output for the foreseeable future, the Bellomy team has established solid protocols so your user experience is much more likely to be free of their negative effects.
Justin Bailey is a member of Bellomy's Strategy Team. This article was written with input from Matthew Gullett, Bellomy SVP of Insights Technology.