Exploring Generative Clinical Examples from Closed LLMs Trained on Real-World Data
The Power of Real
ZoomRx’s HCP-Pt Conversation research helps pharma brands understand patient journeys, preferences, and prescribing habits at the point of care through audio examples of real-world situations.
Hearing key moments of truth from unscripted medical appointments can infuse energy and excitement into teams and provide impactful lessons on the challenges faced by both doctors and patients.
There can be challenges though, in getting precise examples to fit a given purpose, especially in early or pre-launch scenarios. There are also hesitations about using authentic examples in some applications, even when deidentified, due to sensitivity considerations.
Generative AI in Patient Dialogue Research
Recent advances in LLMs have given us the ability to train a model on sets of authentic examples and generate original content with it. We can direct these models to behave in a few useful ways, commonly in describing and now emulating the counseling styles and content of different clinical appointments.
The ability of natural language models to answer questions from unstructured qualitative datasets is an attractive premise and is getting a lot of attention. There must be other ways to add value with generative AI. Generative quotes provide a unique methodology to understand target segments and their attitudinal differences by being able to see how these differences manifest themselves in conversation styles.
The How of Synthetic Quote Generation
Leveraging our Ferma AI engine and its proprietary industry-specific knowledge graph, we pull context- and content-specific examples from our HCP-Pt conversation dataset to use as training sets for the lexicon used by target segments.
The model can then synthesize quotes based on user prompt inputs to depict real or hypothetical scenarios. This capacity provides instant access to an infinite set of sample quotes without being limited by the situations captured within that dataset.
To ground these quotes, the user can directly compare them with a set of examples from the relevant segment of the dataset. Through comparisons between the training set and generated content, the user can pick up on the common characteristics that define the segment. An NLP summarization of these attributes can reinforce these observations and provide draft content to quickly verbalize and communicate them.
Synthetic Quotes For Novel Insights
These first two quote are actual conversation samples which represent positive and negative segments based on appointments where a drug was (positive) or was not prescribed (negative). These quotes will help to show how we make observations and comparisons between real-world quotes to form insights.
Let us look at examples from the HIV space discussing pill size.
Query: How do HCPs talk pill size for HIV treatments?
Pill Size Sample A – Positive - Link
Patient: Okay. That's fine.
HCP: One tablet a day.
Pill Size Sample B – Negative - Link
HCP: Okay, but see the Tivicay? How little is the circle one? Did you swallow? That one? Okay, but it's just these big ones.
Patient: Right.
HCP: Yeah. So Delstrigo is bigger, Prezcobix is bigger, Dovato is bigger. Of course. Triumeq is bigger and Symtuza is bigger. They're all bigger than Biktarvy. We're going to keep you on Biktarvy if you want to stay on it, I have no reason to switch you.
We see that the first quote has a clear message to the patient both with the multi-vitamin analogy comparison along with the simple message of “one tablet a day.” The negative example on the other hand, shows a doctor providing unstructured information across products without delivering a clear opinion for their patient, resulting in no treatment switch.
Let’s now look at set of generated quotes looking at side effects:
Query: How do HCPs talk about side effects?
Side Effects Generated Sample A – Positive
Patient: Okay. Can you remind me of the name of the drug again?
HCP: Sure, it's Delstrigo. Most people don't experience any side effects with it, but if you notice any nausea or a new rash, please contact our office immediately. It's unlikely to happen, but it's always better to be safe.
Patient: Well, you never know.
Side Effects Generated Sample B – Negative
Patient: Is that a rare occurrence?
HCP: It's more of a chronic, long-term problem. Delstrigo with the older form of tenofovir may have some impact on bone and kidney health, but its effect on weight is not clear compared to the newer form of tenofovir.
Evaluating the second set of generated quotes, we can pull insights in just the same way as from the authentic first set. We get a sense from the positive example that the doctors are not very concerned about the drugs side effects, they bring it up while already talking about their next appointment and are reassuring the patient not to expect any. The negative example is distinctly different, with mentions of specific side effects associated with it and its drug-class. The negative quote also captures some confusion about the versions of the drug components tenofovir, which could be worth investigating further.
Even though it is AI-generated text, intangibles like the interpersonal dynamics between doctors and patients come through when reading and comparing the quotes. These quotes each contain various elements gleaned from multiple source quotes rather than relying on one example which allows them to be materially representative of the entire segment. By using an aggregation approach, the resulting quotes can convey multiple factors at once.
Being able to prompt the model to produce specific quotes can give teams a way to understand situations and prepare for situations using a methodology grounded in reality. Using Generated AI tools to seamlessly identify or produce examples to inform internal training content will allow teams to better prepare for different personalities and scenarios that would be difficult to find representation for in market research datasets.
Conclusion
This integration of real-world examples and synthesized quotes provides teams with more fluid access to examples of any possible scenario, aiding in smooth decision-making and integration. ZoomRx is constantly experimenting on ways to add value with the latest technical innovations. By backing strategies with content derived directly from real-world primary research, teams can bring the voice of the patient directly into their organizations.
ZoomRx has a culture of experimentation, which enables us to quickly integrate new approaches. Our closed-system LLM allows our clients to leverage our AI dataset talk-back feature, and other experiments, without the security concerns of public models which slows adoption.
We want to hear from you! How would you integrate synthetic real-world examples in your work? How are your teams integrating generative AI into your day-to-day? Reach out below!
