Medication Counseling with Large Language Models: Balancing Flexibility and Rigidity

The introduction of large language models (LLMs)
has greatly enhanced the capabilities of software agents. Instead
of relying on rule-based interactions, agents can now interact in
flexible ways akin to humans. However, this flexibility quickly
becomes a problem in fields where errors can be disastrous, such
as in a pharmacy context, but the opposite also holds true; a
system that is too inflexible will also lead to errors, as it can
become too rigid to handle situations that are not accounted
for. Work using LLMs in a pharmacy context have adopted
a wide scope, accounting for many different medications in
brief interactions — our strategy is the opposite: focus on a
more narrow and long task. This not only enables a greater
understanding of the task at hand, but also provides insight into
what challenges are present in an interaction of longer nature.
The main challenge, however, remains the same for a narrow
and wide system: it needs to strike a balance between adherence
to conversational requirements and flexibility. In an effort to
strike such a balance, we propose a system meant to provide
medication counseling while juggling these two extremes. We
also cover our design in constructing such a system, with a focus
on methods aiming to fulfill conversation requirements, reduce
hallucinations and promote high-quality responses. The methods
used have the potential to increase the determinism of the system,
while simultaneously not removing the dynamic conversational
abilities granted by the usage of LLMs. However, a great deal of
work remains ahead, and the development of this kind of system
needs to involve continuous testing and a human-in-the-loop. It
should also be evaluated outside of commonly used benchmarks
for LLMs, as these do not adequately capture the complexities
of this kind of conversational system.