ChatGPT holds promise as supplementary tool in clinical pharmacy

Use of chat generative pretrained transformer (ChatGPT) can be of help among clinical pharmacists, but its capacity to handle complex problems needs to be improved and refined, suggests a study.

A team of investigators assessed the performance of ChatGPT in key domains in clinical pharmacy practice, including prescription review, patient medication education, adverse drug reaction (ADR) recognition, ADR causality assessment, and drug counselling. They gathered the questions and clinical pharmacists’ answers from real clinical cases and clinical pharmacist competency assessment.

Responses by ChatGPT were generated by entering the same question into the ‘New Chat’ box of ChatGPT Mar 23 Version. Five licensed clinical pharmacists rated the answers independently on a scale of 0 (completely incorrect) to 10 (completely correct).

A paired 2-tailed Student’s t-test was used to compare the mean scores of ChatGPT and clinical pharmacists. The investigators then descriptively summarized the text content of the answers.

Based on quantitative results, ChatGPT received excellent ratings in drug counselling (ChatGPT vs clinical pharmacist: 8.77 vs 9.50; p=0.0791) but had weak scores in prescription review (5.23 vs 9.90; p=0.0089), patient medication education (6.20 vs 9.07; p=0.0032), ADR recognition (5.07 vs 9.70; p=0.0483), and ADR causality assessment (4.03 vs 9.73; p=0.023).

“The capabilities and limitations of ChatGPT in clinical pharmacy practice were summarized based on the completeness and accuracy of the answers,” the investigators said. “ChatGPT revealed robust retrieval, information integration and dialogue capabilities. It lacked medicine-specific datasets as well as the ability for handling advanced reasoning and complex instructions.”