You start accepting that artificial intelligence (AI) has benefits and decide to try using it in your everyday life. You start small, using it occasionally to answer a question, then eventually find yourself using it throughout the day. The benefits and convenience feel great, and you rely on it more as the days go on. However, although AI has its perks, it is important to be aware that it is not perfect.
Introduction
In pharmacy practice, AI has promising aspects, especially in safety. AI technology is used to investigate complex clinical issues in areas such as data collection, analysis, and the utilization process (Jarab et al., 2023). AI can potentially identify patterns that humans might miss for various reasons, such as fatigue. AI recommendations can save a lot of time and can catch many errors that have the potential to be missed; however, AI-generated medication recommendations do have the possibility of being incorrect, incomplete, or based on flawed data. While AI integration shows great improvement, there are numerous challenges (Li et al., 2025). If these mistakes are not caught before medication is dispensed to the patient, it could lead to the wrong amount being charged to the patient or harm to the patient. Additionally, there are factors that AI cannot decipher. For example, if the patient’s medication list is not recently updated, is incomplete, or contains wrong information, AI may not catch this, leading to a decision based on bad information. Most importantly, AI should never replace the expertise of the pharmacist.
When the Data is Wrong, the Recommendation Can be Wrong
AI’s decision-making accuracy is highly dependent on the information it is given. If given an outdated patient medication list, it doesn’t have a way to determine it is outdated and will make decisions based on the list itself. Undocumented allergies can potentially lead the system into approving a medication that has an ingredient the patient is allergic to, leading to an allergic reaction. If relevant laboratory information is missing, outdated, or unavailable to the system, the AI may make a medication recommendation without considering important factors such as kidney or liver function. Conflicting records can cause the AI to make a mistake, and drug databases may not be up to date. It is very important that the accuracy of data for the AI integration is accurate and recent.
The Automation Bias Problem
Healthcare professionals may be more likely to accept a computer-generated recommendation because they assume that the system may know something they don’t. This could happen if the healthcare team feels that the system should be trusted over their own judgement. When the system is correct multiple times in a row, it is possible that the professionals will start to fully trust the AI integration. On the opposite end, if the AI integration overly scrutinizes and sends multiple minor alerts, the healthcare team may experience alert fatigue and will be more inclined to override without thinking through the decision. Overall, if there is too much trust in AI, pharmacists and other team members may accept anything the computer says without enough scrutiny.
An Alert is Not the Same as a Decision
AI may give an alert that a possible drug interaction is there, but it is up to the pharmacist to truly determine if the interaction is significant. The pharmacist should take into consideration the other factors of the patient that AI might be missing, such as the patient’s dose, timing, indication, kidney function, and other medications that the patient is taking, which might change the interpretation made by AI. As the alert fatigue blog post mentions, some alerts may be technically correct but not clinically meaningful. At the same time, however, a pharmacist should not begin to override alerts without paying much attention if many insignificant ones alert in a row. Clinical context matters, which is not something AI can perfectly take into consideration.
The Patient is More Than a Medication List
Currently, most AI integrations can process medication lists and clinical information, but it may not fully understand the other key factors that come into play when making a decision on someone’s medication. AI models may overlook important contraindications and some regulatory requirements (Li et al., 2025). First, it cannot determine why a patient is taking the medication being reviewed. It also does not know if the patient is taking the medication or if the patient can afford to pick it up at a retail pharmacy. Whether or not the patient understands the instructions is also very important. Additionally, side effects are very unpleasant. Some patients may not report any side effects, while others might discontinue the medication without notifying anyone. AI would not be aware of either of those scenarios. Even if the medication is determined to be safe during the current situation, AI cannot determine if the recommendation is realistic for the patient given all the factors. AI models cannot fully integrate multidimensional information to make decisions on individualized care plans (Li et al., 2025).
When AI Gets it Wrong: Learning from the Error
Everyone involved in the medication-use process has a role in reviewing AI-generated recommendations within their scope of responsibility. Whether it be a pharmacy technician at the beginning of the fill in a hospital, or a nurse who is about to administer the medication, they should review their assigned steps and are responsible for speaking up if something does not look right. AI does not replace human judgment and should never be used to make the final decision without a review.
Identifying the error would be the first step in responding to an error made by AI. Once aware that an error has occurred, the professional should prevent the medication from reaching the patient if it hasn’t been administered already. If it has been administered, the healthcare team member must report the incident, and if it was not, the member should report a near miss. After that, it is important to review how the error occurred and determine if the problem came from the data, model, workflow, or human review. As mentioned previously, updating the system or process can help prevent errors from occurring and can prevent recurrence.
How Can We Make AI-Assisted Medication Use Safer?
As this blog post emphasizes multiple times, human oversight is one of the biggest factors in safety, and all members are responsible for reviewing the integration’s recommendations. In the background, however, regular model validation should occur to ensure the highest accuracy. Transparent reasoning and data quality are also key vitals to check in AI-integrated systems.
Monitoring for errors has the potential to be one of the most important things to do when using AI integration for medication safety, as it can be used to know what to be aware of moving forward and to gather statistics for programming. It is also important to use the most up-to-date model and run bias testing. AI is not perfect when it comes to privacy protection and safety, and can potentially lead to a violation of the patients’ privacy (Jarab et al., 2023).
What Safe AI Implementation Should Look Like
To make AI integration safe, the initial step should be to set clear policies for when AI can and cannot be used. At the same time, defining responsibilities for pharmacists, technicians, nurses, and other involved team members is very useful. As mentioned, nothing should replace the expertise of pharmacists’ review, and therefore there should be human review before dispensing or administration. Regular system checking will help to ensure the system itself has everything it needs to be the safest possible, while monitoring can also help to improve it. Staff should be adequately trained in using the AI and reporting concerns.
Conclusion
AI has real potential to improve medication safety; it can identify patterns, assist with recommendations, and reduce some of the burden on healthcare professionals. It does, however, need to be monitored closely and should not be used without human review. AI should be used as a tool that supports decision-making and should not be used as a replacement for pharmacists or other healthcare professionals. Human review remains essential, particularly when recommendations could affect patient care. Risks to the accuracy include inaccurate or incomplete data, automation bias, alert fatigue, lack of patient-specific context, and errors in the AI system.
Safer AI isn’t necessarily about making AI perfect; it’s about creating systems where errors can be identified before they reach the patient. Effective safeguards, ongoing monitoring, validation, quality data, and human oversight can help make AI-assisted medication use safer. The goal should not be to choose between AI and human expertise, but to find ways for the two to work together safely.
References
Jarab, A. S., Abu Heshmeh, S. R., & Al Meslamani, A. Z. (2023). Artificial intelligence (AI) in pharmacy: an overview of innovations. J Med Econ, 26(1), 1261-1265. https://doi.org/10.1080/13696998.2023.2265245
Li, L., Du, P., Huang, X., Zhao, H., Ni, M., Yan, M., & Wang, A. (2025). Comparative Analysis of Generative Artificial Intelligence Systems in Solving Clinical Pharmacy Problems: Mixed Methods Study. JMIR Med Inform, 13, e76128. https://doi.org/10.2196/76128
Author Bio
I am a healthcare professional and pharmacy technician with experience in pharmacy practice and medication-related topics. I am passionate about medication safety, healthcare education, and making complex healthcare information easily accessible. Through TheMedDose, I share practical insights on medications, pharmacy practice, and patient safety to help the public better understand the impact of medications in everyday care.