Under the Hood: Key Insights from Notable’s Technology Leaders

In the rapidly evolving technology landscape, forward-thinking healthcare leaders are eager to stay up-to-date on advancements to drive their digital transformation.

A panel session during the recent Noteworthy summit provided valuable insights into how AI and automation technology can revolutionize the staff and patient experience. Moderated by Tom Alterman, Notable’s Head of Product, “Under the Hood: Meet Notable's Technology Leaders” featured the Company’s engineering and product leaders in a discussion about AI, software development, security, and more. 

Justin White, Chief Technology Officer; Ryan Pfeffer, Head of Engineering; Kevin Huang, Head of Data Science; Jon Debonis, Head of Security; and Joey Mart, Principal Engineer, discussed key engineering strategies behind Notable’s technology, focusing on ensuring robust data management, security measures, and change management processes that are crucial to ensuring trust and productivity.‍

AI Architecture and Flow Management

The panel discussion centered on how AI is used in Notable’s environment and the methodologies the technology team uses to ensure security and safety.

First, the panel shared that Notable is purposeful and careful about what the AI is asked to do. Notable does not ask the AI to make medical diagnostic decisions. However, an AI may be asked to surface relevant information to enable a healthcare provider to make such a decision. 

Notable’s technology teams also don’t inherently trust the output from an AI. The panel shared that they perform multiple steps to verify the output across a large number of samples and implement different levels of testing and validation, depending on the task. 

Notable’s AI platform utilizes a modular system where each component functions like a public API. This architecture allows for stable interfaces that do not change unexpectedly, ensuring that updates or modifications can occur without disrupting previously deployed systems. 

As healthcare is subject to rigorous legal and compliance standards, the panel members emphasized the importance of auditability and traceability. A standout feature highlighted was the platform's ability to snapshot every iteration of a flow, enabling a comprehensive review of past configurations. This approach not only facilitates compliance but also enhances accountability when legal inquiries arise.

‍Promoting Change with Confidence

The discussion also explored the need for effective change management practices that allow healthcare providers to embrace AI agents as their future workforce.

Pfeffer introduced the concept of a "blue-green deployment strategy.” A blue-green deployment strategy is a software release management technique that aims to minimize downtime and reduce risk when deploying new applications. This approach involves maintaining two identical environments, referred to as the "blue" environment and the "green" environment.

This methodology minimizes risk, empowering customers to pilot changes with a limited population before a full-scale rollout.

Pfeffer said, “As your solid workflow is in place you can actually make changes and explore new flows, and do that lower risk change with a small set of your population before rolling it out wider.”

He continued, “We're really thinking about having change management built in at every layer of the stack to give our customers what they need, which is what we call ‘confidence through the chaos.’ As you are making changes to flows, we're there working with you through that.”

The emphasis on mitigating risks associated with AI implementation resonated throughout the session. The attendees noted that healthcare professionals are often reluctant to adopt new technology due to past experiences or fears of decreased oversight. By systematically integrating change management and promoting iterative improvements, greater confidence is instilled amidst the complexity of healthcare systems.

Third-Party Risk Management and Data Security

In addressing third-party vendor risks, Debonis detailed a few of Notable’s best practices for limiting vendor risk. Debonis shared that critical third parties handling patient data are assessed rigorously, with ongoing evaluations to ensure compliance with industry standards. This process not only safeguards patient information but also fortifies the integrity of the software tool being employed.

Debonis continued, “While all third parties are evaluated by Notable, customers may also add their own third parties and control which of Notable’s optional third parties are enabled for their flows.”

AI in Healthcare: Setting Expectations

One audience member acknowledged the incredible future that AI and automation present in healthcare but was curious about how to establish expectations for what is possible for AI in the near future. 

First, it’s important to know that large language models (LLMs), like those utilized by Notable, improve drastically over time and that, much like human problem-solving, some tasks are easier, like finding a patient name or a medical record number (MRN) within a document, and some are harder. 

Notable practices an overarching development cycle that metaphorically treats AI like a really talented new employee. No matter how smart and talented they are, you're not going to throw them into the system and say, “Good luck, I'll see you in a month!” Employees need instructions, monitoring, performance reviews, and tools to support their work. 

Notable’s development cycle operates similarly. In partnership with a customer, the technology teams provide tailored support and training for the tool, monitor and evaluate performance, and report on outcomes so that when ready, the AI agent can operate independently within a healthcare organization to enhance workflow efficiency and staff productivity. 

Looking Ahead: Building Powerful Partnerships

The session concluded with a glimpse into the future of healthcare powered by AI and automation. Notable envisions a future where, through the thoughtful integration of AI agents, healthcare organizations can experience profound transformation, yielding significant improvements in patient access, quality performance, and staff satisfaction and productivity.