Content area
Model engineers build decision-support systems using Artificial Intelligence (AI) models, expecting that these models will achieve, if not exceed, human expert-level performance in decision-making. To meet this goal, AI models must make accurate decisions, explain reasoning, and ground outputs in comprehensive domain knowledge. This knowledge encompasses all relevant and meaningful information synthesized by experts within the domains these models represent or operate in. However, AI models often lack such knowledge, either because it is missing from training data or algorithms fail to learn correct representations. This dissertation refers to this missing knowledge as latent domain knowledge—information known to experts but latent from the model’s perspective. While many stakeholders possess domain knowledge, this dissertation focuses on model engineers and domain experts, who are most directly involved in developing, evaluating, and interacting with AI models throughout their lifecycle. To address latent knowledge, this dissertation first shows that integrating domain knowledge through collaboration with domain experts significantly improves model performance, outperforming baselines trained solely on expert-provided data. It then shows that users can identify latent knowledge by interactively exploring models through domain-relevant what-if scenarios. Next, it proposes a Bayesian inference–based method to probabilistically evaluate and select effective steering mechanisms (e.g., natural language prompt-based guardrails for Large Language Models (LLMs)) that align model outputs with expert users’ knowledge. Although domain experts may lack AI expertise, they value autonomy and may not want to rely on model engineers for post-deployment monitoring and debugging. To support this, the dissertation introduces interactive steering tools for models that do not accept prompt-based guardrails, enabling users to steer the model’s input–output relationships directly. The Bayesian method validates such steering in the background and retains those that best align outputs with empirical data. These exploration and steering tools are integrated in a data-, domain-, and model-agnostic toolbox named Visual Interactive Model Explorer (VIME). VIME enables users to investigate and steer model behavior toward desired outputs and knowledge. This dissertation details iterative refinements of VIME based on user needs identified through prior literature and formative studies. It presents simplified user evaluation findings on how effectively model engineers and domain experts can identify and introduce latent knowledge, and informs the broader adoption of domain knowledge–aligned AI-based decision-support systems.