Content area
Despite progress in programming tools and environments, developers still struggle to maintain mental models of the systems they build and the context surrounding their code. This dissertation advances a framework for enhancing program understanding and manipulation for all agents working with a codebase—including developers and automated agents such as large language models (LLMs)—through notes on shared context and high-level representations that are attached to code and synchronized with changes as it evolves, e.g., external notes and UIs that stay linked to sections of a program where they are meaningful, with their content updated to remain relevant.
This dissertation first explores the design of high-level programming tools, addressing the specific case of floating-point error analysis with a custom tool developed during the PhD process called Odyssey. Odyssey is a workbench for floating-point analysis that transforms an existing low-level, black-box expression improvement tool (the Herbie floating-point expression rewriting tool[PSSWT15]) to support a high-level, scaffolded analysis process where users can analyze, generate, and iterate on automated suggestions using their own expertise.
Next, considering how contextual information like that from an Odyssey analysis session might be attached to actual floating-point programs, I explore maintaining semantic connections between document sections and metadata using a new technique named Magic Markup. This system uses an LLM to “magically” maintain the positions of external notes on an evolving document without write access through semantic anchoring rather than brittle syntactic approaches.
Finally, I formalize and implement this approach in Codetations, a VS-Code extension that helps developers contextualize documents with rich, interactive notes and tools. Codetations builds on the Magic Markup method to keep notes outside the document while integrating edit-tracking and exposing a rich API for annotations to respond to buffer changes and leverage editor features. In a qualitative evaluation, developers recognize this method as enabling more extensive and useful forms of documentation, and worked examples show the additional context improving code repair performance in LLMs.
Together, these systems indicate a practical path by which existing programming systems can begin to offer powerful, developer-customized tools that meet developers’ needs for contextual information and high-level analysis.