Automating BIM with AI Agents: An Interview with TUM Researchers Changyu Du and Zihan Deng

The world of AI is moving at lightning speed, and its impact on the AEC industry is a topic of intense discussion. While many applications focus on analyzing existing data, a new frontier is emerging: AI agents that can actively perform tasks within complex software.

I recently sat down with Changyu Du, a PhD student at the Technical University of Munich (TUM), and Zihan Deng, a Master’s student soon to begin his PhD at TUM. Their latest research, presented at ICML 2025, explores using computer use agents for automated BIM authoring. We discussed the motivation behind their work, the current hurdles, whether AI will replace architects (spoiler: they don’t think so), and what’s next for this groundbreaking research.

The Elevator Pitch: Why Use AI Agents for BIM?

DataDrivenAEC: What was the initial motivation for this research? What’s the elevator pitch?

Changyu: Our main goal was to explore applying computer use agents to BIM authoring software. We’ve seen them work impressively in gaming and desktop automation, but nobody had really applied them to our domain.

Zihan: As BIM users ourselves, we know how complex these programs can be—they come with thousands of pages of documentation. We wanted to find a more general, vision-based way to simplify the process. Instead of relying on APIs, like in previous Text-to-BIM work, we’re exploring how an AI agent can use the graphical user interface (GUI) just like a human would. The ultimate vision is to make everyone a professional user of BIM software.

*Caption: An overview of the BIMGENT framework, illustrating the agent’s workflow from design to execution. Source: BIMGENT Project Website

A Long Way to Go: Current Challenges and Low Success Rates

DataDrivenAEC: Your paper mentions that the results are still preliminary. How do you measure success, and how far are we from seeing this in practice?

Changyu: We’re still in a very early phase of research. To be transparent, our framework achieved a 32% end-to-end success rate on our mini-building benchmark, which is quite low. The success rate for individual tasks, like creating a single building element, is even lower.

Beyond performance, there are significant hurdles like safety and privacy. You’re essentially letting an agent take control of your computer and software, which is a risky proposition that not every designer would be comfortable with.

Zihan: Right now, our tests are limited to simple, one-floor buildings. We haven’t yet tackled real-world complexities like building regulations or compliance checks. So, while the potential is huge, we have a long way to go before this is ready for a production environment.

The Chicken-and-Egg Problem: Models vs. Data

DataDrivenAEC: What’s the bigger bottleneck right now: the AI models themselves or the lack of AEC-specific training data?

Changyu: It’s really both. We tested off-the-shelf models from major AI labs, but they don’t work well out of the box. They’re designed for general web Browse or desktop tasks and need significant domain adaptation to understand the nuances of BIM.

Zihan: Data is the other massive challenge. Unlike booking a hotel, where a successful click is a clear signal, design lacks a “significant signal” of correctness. There’s no simple ground truth to tell the model, “Yes, that was the right design choice.” Is a building “correct”? Does it meet the design intent? This ambiguity makes it incredibly difficult to create the high-quality, labeled data needed for training a robust model. Our next big step is to figure out how to collect and label this kind of data effectively.

Advice for Firms: The Cost of Innovation

DataDrivenAEC: Given these challenges, what’s your advice for AEC firms wanting to adopt AI? What’s the first step?

Changyu: It’s an interesting dynamic. We, as researchers, have the methodologies but lack massive, real-world datasets. Companies have troves of data hidden in their daily workflows. The key is for firms to start asking if AI can genuinely bring more revenue or reduce costs. If the answer is yes, then it’s worth investing in data scientists who can mine that data for useful applications.

Zihan: There’s a fundamental conflict: AI development is expensive—it requires money, GPUs, and talent. Conversely, AEC firms are primarily focused on reducing project costs. This is why the biggest AI advancements happen at large tech companies, not yet in our industry.

We believe the best path forward is collaboration between industry and academia. Firms provide the data and domain problems, and we provide the research expertise. It’s a win-win.

Will AI Replace Architects?

DataDrivenAEC: This is the question on everyone’s mind. With models like GPT-4o generating impressive floor plans, do you see AI taking over the architect’s job?

Zihan: I admit, when I saw the quality of floor plans GPT-4o could generate, it was a “wow” moment. It made me think about the future of design jobs. From that perspective, it seemed possible.

Changyu: I see it differently. I don’t think we will replace architects. Our goal is to build systems that support architects in making better designs. Architectural design is a super-complex cognitive activity. A floor plan might look good to an engineer like me, but a senior architect would critique it from a dozen different angles we haven’t even considered. We have to respect the deep complexity of the profession.

Zihan: Exactly. We need to treat AI as a tool. A tool helps us reduce time, not reduce people.

The Road Ahead: Next Steps for the Research

DataDrivenAEC: What’s next for your project?

Zihan: I’ll be starting my PhD in October to continue this work. Our roadmap is focused on three key areas: