Automated Building-Truss Structural Analysis from Diagrams Using Fine-Tuned Vision-Language and Language Models: Dataset, Models, and Code

This repository provides the synthetic datasets, prompts, trained model artifacts, training scripts, and end-to-end evaluation code associated with the study “Automated Building-Truss Structural Analysis from Diagrams Using Fine-Tuned Vision-Language and Language Models.” The workflow uses a LoRA-fine-tuned Llama-3.2-11B-Vision-Instruct model to extract structural information from planar truss diagrams and a fine-tuned T5-Large model to convert the extracted description into a structured JSON representation containing nodes, elements, loads, and supports. The repository includes the VLM and T5 training and validation datasets, a held-out end-to-end test set, model checkpoints and configuration files, exact model prompts, training scripts, and an evaluation notebook incorporating FEM and OpenSees-based comparison. The associated study uses 10,000 synthetic training examples, 100 validation examples, and 10 held-out end-to-end evaluation cases.