Hybrid 3D Asset Retrieval via Contrastive Vision-Language Matching and Structured Prompt Parsing

Searching large repositories to retrieve 3D assets for simulation environments can be tedious, especially when dealing with complex metadata. On the other hand, retrieving relevant assets using natural language remains a challenge, as embedding-based approaches often fail to handle prompts with multiple constraints, while metadata-based methods are too rigid for abstract or semantic queries. This paper proposes a hybrid retrieval framework that combines a vision-language model to extract semantic features from asset images with a structured natural-language parser to map user text queries to retrievable attributes. The system employs a dual-path architecture that utilizes contrastive embeddings for semantic matching, while a fine-tuned Phi-3-Mini model extracts explicit attribute-value pairs for metadata filtering. The approach was evaluated on a custom dataset of 500 multi-view 3D assets with structured annotations. Experimental results showed that the hybrid method outperforms standalone retrieval baselines, increasing top retrieval success by more than 55 percent for complex multi-attribute queries. The proposed framework enhances 3D asset retrieval workflows in simulation environments where both accuracy and attribute control are crucial, paving the way for automated environment creation.