| 摘要: |
| 目的 针对当前产品创新设计领域中对基于图像-文本多模态知识支撑创新设计方法研究不足的问题，提出了一套基于图文多模态的产品创新方案设计方法。方法 首先，对设计师的设计草图与文本要求进行预处理，然后引入产品设计知识图谱来促进设计思维的发散和创新;其次，通过微调的生成式预训练变换器模型和扩散模型生成产品方案及其概念图;最后，利用深度多模态设计评估模型对产品设计方案的可行性和市场潜力进行评估。结果 通过产品设计知识图谱，及深度多模态设计评估模型的引入，该设计流程可以生成富有创新性且具备可行性的产品方案。结论 基于图文多模态的产品创新方案设计流程结合了最新的深度学习技术，不仅提高了设计的效率，还为设计师提供了更广阔的创新视角和灵感来源。 |
| 关键词: 图文多模态 深度生成模型 知识图谱 产品创新设计 |
| DOI：10.19554/j.cnki.1001-3563.2024.08.003 |
| 分类号: |
| 基金项目:国家自然科学基金面上(52375254);上海交通大学医工交叉项目(21X010301670) |
|
| Innovative Product Design Schemes Based on Image-text Multi-modal Fusion Reasoning |
|
MA Jin1, FAN Minghao1, MA Liangshan2, HU Jie3
|
|
(1.School of Sensing Science and Technology Shanghai 200240, China; 3. School of Design, Shanghai Jiao Tong University, Shanghai 200240, China;2. Shanghai China Software Computer Systems Engineering Co., Ltd., Shanghai 200001, China)
|
| Abstract: |
| The work aims to propose a novel multi-modal process which integrates both image and text elements for innovative product design to address the issue of insufficient innovation and feasibility in product design schemes within the field of AI-assisted product design. The work begins with preprocessing the designer's sketches and textual requirements, followed by the incorporation of a product design knowledge graph to facilitate divergent thinking and innovation. Subsequently, a fine-tuned generative pre-trained Transformer model and a diffusion model were employed to generate product schemes and their conceptual diagrams. Finally, a deep multi-modal design assessment model was adopted to evaluate the feasibility and market potential of the product design schemes. The results indicated that the introduction of the product design knowledge graph and the deep multi-modal design assessment model enabled the generation of innovative product schemes that also possessed feasibility. In conclusion, this multi-modal approach to innovative product scheme design, leveraging cutting-edge AI and deep learning technologies, not only enhances design efficiency but also provides designers with a broader perspective for innovation and inspiration sources. |
| Key words: multi-modal image and text deep generative models knowledge graph innovative product design |
