The work aims to propose a novel multi-modal process which integrates both image and text elements for innovative product design to address the issue of insufficient innovation and feasibility in product design schemes within the field of AI-assisted product design. The work begins with preprocessing the designer's sketches and textual requirements, followed by the incorporation of a product design knowledge graph to facilitate divergent thinking and innovation. Subsequently, a fine-tuned generative pre-trained Transformer model and a diffusion model were employed to generate product schemes and their conceptual diagrams. Finally, a deep multi-modal design assessment model was adopted to evaluate the feasibility and market potential of the product design schemes. The results indicated that the introduction of the product design knowledge graph and the deep multi-modal design assessment model enabled the generation of innovative product schemes that also possessed feasibility. In conclusion, this multi-modal approach to innovative product scheme design, leveraging cutting-edge AI and deep learning technologies, not only enhances design efficiency but also provides designers with a broader perspective for innovation and inspiration sources.