The work aims to provide a new approach for the extraction and design application of traditional Zhuang brocade color schemes, and to enhance the aesthetic value of design. By combining image processing with the color harmony theory, the pattern was first smoothed and denoised to facilitate subsequent color extraction. Attempt was made to map RGB color space to HSV color space, and then the K-means++clustering algorithm was used to extract feature colors. The color harmony theory was introduced for color beauty calculation, optimizing and outputting new color schemes through harmony. Finally, the feasibility of extracting and optimizing the color of Zhuang brocade patterns was verified by PAD evaluation. The experimental results proved that color extraction of Zhuang brocade based on HSV had higher accuracy. At the same time, the color harmony theory was introduced, and based on the traditional color matching logic of Zhuang brocade, the beauty of the traditional color matching scheme was optimized, and the effectiveness of color optimization application was verified. The conclusion provides a quantitative aesthetic method for optimizing the design of traditional Zhuang brocade color schemes, avoiding strong subjectivity and making the color matching of Zhuang brocade both preserve traditional culture and conform to contemporary aesthetics.