User experience and design outcomes evaluation of a design tool called "Context Connection"

This study recruited 25 designers with more than two years of product design learning experience and hold a 14-days-workshop. This study randomly separated them into Groups A and B, and instructed each group to begin a conceptual design project using the identical design proposal (design an entity product, which has the characteristics of a certain traditional Chinese cultural symbol and uses machine learning technology as one of the product functions). 12 designers made up Group A. They were instructed to use CC for their projects and were asked to quickly prototype by 3D printing for a short user walk-through. With a total of 13 members, Group B were asked to conduct product creative design process based on brainstorming without user study. Before the experiment began, the researcher introduced CC to Group A designers. A midterm display was scheduled for both groups A and B on the eighth day of the workshop to check the progress and answer the questions, and a presenting report was scheduled for the workshop's final day. All designers ultimately finished their work. Following the completion of the product creative designs by 25 designers, a questionnaire was developed to ask Group A to rate their experience on a seven-point Likert scale by answering three questions about "ease of use," two questions about "process of use," two questions about "creative outcomes," and five questions about "knowledge comprehension". On a seven-point Likert scale, all 25 designers were asked to rate how well they understood UCD and how creative they thought they were. Twelve designers from group A were also interviewed about the same four topics as the questionnaire, and their feedback and suggestions were solicited.Twenty industrial design master's or doctoral students were invited to assess the 25 concepts that were presented; all of them were not workshop participants. The 25 concepts were rated on three separate dimensions—"innovation of product function," "innovation of cultural symbol integration," and "overall evaluation of the concept"—using a seven-point Likert scale. This study also conducts a thorough investigation of three presented typical cases.This study conducted descriptive statistical analysis with the data from questionnaires by the software of jamovi (Version 1.6). 12 designers who used CC had high mean scores for the four aspects of ease of use, process of use, creative outcomes and knowledge comprehension (Ease of use M=5.47, SD=1.01. Process of use M=5.46, SD=0.940. Creative outcomes M=5.42, SD=0.900. Knowledge comprehension M=5.75, SD=0.832) . In addition, among all 25 designers, the mean scores of designers who used CC on " understanding of UCD" and " Self-evaluation of creativity " (UCD understanding M=5.75, SD=1.06. Self-evaluation of creativity M=5.25, SD=1.14) were higher than those who did not use CC (UCD understanding M=5.54, SD=0.967. Self-evaluation of creativity M=5.08, SD=1.50).Independent sample T-test and descriptive statistics were performed on the data of design outcomes using the software of jamovi (Version 1.6). The hypothesis is that, the degree of innovation in machine learning functions, cultural symbol integration, and overall concept innovation from other designers are significantly higher in the outcomes of designers who used CC than in the outcomes of designers who did not use CC. The Shapiro-Wilk normality test was passed on all three variables (W is close to 1, p>0.05). The group that used CC (ML function creation Mdn=4.60. Overall evaluation Mdn=4.45) scored significantly higher on evaluations of machine learning function innovation (ML function creation) and overall concept innovation (Overall evaluation) than the group that did not use CC (ML function creation Mdn=4.25. Overall evaluation Mdn=3.95) (ML function creation p=0.004. Overall evaluation p=0.017) . The difference in mean values for the evaluation of innovation in cultural symbol integration between the two groups was not statistically significant (Cultural symbols integration p=0.467), but descriptive statistics indicated that the mean value for the group using CC (M=4.19, SD=0.455) was slightly higher than the mean value for the group.