Prototyping Complex Systems

Role

User Researcher, Interaction Designer

Paper

Jumana Almahmoud, Almaha Almalki, Tarfah Alrashed, and Areej Alwabil. "Prototyping Complex Systems: A Diary Study Approach to Understand the Design Process." In International Conference of Design, User Experience, and Usability, pp. 187-196. Springer International Publishing, 2016. pdf

Challenge

Interconnected engineering systems require a special consideration when designing the interfaces given the specific city of the information displayed to the user and the level of accuracy. Using social-science concepts in HCI provides a systematic framework for the UI design of complex systems.

Solution 

The solution for creating a structured design approach in the research center starts by observing the existing design process. In order to do so, I have introduced diary study on the design time to capture their process. The diary study was applied to explore the design process within the development of a Web-based interactive visualization platform, which is in its early stages of development as part of a project at the Center for Complex Engineering Systems. The project’s team consisted of engineers who were developing the mathematical models behind the engine, and the platform’s designers and developers who were developing the interactive visualization platform. Three designers of this platform participated in the study, which spanned a time frame of four months. In the following subsections, we provide an overview explaining the diary process, the artifacts as communication tools, and the analysis that was conducted as part of this study.

Process

At the beginning of the diary-study process, the facilitator briefed the designers on the tasks required and how to use the diary-logging tool. The tool used was Dscout, an app for moment-based research to collect quantifiable data from users. The app allowed collecting photos, text and numbers momentarily from the designers after each design feedback session. In this study, the goal was to integrate the diary-logging task seamlessly into the design process without causing an interruption to the workflow of the designer. As depicted in following figure, the process started with the designer sharing a prototype with the team to gain feedback. The designer then logged the feedback in the diary tool. The tool collected the data and provided insights for the study facilitator. An interview was conducted when the designers indicated that they faced a problem in the design of the artifact or if the type of artifact has changed from the last feedback session.

Each time the designers logged in their artifact; the diary asked them a set of questions regarding the feedback session. The questions were defined to help the researchers understand the design process for this specific project, and were general enough to accommodate other projects. The questions were chosen to minimize the effort of logging by limiting the number of text inputs and instead giving the participant options and lists to choose from. The following figure shows some of these questions.

The Dscout tool was selected as a diary tool because it provided a convenient interface for the designers to capture images of the artifacts in-situ and a streamlined process for creating diary entries and aligning them in a storyboard across the project’s timeline.

In addition to reporting activities and documenting experiences with artifacts used in the design process, the UX diary in this study was also used by the participants (designers) to discuss design issues or information about the front end of the complex systems’ interfaces. The type of artifact often impacts the participants’ experience while logging data or being interviewed and the researcher’s ability to analyze the data. In this study, there were three types of artifacts used in the design process: sketches, digital prototypes (Axure and Illustrator) and coded prototypes (HTML5/D3.js).

The diary study was supported with feedback interviews conducted with the participants (designers and developers) when the log of the participant indicated a noticeable pattern. These interviews were triggered based on two different incidents that were noticed: first, when designers indicated that they had faced a problem in designing a certain artifact and second, when designers changed the artifact they were using from one logged session to another.

The analysis revealed the variation in the number of participants in the feedback sessions, as they varied from few to many members. It also provided insights into the prototyping process, the issues that the designers face temporal dynamics of perceived usability of the system during the design process. Finally, the study also shows the variation in frequency of usage of artefacts (e.g. digital prototyping was the type of artifact that was mostly used to communicate the designs).

Insights 

The Design Process for Complex Systems When designing for complex systems, the design process tends to adapt to accommodate the complexity of the product to be delivered. Within the interactive visualization platform observed in this study, an interesting pattern emerged during the design process. At the early stage, the designers started by sketching the layout of the tool and when more iterations were required for the digital prototype, the designers worked on the process in parallel. A design workflow was established to build the layout, and another workflow branched to prototype the visualization components.

The visualization prototyping relied on producing coded prototypes, and the reason for this, according to the feedback interviews conducted, is that the design team wanted to speed up the design process to show a working mockup. Thus, the visualization section had to represent real data. The diary-study was effective in understanding why coding was perceived to be the best option to prototype the visualization in this specific context, since sketches or digital prototypes were not sufficient to display the complexity and interaction with such components. Other insights gathered from the data collected were related to temporal dynamics with different artefacts (e.g. findings indicated that designers spent most of the design time producing digital prototypes).

After interviewing the designers to check on their progress and get their feedback, they indicated that using digital prototypes gave them more flexibility to share their designs, apply feedback and then update their designs and share them again, as opposed to sketches that were not enough to convey some of the platform’s complex elements.

 Diary Study as a UX Method in Design

Designers and developers go through an iterative process of prototyping and implementation. In this study, the designers used three different artifacts (sketches, digital prototypes and coded prototypes), and for each artifact, the designers made several versions or drafts to communicate the concept and ideas to the team. As mentioned in the methods section, designers shared their designs with the project’s team members and principal investigators in co-located and remote communication settings. After every session, the designers documented or logged the designs they shared (sketches, prototypes or codes) and their perceived efficacy of the feedback they got in the diary-study’s tool. Based on the interviews we conducted with the designers, we found that this study helped them categorize their design process by going through different types of artifacts and documenting each session they conducted with their team and the feedback they received. The study asked designers to go through different steps, which helped them reflect on key design issues:

1. The type of feedback they got from sharing their design using different artifacts, which had a considerable impact on the next design decision they made and the artifact that they would consider in the prototyping process.

2. The importance of artifacts as a communication tool between designers and other stakeholders in the project.

3. The design considerations that the designers can gain from low-fidelity designs (e.g., sketches) and high fidelity (e.g., Azure prototype and coding).

The designers indicated in the interviews that logging their design process, the artifacts they have to create their designs, the people they shared these designs with and the feedback they received helped them in their next designs. Knowing how different artifacts worked as a communication tool between the designers and the stakeholders, the kind of feedback they received, and logging how difficult each step was, provided the designers with insights regarding what type of artifact could be used to convey their vision and communicate ideas. In addition to planning and communicating, going through the diary-study process helped the designers better understand the time requirements for each task, based on the feedback they received. Based on that, critical decisions were made regarding the artifact type they needed to use next. Going from low-fidelity prototypes to an interactive high-fidelity one was based on evaluating how challenging it was to communicate the designer’s idea to the stakeholders to elicit their feedback.

-