Formative evaluation conducted early on in the design process with low-fidelity prototypes
Summative evaluation conducted with high-fidelity prototypes or final interface
The type of data we collect is related to the type of prototype we are using
Low fidelity prototype requires that the designer collects the data
Tiem to complete a task, count the number of clicks, etc.
High fidelity prototype may produce data of how the system was used
Timestamps of when the user started and ended a session
Log data of how the user interacted with the system
The type of prototype will affect where the evaluation takes place
Low fidelity prototype requires a controlled environment
High fidelity prototype may be deployed in the field
Evaluation of if the design is usable
Design is effective with regards to the degree to which the goals of the task are met
Quantitative data: questionnaires or log data
Qualitative data: an interview with the user
Design is efficient with regards to the execution of task demands
Least amount of time
Least number of clicks
Few or no errors
Other measures of efficiency
Learnability: how easy it is to carry out a task successfully
Measure the amount of time or number of clicks to complete the task and compare these to expert performance
Memorability: how easy it is to remember how to use a product
Compare the amount of time or clicks to complete a task after the user is away from the task for a while
Evaluating user satisfaction
User feedback about their interaction with the design
Cognitive measures: mental effort
Where the steps required to complete the task intuitively?
Emotional measures: feelings experienced
Was the visual layout appealing?
Did they feel frustrated while completing the task?
After completing the task, did the user have negative or positive emotions about the new design?
Evaluating if the design is useful and usable must be in comparison to the Status Quo (current interface) or Objective Outcome (for a completely new interface)