Evaluating a prototype with user data: the Gafael jar opener

(a) With the opener: 9 of 10 users (90%). Without: 7 of 10 (70%). Only U4 could not open it with the opener.

(b) The 7 users who opened it both ways are U2, U3, U5, U6, U7, U9 and U10.

Without: 14 + 6 + 21 + 8 + 25 + 18 + 30 = 122 s. 122 ÷ 7 = 17.4 s.
With: 8 + 7 + 10 + 7 + 11 + 9 + 12 = 64 s. 64 ÷ 7 = 9.1 s.
Cut in time: (122 − 64) ÷ 122 × 100 = 47.5%.

(c) All 10: 5 + 4 + 3 + 2 + 4 + 3 + 4 + 5 + 4 + 4 = 38. 38 ÷ 10 = 3.8. The 8 with arthritis (everyone except U3 and U6): 38 − 3 − 3 = 32. 32 ÷ 8 = 4.0.

(d) PLA: 146 g = 0.146 kg × £20.00 = £2.92. TPU: 0.022 kg × £35.00 = £0.77. Total £3.69.

Point	Evidence	Judgement
S1	Opened 53, 63, 70 and 82 mm lids every time, but slipped 3 times out of 3 on the 43 mm lid. So it works from about 53 mm, not 40 mm.	Partly met
S2	Handle measured 38 mm across, inside 35 to 45 mm.	Met
S3	168 g, which is 18 g (12%) over the 150 g limit. U2 and U4 both said it is heavy.	Not met
S4	9 of 10 users opened the test jar.	Met
S5	47.5% faster for the 7 who could open the jar both ways, above the 30% target.	Met
S6	3.8 for all 10, below 4.0. But exactly 4.0 for the 8 users with arthritis, who are the target users.	Partly met
S7	£3.69, which is £1.19 (47.6%) over the £2.50 limit.	Not met
S8	Jaw warped 2 mm after one dishwasher cycle at 65 °C, because PLA softens at around 60 °C.	Not met
S9	U2: “Looks like a proper kitchen gadget, not a hospital thing.” U7: “The yellow is easy to find in my drawer.” Only 2 users commented, because nobody was asked about the look.	Met, on thin evidence

“Works for everyone”: not supported. U4 could not open the jar with it, and it slipped on the 43 mm lid every time. It worked for 9 of 10 users and 4 of 5 lid sizes.

“Twice as fast”: nearly, on average. 17.4 s ÷ 9.1 s = 1.9 times as fast. But it was not faster for everyone: U3 took 7 s with it and 6 s without. The times also hide the opener’s biggest success: U1 and U8 could not open the jar at all without it, and did with it.

“All 10 users liked it”: not good evidence. “Do you like my opener?”, asked by someone they all know, invites a polite yes. The comfort ratings are more useful, and U4 gave 2 out of 5.

“It meets my specification”: not supported. It fully meets 4 of the 9 points, partly meets 2 and fails 3.

Rewritten conclusion: The Gafael opener let 9 of 10 users open a sealed 63 mm jar, including 2 people with rheumatoid arthritis who could not open it without help, and cut the mean time by 47.5% for the others. It fully meets 4 of my 9 specification points. It is too heavy at 168 g, too expensive at £3.69, not dishwasher safe, and it does not grip lids smaller than about 53 mm.

	Problem	Better method
1	Every user tried without the opener first. The second jar may have been easier because their hands were warmed up or they had practised, so part of the 47.5% may not be the opener at all.	Half the users go with the opener first and half without, chosen at random, so the order effect cancels out.
2	All 10 users know Elin, and she asked “Do you like my opener?” herself. People are kind to someone they know, so the ratings and the “yes” answers are probably too high.	Someone else runs the trial. Users fill in an anonymous card with neutral questions, such as “Rate how easy it was to grip, 1 to 5”, and a space for problems.
3	Only 10 users, all from one lunch club, only 2 without arthritis, and only one jar size in the timed trial. The results may not be the same for other people or other lids.	Test with at least 20 users from more than one group, and time several lid sizes, including 40 to 50 mm.

The Gafael jar opener meets the most important need, opening jars without help, but it is not yet ready for Mrs Hughes to use every day. Its main strength is function: 9 of 10 users opened a sealed jar with it, and U1 said it was “the first jar I’ve opened on my own in months”. For the users who could open a jar anyway, it cut the mean time from 17.4 s to 9.1 s, 47.5% faster. The 38 mm soft grip suits people with arthritis, who gave it a mean comfort rating of 4.0. However, it fails three points that Mrs Hughes herself asked for. At 168 g it is 18 g over the limit, and two users said it felt heavy. It warped in the dishwasher, which matters because she cannot scrub. It also costs £3.69 in materials, 47.6% over budget, which would make it harder to sell. The jaw slipped on a 43 mm lid, and U4 could not close it at all. Overall the prototype proves the idea works, but it needs to be lighter, dishwasher safe and suitable for small lids and small hands before it truly meets the users’ needs.

	Problem and evidence	Modification and how to make it	How to test it
1	Warped 2 mm after one dishwasher cycle at 65 °C (S8). Mrs Hughes: “It has to go in the dishwasher.”	Print the body in PETG, which softens at around 80 °C, instead of PLA. For mass production it would be injection moulded in polypropylene.	5 dishwasher cycles at 65 °C, then measure the jaw with a vernier caliper. Target: no measurable warping.
2	168 g (S3) and £3.69 (S7). “A bit heavy”, “it’s heavy”.	Keep the 180 mm length for leverage and the 38 mm grip, but make the handle hollow with thin internal ribs, printed with less infill.	Weigh it: target 150 g or less. Recalculate the cost from the new masses: target £2.50 or less. Repeat the 5 lid tests to check it is still strong enough.
3	Slipped on the 43 mm lid 3 times out of 3 (S1). U4: hands “too small to close it round the lid”.	Add a stepped jaw with a smaller inner step for 40 to 52 mm lids, so the user does not have to squeeze as far. Sand off the 1 mm seam ridge and round the handle edge, because U10 said it “digs in”.	Three tries on 40 mm and 43 mm lids. Ask U4 and U10 to try the new version and rate it on an anonymous card.