Be Weird

HP's recent problem with facial recognition software failing to recognize the face of a black man underscores the importance of good quality assurance (QA) testing. Regardless of whether it was just an issue of too much backlight or something else with the algorithm, there is a problem here that should have been caught.

Recognition of different races is something that should have been pretty obvious to test. Recognition in poor lighting situations is another obvious one. HP likely did test both of these items.

It is just as important for QA teams to test things that aren't quite so obvious, such as the combination of race and poor lighting. What if you throw glasses, a hat, and headphones into the mix? What about glittery makeup? Facial hair? Vibration from using a laptop with the camera while riding as a passenger in a car or bus?

How far do you take it? Where are the reasonable limits?

For a good QA team, that's a trick question. There are no limits.

I remember finding some pretty obscure stuff when I did QA testing on videoconferencing units for Sorenson Technologies. The main product I worked on was sold as the D-Link i2eye. We stuck them in the refrigerator and under hot pads to test extreme temperatures. We'd leave a call going for 48 hours straight. We affectionately referred to one bug I found as the Kevin Bacon bug, referring to his movie Hollow Man, wherein his character turned invisible. I figured out how to send video from a third party unit to a unit that was in a call with someone else. The unit received video from both sources and mixed them together, so you'd get something that looked kind of like a semi-transparent shape moving around on the background of the other video.

What did they do about the bug I found? Nothing. They determined it was not likely enough to actually happen to warrant setting up the unit to filter where it received video from. It didn't matter to me. My job wasn't to determine what the programmers were to work on. It was to do weird stuff and report the results. Now that I manage programmers and QA testers, it is my job to prioritize what gets worked on and to remind the QA testers to stay weird.