Big Data Causes Night Blindness

March 02, 2014 • 2 minutes to read

I like to compare mining big-data for UX insights to shining a flashlight in a dark room. The beam of light from a handheld light will brightly illuminate the darkness. The harsh beam brings highlights and shadows into vivid relief. In some ways the definition is even higher than under natural conditions.

Quantitative analysis is a powerful tool for any UX practitioner. Traffic logs and multivariate tests offer insights into user behavior with less subjectivity and bias. Usability labs, interviews and surveys can provide color around user needs but objective measurement provides proof. Mining big-data can empower the novice UX research but it needs a wide area of focus to truly shine.

On the other hand the narrow beam limits your area of focus. Only a single vector is visible at a time. A sharp contrast forms between the lit area and the darkness surrounding it. This contrast creates fovea blindness as the eye shifts focus from cones to rods. This is the same phenomenon experienced by stage performers and put to use in one way mirrors.

The UX practitioner running quantitative tests faces data foveal blindness. Focusing on that data behind a single interaction can block you from seeing downstream effects. For example: Clarifying the pricing of your signup will cause near term and longer effects you need to be aware of. The close-range impact may be a decrease in purchase conversions; but at the same time there can be corresponding decrease in churn downstream. These aren’t equivalent changes because churn is always more expensive than lost sales. It’s a dissatisfied customer that requires support and maintenance.

Making sense of the complex interaction between short-range and long-range impact of design changes is similar to navigating a dark room. A flashlight will illuminate your current vector but that vector is only valuable if it’s the correct one. A dark room with multiple dimensions has multiple pathways through it. Determining the correct pathway requires understanding the surrounding space. Moving that flashlight left and right will illuminate other potential vectors. New opportunities and dangers are revealed. A richer understanding enables you to purposefully change direction if needed.

The savvy UX practitioner needs to shine their beam of analysis in multiple directions to understand a problem. The insights from a single metric will always pale in comparison to triangulating that insight with multiple metrics. Deeper insights will help you improve your designing by illuminating the most valuable problems it faces.