We might be ignoring the biggest variable in the room....

In Market Research, we obsess over data quality. We try to fight bots, fraud, and measurement error to make sure our 'raw material' is clean. But we might be ignoring the biggest variable in the room... the researcher....

A recent landmark study in Science Advances proves that clean data isn’t a silver bullet: researchers gave the exact same dataset to 71 independent teams. 𝗔𝗻𝗱 𝗱𝗲𝘀𝗽𝗶𝘁𝗲 𝘂𝘀𝗶𝗻𝗴 𝗶𝗱𝗲𝗻𝘁𝗶𝗰𝗮𝗹 𝗱𝗮𝘁𝗮, 𝘁𝗲𝗮𝗺𝘀 𝗿𝗲𝗮𝗰𝗵𝗲𝗱 𝘄𝗶𝗹𝗱𝗹𝘆 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗰𝗼𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻𝘀 𝗯𝗮𝘀𝗲𝗱 𝗼𝗻 𝘁𝗵𝗲𝗶𝗿 𝗼𝘄𝗻 𝘀𝘂𝗯𝗷𝗲𝗰𝘁𝗶𝘃𝗲 𝘁𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗰𝗵𝗼𝗶𝗰𝗲𝘀 𝗮𝗻𝗱 𝘀𝘂𝗯𝗰𝗼𝗻𝘀𝗰𝗶𝗼𝘂𝘀 𝗯𝗶𝗮𝘀𝗲𝘀.

It turns out "data-driven" insights are often "researcher-steered".... If 71 experts can’t agree on one truth from one dataset, how do we ensure our commercial recommendations are actually robust? How sensitive are the conclusions to the choices the researchers made?

Methodological transparency and internal 'red teaming' (have a second researcher try to "break" your conclusion using the same data) might be a good start. 𝗪𝗵𝗮𝘁 𝗲𝗹𝘀𝗲 𝗵𝗮𝘃𝗲 𝘆𝗼𝘂 𝘀𝗲𝗲𝗻 𝘁𝗵𝗮𝘁 𝗰𝗮𝗻 𝗿𝗲𝗱𝘂𝗰𝗲 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵𝗲𝗿-𝗱𝗿𝗶𝘃𝗲𝗻 𝗯𝗶𝗮𝘀𝗲𝘀?

------------

Link to the study:https://lnkd.in/eCfAwWfk

There is also an interesting Youtube video with the lead researcher further exploring this phenomenon:https://lnkd.in/ey44fTVS