Traditional market research, based on clipboards and questions, has a glaring weakness, according to Carl Wong, who has worked in the industry for more than 20 years.

He says people cannot answer some questions honestly because they are not aware of their own deep underlying motives.

"We are all influenced by things we're not able, or willing, to reflect on at the time a purchasing decision is taken or when we answer a survey question," says Mr Wong.

That gap in our understanding is being filled by a data-rich world that generates masses of information about our inner desires through tracking cookies and a host of other clues we leave behind as we trek across websites that pique our interest.

In addition, market researchers can access massive amounts of computing power and data storage using low-cost cloud computing services, which offer data processing and storage that can be quickly scaled-up or down, depending on the customer's needs.

This tech-fuelled revolution is shedding light on how we conceal our innermost urges from ourselves - for example why we are drawn to the exploits of certain public figures and celebrities.

Jason Brownlee researches consumer behaviour on a grand scale, fuelled by the social media data explosion and the resulting caches of big data.

He says people cannot answer some questions honestly because they are not aware of their own deep underlying motives.

"We are all influenced by things we're not able, or willing, to reflect on at the time a purchasing decision is taken or when we answer a survey question," says Mr Wong.

That gap in our understanding is being filled by a data-rich world that generates masses of information about our inner desires through tracking cookies and a host of other clues we leave behind as we trek across websites that pique our interest.

In addition, market researchers can access massive amounts of computing power and data storage using low-cost cloud computing services, which offer data processing and storage that can be quickly scaled-up or down, depending on the customer's needs.

This tech-fuelled revolution is shedding light on how we conceal our innermost urges from ourselves - for example why we are drawn to the exploits of certain public figures and celebrities.

Jason Brownlee researches consumer behaviour on a grand scale, fuelled by the social media data explosion and the resulting caches of big data.