Their mission was to test all of the popular neuromarketing technologies and see which could actually predict consumer behavior after viewing the ads. The results may surprise you!
Welcome to the Brainfluence Podcast with Roger Dooley, author, speaker and educator on neuromarketing and the psychology of persuasion. Every week, we talk with thought leaders that will help you improve your influence with factual evidence and concrete research. Introducing your host, Roger Dooley.
Roger Dooley: Welcome to the Brainfluence Podcast. I’m Roger Dooley. I’m excited to introduce this week’s special guests. They are researchers at Temple University’s Center for Neural Decision Making at the Fox School of Business. The three guests are Angela Dimoka, Paul Pavlou, and Vinod Venkatraman. What I’m going to do is let each of you folks give a very short summary of who you are and what you do.
Angelika Dimoka: Good afternoon. This is Angelika Dimoka. I’m an Associate Professor at Department of Marketing in Fox School of Business at Temple University. I’m also the Director of the Center for Neural Decision Making.
Roger Dooley: Great. Let’s see. How about Paul next?
Paul Pavlou: Yes. My name is Paul Pavlou, good afternoon, Roger, and it’s a great pleasure to be talking to you. I’m Professor of Information Technology and Strategy and I’m also the Associate Dean of Research in Doctoral Programs at the Fox School of Business.
Roger Dooley: Great, and Vinod.
Vinod Venkatraman: My name is Vinod Venaktraman. I’m an Assistant Professor in the Marketing Department at Fox School of Business and I study a lot of decision making and how different methods like eye tracking and FMRI help us to understand how and what kind of decisions people make.
Roger Dooley: Great. Sounds like there’s some really talented folks here. I know that our listeners are going to be really interested because we don’t often hear from academics that are doing this kind of research. This is really exciting. Let’s start at the beginning and talk a little bit about the Center for Neural Decision Making or CNDM. What does the origin of this unit? Was there a visionary behind it? Is it a foundation? How did it get started?
Angelika Dimoka: Roger, it started back in 2008 when I joined Temple University. At that point, it was the center was, as you said, like a vision. It was something that I wanted to great. Of course, I never imagine it would be what it is right now. At that point, it was just me and the support that I was having from the Dean’s Office here at Fox School to create something in a business school that could completely revolutionize the field of marketing, especially specifically the field of advertising.
Bringing my expertise from neuroscience to the Business School, one of the ideas that I had when I came here was to bring all of this expertise to understand how consumers make decisions, especially how consumers’ brains are reacting and functioning when they’re interacting with different marketing or advertising stimuli.
After that, I had the pleasure that Vinod Venkatraman joined us a couple of years later and with the support, of course, of Paul, who has been involved with the center right at the beginning of it, we have created a center that is worldwide recognized for the research that we do here at Fox.
Roger Dooley: Great. Academics have often felt neuromarketing was a sketchy field, maybe somewhere a little bit more reliable than parapsychology or something. Do you get any brief from your colleagues about doing neuromarketing research or is there a gradual realization that, okay, some companies may have made wild claims or individuals but in fact that this is an area that’s suitable for serious work?
Vinod Venkatraman: It’s a great question because in academia, some of us refer to neuromarketing as a taboo word. We actually refer it more formally as decision neuroscience or consumer neuroscience simply for the reasons that you had just stated. I think part of it comes from the growth in the industry side of how people have taken some of the findings in the neuroscience area and made these big, broad claims about what these neuroscience claims can bring to the marketing field.
The general grief is that some of these haven’t been explicitly tested and could be much more farther claims than what they potentially could be. That’s a little bit of history over there but now I think what the field has seen, at least in the academic side, is a lot more concerted effort to do more research that’s applied so that we can specifically start addressing that exact question. What is it that really neuroscience can coordinate? What is it that neuroscience can really help understand about real-world decisions? More importantly, how can we use some kind of academic rigor in answering some of these questions that the industry is interested in?
Roger Dooley: Very good. I think it’s coming around because we’re seeing more academic dipping their toe in this water, not necessarily using the term neuromarketing for the reasons you mentioned but doing, in essence, what most people would accept as being neuromarketing-type work. This is going to be great for the industry, as well, because one thing that has been lacking is the academic research base. There just is not really been … There hasn’t been academic work out there to show that these techniques work.
Customers of neuromarketing firms are really relying solely on the data that the neuromarketing firms themselves provide, which often isn’t published and then also. It’s top secret client stuff because they don’t want their secret sauce revealed and so it’s exciting to see this going on. Beyond the advertising studies that you do there at the CNDM, what other kinds of projects? Decision making is a pretty broad area. Are there other fields of study that you’re working on?
Vinod Venkatraman: Yeah. Personally, I am very interested in the field of, say, consumer financial decision making right now. Let’s say things like how do people choose to decide what kind of retirement funds to invest in. A part of my interest is in annuities. When you basically retire, how do you de-cumulate your savings? Do you purchase annuities? What are some factors that lead you to make those kinds of decisions? I try to study this broadly again from the perspective of multiple methods and what can each method bring to the table and ultimate goal with some of that research is on what’s called chart architecture. How can we design better questions or better formats to help people make better decisions in some of these areas.
Again, the nature of the research is more applied to state findings that you can learn from neuroscience or eye tracking and biometrics and see how can we apply it to some really important real-world scenario. In this case, being consumer financial decisions.
Roger Dooley: How do you see the divide between neuroscience and then traditional psychology or social science research or behavior research? Because I guess my opinion has been that people are people, humans are humans, and that each field of study is really shining maybe a little bit different light on the same thing. Bob Cialdini just published a new book on some really fascinating topics, including some of the sorts of financial decision making and so on that are the nudge-type stuff that gets people to behave in a certain way and most of that work is based purely on behavior research but you’re looking at it from the neuroimaging side of things. How do you bring those two together or do you?
Vinod Venkatraman: Yeah. To me, the basic difference is outcome versus process difference. A lot of the behavioral stuff just focuses on the outcome and we try to see what are some things that can explain the outcome or how can we match the outcome? Ultimately, that’s where we want to get there but what a lot of us believe is understanding something about the mechanism or the process can give us better starting points to do the nudges or to help people make better decisions. I think that’s the primary difference.
We are no longer satisfied by just knowing what people do but it’s more we want to know why they do what they do, and that’s where a lot of these methods bring something new to the table.
Roger Dooley: Right. Yeah.
Vinod Venkatraman: I want to add something.
Paul Pavlou: Actually, of course, coming from a different perspective, as I mentioned, my expertise is in information technology and strategy, and I’m not a neuroscientist by training but I guess my reaction to your question is more about the neuroscience provides some very powerful tools that allow us to look into the human body in terms of physiological and neuroimaging tools into the brain, and those complement what we can use from with behavioral economic or other traditional approaches.
How I see this integrate value they provide is that they give us some additional tools to investigate more into these underlying mechanism as Vinod just said, into a neural mechanism, how the process really takes place in terms of understanding behavior, economic impacts, and so forth. Of course, all of us here actually also investigate the behavioral aspects and we complement them with this additional tool. All of us here were trained also in terms of behavior, economics, social methods, in addition to all these neuroscience methods complement and give us some additional power to investigate similar phenomena.
Roger Dooley: Great. Let’s talk a little bit about the ARF study that is now going to be part of your publication. I remember the first neurostandards effort by ARF a few years ago need it was a valiant effort that was I think originally intended to set some standards for neuromarketing techniques and it really didn’t come up with any standards and didn’t pick any technology winners or losers. The final report kind of read like a negotiated treaty where every technology got a nod.
Of course, at that point, they didn’t really, I don’t think, had the data to say, “Okay, hey, this is what you should use, this is what you shouldn’t use.” Tell us about the new study and what its objectives were and what’s been discovered so far.
Angelika Dimoka: Absolutely. Actually how this study was started, the neuro 2.0, was just because the neuro 1.0 didn’t have any conclusive results and there were a lot of challenges, it was a well-executed study but there were a lot of challenges on that one. At that point ARF came to us, actually, in one of the conferences that we organized, they came to us and they start challenging us how we would do it differently, and we expressed to them in many ways how we would do this study differently, and then they came back and said, “All right, why don’t you do it?” Then you can show us. They wanted to bring pretty much the academic rigor into this study and have all the different neuroficial logical measures tested in one group, in one facility, and then have a completely separate group to evaluate the results of the neuroficial logical data that we collected.
So really one of the important things that they wanted to avoid was to have any influences because if you remember in neuro 1.0, every company they executed their own study and they provided their own results. Of course, they had reviewers, but still, there wasn’t much information about how these studies were executed and the agreement that we made with them was that we will conduct all the studies here, the temple, we will collect all the data here and then we will provide that data to an independent group and that is the NYU group, who will be doing the econometric analysis to see which measure performs better.
That is an … Actually, I’ll pass it to Vinod, also, because I would like him to talk a little bit about this study. He is actually the first author of an article that just came out in the Journal of Marketing Research and he can tell a little bit about what this study is and what was the purpose of the study.
Vinod Venkatraman: Yeah. I think just to add to what Angelika was saying. I think the key here was to do more carefully controlled study where all the methods are being treated equally in terms of the protocol, every method, the protocol is exactly the same so you get different measures and then we had a very specific constraint prediction problem here. What we did is for all these ads we collected real-world performance data based on what happened to the product that was featured in the ad, and that’s where the folks from NYU who are experts in this area were involved so they collected this data and they came up with some kind of measures of ad elasticities, which essentially transfers to what’s successful with the ad.
Because we had a carefully-controlled protocol that was same across all the different methods, we could now meaningfully create a model where we put all these different methods or measures from these different methods and see which one was one of the strongest predictors of what happens out in the real world. That was the gist of the study and that was how it was different from neuro 1.0 where everybody could do whichever protocol that they were using at that point of time.
Roger Dooley: Right. Well Vinod, how are you sure that you’re doing each of these technologies right? Because if I’m, say, a company that has expertise in facial coding or EEG or whatever, I might say, “Gee, you guys didn’t get the right results because we’re the experts and we know how to do it right.” Did you involve these companies in or how do you make sure that you’re doing it in the best commercial way?
Angelika Dimoka: That’s a great question and as you already know and you hinted in the question, companies, even if they’re using the same technology, they have their own ways in which they use the technology, which in most cases is also proprietary. The way we did this study was we had two objectives. One is we would do it with as much academic rigor as possible. If you were doing an academic study, how have academics treated these different methodologies? How have they used these methods? What can we learn from that?
Because at the end of the day a lot of these companies’ proprietary data or proprietary software are based on existing academic research. We went back to the roots. We were like, if this is what academics did first, let’s try to follow that. At the same time, we also worked closely with some people that use these measures out in the industry. Part of it was we made some linkages and collaborations with companies that are currently doing this kind of research and we went and talked to and we said, “Hey, want to do this study. This is what we are thinking. Do you think you would have done something differently or at least without biasing what we are doing, do you think I should ask some additional questions? Do you I think should be looking at some additional measures that we haven’t thought about within the framework of what we are doing?”
This was helped a lot by RF2 because they had access to a lot of other companies that were doing this kind of helped, so they helped facilitate some of these early discussions and this is watch before the protocol was even created, we went through some lengthy pilot testing to make sure we’re collecting all the data that was necessary and made sure everything was accessible.
The other thing about the way the study was set up is once we collect the raw data, it was made accessible to people that wanted to do their own kind of analysis within there. That was the thing that where they could come up and say, “This is the kind of measure that we use. Can you plug this particular measure into the data that you have collected and see if the results are going to be any different?” Some of those things are still ongoing but that’s the brief answer to how did we try to account for the fact that all these things could be different.
Roger Dooley: Okay. Great. You mentioned, too, that the measure of success is performance in the marketplace. Right? How does that work and how do you determine that it was the ad that really worked versus some other factor. That’s been the traditional problem in evaluating advertising except in some very limited scope type things like website conversion where you can run very clean tests of what people are seeing. In the real world, sales are affected advertising, by external factors, by weather, by consumer mood changes and all kinds of factors.
Paul Pavlou: Actually, that’s, again, a very important question. Of course, there are many years of research in both academia and practice in the marketing and advertising field to assess how effective advertising is. In this case, as Vinod briefly mentioned earlier, our colleagues at NYU calculated the ad elasticities, which is a term that suggests how effective is the spending in advertising for a given time period and how much changes in sales during that period.
Of course, I will not claim that I’m an expert in this area but at least in terms of this interaction and trying to understand better our dependent variable. The basic idea is that the accumulated from the various companies that participate in this study, two to three years’ worth of data, sometimes monthly, sometimes weekly sales data for the products. They got all the information about the ad, the spending in different markets, and they were actually very encompassing in terms of studying that while accounting for numerous control variables to make sure that the various things you mentioned like weather or different conditions or various aspects actually are accounted for.
Those data are actually checked against any existing data this companies had. They were compared against other forms of ads. For example, what is the average effectiveness of ads based on the ad elasticities. IN general, it was a very appropriate measure.
Of course, I’d like to add that we try to maintain a clear separation between the work that our colleagues did at NYU in terms of getting the ad elasticities and the work that we conducted here at the Center of Neural Decision Making in terms of collecting the data, and the idea, essentially, we give them our measures across all these different methods and we said, “Okay, which ones are the most likely to predict your dependent variable?” We tried to stay agnostic to how these dependent variable was predicted.
That was the basic idea, and of course, we try to use the … Our colleagues at NYU tried to use the state-of-the-art practices to calculate the dependent variable with all these verifications and then they engaged in these econometrics, sophisticated analysis to see which method is the most predictive, in terms of the ad elasticities.
Roger Dooley: That leads right into the next question. How did the various technologies stack up against each other?
Vinod Venkatraman: The good thing about this whole thing was that the traditional methods, like the paper and pencil test that a lot of people tend to do was still a pretty good predictor of what happens in the real market. What we found was … Then we asked our question. Given everybody is going to do these kinds of paper and pencil tests, how do these additional methods contribute something more than what you would have found with the traditional measures? What we find in this study is that there is this one region in the brain, which is the ventral striatum that was the strongest contributor in addition to the paper and pencil measures.
In other words, the amount of variance that you can explain in the substance of an ad almost doubles by including this one particular measure from the brain, which is the ventral striatum.
Roger Dooley: What is that showing in terms of the consumer reaction to the ad if there is activity in that area? Does that mean they like the ad? That they like the product being advertised? That’s been another traditional problem in their marketing is yeah, we can see these things going on in the brain but what does it actually mean? We can see that okay, there’s engagement going on, there’s emotion, but what is that emotion? Maybe you can say, okay, the viewer likes this but what are they really liking? Are they liking the cute puppy? Are they liking the beer brand and so on?
Vinod Venkatraman: That’s a really interesting question, and like you said, that’s one of the big challenges, always, in most of the research that uses this area. One of the things about this particular area is it’s really well characterized in terms of what it does and it’s really popular in the literature for being associated with some kind of reward processing. It’s one of the primary targets for the dopaminogic neuron system, which is the primary neurons in the brain that are associated with reward. It’s also associated a lot with, say, motivation and stuff. Our thinking at this point is that it’s got something to do with desirability for the products that are featured in the ad.
The things you point out like liking and stuff, it’s a lot more easier to just ask that question, “Do you like the product?” Or, “Do you like the ad?” People can answer it a lot more easily but the challenge always has been with things like purchase intent or desirability, like how much do you desire this product or how likely are you to purchase this product? Those kinds of questions are a little bit more difficult for people to answer in a simple paper and pencil test.
What we believe is happening here is we are getting some kind of measure of these kinds of motivations or desirability for the product that’s actually featured in the ad that’s indexed by the level of activation of the ventral striatum and that’s probably one of the reasons why it’s such a strong predictor of advertising success down the road.
Roger Dooley: The one technology that actually shows what’s going on in the ventral striatum is FMRI, right?
Vinod Venkatraman: Yes. This data is based on FMRI. You’re right. Yeah.
Roger Dooley: What about all these other techniques, whether it’s implicit association or EEG? So far, is your data suggesting that these studies may be predictive but not necessarily more predictive than simply asking consumers?
Angelika Dimoka: Yes. That’s what we found in the study. We found that these brain area is the most predictive but we have to acknowledge that this study has some limitations, some limitations that actually could minimize the effect of the other methods. For example, other methods like EEG or the biometrics measures and skin conductance and heart rate have the advantage that you can look at moment-to-moment changes that are happening in the body for the purpose of the study because we were comparing everything with the traditional behavioral measures, which usually are measured after people watch the commercials, we follow the same process for all the neuroficial logical measures that we use.
Pretty much, we operate across the 30 seconds of the commercials and we’re trying to see another ad’s effect because of that commercial. That put some of the methodologies in disadvantage, especially biometrics and EEG. We don’t want to make any strong conclusions that FMRI is the way to go if you want to predict advertising success. Yes, we found in this study that performed much better than the other ones, but of course, many other studies have to take place to see what the other methods can contribute, as well.
Roger Dooley: Yeah. It seems like some of the other techniques might have specific purposes. For instance, if a brand is trying to figure out if their image is youth-oriented or health-oriented or something like that, one of the timed response-type tests where people view images and have to respond within a fraction of a second to see which image represents the brand more closely in their minds and they have to do that before they can consciously think about it.
A technique might be good at getting that particular kind of information but not necessarily predicting whether version A of an ad is better than version B.
Vinod Venkatraman: You’re absolutely right. In terms of, yeah, we use specifically only study 30-second TV spots, so that has a clear pattern in terms of the kind of responses you’re going to get. Even within there, going back to the point. If the objective was to come up with an interesting credo, for example, where you have different options for the seconds within the 30 seconds you want to change somewhere in the 15 seconds some data, I believe the moment-to-moment ones like biometrics and EEG would have much stronger prediction or help in those cases.
Here we were specifically interested in what was the effect of the entire 30 seconds of the ad and whether or not it can predict sales performance and there we basically find that this measure from this particular brain area was the strongest predictor.
Roger Dooley: I guess one that is a little bit of a disappointment to me, not in the work that you’ve done, but the fact that FMRI seems to be biggest value add from a prediction standpoint means that it is going to be probably for a while outside the budgets of many smaller advertisers. Some of the other techniques can be done in pretty quickly and using very low-cost equipment but FMRI machines are not all that available and they tend to be very expensive. Do you see hope for that in the future? Is available and cost coming down in a really significant way for at least in the near future we’re going to be limited to these pretty costly studies.
Paul Pavlou: Roger, I can provide my input into this. Obviously, these techniques are as easy as paper and pencil. Of course, they take some more expensive machinery and, of course, the data collection, the data processing are not trivial, either. However, the basic idea is that besides advertising and marketing, there are so many other uses of FMRI and the other neuroimaging techniques. Obviously, the cost has gone down considerably.
In terms of expertise, we can actually design very sophisticated studies with relatively few subjects and be able to get some very reliable results with a small set of subjects without an excessive cost. Of course, I will let my colleague discuss more specific about the cost. However, on a broader picture, Super Bowl was a few days ago and if 30-second ads were $4.5 million in terms of, and pretty much in this case it’s to test how effective the ad would be.
The cost of an FMRI study, which is in the thousands, not even close to the tens of thousands, then it’s something that it’s a tiny amount compared to how expensive TV advertising is. It’s all relative, of course, as my colleague said. This provides additional tests about the effectiveness of the ad and the next stage you see is how our knowledge and wisdom techniques, activation in different brain areas can predict how the advertising works.
That’s one perspective that we used.
Vinod Venkatraman: Yes, to put this perspective based on what Paul was saying. This whole study was done with a $300,000 grant from ARF. Paul said a 30-second commercial in Super Bowl was 4.5 million. I don’t think it’s as enormously high as some people make it out to be. Yes, it’s higher than most other methodologies, but one of the things that the field is actively trying to do now is to see reliably can we predict some of these things with smaller samples. Most FMRI studies use 30 subject samples and what we are trying to see is if we can somehow reliably demonstrate that the data from these 30-second samples can predict performance at the population level, which we have some details, some data within our own paper.
I think if the field can establish that better, I think that would drastically bring some of these costs down because you’re not looking to get these large target samples and stuff for your experiments but you can make the same kind of conclusions and even more with much small smaller.
Angelika Dimoka: Apart from that, also, the costs there are for the scanning time is going down. The accessibility of MRI scan also is becoming easier and easier. Actually, most of the top business schools right now in the United States and in many countries, as well, they do have people that they do this type of research similar with what we do, and all of us, we have access to scanners and, as I said, the rates are dropping. I don’t see that this will be the challenging anymore, that this methodology is very expensive.
Roger Dooley: Angelika, you wrote a paper about using FMRI in social science. Is the sample size a concern there, too? Because one thing that I think has hindered use of FMRI beyond the cost commercially has been the consumer products companies really don’t like to see 10 or 20 or 30 sample sizes. They like to see hundreds with broad demographic representation, different ethnic groups, and all that kind of stuff. How would you bring those together? Can you really achieve results with relatively small sample sizes?
Angelika Dimoka: It depends. Yes and no. Yes in cases like this article, that we’re discussing about. We were able to predict out of some 30 subjects, we were able to predict the responses of thousands of subjects, and we do see many other subjects. We have the power to predict how the rest of the population will behave out of these small sample sizes. Of course, we have to be more careful of when we want to generalize our results, what kind of sample we are using.
Paul Pavlou: Going back to the point that you’re raising. If you want to do things like segmentation and stuff, then obviously you need larger samples where you’re really interested in how each individual responding differently to the same scenario, but if you’re trying to make more general predictions, then essentially these sample sizes are sufficient based on what we know from the neuroscience literature so far.
Roger Dooley: Great. We’re just about out of time. Let me remind our listeners that we’ve speaking with Angelika Dimoka, Paul Pavlou, and Vinod Venkatraman, all from Temple University’s Center for Neural Decision Making. The Web address for the Center is long but if you navigate to fox.temple.edu, you should be able to find it pretty easily. We will, of course, have links to the CNDM’s website as well as links for our guests on the show notes page at rogerdooley.com/podcast.
There will be a text version of this podcast there, too. Angelika, Paul, and Vinod. Thanks for being on the show.
Paul Pavlou: Thank you so much, Roger. It was a great pleasure talking to you.
Angelika Dimoka: Thank you, Roger. Thank you very much.
Vinod Venkatraman: Thank you, Roger. It was a great pleasure talking to you.
Roger Dooley: Great. Looking forward to seeing what’s coming next from you folks. Bye, now.
Vinod Venkatraman: Thank you.
Angelika Dimoka: Bye.
Thank you for joining me for this episode of the Brainfluence Podcast. To continue the discussion and to find your own path to brainy success, please visit us at rogerdooley.com.