Thinkers50 Curated LinkedIn Live with Dashun Wang | The Science of Science

Dashun Wang is an associate professor of management and organizations at the Kellogg School of Management, where he is the founding director of the Center for Science of Science and Innovation. He is also a core faculty member at the Northwestern Institute on complex systems.

Transcript:

Stuart Crainer:
Hello, welcome to the Thinkers 50 Radar, 2021 series, brought to you with LinkedIn life. I’m Stuart Crainer.

Des Dearlove:
And I’m Des Dearlove, and we are the founders of Thinkers 50, the world’s most reliable resource for identifying, ranking and sharing the leading management ideas of our age, ideas that can make a real difference in the world.

Stuart Crainer
Our belief in the power of ideas has been the foundation of our work, since we launched the first ever global ranking of management thinkers in 2001. We’ve published a new Thinkers 50 ranking every two years since, and it remains the premier ranking of its kind.

Des Dearlove:
So we are excited that 2021, a year in which fresh thinking and human ingenuity are more important than ever, is also a Thinkers 50 year.

Stuart Crainer
Nominations are now open for both the ranking of management thinkers and the distinguished achievement awards, which the Financial Times very perceptively calls the Oscars of management thinking.

Des Dearlove:
Short lists for the Thinkers 50 awards will be announced during the summer and then the year’s finale on the 15th and 16th of November will bring all the excitement of a new ranking and the naming of our Thinkers 50 2021 award recipients.

Stuart Crainer
In this series of 30 minute webinars, we want to showcase some of the freshest and most interesting ideas and to bring you the new voices of management thinking.

Des Dearlove:
We want to inspire you to seize this moment to create a better future for you and your organization.

Stuart Crainer
Our guest today will certainly help along the way. His name is Dashun Wang an associate professor of management and organizations at the Kellogg School of Management, where he is the founding director of the center for the Science of Science and Innovation. He’s also a core faculty member at the Northwestern Institute on Complex Systems.

Des Dearlove:
Dashun’s current research focuses on the science of science, a quest to turn scientific methods and curiosities upon ourselves. His research hopes to use and develop tools from complexity sciences, and artificial intelligence to explore the opportunities for innovation, and promises of prosperity offered by the data explosion in science. His first book, “The Science of Science” is now available.

Stuart Crainer
Dashun, welcome and thank you for joining us. I think now’s the time to plug “The Science of Science” book. You can hold it up on camera-

Dashun Wang:
Thank you so much, now it’s a shameless but obligatory plug. The book is now available on Amazon if you want to order a physical copy. Pick the paperback, which is just as good as the hard copy, but much, much cheaper. At the same time I’m also excited to say, part of the Open Science Movement, is I’m also excited to be releasing the book, the PDF version, freely available online in the next few weeks. So if you don’t want to [inaudible] book, trace me on my website and hopefully the book will be available, free to everybody to read.

Stuart Crainer
That sounds a really great innovation. So what’s going to happen today is our usual format. We’re going to hand over to Dashun who’s going to talk for 10 minutes or so, and then we’re going to have a discussion about his ideas and the science of science after that. So Dashun, the virtual stage is yours.

Dashun Wang:
Well, thank you so much. Thank you everybody for having me. It’s a real pleasure to be here. And I thought today… I’m a recovering physicist in a business school, and I thought, given the 15 minutes that we have today, first of all, is to show you some of the new thinkings we have around failures and it’s relationship to success. And then my goal in the first part of the presentation is to present some of the new data that we have, and hopefully that helps anchor some of the discussions following the presentation.
So truth be told, I started my career trying to understand success, and trying to look at these people’s ideas, organizations, and their achievements, and trying to understand the patterns behind them. And then as I studied them for about five to six years, and many of my colleagues also joined me in that venture, I started to realize a big flaw in this thinking, that is, what we’re really ignoring is the many failures that are also happening at the same time.

So in many ways… That’s why about a couple of years ago, I told my group members that I’m actually now going to study failure rather than success. And for about three months, nobody believed me, that I was serious. So the overall thinking here is very simple, is to think about, while we may have succeeded in understanding success, we have failed to really understand failures.

So after a couple of years now, I’m happy to report that we have now gathered several different kinds of new data and built several different kinds of new theoretical frameworks that have broadly improved our understanding of failure on several dimensions. For the interest of time today, I’m going to present some data from joined three different papers, including papers published in Nature and Nature Communications, as well as working papers that has yet to be published, to try to show you some data that improves our thinkings around two dimensions of failures in particular.
First is to think about the consequence of failures. Do failures matter? What’s the consequence of early failure on long-term career outcomes? And second is to think about other impending signals in failures that eventually lead to success. Are there things we can learn from failures that can help us succeed sooner? What are they and how?
So first off, the consequence of failure, how big a difference does a failure make in your career? And so here, I want to focus on a specific kind of failure that has fascinated me for a long time, namely near miss. Near miss is the kind of failures, almost, but it didn’t quite make it. So this happens all the time in sports, as well as in every other domain, so the question I want to ask is, if you had an early career near miss, how big a difference in the future does it make?

The opportunity came when NIH came to me and offered me their data sets that requires over 700,000 competing grant applications, including both funded and unfunded applications. In this case, NIH is the largest funder for biomedical research in the world, with an annual budget now close to $40 billion. And in this case, NIH is an ideal example for us to really study failure. So excellent setting, in the sense that the way NIH ranks their grant proposals is through peer review panels, that is each proposal is graded by a certain score. And then NIH basically sorts these grant proposals and funds them one by one, until one point they run out of money. So that creates this obvious arbitrary cutoff, what they call pay line, where it separates out these two very interesting populations of junior PIs. Here I’m focusing on junior professors, think about people starting out their life within the past three years.

And in this case, this is what economists called the identical twins, between narrow winners and near misses. Before that day, both of them basically are identical people in both observable and non observable ways. But after that day, there’s a big difference between the two. Narrow winners now have a million dollars for the next five years, to conduct the research that they proposed. Near misses at that point have nothing.
So, I want to think about, many years later, imagine these two groups of people come back for a job interview. Who do you hire? So in this case, the answer appears to be quite simple. We want to think about, even though they were the same people, this lucky event tilted the scale where one succeeded when the other one didn’t. So we should hire the one who succeeded because we know rich gets richer, success begets success, winning begets more winnings.

But, in this case, we just wanted to look at the data and let’s just examine for the two groups of people over the next 10 years, out of the papers they publish, how good are these papers? So we started [inaudible] their publications, and here we’re going to measure the impact of a paper using the ISO standard measurements, in the field of science of science, called heat paper probability. It measures, out of all the papers you published, what’s the fraction of them becoming a hit paper in your field a year, defined as top five percentile citations in the same field a year. What we see is that narrow winners, in the next five years, publish hit papers at 13%, much, much higher than the baseline 5% rate. So this is consistent to a competitive process at NIH. Being a narrow winner at NIH is not a walk in the park, so these people are substantially better than the same people in their field.

But when we look at near misses, we see they publish a hit paper in the next five years at 16%. If we look at their papers in the next five years, from year six to 10, we see basically a similar scale. So, what we see here is paradoxically, somehow near misses systematically outperform narrow wins, in the future. When we found this finding, I was like, “This can’t be right. There must be something wrong.” So we varied the measurements across all sorts of measurements that we can think of. But every time we measured it from the data, we arrived at the same conclusion. Interestingly, if you look at the number of papers these two groups have published, you see they’ve published a similar number of papers. So what this means is that near misses had fewer initial grants by design, yet they ultimately published as many papers and most surprisingly, they produced the work that garnered a substantially higher impact.

So a natural question at this point is, what is going on here? And a common hypothesis, called a screening hypothesis, it goes something like this, is that early career failure is actually of consequence. So when that happens, it differentially removed people from two sides of the aisle. So in this case, it then raises the question of this idea of quit versus grit, is to say, well, is this just a better half of the population left? Or they become a better version of themselves? One contribution of this work we did was to be able to move beyond this hypothesis to show that this screening hypothesis, while it may account for some of the relationship, it’s not sufficient to explain what we observed here.

So as a whole, then this actually offers among the first empirical evidence, consistent with Nietzsche’s classic phrase, of what doesn’t kill you makes you stronger. Overall, this is trying to show that, while the traditional wisdom of winning begets more winnings, it seems like the story is more complex than that. So before I actually get too excited about this, let me show you another domain of data, try to help us think about this. Because if this is true, then it must be also true for other domains. So let’s think about a domain that we’re familiar with. I think about these winners, all winners, standing on a podium, accepting their Olympics medals after years of training. So this is a familiar picture, but what I want to bring your attention, is to think about the losers that are not in this picture.

In other words, let me just subtract this with this abstract view of all these winners that we’re familiar with, but for every three people standing on the podium, there’s always a number four that we cared so little about. And this is really interesting because I was trying to find a picture of the number four online, but I couldn’t really find it. Instead, I find so many pictures about the reward ceremony, everywhere. But that raises a very interesting question, is that given in this case, bronze medalists objectively outperform the first place counterparts, then if I compare these two people in the future, who will perform better in the future? So then that prompt me and my [inaudible] along with other collaborators to collect systematically all the data about Olympics and world-class competitions.

And here we focused on young athletes, again, below the age of 21. And we categorized them based on their ranking in their initial competition final. I don’t want to just compare their future performance. If we just look at the winners, namely gold, silver, bronze medalist, we see a familiar pattern, that is, if I look at people who started a gold medalist, how many medals do you win in the next four years? What we see is the familiar pattern that past winners is expected to be a bigger future winner. Just knowing your starting point has yields substantial predictive power over how well you will do in the future. So if we look at the number five and number six, in this case, we see they follow a similar trend. So it’s almost as if these curves follow a certain trend.
Except when I look at number four, it seems to actually jump out of the pack. In other words, number four seems to be achieving more than we would expect otherwise. While the explanation for this is to say, while maybe you didn’t make a podium at the previous competition, then you selectively go to less competitive competitions in the next rounds. So what we wanted to look at next, is to say, why don’t we pick two people in one competition, and look, when they meet again on the same starting line, run head to head with each other, who performs better? So when we do that, when we look at the number one and number two, gold versus silver, we see gold medal is more likely to win in the future. We look at number two and number three, we see the same pattern. If we look at number five, number six, or number four, and number five, we’re seeing the same pattern. Past winners are more likely to be future winners.

Except between number three and number four, where we see a reversal of performance, where the fourth place finishers are more likely to upset their bronze model counterparts. Here are interesting and surprising results I want to leave you with for this part of the presentation, is to think about extraordinary performance. If I take these people who start at a certain rank, 1, 2, 3, 4, 5, 6, who are more likely to break a record in terms of world record, Olympic record or championship record? What we see is that the first group, top group, is the gold medalist, but what’s really interesting is that the second group is neither the silver medal or the bronze medalist, but the fourth place finishers. So the initial fourth place finishers seems to be the second most likely group for record breaking performance. And once you have this data, then it’s easy to cherry pick on some examples. For example, a famous example would be Asafa Powell, who is a very famous sprinter. He set a 100 meter world record twice. Over the years, many gold medal, silver medal, bronze medal performances.

But what was interesting is that when he first started his career, he had [inaudible] on the number four. He actually placed fourth in both the 100 and 200 meter sprint. So overall, these findings begin to establish failure as a counterintuitive predictor for future success. And overall it shows that for those who persevere, early failure should not be taken as a negative signal, but perhaps rather the opposite. So hopefully this first of all will give you some optimistic picture to think about when you had a setback… what’s interesting about failure is that no one’s immune to it. So when that occurs, hopefully this will not be your first reaction anymore. And rather you will think about something around these lines, and remember that for innovators and managers, it’s important to keep in mind that winners misclassified as losers today could end up being the bigger winner tomorrow.

So in the next two minutes, let me show you another research to think about. If failure and success happens all the time, can we learn from past failures and lead to success? So here I’m quoting, Michael Jordan’s famous saying, is that, “I’ve failed over and over again in my life. And that’s why I succeed.” So the reality is that only some history of failures lead to success. So which failure patterns ultimately yield results? Conventional [inaudible] here centers around the luck or differences between winners and losers, because of their learning strategies or individual characteristics. But our research published in Nature shows that this is not so simple. In particular, here we study failures across three different domains in terms of NIH investigators trying to apply for a grant and then failed, and then try again, failed, and eventually land that grant. Or think about entrepreneurs that found a startup that failed, found another startup, failed, and eventually found a startup that succeeded, or not.

Or here, we also look at a very unconventional domain of research, where we look at the terrorist organizations. In this case, we look at terrorist groups that launch an attack, didn’t kill anyone, didn’t claim any casualty, and launch another attack, eventually as they move towards the lethal attacks. So across all these three domains, we want to build a simple but general framework to capture this. And the way I want to think about this is to think about a model of how successive attempts build on one another. So the key to this model is to view failure as a wonderful thing that offers two crucial assets. That is, first, it offers experience, and second, it offers you some feedback. So imagine each of the times include multiple components, like business startup requires a viable idea, revenue model, operational practices, or leadership teams, and others.

So failure in this domain and not achieving an IPO will help us understand whether we should reuse certain components, for example, speaking with the same team or the same revenue model, and take the past failure as a feedback to think about, how do we initiate another attempt? So once we assume this model and the model makes a rather surprising prediction, is to show that how much we benefit from failure actually doesn’t follow a smooth task, but a very abrupt one characterized by tipping points, that separates stagnation and successful region. Very much similar to the physical transition for water in the phase transition sense. So in the stagnation region, people learn very little from past failures. So think about entrepreneurs learning only from the most recent startups while ignoring all previous failures. So those in this region hit an early stagnation point, failing to engage in intelligent improvement.

In many cases, they threw out prior times altogether, though some times may have been on target and advantageous. So this stagnation region is similar to water’s physical phase of ice. Think about when you’re going from a temperature, going from minus 40 degrees Celsius to minus 10 degrees. Ice remains solid. But then if you learn just a few more failures, you quickly transition, [inaudible] past the tipping point. Now you are entering the regime of progression and success. So in this case, innovators that use feedback to engage in intelligence improvement, yield incrementally better attempts, and ultimately to victory. So those who eventually succeed, fail much faster and increasingly faster with each attempt, reusing previous components and rather discarding the baby with the bath water, as your stagnant peers do. So more importantly than this [inaudible] tipping points suggest that small changes can make a big difference.

So in this model, if we think about, people are along this critical point, they may be very similar in their luck or learning styles or productivity, but just like water going from minus one degree to plus one degree, the temperature difference may be negligible, but the reality is very, very different. One may be successful and another maybe keeps trying, but not seeing the light of the tunnel. So despite the simplicity of the model and the diversity in the domains we studied, we find that people across different fields, Paris entrepreneurs or scientists, predictably divert into success and stagnation groups described here. Although both groups are of a similar size and initial characteristics from the beginning. So the idea of tipping points tells us that even in the absence of distinguishing characteristics, individuals and organizations may experience fundamentally different outcomes. The two groups may follow fundamentally different temporal patterns that are distinguishable at a very early stage.

So taken together, Thomas Edison, who is probably the most famous one to fail many, many times, had this phrase of, “Many of life’s failures are people who didn’t realize how close they were to success when they gave up.” So hopefully this will help us understand better [inaudible] how close we are to success, as we fail over and over. Thank you so much. It’s so great to share at least one direction of the overall research agenda, around science of science, to think about how do we use the new data and different computational methods to better understand issues around innovation and science. Feel free to check out my website for other related work as well. Thank you very much.

Stuart Crainer
Thank you, Dashun. Let us have any questions. People have joined us from throughout the world. Good to see everybody here. What a fascinating subject, it’s amazing. It’s so neglected, and we’re so obsessed with success. Dashun, a practical question, what does this mean for organizations, in their hiring in particular? Because from what you’re saying, they should be hiring people who’ve experienced near misses, effectively.

Dashun Wang:
Yes, I think so. Very good question, actually. So maybe it’s easier to think about my own practice. This research actually improved my own practice, as I hire people regularly in my team. There’s this cliche about the idea that we want to ask people, tell me about your failure experience. And usually this traditionally has been a cliche, in terms of thinking about… It seems like an interesting question to ask, but this data does tell us that these failures, and this is a very important qualifier, is that people who manage to persevere after failure, and the failure could actually become a predictor for future success. So in this case then I elevated the importance of failure, in terms of the interviewing process, is to think about specifically, be very serious of studying, ask candidates about their past failure experience, and think about what are things they learn from that experience.

So on my implication, yes, absolutely. I think this changed my view about the importance of failing early in your career, and become a predictor for future success. And there are many behavioral research, that shows why it could be so. Just to think about this idea that failure may teach many viable lessons that otherwise are difficult to learn. So that’s been very, very important. And another one that’s an equally interesting implication for me, is to also think about how these results change my view about success. I recently had this experience with my student, where she had a great paper published very early on in her career.

But when we published a paper, we actually sit down and think about, are we narrow winners in this case? And how should we think about this winning? Because I feel like in my experience, what’s interesting about narrow winners, is that most narrow winners don’t think they’re narrow winners, but in reality, a lot of winning may just be split by a hair and you are lucky to be on the winning side. And so that also has implications, to think about how to we internalize and conceptualize this winning to continually improve ourselves and not just become complacent?

Des Dearlove:
Gosh, absolutely fascinating stuff, just getting your head around it. I’ve got some questions coming in. Terry wants to know, is there a link to developing more grit and persistence through experiencing failures?

Dashun Wang:
Yeah. So that’s an excellent question. And I think it relates to phenomenal work done by Angela Duckworth, I’d recommend the book, great, to everybody. In this research, we have not surveyed people in terms of their grit factors and psychological factors. What we’re observing is, when they fail, how do they initiate another attempt? So in this respect, I think there are some things to learn around this research, to think about how do we fail more intelligently? I think a very important idea here is to think about the model, in terms of taking into account past failure experiences. I think what’s very important is to realize your past failure experience can give you experience as well as feedback. And what you want is to try to make sure your eye’s on the prize, try to update the components that didn’t work so well, but keeping the ones that worked well.

So in other words, you actually want to think about managing incremental improvements, so that you can retain what worked well, only updating what doesn’t work out. What’s interesting about the failure group that we observed, in other words, the group that tried over and over, but didn’t achieve success, that group of people didn’t work less. Actually they worked more, they’re actually less efficient. What happened there is that they basically started from scratch, not learning enough from past experience. So what’s interesting here is to also think about this idea of work smarter, but not harder. Try to diagnose yourself, how this is going. For managers, it could also be relevant, because as a manager, this also gives us a tool to diagnose, as we fail over and over, how well we’re doing. Are we in the stagnation region or are we in the progression region? The mantra of Silicon Valley, is to say fail fast. But I think the key here is to think about fail faster, not just fail fast, but needs to fail increasingly fast as an indication for your learning experience, your learning from failure.

Stuart Crainer
Joel Malard’s got an interesting question. Do failures lead to a wider and more robust network of mentors and collaborators? I suppose-

Dashun Wang:
That’s-

Stuart Crainer
Yeah. Narrow winners might think they don’t need anyone else. They’ve got it cracked.

Dashun Wang:
That is another fascinating question. Think this is actually an active area for research. I think it’s very likely, across all the theoretical frameworks, that this is likely to happen. Some behavioral research suggests that when people fail, they enable wider search process and that is consistent with this idea. You may be seeking out more advantageous mentors or collaborators. We don’t yet have the evidence to show this yet, but I think this is one of the many things that this research enables us, next, to look further into the data.

Des Dearlove:
Okay. Margaret is asking, can you please provide the information on the website? We will post that at a later day. Stay posted. And I’m sure you’ll be able to share that information also. But I think the interesting thing here is we are actually quantifying that we do indeed learn more from failure than we possibly do from success, but can you talk a little bit more about the two elements, experience and feedback? How do these people that go on to then succeed, your number fours, if you like, how do they interpret the feedback? How do we avoid just making the same mistakes again and again?

Dashun Wang:
Correct. And also individuals will be able to infer differently from their past experience, and also some past experiences, the feedback may be more ambiguous than others. And these are all excellent questions. And I think what we’re taking here is a very simplistic approach, is to think about, on average, what might happen. And what’s important here is to think very carefully and strategically about your past experience. For example, I experience failure as a daily basis, and that’s part of why I feel like I have some expertise to study this, and motivation to do so. But every time I fail, I want to think back to, what are the different components that I included in this attempt, and what are the components that I do well, or not, based on the older feedback I can get and then try to diagnose myself to think about, when I try again, what are the components I should try to retain, what are the components I should focus my energy and try to improve?

Stuart Crainer
A number of people have commented that, if only the education system had access to your research and understanding of your research because presumably that would fundamentally change the way we approach education.

Dashun Wang:
I’m very delighted to hear that. I fully agree, but… These are all the new data that we’re learning and hopefully we’ll be able to bring this to wider and wider audience and this audience and the different institutions can also help us stress test these results and test the generalizability of these results, as well. We ourselves are working on the generalizability of these results and we’re seeing several very, very promising signals from ongoing work that this seems to be generalizable from different domains.

Des Dearlove:
Dashun, good point from Joti here, saying our corporate performance systems are allergic to failure. I think we’ve somehow trained ourselves to make ourselves allergic to failure. How can we change that? Because it’s a mindset thing, isn’t it?

Dashun Wang:
Yes. It is a mindset thing. The greatest thing I learned as a non-native speaker, first when I learned English, is that you say hello by saying, how are you? And then the answer is uniformly fine, great, fantastic, awesome, cannot be better, in reality, no matter what you’re going through. So this is a big deal, very clearly, not just in the corporate culture, but also just in the culture more generally. This actually raises questions for, what should we do as an individual and what should we do as a manager? As an individual, to me, it’s also been very educational, this own research. For an individual, it’s important to realize that reality is necessarily rosier than it really is. Everybody that has Facebook will understand. All your friends are either on the beach or some five star hotel, exotic place at all times, but that’s not really what’s happening.

So we have to realize, the world is… all the rare successes are overrepresented in the world. For example, if you look at a LinkedIn or resume, it’s completely conclusively composed of just successful experiences. At the same time, the common failures are already underrepresented in the data, sometimes even forgotten and or ignored. So for an individual it’s very important to think about these selection bias in the data. And then think about how do you learn from this data that has inherently substantial consequences on selection biases. How do you learn from them and update your own prior? So as an individual, to me, that’s been a very, very important point for me, to think about how to conceptualize success and failure.

Stuart Crainer
Dashun, unfortunately we’re out of time. This is just a taster of Dashun Wang’s fantastic research. Failure is such a fertile area. It’s just amazing, isn’t it? And it’s universal. It’s riveting. It’s human. And I think the way you conceptualize it makes it very accessible and challenges a lot of very fundamental ideas. So thank you very much for joining us today. Check out Dashun’s book, “The Science of Science”. There’s no need for him to plug it again. It’s okay. I’ve got that covered. And check out his website for all these details, dashunwang.com. Dashun, thank you very much for joining us, and thank you all for joining us, and we look forward to see you again next week. Thank you.