andres.thor / Foter / CC BY-NC-SA

I went to the dentist today for a regular checkup. Everything was fine, except for the wisdom teeth. They were coming in at a crooked angle, so the dentist recommend that I get all four of them pulled out as soon as possible. Thus, an hour later, I found myself in the surgeons office, waiting for my wisdom teeth to be extracted.

By far the worst part of the process was the injection of the local anesthetic. I felt considerable pain, noticed that my heart started racing, and felt nauseous as if I were going to faint.

Although I’m sure that these symptoms are common for people getting their wisdom teeth removed,  I can’t help but feel that I made the situation worse by 1) eating a big meal of butternut squash about half an hour beforehand (which most likely added to the nausea), and 2) consuming a rare cup of coffee in the morning (which probably added to the increased heart rate).

The lesson that I’m taking away from this is variance reduction. You can never completely plan for the future, but it is certainly possible to reduce the variability of the possible outcomes. If I had avoided the caffeine and the big meal I still would have experienced pain during the procedure, but I’m pretty sure that my reaction would have been less extreme.

In general, I think there is always a tension between a desire for stability, and a striving for unpredictable gains.

On the one hand coffee can lead to an increase in creativity, but on the other hand it can also make your behavior more erratic. Likewise, being an entrepreneur opens the possibility of making millions of dollars with a big idea, but at the same time it also leaves you with a far more variable income than a standard office job would.

For me, I think I prefer the route of stability  Something about knowing what will happen day after day is attractive to me. The lack of variance allows you to build incrementally without too much risk of catastrophic failure. Then again, I’m aware that I may be falling into a cognitive bias here. Studies show that people prefer weather that is worse but less variable (e.g. 45 degrees plus or minus 5 degrees), than whether which is always better but has more variability(e.g. 75 degrees plus or minus 15 degrees). There is something about variability that is inherently distressful, and this distress may lead to bad decisions.

Is it possible to retain unpredictability so that you have the chance of massive gains while still maintaining a high degree of stability? I’m not sure. They don’t seem to be completely incompatible, but combining them is something I’m still trying to work out.

—

Do you tend to prefer stability or do you strive for unpredictability in your life? Perhaps this is a personality thing. Please share your thoughts in the comments below.

“Whoever sees the web of cause and effect, sees the way”–mahàhatthipadopamasutta The Great Elephant Footprint Sutra

In this post I want to share some of my recent thoughts about the nature of wisdom. This article is fairly dense and long, and some of the ideas definitely need to be developed more. With this in mind, I’d encourage you to actively challenge my thesis and post your thoughts in the comments below.

First a Definition

Since wisdom is such a vague term that is used in many ways I’d like to begin by clarifying what I mean by wisdom.

To me, wisdom means being able to make sound decisions about practical life scenarios. Specifically, wise people are sensitive to a problems complexity and lack of structure and combine this sensitivity with an understanding of their own limitations. In addition, they have an equally distinctive capacity for empathy and are able to process the specific emotional and motivation cues of the people involved in a given situation.

With this definition in mind, I’d now like to present my hypothesis about wisdom.

My Wisdom Hypothesis: Wisdom results from an ability to distinguish between a large number of possibilities for what a situation means. Wise people draw on a huge array of knowledge about practical life events, and this knowledge enables them to make sound judgments. In technical terms we can say that their judgments contain more information than the judgments of most people, because their judgments are based on a deeper understanding of the structure of cause and effect that governs events in the world.

In the remainder of this post I will try and defend this hypothesis. First, I will present an example of a light-detector which will highlight the concept of discrimination. Then, I will discuss the idea of a web of knowledge. Finally, I will try and explore some practical tips for cultivating wisdom.

The Light Detector Example

I’d like you to consider the following thought experiment taken from an influential paper on consciousness by the psychiatrist Giulio Tononi:

You are facing a blank screen that is alternately on and off, and you have been instructed to say “light” when the screen turns on and “dark” when it turns off. A photodiode—a simple light-sensitive device—has also been placed in front of the screen. It contains a sensor that responds to light with an increase in current and a detector connected to the sensor that says “light” if the current is above a certain threshold and “dark” otherwise….When you distinguish between the screen being on or off, you have the subjective experience of seeing light or dark. The photodiode can also distinguish between the screen being on or off, but presumably it does not have a subjective experience of light and dark. What is the key difference between you and the photodiode?

According to Tononi, the key difference has to do with the number of alternatives that you rule out when you decide whether the screen is light or dark. When the blank screen turns on, the photodiode only has to distinguish between two alternatives: either the screen is ‘light’ or it is ‘dark’. When you look at the screen, on the other hand, not only are you distinguishing between light and dark, you are also perceiving whether or not the screen contains color, motion, objects and a whole host of other visual features.

This is significant because it means that your judgement contains a lot more information than the light-detectors. Because you have had to rule out so many alternatives about what the screen is not, we can learn a lot more by listening to your response than by looking at how the light-detector responded– no matter how we program the light-detector we can never learn about motion, color, or objects.

The light-detector example is important because the principals that are at work here bear a direct analogy to the differences that make people wise.

I’d like to argue that what makes someone wise is their ability to distinguish between a large number of possibilities for what a situation means. In contrast to an average person, who may have a relatively small number of experiences to draw upon, wise people have huge amount of practical life experience from which they can draw sound judgments. Just as our increased capacity to discriminate between alternatives enables us to perceive more than a light-detector, so a wise persons superior knowledge about practical life events enables them to make better judgments than most people.

But how does someone develop wisdom?

Given this framework about what wisdom is, I think we can make a number of deductions about how someone can develop wisdom.

The first thing to note is that wisdom is highly dependent upon the number of experience you have.

The ability to discriminate between many alternatives, and deduce a lot of information from a particular scenario, is predicated on the number of experiences that you have to draw upon. This is a possible explanation for why wisdom seems to be directly linked to age. The reality is that in general older people have been exposed to a greater number, and a greater range of life experiences than their younger counterparts. This, of course, means that they can draw from a greater pool of knowledge when having to make important judgments.

In addition to the sheer number of experiences that someone has, it seems that the variety of experiences is important too.

Although a fifty year undoubtedly has had more experiences than a twenty-five year old, the twenty-five year old may be able to make up for the deficit by seeking out diverse experiences. Specifically, experiences like travelling the world, experimenting with a number of jobs, majoring in more than one subject, or creating a startup, would probably lead to an accumulation of more wisdom than simply working the same office job for an equivalent length of time. It’s not that the number of experiences is different, it’s the variety that counts.

One way of thinking about this distinction is by imagining our experiences as forming a web of knowledge. When we add to our existing knowledge by having novel experiences, we extend the range of our web, reaching into previously uncharted territories. In contrast, when we have experiences that are similar to what we have experienced in the past, we merely strengthen the existing connections in our web without gaining the ability to see into new territory. To be wise we must maximize the size of this web, while still keeping the internal connections relatively dense.

Strengthening the Web

As a final thought, I want to consider some ways that we can increase our capacity for wisdom by strengthening our web of knowledge. Three thoughts come to mind:

First, it seems that to some extent our web of knowledge can be grown through practical training.

In our culture, counselors and psychiatrists are often associated with having a high degree of wisdom. Although to some extent this may be caused by innate personality traits, it seems equally possible that a lot of their wisdom is a result of the their specific career training. I do not have too much experience in the field, but it seems likely that practices such as emphatic listening, developing an understanding of the biology behind peoples behaviors, and learning about therapies such as cognitive behavioral therapy can increase your wisdom.

Second, it seems that wisdom can be cultivated by actively seeking out diverse life experiences. 

Most people would agree that a foreign-diplomat who had traveled the world learning about a diverse range of cultures would be better equipped to make life decision than an equally intelligent person who had spent twenty years doing the same job. Of course, the latter would likely have a greater capacity for wise decision within the specific domain of their work, but in general the diplomat would have better judgement across a range of life decisions.

Third, there seems to be a personality attribute to being wise.

Having a sizable web of knowledge is probably necessary for wisdom, but it may not be sufficient. Specifically, a close-minded personality may be destructive for accessing the full range of stored experiences. A mind-set of openness, and a willingness to savor, and then draw upon events from the past most likely helps with being wise.

Undoubtedly wisdom is a complicated and multifaceted phenomenon. But nevertheless is seems that by identifying the traits that make someone wise we can put ourselves in a better position to cultivate a wise disposition.

What are your thoughts about what makes someone wise? Are you convinced by my analysis, or do you disagree? Please share you thoughts in the comments below.

In perhaps the world’s most obvious study Dan Arierly, a psychology professor at Duke University and author of the book ‘Predictably Irrational’, showed that male students who are sexually aroused have impaired judgement when it comes to sex. When asked, “would you always use a condom if you didn’t know the sexual history of a new partner”, 88% of students replied ‘No’ in the non-aroused condition, whereas only 69% of students replied ‘No’ when aroused. Similarly, when asked, “would you encourage your date to drink to increase the chance that she would have sex with you”, 46% of students said ‘Yes’ when not aroused compared to 63% of students who said ‘Yes’ when aroused.

The key point of this study is that our decision making is strongly affected by situational  forces. Although we believe that we can reason well about future events, the reality is that our emotions have a profound affect on our decisions, and it’s hard to forecast how we will behave without being in the appropriate situation.

In this post, I want to explore some ideas for counteracting our bias of underestimating the role of emotions. Not only is this skill important when making decision that obviously have a strong emotional impact– decisions about sex, whether to deploy soldiers, how to deal with breakups etc…–   it is also important when considering more mundane things like how much work we can get done during a day and whether we will have the willpower to avoid cookies.

1) Think Worse Case

My first suggestion is that it’s a good idea to think about the worst-case scenario. A lot of the time we make decisions based on the readiness with which an idea comes to mind. In psychology this is called the availability heuristic, and although it is a good strategy to use when we are pressed for time, it can often lead to less than optimal results.

For example, in one of my classes I am currently building a computer simulation to try and determine how many ambulances are needed to meet the needs of a particular city. The data I have includes the average time that it takes for an ambulances to get to a scene as well as some numbers indicating how much the arrival times vary. Although my first reaction was to use the average call time in my model– this number is the most likely after all– my professor pointed that in fact, given the costs of under predicting the number of ambulances needed, it makes a lot more sense to use the 95th percentile for the arrival times. In this case my first reaction could have lead to a disastrous result.

As a general rule, it’s probably a good idea to add 20% when making predictions. Although this rule shouldn’t be applied to all situations– i.e. when truth and accuracy are the most important considerations– it nevertheless seems to be a good way to add a valuable margin of safety.

2) Don’t Make it Personal

My second suggestion is that it’s best not to think about what you would do, but instead to think about what a friend would do in a situation. Despite the fact that we are remarkably poor predictors of our own behavior– 87% of academics think that their work is better than average for their field for example– it turns out that we are often very good at predicting what other people will do.

In one study, conducted by David Dunning at Cornell, students said that, if given the chance, they would donate roughly half of their $5 fee for participating in an experiment to charity, but predicted that most students would donate only $1.80. In truth, when given the chance to donate, students only gave $1.53. The conclusion: our predictions are a lot better when they are predictions about others, not ourselves.

3) Don’t Count on Will Power

My final suggestion is to try and plan your behavior so that willpower is not needed. A lot of the time we plan for events when we are feeling positive and in a good mood. In these moods we feel like we can do anything. Avoiding desert, no problem! Exercising every day, seems easy enough! The problem is that states of feeling extremely productive are the exceptions rather than the norm.

The only way I see to deal with this is to try and create extremely simple habits that rely on environmental cues. For example, if you want to increase the time that you spend doing school work, one simple habit is to get into the routine of going to office hours. The environmental trigger of being surrounded by people who are meant to help you with your work makes it very easy to be productive. Similarly, if you want to start an exercise routine, making it the first thing you do in the morning is a great way to make it a habit without having to think about it.

Although it’s unlikely that we can ever become perfect predictors of our behavior, it does seem possible to make serious improvements. If you seek out feedback to understand where your beliefs diverge from reality, and design behaviors that are simple to follow, then it’s likely that you can eliminate a lot of the risks of varying emotional states.

Earlier today I listened to a RadioLab podcast in which the neurologist Oliver Sacks talked about one of his patients who, to put it bluntly, was slow.

Describing the motion of this patient, Sacks explained how the patients hand would seem to hover in the air moving imperceptibly slow as if the patient were a glacier changing at the slowest possible rate. To get a feeling for just how slow we’re talking, Sacks explained how it took two hours just for the patient to wipe his nose!

When asked about this slowness the patient seemed indignant. He had no appreciation that he was moving at a slower rate than usual, and, “seemed utterly shocked,” that the movement had taken two hours.

Although this example is a fairly extreme case, I think it shows something quite profound about human nature: the fact that we often have blind spots that are impossible for us to see.

This raises the question of how well we know ourselves. We often feel that we know ourselves better than anyone else does because we have privileged access to the subjective experience of what it’s like to be us. It seems, however, that sometimes this information can bias our perception.

Consider another example, this time coming from academia. During my freshman year I took a course that was a series of seminars related to introductory psychology. The graduate student who was teaching the course, Erik Helzer, explained a study he was conducting that explored how students predict their future performance on exams.

Having just taken the first exam in Psych 101 imagine that you are asked to predict your performance on exam two. At the same time a friend of yours, who  knows what you scored on exam one, is also asked to predict your score on exam two. Who do you think will make the better preditiction?

It turns out that your friend will make a better prediction than you. This is because, as Erik explains, “peers tend to avoid the degree of over-optimism so often seen in self-predictions.” In other words, while our peers base their predictions pretty much exclusively on our past performance, we tend to rely a lot more on how we think we will improve. In general, however, we overestimate how much we will improve and this leads to erroneous predictions.

I’m not suggesting that our overconfidence is a bad think. A bit of reality distortion can often go a long way towards stimulating creativity and getting stuff done. Nevertheless, in some situations it is better to have a clear view of how things really are.

For example, if you’re trying to loose weight then it’s easy to convince yourself that you’re doing great on the new program when in reality you may only have lost a minimal amount of weight. Likewise, it’s easy to be overconfident about investments, to think that we are doing better in school than we really our, or even to believe that we are more liked than is really the case.

Instead of blindly relying on your intuitions, I think that in some situations it is better to rely on objective measures to evaluate how you are doing.

Instead of studying until you ‘feel good’ about the upcoming test, it might be more helpful to set an objective measure such as getting 85% on a past prelim. Similarly if you’re trying to become more extroverted, an objective measure, such as comments from a public speaking club or data on the number of new people you talk to each day, is probably better than relying on intuition.

The key point is to find a measure that will prevent you from becoming a victim of your own internal biases. If you were measuring someone else’s performance what metrics would you use? Determine a set of measures, then stick to them. Keep yourself honest!

One of the defining features of the modern world is the huge range of choice. Go into a standard supermarket and you will face a choice between 40,000 different products, look at the course catalog for a major University and you will see upwards of a thousand courses listed, spend a couple of minutes online and you will be exposed to hundreds of different media sources, millions of videos, and a multitude of advertisements.

In general I think that more choice is a good thing. Competition drives innovation and brings down costs for consumers. It also means a more diverse array of goods so even people with special needs can find something that works for them.

Despite this, however, I think that too much choice can often lead to problems. Specifically, an issue that I’ve noticed is that the more choices we have, the more bad choices appear, and consequently the greater our chances become of making choices we regret.

For example, think about the range of drinks that are available to us. At lunch time, you most likely have the choice between a couple dozen different drinks. Fruit juices, fizzy drinks, caffeinated beverages, flavored waters, and milk are usually available in various assortments. The problem with this is that the vast majority of these drinks will do you more harm than good.

Similarly if we take the example of internet browsing, even though there are a huge number of great sites out there providing unprecedented access to knowledge about how the work works, the reality is that there is a far greater number of sites that, to put it lightly, are utter garbage.

Now, it’s certainly true that the web pages we view are not completely random, nor is our selection of drinks at lunch time completely arbitrary. In most cases the majority of our decisions are restricted to a relatively small bunch of alternatives. In terms of what I drink, I rarely venture beyond water, tea, coffee, and the occasional sugary drink from Starbucks. Similarly, as the tabs of my Google Chrome home site indicate, the majority of my browsing is split between the BBC, Facebook, and various sites related to my academic work.

So here’s my question: how can we alter our set of habitual behaviors for the greatest positive effect? I have a hypothesis that in general too much choice is leading us into making too many bad decisions. In the rest of this post, I want to try and explore how restricting the range of behaviors available to us can be beneficial.

Let’s think more about the drink example. For the sake of argument, let’s assume that a person needs to consume a base level of water each day. Further let’s assume that the source of this water is distributed between the liquids that they habitually consume. In my case we could say that the base amount of water I need each day is distributed among the coffee, tea, water, and sugary drinks that I regularly consume.

Now, one way of categorizing these drinks is as good and bad. In my case I would categorize coffee and sugary drinks as bad (coffee makes me anxious and sugary drinks provoke an annoying insulin spike) and water and tea as good. Given this categorization it seems that one way to improve our behaviors is simply by limiting our possibilities to only the good things. If I only consumer water and tea, then of course this would be better than if I consumed a mix of water, tea, coffee, and sugary drinks.

This, I hope, is a fairly obvious observation. Implementing the idea of restricting your behaviors in real life, however, can often be a challenge.

For one thing, it is sometimes difficult to know what our current range of behaviors is. Could you accurately list the top ten things that you do in your leisure time? Do you know the distribution of your web browsing? What types of books have your read over the past year, and which of those books have been good? My guess, is that most of us are unable to answer these questions.

Another problem, is that it is often hard to define which of our behaviors are good and which ones bad. Rarely do most of us think about which of our leisure activities we actually enjoy. Moreover, in the example of something like coffee consumption, arguments can often be made either way. Yes coffee can cause anxiety, but it can also boost metabolism, and lead to improved sports performance.

These problems not withstanding I still think that a lot of the time it can be beneficial to modify the range of our behaviors. For me, I think I could improve my quality of life by thinking hard about what books I spend my time reading, what objects I have in my room, what foods I let myself eat, how I spend my recreational time, and what websites I let myself browse. In each of these situations more restriction is needed.

If you’re interested in restricting your range of behaviors, here’s some tips on how it can be done:

1) Make if hard for yourself to do the bad things. If you want to avoid visiting time wasting sites consider investing in an app that blocks your access to websites after a certain amount of time. In chrome stay focused works well for this. If you notice that yo spend too much time on your phone consider turning it off and hiding it. If you are spending too much money on a credit card think about experimenting with paying in cash.

2) Make it easy to do the good things. If you want to drink more water, consider carrying two bottles with you. If you want to eat more fruit, think about buying a fruit bowl. If you want to exercise more, wear your exercise clothes more often and put your running shoes in a prominent location.

3) Don’t eliminate, replace. Often the elimination of a behavior creates a void that must be filled with something else. Your base line level of water consumption needs to be filled from some sources. Likewise your leisure time has to be filled with something. If cutting out your bad behavior might create a void, make sure you think of a replacement. If you think cutting out cookies will make you hungry, then make sure you have some carrots or nuts to fill you up instead.

You follow the path ahead of you. A new idea comes. Read a book, research online, walk around, go to a movie, try and make an observation. You feel inspired, or you know at least that you want to feel that way.

You continue, following the lead, seeing where it will go. You attend a new meeting, a new group, new personalities. You get a new idea, a new direction, something more to research and the process continues.

Okay, you think. I’m making progress. I’m learning. I’m doing something. At least I’m not sitting around watching sitcoms on TV.

But then, the thought, Is this the best I can do? Am I actually learning anything at all? What was in that book? Argghh, I can’t remember a word of it.

Reset. Planning. Thinking. Clarifying direction. There’s a path and you just need to find it. Keep plugging away, keep looking for it. As long as you’re doing something that has a non-zero probability you’ll eventually find it. It’s only a matter of time.

Your energy changes. You feel sharp and up, like you can climb a mountain, attack an edifice of ideas. Then it changes, you feel just normal. Not bad, not good, just normal. Just working away, nothing special.

And so it continues. Keep working, keep moving, keep learning.

You’re aware that the clock is ticking. But what can you do? Keep trying. Take in the situation where you are. Enjoy it, knowing it’s all you’ve got.

Breath, think, continue. It’s time to move on.

One of the first things you need to do when learning new material is to ask tonnes of questions. How does this work? Why does that follow? What is the main argument here? Is there a good metaphor that would explain this?

The question is the first step and it opens you up to discovering deeper insights about the material.

The question, however, isn’t enough. The learning comes when you grapple with the solution to the question. If you ask questions during conversations or while in lecture, don’t just stop at the question. Listen to the response and then try to form your own answers too.

The question is a good start, but the journey towards the answer is the real place to focus.

Recently I set myself the goal of loosing about five pounds of weight. In order to measure my progress one thing I’m doing is tracking the food I eat. I don’t do this everyday, but perhaps will do it for a couple of days every few weeks.

One observation I’ve made is that after a good day, where I eat nutrious meals, plenty of vegies and don’t overeat, I tend to follow it up immediately with a worse day. Similarly, I’ve noted that if I have a bad day, where I might indulge in one too many sugary sweets or have a few too many grain based meals, I will often follow it up with a day that is above average.

My first reaction to this data was to fall back on the willpower hypothesis. Bad days follow good days because willpower is impossible to sustain for large periods of time. Similarly good days follow bad days because the bad days allow the possibility to replenish will power.

Although this explanation sounds logical and at face value is extremely appealing, I now think that there might be an easier explanation, a statistical phenomenon called regression to the mean.

Let me quickly explain what that is. Regression toward the mean is the phenomenon that if a measurement is extreme on one trial then on the next trial it will tend to be closer to the average. One way to think of this is by visualizing the normal distribution (also known as the bellshape curve). If on one trial you get a result that is on the extreme edge of the curve, then it is likely that your next measurement will be closer to the center of the distribution simply because the ‘bell’ is fattest in the middle.

I think that we often try and explain our inability to sustain good habits by reverting to the willpower hypothesis. But the fact is that if we’ve been studying extremely effectively for a long period of time, or have been exercising well, or having been following our diets closely, then eventually we are bound to experience a dip in our performance. The true explanation for this isn’t a lack of power, it’s simply that we’ve been operating at the extremes of our abilities and are therefore statistically likely to experience a drop in performance.

One psychologist who has explored regression to the mean closely in his work is the author of the best selling book, Thinking, Fast and Slow, Daniel Khaneman. In one 1974 paper published in Science, Khaneman and his collaborator Amos Tversky show how failure to appreciate regression to the mean can lead to erroneous conclusions:

“The failure to recognize the import of regression can have pernicious consequences, as illustrated by the following observation. In a discussion of flight training, experienced instructors noted that praise for an exceptionally smooth landing is typically followed by a poorer landing on the next try, while harsh criticism after a rough landing is usually followed by an improvement on the next try. The instructors concluded that verbal rewards are detrimental to learning, while verbal punishments are beneficial, contrary to accepted psychological doctrine. This conclusion is unwarranted because of the presence of regression toward the mean. As in other cases of repeated examination, an improvement will usually follow a poor performance and a deterioration will usually follow an outstanding performance, even if the instructor does not respond to the trainee’s achievement on the first attempt. Because the instructors had praised their trainees after good landings and admonished them after poor ones, they reached the erroneous and potentially harmful conclusion that punishment is more effective than reward.”

I like this example because it shows how simple statistical facts can often provide the simplest explanations for our data.

By thinking about regression to the mean I think that we can plan better for the eventual dip that follows any good period of performance. Instead of interpreting our off days as representing a lack of willpower, I think it’s more freeing just to accept them as a result of statistics. Just because we can’t sustain performance at our max for long periods of time does not mean that we can’t improve our max level of performance in the long run.

Think of performance like economic growth. On any one day we should expect the possibility of either a good performance or a bad one. This is the result mainly of random probability and can’t be predicted. If, however, we continue to improve productivity in general and focus on developing our strengths then we can expect that in the long run we will experience sustained growth.

When it comes to my education, I want to be a generalist.

Perhaps the main reason that I decided to come to the US for my undergraduate education is that I wanted to study multiple disciplines. In much of the world education is geared towards mastering one discipline only. In the UK you apply to universities with your major already declared. You apply to study history, or maths, or chemistry  and that one discipline is pretty much all you get.

I’m fortunate, however, that in my first two years at Cornell I’ve already been exposed to a number of fields. I’ve dabbled in philosophy, sociology, mathematics, engineering and psychology. Learning about a number of disciplines is something I very much enjoy, and I think that this search for breadth is something I want to continue.

It seems however that being a generalist can create problems when it comes to employment. It is well known that the biggest rewards come from mastery of a discipline. In general, someone who knows a little about a lot of disciplines will find it harder to gain work than a specialist. This is because in any one field the specialist’s knowledge will trump what the generalist knows.

On the other hand, it seems that some of the most successful people in history have been generalist. Think about da Vinci, Jefferson, or more recently Steve Jobs. Each of these people mastered a number of fields during their life time. Moreover, a considerable number of the advances in modern science are discovered by finding links across disciplines.

Nevertheless, I think that success as a generalist is the exception rather than the rule. The reality is that if you spread your studying too broadly then you end up being distinctly average at everything.

So how can you avoid this problem?

I think that the author Sebastian Marshall may have hit upon the solution in his article ‘What Separates a Generalist from a Dabbler?‘ To be a successful generalist, Marshall claims, you must ship! You must produce output in each of the fields you dabble in, instead of moving on before having produce anything.

This insight is something that makes a lot of sense to me. I certainly have a tendency to just dabble in fields and avoid the hard work of actually producing something. In the past I’ve tried improv, dancing, playing the ukulele, building websites, and a whole lot of other things. But in each of these situations I’ve never reached the point where I actually produced something.

Going forward, my goal will be to produce something whenever I enter a new field. Instead of just learning more about statistics, I will try to apply this knowledge to creating real world simulations. Instead of just reading up about new psychological concepts, I will focus on actually writing about them.

I think it’s true that the mindset of a producer and a consumer is significantly different. If you want to a be a generalist, make sure that you get into the producer mindset as often as possible. Yes, study multiple fields. Yes, learn about new disciplines. But at the same time make sure you put in the hard work to actually link those disciplines together. Write something, give a talk, write a program, produce a youtube video. Just make sure you ship something!

Consider an experiment where 100 people, each in their own cubicle, are asked to rank three different types of chocolate bars based on their taste. Imagine that 54 people prefer chocolate bar A, 38 people prefer chocolate bar B, and only 8 people prefer chocolate bar C. Given these results most people would be happy to assert that chocolate bar A is the best tasting chocolate bar– I know I would.

Now consider a similar, but slightly different situation. One hundred people are asked to rank three different types of tea. This time however the participants vote one at a time and after each person votes a leader board is updated indicating the current rankings of the tea. If after the first 20 people, for example, tea A had 8 votes, tea B had 6 votes, and tea C also had 6 votes then, person number 21 will have access to this information. Imagine that in a similar way to our first experiment, the results show that 54 people prefer tea A, 38 people prefer tea B, and 8 people prefer tea C. Can we still conclude that tea A is the best tea? This time it is unclear.

The difference between the two experiments is a matter of independence. In the first experiment each of the participants made their decisions independently in their own cubicles. But in the second experiment, each participant (except the first one) also has the knowledge of the votes of previous people. This difference is important because the lack of independence introduces a confound. It is not clear whether tea A is popular because it was objectively the best tea, or whether it became popular as a result of people copying the votes of people who came before them.

In this article my goal is to examine our notion of popularity. We often think that the most popular things are popular because they are the best. It might be, however, that effects of social-influence (like in our tea experiments) may actually be playing an important role.

Let’s make this discussion more concrete. Consider a study published in Science in which a group of experimenters invited 14,000 particpants to register at a Web Site called Music Lab where they were asked to listen, rate and, if they chose, download songs by bands they had never heard of. Some of the participants were only shown the names of the songs and the bands, while other participants were also shown how many times each song had been downloaded by other participants. This second group of people was labeled the “social influence” group and was split further into eight parallel “worlds” so that participants could only see the downloads of other people in their world. At the start of the experiment all of the rankings of the songs were set at zero. Then as the experiment began each world developed it’s own set of rankings independent of the others.

This setup allowed the experiments to test the notion of popularity in the following way: if people know what they like regardless of what they think other people like, then the most popular songs in the independent condition should be the same songs that are popular in the social influence condition.

The findings were the exact opposite. The particular songs that became hits were different in each of the worlds. In other words popularity was not just the result of “intrinsic” quality as measured in the independent condition but was also affected by social influence in each of the worlds. As one of the experiments put it, “introducing social influence into human decision making made the hits more unpredictable.”

But what was the scale of this effect. Although the experimenters found that “good” songs were rated better on average than “bad” songs, it was still found that the impact of a listener’s own reactions is easily overwhelmed by his or her reactions to others. One song, “Lockdown”, by 52metro, for example, was ranked 26th out of 48 in the independent condition, but was the No.1 song in one of the social-influence worlds and 40th in another.

This finding has surpisingly deep consequences for our understanding of popularity. Consider the comments of one of the experiments, Duncan Watts:

“Because the long-run success of a song depends so sensitively on the decisions of a few early-arriving individuals, whose choices are subsequently amplified and eventually locked in by the cumulative-advantage process, and because the particular individuals who play this important role are chosen randomly and may make different decisions from one moment to the next, the resulting unpredictability is inherent to the nature of the market. It cannot be eliminated either by accumulating more information — about people or songs — or by developing fancier prediction algorithms, any more than you can repeatedly roll sixes no matter how carefully you try to throw the die.”

In other words, it seems that often popularity is more the result of random events that occur early on than of any innate quality.

Profound as this experiment may be, it is not an entirely new observation. Self-fulfilling prophecy’s are already a well known phenomenon in a number of fields. In medicine it is know that a placebo– a drug with no active medical ingredients– can still have a positive effect on patients if they believe it will have a positive effect. Similarly, in education if a teacher believes that she is teaching a group of high acheiving students then those students will subsequently perform better on tests.

In fact, it seems that social-influence effects plays a role in any decision that is the least part social.

As a final thought consider the notion of popularity as it applied in high school. (It is often rumored that the careers of social sciences are based on explaining their high school experiences.)  Although some of you may have occupied the enviable position of one of the popular kids in high school a lot of you probably felt left out on the side. This, however, may not have been your fault. It is quite possible that small friendships formed through random events in the early years of high school may have set the hierarchies of popularity in an almost completely arbitrary manner.

On a more serious note, if nothing else, I think that this research should create a sense of humility. To those who have done well, yes it is true that you are most likely better than most at what you do. To some extent though, it is likely that random social-effect also played a role in your success. Success is never just an individuals making, randomness always plays an important role too.