What is a Black Swan?
First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact (unlike the bird). Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable.
Unexpected innovations
Count the significant events, the technological changes, and the inventions that have taken place in our environment since you were born and compare them to what was expected before their advent. How many of them came on a schedule?
The killer secret method…
Think about the “secret recipe” to making a killing in the restaurant business. If it were known and obvious, then someone next door would have already come up with the idea and it would have become generic.
Inability to predict
What is surprising is not the magnitude of our forecast errors, but our absence of awareness of it. This is all the more worrisome when we engage in deadly conflicts: wars are fundamentally unpredictable
How to deal with uncertainty
The strategy for the discoverers and entrepreneurs is to rely less on top-down planning and focus on maximum tinkering and recognizing opportunities when they present themselves.
Focus on general, not precise…
Another related human impediment comes from excessive focus on what we do know: we tend to learn the precise, not the general.
Confirmation bias
I call this overload of examples naïve empiricism — successions of anecdotes selected to fit a story do not constitute evidence. Anyone looking for confirmation will find enough of it to deceive himself—and no doubt his peers.
Umberto Eco’s Anti-library
Read books are far less valuable than unread ones. The library should contain as much of what you do not know as your financial means, mortgage rates, and the currently tight real-estate market allow you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menacingly.
Focusing on the known
So this tendency to offend Eco’s library sensibility by focusing on the known is a human bias that extends to our mental operations. People don’t walk around with anti-résumés telling you what they have not studied or experienced (it’s the job of their competitors to do that), but it would be nice if they did.
Knowledge is not a treasure
Let us call an antischolar—someone who focuses on the unread books, and makes an attempt not to treat his knowledge as a treasure, or even a possession, or even a self-esteem enhancement device—a skeptical empiricist.
3 facets of the Black Swan problem:
History is unexplainable
History is opaque. You see what comes out, not the script that produces events, the generator of history. There is a fundamental incompleteness in your grasp of such events, since you do not see what’s inside the box, how the mechanisms work.
Human mind fallacy
The human mind suffers from three ailments as it comes into contact with history, what I call the triplet of opacity. They are: the illusion of understanding, or how everyone thinks he knows what is going on in a world that is more complicated (or random) than they realize; the retrospective distortion, or how we can assess matters only after the fact, as if they were in a rearview mirror (history seems clearer and more organized in history books than in empirical reality); and the overvaluation of factual information and the handicap of authoritative and learned people, particularly when they create categories—when they “Platonify.”
Accepting that we cannot predict or explain
our minds are wonderful explanation machines, capable of making sense out of almost anything, capable of mounting explanations for all manner of phenomena, and generally incapable of accepting the idea of unpredictability.
Worth is a diary
While we have a highly unstable memory, a diary provides indelible facts recorded more or less immediately; it thus allows the fixation of an unrevised perception and enables us to later study events in their own context.
Did people predict WWII? Not at all…
Still, encountering Shirer’s book provided me with an intuition about the workings of history. One would suppose that people living through the beginning of WWII had an inkling that something momentous was taking place. Not at all.
Cabdrivers vs experts
I noticed that very intelligent and informed persons were at no advantage over cabdrivers in their predictions, but there was a crucial difference. Cabdrivers did not believe that they understood as much as learned people — really, they were not the experts and they knew it.
Everyone will come to know the current news and details
It came to my notice that almost everybody was acquainted with current events in their smallest details. The overlap between newspapers was so large that you would get less and less information the more you read.
When categorizing is dangerous
Categorizing is necessary for humans, but it becomes pathological when the category is seen as definitive, preventing people from considering the fuzziness of boundaries, let alone revising their categories.
When the Right became capitalists and the Islamists became Socialists
By some farcical turn of events, in that civil war of Lebanon, Christians became pro-free market and the capitalistic system—i.e., what a journalist would call “the Right”—and the Islamists became socialists, getting support from Communist regimes (Pravda, the organ of the Communist regime, called them “oppression fighters,” though subsequently when the Russians invaded Afghanistan…
Unscalable professions
How did career advice lead to such ideas about the nature of uncertainty? Some professions, such as dentists, consultants, or massage professionals, cannot be scaled: there is a cap on the number of patients or clients you can see in a given period of time… In these professions, no matter how highly paid, your income is subject to gravity. Your revenue depends on your continuous efforts more than on the quality of your decisions.
Evolution and scalability
I am convinced that the process started much, much earlier, with our DNA, which stores information about our selves and allows us to repeat our performance without our being there by spreading our genes down the generations. Evolution is scalable: the DNA that wins (whether by luck or survival advantage) will reproduce itself, like a bestselling book or a successful record, and become pervasive.
Thinking > Making. Having said that making is a pre-requisite to thinking. E.g. Shenzhen.
There is more money in designing a shoe than in actually making it: Nike, Dell, and Boeing can get paid for just thinking, organizing, and leveraging their know-how and ideas while subcontracted factories in developing countries do the grunt work and engineers in cultured and mathematical states do the noncreative technical grind.
Mediocristan vs Extremistan
I can state the supreme law of Mediocristan as follows: When your sample is large, no single instance will significantly change the aggregate or the total… In Extremistan, inequalities are such that one single observation can disproportionately impact the aggregate, or the total.
Tyranny of the collective or the singular
Mediocristan is where we must endure the tyranny of the collective, the routine, the obvious, and the predicted; Extremistan is where we are subjected to the tyranny of the singular, the accidental, the unseen, and the unpredicted.
Knowledge and healthy skepticism
While the ancient skeptics advocated learned ignorance as the first step in honest inquiries toward truth, later medieval skeptics, both Moslems and Christians, used skepticism as a tool to avoid accepting what today we call science.
No evidence !== Evidence of none
An acronym used in the medical literature is NED, which stands for No Evidence of Disease. There is no such thing as END, Evidence of No Disease.
What is wrong vs what is right
But it remains the case that you know what is wrong with a lot more confidence than you know what is right. All pieces of information are not equal in importance. Popper introduced the mechanism of conjectures and refutations, which works as follows: you formulate a (bold) conjecture and you start looking for the observation that would prove you wrong.
Continuously search to prove your own hypothesis wrong!
Scientists believe that it is the search for their own weaknesses that makes them good chess players, not the practice of chess that turns them into skeptics. Similarly, the speculator George Soros, when making a financial bet, keeps looking for instances that would prove his initial theory wrong. This, perhaps, is true self-confidence: the ability to look at the world without the need to find signs that stroke one’s ego.
Our recollection of memories are also biased
Narrativity can viciously affect the remembrance of past events as follows: we will tend to more easily remember those facts from our past that fit a narrative, while we tend to neglect others that do not appear to play a causal role in that narrative.
Natural disasters > Terrorism
Terrorism kills, but the biggest killer remains the environment, responsible for close to 13 million deaths annually. But terrorism causes outrage, which makes us overestimate the likelihood of a potential terrorist attack—and react more violently to one when it happens. We feel the sting of man-made damage far more than that caused by nature.
Experiments > Storytelling
The way to avoid the ills of the narrative fallacy is to favor experimentation over storytelling, experience over history, and clinical knowledge over theories.
Empirical records..
when you look at the empirical record, you not only see that venture capitalists do better than entrepreneurs, but publishers do better than writers, dealers do better than artists, and science does better than scientists
Praying protects? The Survivorship bias
One Diagoras, a nonbeliever in the gods, was shown painted tablets bearing the portraits of some worshippers who prayed, then survived a subsequent shipwreck. The implication was that praying protects you from drowning. Diagoras asked, “Where were the pictures of those who prayed, then drowned?”
Silent evidence
But this number ignores the silent cemetery of species that came and left without leaving traces in the form of fossils; the fossils that we have managed to find correspond to a smaller proportion of all species that came and disappeared. This implies that our biodiversity was far greater than it seemed at first examination.
Selling avoidance…
It is much easier to sell “Look what I did for you” than “Look what I avoided for you.”
I do not know
My biggest problem with the educational system lies precisely in that it forces students to squeeze explanations out of subject matters and shames them for withholding judgment, for uttering the “I don’t know.” Why did the Cold War end? Why did the Persians lose the battle of Salamis? Why did Hannibal get his behind kicked?
Be suspicious of the reasons
Note here that I am not saying causes do not exist; do not use this argument to avoid trying to learn from history. All I am saying is that it is not so simple; be suspicious of the “because” and handle it with care—particularly in situations where you suspect silent evidence.
The supreme act of war is to subdue the enemy without fighting - Sun Tzu
I came out of the meeting realizing that only military people deal with randomness with genuine, introspective intellectual honesty—unlike academics and corporate executives using other people’s money. This does not show in war movies, where they are usually portrayed as war-hungry autocrats. The people in front of me were not the people who initiate wars. Indeed, for many, the successful defense policy is the one that manages to eliminate potential dangers without war, such as the strategy of bankrupting the Russians through the escalation in defense spending.
News-worthy vs empirical
Train your reasoning abilities to control your decisions; nudge System 1 (the heuristic or experiential system) out of the important ones. Train yourself to spot the difference between the sensational and the empirical. This insulation from the toxicity of the world will have an additional benefit: it will improve your well-being.
Discoveries are surprises
When I ask people to name three recently implemented technologies that most impact our world today, they usually propose the computer, the Internet, and the laser. All three were unplanned, unpredicted, and unappreciated upon their discovery, and remained unappreciated well after their initial use. They were consequential.
Knowledge and confidence…
The Australians had actually built a symbol of the epistemic arrogance of the human race. The story is as follows. The Sydney Opera House was supposed to open in early 1963 at a cost of AU $ 7 million. It finally opened its doors more than ten years later, and, although it was a less ambitious version than initially envisioned, it ended up costing around AU $ 104 million… True, our knowledge does grow, but it is threatened by greater increases in confidence, which make our increase in knowledge at the same time an increase in confusion, ignorance, and conceit.
Arrogance
Epistemic arrogance bears a double effect: we overestimate what we know, and underestimate uncertainty, by compressing the range of possible uncertain states (i.e., by reducing the space of the unknown).
More knowledge == worse off
The more information you give someone, the more hypotheses they will formulate along the way, and the worse off they will be. They see more random noise and mistake it for information. The problem is that our ideas are sticky: once we produce a theory, we are not likely to change our minds—so those who delay developing their theories are better off.
Reading weekly
Remember that we are swayed by the sensational. Listening to the news on the radio every hour is far worse for you than reading a weekly magazine, because the longer interval allows information to be filtered a bit.
Increase in knowledge frequency leads to increase in confidence, but not true knowledge
The increase in the information set did not lead to an increase in their accuracy; their confidence in their choices, on the other hand, went up markedly. Information proved to be toxic. I’ve struggled much of my life with the common middlebrow belief that “more is better”—more is sometimes, but not always, better. This toxicity of knowledge will show in our investigation of the so-called expert.
Complexity does not mean accuracy
Makridakis and Hibon reached the sad conclusion that “statistically sophisticated or complex methods do not necessarily provide more accurate forecasts than simpler ones.”
Forecasting is for anxiety relief
Our forecast errors have traditionally been enormous, and there may be no reasons for us to believe that we are suddenly in a more privileged position to see into the future compared to our blind predecessors. Forecasting by bureaucrats tends to be used for anxiety relief rather than for adequate policy making.
Accidental discoveries
If you think that the inventions we see around us came from someone sitting in a cubicle and concocting them according to a timetable, think again: almost everything of the moment is the product of serendipity. The term serendipity was coined in a letter by the writer Hugh Walpole, who derived it from a fairy tale, “The Three Princes of Serendip.” These princes “were always making discoveries by accident or sagacity, of things which they were not in quest of.”
Most important advances
Sir Francis Bacon commented that the most important advances are the least predictable ones, those “lying out of the path of the imagination.” Bacon was not the last intellectual to point this out.
Example of accidental discoveries: Background Microwave discovery
True, Fleming was looking for “something,” but the actual discovery was simply serendipitous. Furthermore, while in hindsight the discovery appears momentous, it took a very long time for health officials to realize the importance of what they had on their hands. Even Fleming lost faith in the idea before it was subsequently revived. In 1965 two radio astronomists at Bell Labs in New Jersey who were mounting a large antenna were bothered by a background noise, a hiss, like the static that you hear when you have bad reception. The noise could not be eradicated—even after they cleaned the bird excrement out of the dish, since they were convinced that bird poop was behind the noise. It took a while for them to figure out that what they were hearing was the trace of the birth of the universe, the cosmic background microwave radiation. This discovery revived the big bang theory, a languishing idea that was posited by earlier researchers.
Looking for evidence
As happens so often in discovery, those looking for evidence did not find it; those not looking for it found it and were hailed as discoverers.
Maximize exposure
Louis Pasteur’s adage about creating luck by sheer exposure. “Luck favors the prepared,” Pasteur said, and, like all great discoverers, he knew something about accidental discoveries. The best way to get maximal exposure is to keep researching. Collect opportunities on that, later.
Expectations
There is actually a law in statistics called the law of iterated expectations, which I outline here in its strong form: if I expect to expect something at some date in the future, then I already expect that something at present.
Predicting the future
But there is a weaker form of this law of iterated knowledge. It can be phrased as follows: to understand the future to the point of being able to predict it, you need to incorporate elements from this future itself. If you know about the discovery you are about to make in the future, then you have almost made it.
Pretense of knowledge
It was eloquently called “The Pretense of Knowledge,” and he mostly railed about other economists and about the idea of the planner. He argued against the use of the tools of hard science in the social ones, and depressingly, right before the big boom for these methods in economics.
Collective gathering of infoL Wikipedia?
One single institution, say, the central planner, cannot aggregate knowledge; many important pieces of information will be missing. But society as a whole will be able to integrate into its functioning these multiple pieces of information. Society as a whole thinks outside the box.
Forecasts and projections
It is why I fear governments and large corporations—it is hard to distinguish between them. Governments make forecasts; companies produce projections; every year various forecasters project the level of mortgage rates and the stock market at the end of the following year. Corporations survive not because they have made good forecasts, but because, like the CEOs visiting Wharton I mentioned earlier, they may have been the lucky ones.
Minimal theorizing
The empirics practiced the “medical art” without relying on reasoning; they wanted to benefit from chance observations by making guesses, and experimented and tinkered until they found something that worked. They did minimal theorizing.
Predicting all our future actions… is that really so good?
If I can predict all of your actions, under given circumstances, then you may not be as free as you think you are. You are an automaton responding to environmental stimuli. You are a slave of destiny. And the illusion of free will could be reduced to an equation that describes the result of interactions among molecules.
Chance is too fuzzy
The notion of future mixed with chance, not a deterministic extension of your perception of the past, is a mental operation that our mind cannot perform. Chance is too fuzzy for us to be a category by itself. There is an asymmetry between past and future, and it is too subtle for us to understand naturally.
Forward vs Backward process
However, from the pool of water you can build infinite possible ice cubes, if there was in fact an ice cube there at all. The first direction, from the ice cube to the puddle, is called the forward process. The second direction, the backward process, is much, much more complicated. The forward process is generally used in physics and engineering; the backward process in non-repeatable, non-experimental historical approaches.
Stay empirical
The empirical doctor’s approach to the problem of induction was to know history without theorizing from it. Learn to read history, get all the knowledge you can, do not frown on the anecdote, but do not draw any causal links, do not try to reverse engineer too much — but if you do, do not make big scientific claims.
The narrative fallacy
The more we try to turn history into anything other than an enumeration of accounts to be enjoyed with minimal theorizing, the more we get into trouble. Are we so plagued with the narrative fallacy?
What to avoid…
What you should avoid is unnecessary dependence on large-scale harmful predictions—those and only those. Avoid the big subjects that may hurt your future: be fooled in small matters, not in the large. Do not listen to economic forecasters or to predictors in social science (they are mere entertainers), but do make your own forecast for the picnic. By all means, demand certainty for the next picnic; but avoid government social-security forecasts for the year 2040.
Personal motto
This same point can be generalized to life: maximize the serendipity around you.
Incremental random changes
Trial and error means trying a lot. In The Blind Watchmaker, Richard Dawkins brilliantly illustrates this notion of the world without grand design, moving by small incremental random changes. Note a slight disagreement on my part that does not change the story by much: the world, rather, moves by large incremental random changes.
How does military prepare?
The impulse on the part of the military is to devote resources to predicting the next problems. These thinkers advocate the opposite: invest in preparedness, not in prediction. Remember that infinite vigilance is just not possible.
The success of the free markets
Effectively, if free markets have been successful, it is precisely because they allow the trial-and-error process I call “stochastic tinkering” on the part of competing individual operators who fall for the narrative fallacy—but are effectively collectively partaking of a grand project.
Spreading not because it is the best
A great illustration of preferential attachment can be seen in the mushrooming use of English as a lingua franca—though not for its intrinsic qualities, but because people need to use one single language, or stick to one as much as possible, when they are having a conversation. So whatever language appears to have the upper hand will suddenly draw people in droves; its usage will spread like an epidemic, and other languages will be rapidly dislodged.
Too many too same
We have moved from a diversified ecology of small banks, with varied lending policies, to a more homogeneous framework of firms that all resemble one another. True, we now have fewer failures, but when they occur … I shiver at the thought. I rephrase here: we will have fewer but more severe crises.
Studying the nature
So being old implies a higher degree of resistance to Black Swans, though, as we saw with the turkey story, it is not a guaranteed proof—older is almost always more solid, but older is not necessarily perfect. But a few billion years is vastly more proof than a thousand days of survival, and the oldest system around is clearly Mother Nature.
Historia
They are also the only group of people to use philosophy for anything useful. They proposed historia: maximal recording of facts with minimal interpretation and theorizing, describing of facts without the why, and resisting universals.
Learning from nature: redundancies
First, Mother Nature likes redundancies, three different types of redundancies. The first, the simplest to understand, is defensive redundancy, the insurance type of redundancy that allows you to survive under adversity, thanks to the availability of spare parts.
Avoid over overspecialization
Mother Nature does not like overspecialization, as it limits evolution and weakens the animals.
Small units
Mother Nature does not limit the interactions between entities; it just limits the size of its units. (Hence my idea is not to stop globalization and ban the Internet; as we will see, much more stability would be achieved by stopping governments from helping companies when they become large and by giving back advantages to the small guy.)
The case for not reducing volatility
the idea is not to correct mistakes and eliminate randomness from social and economic life through monetary policy, subsidies, and so on. The idea is simply to let human mistakes and miscalculations remain confined, and to prevent their spreading through the system, as Mother Nature does. Reducing volatility and ordinary randomness increases exposure to Black Swans — it creates an artificial quiet.
Having a variety of exercise
This is another application of the barbell strategy: plenty of idleness, some high intensity. The data shows that long, very long walks, combined with high-intensity exercise outperform just running. I am not talking about “brisk walks” of the type you read about in the Health section of The New York Times. I mean walking without making any effort.
When hunger is good
But these lifestyle ideas do not come from mere self-experimentation or some quack theory. All the results were completely expected from the evidence-based, peer-reviewed research that is available. Hunger (or episodic energy deficit) strengthens the body and the immune system and helps rejuvenate brain cells, weaken cancer cells, and prevent diabetes.
Let it happen and be confined
By a variant of the same reasoning we can see how the fear of volatility I mentioned earlier, leading to interference with nature so as to impose “regularity,” makes us more fragile across so many domains. Preventing small forest fires sets the stage for more extreme ones; giving out antibiotics when it is not very necessary makes us more vulnerable to severe epidemics—
Let it die early
Which brings me to another organism: economic life. Our aversion to variability and desire for order, and our acting on those feelings, have helped precipitate severe crises. Making something artificially bigger (instead of letting it die early if it cannot survive stressors) makes it more and more vulnerable to a very severe collapse—as I showed with the Black Swan vulnerability associated with an increase in size.
2 different views, 2 different probabilities
The notion that two people can have two different views of the world, then express them as different probabilities remained foreign to the research. So it took a while for scientific researchers to accept the non-Asperger notion that different people can, while being rational, assign different probabilities to different future states of the world. This is called “subjective probability.”
Long run
Few understand that there is generally no such thing as a reachable long run except as a mathematical construct to solve equations; to assume a long run in a complex system, you need to also assume that nothing new will emerge.
Rare events
the rarer the event, the less we know about its role—and the more we need to compensate for that deficiency with an extrapolative, generalizing theory. It will lack in rigor in proportion to claims about the rarity of the event. Hence theoretical and model error are more consequential in the tails; and, the good news, some representations are more fragile than others.
Probability distribution vs events
As we showed with the undecidability theorem and the self-reference argument, in real life we do not observe probability distributions. We just observe events. So I can rephrase the results as follows: we do not know the statistical properties—until, of course, after the fact.
Black Swan effects
Associated with the previous fallacy is the mistake of thinking that my message is that these Black Swans are necessarily more probable than assumed by conventional methods. They are mostly less probable, but have bigger effects.
4 Quadrants:
False preference
Linked to this need for positive advice is the preference we have to do something rather than nothing, even in cases when doing something is harmful.
State of Science
Unfortunately such lack of rigor pervades the place where we expect it the least: institutional science. Science, particularly its academic version, has never liked negative results, let alone the statement and advertising of its own limits. The reward system is not set up for it. You get respect for doing funambulism or spectator sports—following the right steps to become “the Einstein of Economics” or “the next Darwin” rather than give society something real by debunking myths or by cataloguing where our knowledge stops.
Some generic todos:
Tackling black Swan
Knowing where you are headed is important
I wanted to prepare myself for where I will go next. This is my plan B. I kept looking at the position of my own grave. A Black Swan cannot so easily destroy a man who has an idea of his final destination. I felt robust.
Learning Stoicism
Seneca was the great teacher and practitioner of Stoicism, who transformed Greek-Phoenician Stoicism from metaphysical and theoretical discourse into a practical and moral program of living, a way to reach the summum bonum, an untranslatable expression depicting a life of supreme moral qualities, as perceived by the Romans.
Ricker and poorer and Stoicism
And, just as it is harder to have good qualities when one is rich than when one is poor, it is harder to be a Stoic when one is wealthy, powerful, and respected than when one is destitute, miserable, and lonely.