Data and Goliath

date May 17, 2017
authors Bruce Schneier
reading time 40 mins

Every web developer / engineers working with the Internet should read this so that we are aware of what we are engineering, so that we can effectively build transparent and accountable data collection, so that we build systems that uses encryption and safety vulnerability declaration.

Further readings

  1. OECD Guidelines on the Protection of Privacy and Transborder Flows of Personal Data
  2. Madrid Privacy Declaration (2009)
  3. Plugins: Lightbeam, Privacy Badger, Disconnect, Ghostery, FlashBlock

The Code of Fair Information Practices is based on five principles:

  1. There must be no personal data record-keeping systems whose very existence is secret.
  2. There must be a way for a person to find out what information about the person is in a record and how it is used.
  3. There must be a way for a person to prevent information about the person that was obtained for one purpose from being used or made available for other purposes without the person’s consent.
  4. There must be a way for a person to correct or amend a record of identifiable information about the person.
  5. Any organization creating, maintaining, using, or disseminating records of identifiable personal data must assure the reliability of the data for their intended use and must take precautions to prevent misuses of the data.

Daily observations

Bargain with the cell carrier

Yet every morning when you put your cell phone in your pocket, you’re making an implicit bargain with the carrier: “I want to make and receive mobile calls; in exchange, I allow this company to know where I am at all times.” The bargain isn’t specified in any contract, but it’s inherent in how the service works.

Location based profiling for protests and demonstrations

In 2014, the government of Ukraine sent this positively Orwellian text message to people in Kiev whose phones were at a certain place during a certain time period: “Dear subscriber, you have been registered as a participant in a mass disturbance.” Don’t think this behavior is limited to totalitarian countries; in 2010, Michigan police sought information about every cell phone in service near an expected labor protest. They didn’t bother getting a warrant first.

Collection of personal data

The computers you interact with are constantly producing intimate personal data about you. It includes what you read, watch, and listen to. It includes whom you talk to and what you say. Ultimately, it covers what you’re thinking about, at least to the extent that your thoughts lead you to the Internet and search engines. We are living in the golden age of surveillance.

Price of convenience

Google’s chairman Eric Schmidt and its director of ideas Jared Cohen laid it out in their 2013 book, The New Digital Age. Here I’m paraphrasing their message: if you let us have all your data, we will show you advertisements you want to see and we’ll throw in free web search, e-mail, and all sorts of other services. It’s convenience, basically. We are social animals, and there’s nothing more powerful or rewarding than communicating with other people.

Surveillance enables discrimination

Mass surveillance is dangerous. It enables discrimination based on almost any criteria: race, religion, class, political beliefs. It is being used to control what we see, what we can do, and, ultimately, what we say. It is being done without offering citizens recourse or any real ability to opt out, and without any meaningful checks and balances.

Ubiquitous mass surveillance

It’s easy to focus on how data is collected by corporations and governments, but that gives a distorted picture. The real story is how the different streams of data are processed, correlated, and analyzed. And it’s not just one person’s data; it’s everyone’s data. Ubiquitous mass surveillance is fundamentally different from just a lot of individual surveillance, and it’s happening on a scale we’ve never seen before.

Saving everything, instead of figuring out what to save

Two years later, after a court battle, Facebook sent him a CD with a 1,200-page PDF: not just the friends he could see and the items on his newsfeed, but all of the photos and pages he’d ever clicked on and all of the advertising he’d ever viewed. Facebook doesn’t use all of this data, but instead of figuring out what to save, the company finds it easier to just save everything.

Suspect or not - collect information from all

This is mass surveillance, impossible without computers, networks, and automation. It’s not “follow that car”; it’s “follow every car.” Police could always tail a suspect, but with an urban mesh of cameras, license plate scanners, and facial recognition software, they can tail everyone—suspect or not.

Bargain of accepting surveillance

The overwhelming bulk of surveillance is corporate, and it occurs because we ostensibly agree to it. I don’t mean that we make an informed decision agreeing to it; instead, we accept it either because we get value from the service or because we are offered a package deal that includes surveillance and don’t have any real choice in the matter.

Feudal - peasant relationship

That’s primarily because we’re not customers. We’re products those companies sell to their real customers. The relationship is more feudal than commercial. The companies are analogous to feudal lords, and we are their vassals, peasants, and—on a bad day—serfs. We are tenant farmers for these companies, working on their land by producing data that they in turn sell for profit.

Wasted tax

We taxpayers are wasting billions on mass-surveillance programs, and not getting the security we’ve been promised. More importantly, the money we’re wasting on these ineffective surveillance programs is not being spent on investigation, intelligence, and emergency response: tactics that have been proven to work.

Internet is too big for government to not care anymore

Twenty years ago, few governments had any policies regulating the Internet. Today, every country does, and some of them are pretty draconian. This shouldn’t come as a surprise; the Internet became too big a deal for governments to ignore.

Cannot depend on Facebook / Google to lead the way

The companies that most extensively collect our data believe in the potential for massive increases in advertising revenue. Internet advertising might be a $ 125 billion business worldwide, but it’s still only 25% of the advertising market. Companies like Google and Facebook have their eyes on the advertising money spent on television (40%) and in newspapers and magazines (36%). They have a lot of money invested in the value of big data—collecting everything and then figuring out what to do with it later—and will not switch gears easily.


If you have nothing to hide, then you have nothing to fear??

Defenders of surveillance—from the Stasi of the German Democratic Republic to the Chilean dictator Augusto Pinochet to Google’s Eric Schmidt—have always relied on the old saw “If you have nothing to hide, then you have nothing to fear.” This is a dangerously narrow conception of the value of privacy. Privacy is an essential human need, and central to our ability to control how we relate to the world.

Mass surveillance is a security benefit??

Government mass surveillance is often portrayed as a security benefit, something that protects us from terrorism. Yet there’s no actual proof of any real successes against terrorism as a result of mass surveillance, and significant evidence of harm. Enabling ubiquitous mass surveillance requires maintaining an insecure Internet, which makes us all less safe from rival governments, criminals, and hackers.

Eavesdropping vs surveillance

Now imagine that you asked the detective to put that person under surveillance. You would get a different but nevertheless comprehensive report: where he went, what he did, who he spoke with and for how long, who he wrote to, what he read, and what he purchased. That’s metadata. Eavesdropping gets you the conversations; surveillance gets you everything else.

What does a meta data tell?

Telephone metadata alone reveals a lot about us. The timing, length, and frequency of our conversations reveal our relationships with others: our intimate friends, business associates, and everyone in-between.

Corporate vs government surveillance

The result is that corporate and government surveillance interests have converged. Both now want to know everything about everyone. The motivations are different, but the methodologies are the same. That is the primary reason for the strong public-private security partnership

Maximising addictiveness

Websites that profit from advertising spend a lot of effort making sure you spend as much time on those sites as possible, optimizing their content for maximum addictiveness. The few sites that allow you to opt out of personalized advertising make that option difficult to find.

Choice of a feudal lord

These are the tools of modern life. They’re necessary to a career and a social life. Opting out just isn’t a viable choice for most of us, most of the time; it violates what have become very real norms of contemporary life. And choosing among providers is not a choice between surveillance or no surveillance, but only a choice of which feudal lords get to spy on you.

Surveillance means control

Someone who knows things about us has some measure of control over us, and someone who knows everything about us has a lot of control over us. Surveillance facilitates control. Manipulation doesn’t have to involve overt advertising. It can be product placement ensuring you see pictures that have a certain brand of car in the background. Or just an increase in how often you see that car. This is, essentially, the business model of search engines.

Past hangs on forever

One-fourth of American adults have criminal records. Even minor infractions can follow people forever and have a huge impact on their lives—this is why many governments have a process for expunging criminal records after some time has passed. Losing the ephemeral means that everything you say and do will be associated with you forever.

Attacking is easier because…

  • It’s easier to break things than to fix them.
  • Complexity is the worst enemy of security, and our systems are getting more complex all the time.
  • The nature of computerized systems makes it easier for the attacker to find one exploitable vulnerability in a system than for the defender to find and fix all vulnerabilities in the system.
  • An attacker can choose a particular attack and concentrate his efforts, whereas the defender has to defend against every possibility.
  • Software security is generally poor; we simply don’t know how to write secure software and create secure computer systems. Yes, we keep improving, but we’re still not doing very well.
  • Computer security is very technical, and it’s easy for average users to get it wrong and subvert whatever security they might have.

Consequences of leaving vulnerabilities unpatched

An arms race is raging in cyberspace right now. The Chinese, the Russians, and many other countries are also hoarding vulnerabilities. If we leave a vulnerability unpatched, we run the risk that another country will independently discover it and use it in a cyberweapon against us and our allies.

Making you forget about privacy

Companies like Facebook prefer it this way. They go out of their way to make sure you’re not thinking about privacy when you’re on their site, and they use cognitive tricks like showing you pictures of your friends to increase your trust.

Social contract with digital sovereign

In her book Consent of the Networked, journalist and digital rights advocate Rebecca MacKinnon makes this point: “No company will ever be perfect—just as no sovereign will ever be perfect no matter how well intentioned and virtuous a king, queen, or benevolent dictator might be. But that is the point: right now our social contract with the digital sovereigns is at a primitive, Hobbesian, royalist level. If we are lucky we get a good sovereign, and we pray that his son or chosen successor is not evil.

Fallacy to believe we live in unique times

It’s just not true. It’s a common psychological fallacy to believe that we live in unique times, that our challenges are totally unlike anything that came before and necessitate ignoring any of the societal controls we previously put in place to check the powers of governmental authorities. President Lincoln succumbed to the fallacy when he suspended habeas corpus during the Civil War. President Wilson did so when he arrested and deported Socialists and labor leaders just after World War I.

Government increasing power

And then the laws will change to give them even more authority. Jack Goldsmith again: “The government will increase its powers to meet the national security threat fully (because the People demand it).”

Counter-intuition uncommon sense

More intrusive, more hidden

As surveillance fades into the background, it becomes easier to ignore. And the more intrusive a surveillance system is, the more likely it is to be hidden.

Future of hidden surveillance

In a sense, we’re living in a unique time in history; many of our surveillance systems are still visible to us. Identity checks are common, but they still require us to show our ID. Cameras are everywhere, but we can still see them. In the near future, because these systems will be hidden, we may unknowingly acquiesce to even more surveillance.

Anonymity cannot guarantee that it’s easy to hide

We might naïvely think that there are so many of us that it’s easy to hide in the sea of data. Or that most of our data is anonymous. That’s not true. Most techniques for anonymizing data don’t work, and the data can be de-anonymized with surprisingly little information.

We are all pretty similar - how anonymous does not work

It’s counterintuitive, but it takes less data to uniquely identify us than we think. Even though we’re all pretty typical, we’re nonetheless distinctive. It turns out that if you eliminate the top 100 movies everyone watches, our movie-watching habits are all pretty individual. This is also true for our book-reading habits, our Internet-shopping habits, our telephone habits, and our web-searching habits. We can be uniquely identified by our relationships.

Why is surveillance a business model?

Surveillance is the business model of the Internet for two primary reasons: people like free, and people like convenient. The truth is, though, that people aren’t given much of a choice. It’s either surveillance or nothing, and the surveillance is conveniently invisible so you don’t have to think about it. And it’s all possible because US law has failed to keep up with changes in business practices.

Uncanny valley of advertising

There’s a concept from robotics that’s useful here. We tend to be comfortable with robots that obviously look like robots, and with robots that appear fully human. But we’re very uncomfortable with robots that look a lot like people but don’t appear fully human. Japanese roboticist Masahiro Mori called this phenomenon the “uncanny valley.” Technology critic Sara M. Watson suggested that there’s a similar phenomenon in advertising. People are okay with sloppily personalized advertising and with seamless and subtle personalized advertising, but are creeped out when they see enough to realize they’re being manipulated or when it doesn’t match their sense of themselves.

Unpractical advise

If they don’t like it, they shouldn’t do it. This advice is not practical. It’s not reasonable to tell people that if they don’t like the data collection, they shouldn’t e-mail, shop online, use Facebook, or have a cell phone.

Where else is the data used?

Crime and terrorism provide justifications for surveillance, but this data is also used against Russian journalists, human rights activists, and political opponents.

False justification

We labeled the Chinese actions “cyberattacks,” sometimes invoking the word “cyberwar.” After Snowden revealed that the NSA had been doing exactly the same thing as the Chinese to computer networks around the world, the US used much more moderate language to describe its own actions—terms like “espionage,” or “intelligence gathering,” or “spying”—and stressed that it is a peacetime activity.

Encryption used both ways

Encryption allows the good guys to communicate without being eavesdropped on by the bad guys, and also allows the bad guys to communicate without being eavesdropped on by the good guys. And the same facial recognition technology that Disney uses in its theme parks to pick out photos its patrons might want to buy as souvenirs can identify political protesters in China, and Occupy Wall Street protesters in New York.

How security cannot be guaranteed once there is a back door

The FBI’s ultimate goal is government prohibition of truly secure communications. Valerie Caproni, the general counsel for the FBI, put it this way in 2010: “No one should be promising their customers that they will thumb their nose at a US court order. They can promise strong encryption. They just need to figure out how they can provide us plain text.” Translation: you can’t actually provide security for your customers.

Data and evidence gathering

Lavrentiy Beria, head of Joseph Stalin’s secret police in the old Soviet Union, declared, “Show me the man, and I’ll show you the crime.” Both were saying the same thing: if you have enough data about someone, you can find sufficient evidence to find him guilty of something.

Ubiquitous crime == anyone can be made into a crimial

Ubiquitous surveillance means that anyone could be convicted of lawbreaking, once the police set their minds to it. It is incredibly dangerous to live in a world where everything you do can be stored and brought forward as evidence against you at some later date.

Systems used against us without our knowledge

Another harm of government surveillance is the way it leads to people’s being categorized and discriminated against. George Washington University law professor Daniel Solove calls the situation Kafkaesque. So much of this data is collected and used in secret, and we have no right to refute or even see the evidence against us. This will intensify as systems start using surveillance data to make decisions automatically.

Social control

Jeremy Bentham’s key observation in conceiving his panopticon was that people become conformist and compliant when they believe they are being observed. The panopticon is an architecture of social control.

What happens with constant surveillance?

When we are constantly under the threat of judgment, criticism, and correction for our actions, we become fearful that—either now or in the uncertain future—data we leave behind will be brought back to implicate us, by whatever authority has then become focused upon our once-private and innocent acts. In response, we do nothing out of the ordinary. We lose our individuality, and society stagnates. We don’t question or challenge power. We become obedient and submissive. We’re less free.

Powerful system

A system that is overwhelmingly powerful relies on everyone in power to act perfectly—so much has to go right to prevent meaningful abuse. There are always going to be bad apples—the question is how much harm they are allowed and empowered to do and how much they corrupt the rest of the barrel. Our controls need to work not only when the party we approve of leads the government but also when the party we disapprove of does.

Case for Internet freedom

That, in short, would be a disaster. The Internet is fundamentally a global platform. While countries continue to censor and control, today people in repressive regimes can still read information from and exchange ideas with the rest of the world. Internet freedom is a human rights issue, and one that the US should support.

Employee-employer data exchange

The potential for intrusiveness increases considerably when it’s an employer–employee relationship. At least one company negotiated a significant reduction in its health insurance costs by distributing Fitbits to its employees, which gave the insurance company an unprecedented view into its subscribers’ health habits. Similarly, several schools are requiring students to wear smart heart rate monitors in gym class; there’s no word about what happens to that data afterwards. In 2011, Hewlett-Packard analyzed employee data to predict who was likely to leave the company, then informed their managers.

Changing public discourse

A truly sinister social networking platform could manipulate public opinion even more effectively. By amplifying the voices of people it agrees with, and dampening those of people it disagrees with, it could profoundly distort public discourse.

Peril of filter bubble

The first listing in a Google search result gets a third of the clicks, and if you’re not on the first page, you might as well not exist. The result is that the Internet you see is increasingly tailored to what your profile indicates your interests are. This leads to a phenomenon that political activist Eli Pariser has called the “filter bubble”: an Internet optimized to your preferences, where you never have to encounter an opinion you don’t agree with. You might think that’s not too bad, but on a large scale it’s harmful. We don’t want to live in a society where everybody only ever reads things that reinforce their existing opinions, where we never have spontaneous encounters that enliven, confound, confront, and teach us.

Moving from personal to mass data Hacking

A dozen years ago, the risk was that the criminals would hack into your computer and steal your personal data. But the scale of data thefts is increasing all the time. These days, criminals are more likely to hack into large corporate databases and steal your personal information, along with that of millions of other people. It’s just more efficient. Government databases are also regularly hacked.

The case for being ephemeral - is that why the young were preferring Snapchat?

Forgetting is an important enabler of forgiving. Individual and social memory fades, and past hurts become less sharp; this helps us forgive past wrongs. I’m not convinced that my marriage would be improved by the ability to produce transcripts of old arguments. Losing the ephemeral will be an enormous social and psychological change, and not one that I think our society is prepared for.

Watching everything yields nothing

Each alert requires a lengthy investigation to determine whether it’s real or not. That takes time and money, and prevents intelligence officers from doing other productive work. Or, more pithily, when you’re watching everything, you’re not seeing anything.

NOBUS is not possible

In congressional testimony, former NSA director Michael Hayden introduced the agency jargon NOBUS, “nobody but us”—that is, a vulnerability that nobody but us is likely to find or use. The NSA has a classified process to determine what it should do about vulnerabilities. The agency claims that it discloses and closes most of the vulnerabilities it finds, but holds back some—we don’t know how many—that it believes are NOBUSes.

Backdoor systems are not secure

Deliberately created vulnerabilities are very risky, because there is no way to implement backdoor access to any system that will ensure that only the government can take advantage of it. Government-mandated access forces companies to make their products and services less secure for everyone.

Secrecy has to be time-limited

Even when technologies are developed inside the NSA, they don’t remain exclusive for long. Today’s top-secret programs become tomorrow’s PhD theses and the next day’s hacker tools. Techniques first developed for the military cyberweapon Stuxnet have ended up in criminal malware.

Security can only be achieved without privacy?

Often the debate is characterized as “security versus privacy.” This simplistic view requires us to make some kind of fundamental trade-off between the two: in order to become secure, we must sacrifice our privacy and subject ourselves to surveillance.

Secrets are getting harder to keep

Privacy-law scholar Peter Swire writes about a declining half-life of secrets. What he observed is that, in general, secrets get exposed sooner than they used to. Technology is making secrets harder to keep, and the nature of the Internet makes secrets much harder to keep long-term. The push of a “send” button can deliver gigabytes across the Internet in a trice. A single thumb drive can hold more data every year. Both governments and organizations need to assume that their secrets are more likely to be exposed, and sooner, than ever before.

Weaken enemy’s network and protect our own is not possible!

We cannot simultaneously weaken the enemy’s networks while still protecting our own. The same vulnerabilities used by intelligence agencies to spy on each other are used by criminals to steal your financial passwords. Because we all use the same products, technologies, protocols, and standards, we either make it easier for everyone to spy on everyone, or harder for anyone to spy on anyone.

What cannot be protected even with encryption

Most metadata can’t be encrypted. So while you can encrypt the contents of your e-mail, the To and From lines need to be unencrypted so the e-mail system can deliver messages. Similarly, your cell phone can encrypt your voice conversations, but the phone numbers you dial, the location of your phone, and your phone’s ID number all need to be unencrypted.

Data surveillance

Computers handling data

Computers can’t abstractly reason nearly as well as people, but they can process enormous amounts of data ever more quickly. (If you think about it, this means that computers are better at working with metadata than they are at handling conversational data.) And they’re constantly improving; computing power is still doubling every eighteen months, while our species’ brain size has remained constant. Computers are already far better than people at processing quantitative data, and they will continue to improve.

Finding burner phones

The NSA has a program where it trawls through cell phone metadata to spot phones that are turned on, used for a while, and then turned off and never used again. And it uses the phones’ usage patterns to chain them together. This technique is employed to find “burner” phones used by people who wish to avoid detection.

Why Google and Facebook collects more data?

One analysis of 2013 financial reports calculated that the value of each user to Google is $ 40 per year, and only $ 6 to Facebook, LinkedIn, and Yahoo. This is why companies like Google and Facebook keep raising the ante. They need more and more data about us to sell to advertisers and thereby differentiate themselves from the competition.

We like the convenience

We want our calendar entries to automatically appear on all of our devices. Cloud storage sites do a better job of backing up our photos and files than we can manage by ourselves; Apple has done a great job of keeping malware out of its iPhone app store. We like automatic security updates and automatic backups; the companies do a better job of protecting our devices than we ever did. And we’re really happy when, after we lose a smartphone and buy a new one, all of our data reappears on it at the push of a button.

How NSA became the one to surveil everyone

The NSA became more focused on defense and more open. But eavesdropping acquired a new, and more intense, life after the terrorist attacks of 9/ 11. “Never again” was an impossible mandate, of course, but the only way to have any hope of preventing something from happening is to know everything that is happening. That led the NSA to put the entire planet under surveillance.

Surveilling everything

We spy on foreign governments and on people who are their agents. But the terrorist enemy is different. It isn’t a bunch of government leaders “over there”; it’s some random terrorist cell whose members could be anywhere. Modern government surveillance monitors everyone, domestic and international alike.

Using same technologies

There used to be Russian electronics, radios, and computers that used Russian technology. No more. We all use Microsoft Windows, Cisco routers, and the same commercial security products. You can buy an iPhone in most countries.

Corporate and government surveillance

Corporate surveillance and government surveillance aren’t separate. They’re intertwined; the two support each other. It’s a public-private surveillance partnership that spans the world.

How the corporate world helped NSA indirectly

The Snowden documents made it clear how much the NSA relies on US corporations to eavesdrop on the Internet. The NSA didn’t build a massive Internet eavesdropping system from scratch. It noticed that the corporate world was already building one, and tapped into it.

Who’s helping to build these surveillance systems?

The French company Bull SA helped the Libyan government build its surveillance center. Nigeria used the Israeli firm Elbit Systems. Syria used the German company Siemens, the Italian company Area SpA, and others. The Gadhafi regime in Libya purchased telephone surveillance technology from China’s ZTE and South Africa’s VASTech. We don’t know who built the Internet surveillance systems used in Azerbaijan and Uzbekistan, but almost certainly some Western companies helped them.

How accurate was the profiling?

The second is “signature strikes,” where unidentified individuals are targeted on the basis of their behavior and personal characteristics: their apparent ages and genders, their location, what they appear to be doing. At the peak of drone operations in Pakistan in 2009 and 2010, half of all kills were signature strikes. We don’t have any information about how accurate the profiling was.

Using past data, taken out of context

This fear of scrutiny isn’t just about the present; it’s about the past as well. Politicians already live in a world where the opposition follows them around constantly with cameras, hoping to record something that can be taken out of context.

Secrecy keeps expanding

In World War I, we were concerned about the secrecy of specific facts, like the location of military units and the precise plans of a battle. In World War II, we extended that secrecy to both large-scale operations and entire areas of knowledge. Not only was our program to build an atomic bomb secret; the entire science of nuclear weaponry was secret. After 9/ 11, we generalized further, and now almost anything can be a secret.

Surveillance driven HR

A lot of this comes from a new field called “workplace analytics,” which is basically surveillance-driven human resources management. If you use a corporate computer or cell phone, you have almost certainly given your employer the right to monitor everything you do on those devices.

Mathematics of detection

The reason lies in the mathematics of detection. All detection systems have errors, and system designers can tune them to minimize either false positives or false negatives. In a terrorist-detection system, a false positive occurs when the system mistakenly identifies something harmless as a threat. A false negative occurs when the system misses an actual attack.

More false negatives than ever

Because terrorist attacks are so rare, false positives completely overwhelm the system, no matter how well you tune. And I mean completely: millions of people will be falsely accused for every real terrorist plot the system finds, if it ever finds any.

New changes for good

Courts and regulations will stop it

Challenges along both of those fronts are being debated in the courts right now. I believe that eventually much of what the NSA is currently doing will be stopped by the courts, and more of what the NSA is currently doing will be stopped by new legislation. Of course, by then Americans will have been subject to decades of extensive surveillance already, which might well have been the agency’s strategy all along.

Trust is being lost

Chambers’s comments echo the third aspect of the competitiveness problem facing US companies in the wake of Snowden: they’re no longer trusted. The world now knows that US telcos give the NSA access to the Internet backbone and that US cloud providers give it access to user accounts. The world now knows that the NSA intercepts US-sold computer equipment in transit and surreptitiously installs monitoring hardware.

Paying for privacy

People are now much more cognizant of who has access to their data, and for years there have been indications that they’re ready to pay for privacy. A 2000 study found that US Internet spending would increase by $ 6 billion a year if customers felt their privacy was being protected when they made purchases.

Targeted surveillance » mass surveillance

Whenever we learn about an NSA success, it invariably comes from targeted surveillance rather than from mass surveillance. One analysis showed that the FBI identifies potential terrorist plots from reports of suspicious activity, reports of plots, and investigations of other, unrelated, crimes.

Encryption is easier

Encryption, and cryptography in general, is the one exception to this. Not only is defense easier than attack; defense is so much easier than attack that attack is basically impossible.

Note of conclusion

I often turn to a statement by Rev. Martin Luther King Jr: “The arc of history is long, but it bends toward justice.” I am long-term optimistic, even if I remain short-term pessimistic. I think we will overcome our fears, learn how to value our privacy, and put rules in place to reap the benefits of big data while securing ourselves from some of the risks.

Choose widespread encryption

Remember the economics of big data: just as it is easier to save everything than to figure out what to save, it is easier to spy on everyone than to figure out who deserves to be spied on. Widespread encryption has the potential to render mass surveillance ineffective and to force eavesdroppers to choose their targets. This would be an enormous win for privacy, because attackers don’t have the budget to pick everyone.

Prioritize security

By prioritizing security, we would be protecting the world’s information flows—including our own—from eavesdropping as well as more damaging attacks like theft and destruction. We would protect our information flows from governments, non-state actors, and criminals. We would be making the world safer overall.


We also need transparency in the algorithms that judge us on the basis of our data, either by publishing the code or by explaining how they work.

Institutional transparency

Individual privacy increases individual power, thereby reducing that power differential. That’s good for liberty. It’s exactly the same with transparency and surveillance. Institutional transparency reduces the power imbalance, and that’s good. Institutional surveillance of individuals increases the power imbalance, and that’s bad.


Contrary to what many government officials argue, warrants do not harm security. They are a security mechanism, designed to protect us from government overreach. Secret warrants don’t work nearly as well. The judges who oversee NSA actions are from the secret FISA Court. Compared with a traditional court, the FISA Court has a much lower standard of evidence before it issues a warrant.

External auditor

To rectify this, an external auditor is essential. Making government officials personally responsible for overreaching and illegal behavior is also important. Not a single one of those NSA LOVEINT snoops was fired, let alone prosecuted. And Snowden was rebuffed repeatedly when he tried to express his concern internally about the extent of the NSA’s surveillance on Americans.

Journalism and protection

Additionally, we need laws that protect journalists who gain access to classified information. Public disclosure in itself is not espionage, and treating journalism as a crime is extraordinarily harmful to democracy.

Targeting surveillance only

The problem is electronic surveillance on the entire population, especially mass surveillance conducted outside of a narrow court order. As we saw in Chapter 11, it doesn’t make us any safer. In fact, it makes us less safe by diverting resources and attention from things that actually do make us safer. The solution is to limit data collection and return to targeted—and only targeted—surveillance.

Separating civilian and army

One of the great political achievements of the late nineteenth century was the separation of civilian government from the military. Both history and current politics have demonstrated the enormous social damage that occurs when generals are in charge of a country. Separating the two, as many free and democratic countries worldwide do, has provided space for liberty and democracy to flourish.

Military offense should be treated as such

Offensive military operations in cyberspace, be they espionage or attack, should remain within the purview of the military. In the US, that’s Cyber Command. If we’re going to attack another country’s electronic infrastructure, we should treat it like any other attack on a foreign country. Not simple espionage (cyber or real world), but as an attack. Such operations should be recognized as offensive military actions, correspondingly approved at the highest levels of the executive branch, and should be subject to the same international law standards that govern acts of war in the offline world.

Public spun off

The NSA’s defensive capabilities in cryptography, computer security, and network defense should be spun off and become much more prominent and public.

Raising the cost of privacy breaches

By raising the cost of privacy breaches, we can make companies accept the costs of the externality and force them to expend more effort protecting the privacy of those whose data they have acquired. We’re already doing this in the US with healthcare data; privacy violations in that industry come with serious fines.

Public declarations

One way to help would be to require companies to inform users about all the information they possess that might have been compromised.

Algorithms cannot be trade secrets anymore

Other problems arise when corporations treat their underlying algorithms as trade secrets: Google’s PageRank algorithm, which determines what search results you see, and credit-scoring systems are two examples.

Rights based vs permission based

The last thing we want is for the government to start saying, “You can only do this and nothing more” with our data. Permissions-based regulation would stifle technological innovation and change. We want rights-based regulation—basically, “You can do anything you want unless it is prohibited.”


One intriguing idea has been proposed by University of Miami Law School professor Michael Froomkin: requiring both government agencies and private companies engaging in mass data collection to file Privacy Impact Notices, modeled after Environmental Impact Reports. This would serve to inform the public about what’s being collected and why, and how it’s being stored and used. It would encourage decision makers to think about privacy early in any project’s development, and to solicit public feedback.

Opt-in first!

One place to start is to require opt-in. Basically, there are two ways to obtain consent. Opt-in means that you have to explicitly consent before your data is collected and used. Opt-out is the opposite; your data will be collected and used unless you explicitly object. Companies like Facebook prefer opt-out, because they can make the option difficult to find and know that most people won’t bother. Opt-in is more fair, and the use of service shouldn’t be contingent on allowing data collection.

Classifying as fiduciaries

In order to motivate companies to become fiduciaries, governments could offer certain tax breaks or legal immunities for those willing to accept the added responsibility. Perhaps some types of business would be automatically classified as fiduciaries simply because of the large amount of personal information they naturally collect: ISPs, cell phone companies, e-mail providers, search engines, social networking platforms.

Declaring that order was never received - not declaring it will indicate it did!

Other companies are employing “warrant canaries” to try to get around legal gag orders. Starting in 2013, Apple’s transparency reports contain this sentence: “Apple has never received an order under Section 215 of the USA Patriot Act.” The idea is that if it ever receives such an order it will be prohibited from disclosing it, but it could remove the sentence as a signal to watchful readers.

More encryption!

Several large e-mail providers are now encrypting e-mail as it flows between their data centers. Other companies are doing more to encrypt communications between them and their users and customers. Both iPhones and Android phones are encrypted by default.

International standards body

We don’t want the governments of China and Russia to decide what censorship capabilities are built into the Internet; we want an international standards body to make those decisions. We don’t want Facebook to decide the extent of privacy we enjoy amongst our friends; we want to decide for ourselves. All of these decisions are bigger and more important than any one organization.

Using Tor anonymity

The current best tool to protect your anonymity when browsing the web is Tor. It’s pretty easy to use and, as far as we know, it’s secure. Similarly, various proxies can be used to evade surveillance and censorship. The program Onionshare anonymously sends files over the Internet using Tor.

Help NSA figure out

Yes, we need to figure out how much we want the NSA in all of our networks. But we also need to help the NSA not want to get into all of our networks. If we can give governments new ways to collect data on hostile nations, terrorist groups, and global criminal elements, they’ll have less need to go to the extreme measures I’ve detailed in this book. This is a genuine call for new ideas, new tools, and new techniques.


Designing appropriate surveillance

There are also times when we need to design appropriate surveillance into systems. We want shipping services to be able to track packages in real time. We want first responders to know where an emergency cell phone call is coming from. We don’t use the word “surveillance” in these cases, of course; we use some less emotionally laden term like “package tracking.”

Declared surveillance with minimum data and minimum time

The general principle here is that systems should be designed with the minimum surveillance necessary for them to function, and where surveillance is required they should gather the minimum necessary amount of information and retain it for the shortest time possible.


You can think of the difference between tactical and strategic oversight as the difference between doing things right and doing the right things. Both are required. Neither kind of oversight works without accountability. Those entrusted with power can’t be free to abuse it with impunity; there must be penalties for abuse.

Resilient system

In systems design, resilience comes from a combination of elements: fault-tolerance, mitigation, redundancy, adaptability, recoverability, and survivability. It’s what we need in the complex and ever-changing threat landscape I’ve described in this book.

Here is a list of different types of data you produce on a social networking site.

  • Service data
  • Disclosed data
  • Entrusted data
  • Incidental data
  • Behavioral data
  • Derived data

Doing surveillance the right way

More importantly, we need to support legitimate surveillance, and work on ways for these groups to do what they need to do without violating privacy, subverting security, and infringing upon citizens’ right to be free of unreasonable suspicion and observation.