Shyam Sankar: The rise of human-computer cooperation

Translator: Joseph Geni
Reviewer: Morton Bast I’d like to tell you about two games of chess. The first happened in 1997, in which Garry Kasparov, a human, lost to Deep Blue, a machine. To many, this was the dawn of a new era, one where man would be dominated by machine. But here we are, 20 years on, and the greatest change in how we relate to computers is the iPad, not HAL. The second game was a freestyle chess tournament in 2005, in which man and machine could enter together as partners, rather than adversaries, if they so chose. At first, the results were predictable. Even a supercomputer was beaten by a grandmaster with a relatively weak laptop. The surprise came at the end. Who won? Not a grandmaster with a supercomputer, but actually two American amateurs using three relatively weak laptops. Their ability to coach and manipulate their computers to deeply explore specific positions effectively counteracted the superior chess knowledge of the grandmasters and the superior computational power of other adversaries. This is an astonishing result: average men, average machines beating the best man, the best machine. And anyways, isn’t it supposed to be man versus machine? Instead, it’s about cooperation, and the right type of cooperation. We’ve been paying a lot of attention to Marvin Minsky’s vision for artificial intelligence over the last 50 years. It’s a sexy vision, for sure. Many have embraced it. It’s become the dominant school of thought in computer science. But as we enter the era of big data, of network systems, of open platforms, and embedded technology, I’d like to suggest it’s time to reevaluate an alternative vision that was actually developed around the same time. I’m talking about J.C.R. Licklider’s human-computer symbiosis, perhaps better termed “intelligence augmentation,” I.A. Licklider was a computer science titan who had a profound effect on the development of technology and the Internet. His vision was to enable man and machine to cooperate in making decisions, controlling complex situations without the inflexible dependence on predetermined programs. Note that word “cooperate.” Licklider encourages us not to take a toaster and make it Data from “Star Trek,” but to take a human and make her more capable. Humans are so amazing — how we think, our non-linear approaches, our creativity, iterative hypotheses, all very difficult if possible at all for computers to do. Licklider intuitively realized this, contemplating humans setting the goals, formulating the hypotheses, determining the criteria, and performing the evaluation. Of course, in other ways, humans are so limited. We’re terrible at scale, computation and volume. We require high-end talent management to keep the rock band together and playing. Licklider foresaw computers doing all the routinizable work that was required to prepare the way for insights and decision making. Silently, without much fanfare, this approach has been compiling victories beyond chess. Protein folding, a topic that shares the incredible expansiveness of chess — there are more ways of folding a protein than there are atoms in the universe. This is a world-changing problem with huge implications for our ability to understand and treat disease. And for this task, supercomputer field brute force simply isn’t enough. Foldit, a game created by computer scientists, illustrates the value of the approach. Non-technical, non-biologist amateurs play a video game in which they visually rearrange the structure of the protein, allowing the computer to manage the atomic forces and interactions and identify structural issues. This approach beat supercomputers 50 percent of the time and tied 30 percent of the time. Foldit recently made a notable and major scientific discovery by deciphering the structure of the Mason-Pfizer monkey virus. A protease that had eluded determination for over 10 years was solved was by three players in a matter of days, perhaps the first major scientific advance to come from playing a video game. Last year, on the site of the Twin Towers, the 9/11 memorial opened. It displays the names of the thousands of victims using a beautiful concept called “meaningful adjacency.” It places the names next to each other based on their relationships to one another: friends, families, coworkers. When you put it all together, it’s quite a computational challenge: 3,500 victims, 1,800 adjacency requests, the importance of the overall physical specifications and the final aesthetics. When first reported by the media, full credit for such a feat was given to an algorithm from the New York City design firm Local Projects. The truth is a bit more nuanced. While an algorithm was used to develop the underlying framework, humans used that framework to design the final result. So in this case, a computer had evaluated millions of possible layouts, managed a complex relational system, and kept track of a very large set of measurements and variables, allowing the humans to focus on design and compositional choices. So the more you look around you, the more you see Licklider’s vision everywhere. Whether it’s augmented reality in your iPhone or GPS in your car, human-computer symbiosis is making us more capable. So if you want to improve human-computer symbiosis, what can you do? You can start by designing the human into the process. Instead of thinking about what a computer will do to solve the problem, design the solution around what the human will do as well. When you do this, you’ll quickly realize that you spent all of your time on the interface between man and machine, specifically on designing away the friction in the interaction. In fact, this friction is more important than the power of the man or the power of the machine in determining overall capability. That’s why two amateurs with a few laptops handily beat a supercomputer and a grandmaster. What Kasparov calls process is a byproduct of friction. The better the process, the less the friction. And minimizing friction turns out to be the decisive variable. Or take another example: big data. Every interaction we have in the world is recorded by an ever growing array of sensors: your phone, your credit card, your computer. The result is big data, and it actually presents us with an opportunity to more deeply understand the human condition. The major emphasis of most approaches to big data focus on, “How do I store this data? How do I search this data? How do I process this data?” These are necessary but insufficient questions. The imperative is not to figure out how to compute, but what to compute. How do you impose human intuition on data at this scale? Again, we start by designing the human into the process. When PayPal was first starting as a business, their biggest challenge was not, “How do I send money back and forth online?” It was, “How do I do that without being defrauded by organized crime?” Why so challenging? Because while computers can learn to detect and identify fraud based on patterns, they can’t learn to do that based on patterns they’ve never seen before, and organized crime has a lot in common with this audience: brilliant people, relentlessly resourceful, entrepreneurial spirit — (Laughter) — and one huge and important difference: purpose. And so while computers alone can catch all but the cleverest fraudsters, catching the cleverest is the difference between success and failure. There’s a whole class of problems like this, ones with adaptive adversaries. They rarely if ever present with a repeatable pattern that’s discernable to computers. Instead, there’s some inherent component of innovation or disruption, and increasingly these problems are buried in big data. For example, terrorism. Terrorists are always adapting in minor and major ways to new circumstances, and despite what you might see on TV, these adaptations, and the detection of them, are fundamentally human. Computers don’t detect novel patterns and new behaviors, but humans do. Humans, using technology, testing hypotheses, searching for insight by asking machines to do things for them. Osama bin Laden was not caught by artificial intelligence. He was caught by dedicated, resourceful, brilliant people in partnerships with various technologies. As appealing as it might sound, you cannot algorithmically data mine your way to the answer. There is no “Find Terrorist” button, and the more data we integrate from a vast variety of sources across a wide variety of data formats from very disparate systems, the less effective data mining can be. Instead, people will have to look at data and search for insight, and as Licklider foresaw long ago, the key to great results here is the right type of cooperation, and as Kasparov realized, that means minimizing friction at the interface. Now this approach makes possible things like combing through all available data from very different sources, identifying key relationships and putting them in one place, something that’s been nearly impossible to do before. To some, this has terrifying privacy and civil liberties implications. To others it foretells of an era of greater privacy and civil liberties protections, but privacy and civil liberties are of fundamental importance. That must be acknowledged, and they can’t be swept aside, even with the best of intents. So let’s explore, through a couple of examples, the impact that technologies built to drive human-computer symbiosis have had in recent time. In October, 2007, U.S. and coalition forces raided an al Qaeda safe house in the city of Sinjar on the Syrian border of Iraq. They found a treasure trove of documents: 700 biographical sketches of foreign fighters. These foreign fighters had left their families in the Gulf, the Levant and North Africa to join al Qaeda in Iraq. These records were human resource forms. The foreign fighters filled them out as they joined the organization. It turns out that al Qaeda, too, is not without its bureaucracy. (Laughter) They answered questions like, “Who recruited you?” “What’s your hometown?” “What occupation do you seek?” In that last question, a surprising insight was revealed. The vast majority of foreign fighters were seeking to become suicide bombers for martyrdom — hugely important, since between 2003 and 2007, Iraq had 1,382 suicide bombings, a major source of instability. Analyzing this data was hard. The originals were sheets of paper in Arabic that had to be scanned and translated. The friction in the process did not allow for meaningful results in an operational time frame using humans, PDFs and tenacity alone. The researchers had to lever up their human minds with technology to dive deeper, to explore non-obvious hypotheses, and in fact, insights emerged. Twenty percent of the foreign fighters were from Libya, 50 percent of those from a single town in Libya, hugely important since prior statistics put that figure at three percent. It also helped to hone in on a figure of rising importance in al Qaeda, Abu Yahya al-Libi, a senior cleric in the Libyan Islamic fighting group. In March of 2007, he gave a speech, after which there was a surge in participation amongst Libyan foreign fighters. Perhaps most clever of all, though, and least obvious, by flipping the data on its head, the researchers were able to deeply explore the coordination networks in Syria that were ultimately responsible for receiving and transporting the foreign fighters to the border. These were networks of mercenaries, not ideologues, who were in the coordination business for profit. For example, they charged Saudi foreign fighters substantially more than Libyans, money that would have otherwise gone to al Qaeda. Perhaps the adversary would disrupt their own network if they knew they cheating would-be jihadists. In January, 2010, a devastating 7.0 earthquake struck Haiti, third deadliest earthquake of all time, left one million people, 10 percent of the population, homeless. One seemingly small aspect of the overall relief effort became increasingly important as the delivery of food and water started rolling. January and February are the dry months in Haiti, yet many of the camps had developed standing water. The only institution with detailed knowledge of Haiti’s floodplains had been leveled in the earthquake, leadership inside. So the question is, which camps are at risk, how many people are in these camps, what’s the timeline for flooding, and given very limited resources and infrastructure, how do we prioritize the relocation? The data was incredibly disparate. The U.S. Army had detailed knowledge for only a small section of the country. There was data online from a 2006 environmental risk conference, other geospatial data, none of it integrated. The human goal here was to identify camps for relocation based on priority need. The computer had to integrate a vast amount of geospacial information, social media data and relief organization information to answer this question. By implementing a superior process, what was otherwise a task for 40 people over three months became a simple job for three people in 40 hours, all victories for human-computer symbiosis. We’re more than 50 years into Licklider’s vision for the future, and the data suggests that we should be quite excited about tackling this century’s hardest problems, man and machine in cooperation together. Thank you. (Applause) (Applause)

Glenn Chapman


  1. good talk but i feel like he should have explained about how things like data mining would evolve in the future. There were just to many example of the present, heck there were somethings which I already knew lol.

  2. I disagree with some of the negative comments about this video. His thesis is valuable to policy managers and programmers. His outlook is valuable as many of us get caught in false dichotomies.

  3. At 4:14, it looks like the sphere shaped robot is giving it the middle finger, did anyone else notice this or was it just me? It was probably just me..

  4. I hate the loud intro and outro of TED talks, especially the outro. Keep it down will you? You wake up my neighbors.

  5. To everyone interested in this topic, I recommend a book called "Natural-Born Cyborgs" by Andy Clark. In general, look into "extended cognition" and "situated cognition."

  6. These educated persons and marvels they help to produce are not original. World that manufactured them is.

  7. …and by "in place" you mean? As of today, no quantum computer have been used for other than experimentation. They are certainly not commercially available.

  8. I think his thesis is pretty meaningless actually. Computers have ALWAYS been cooperating with us and extending our abilities. Excel helps us calculate better and make decisions we couldn't do on our own, computers predict the weather and we interpret the results – there's nothing new here.

  9. the reason I didn't put the name because that it is not really 100% known..
    and there are slightly variable versions of the quote.
    This site explains:
    ——-benshoemate-com /2008/11/30/einstein-never-said-that/
    anyway, the closest match is:
    Robert O' Conner, April 1968
    but some attribute it to Leo M. Cherne
    lol. I guess its hard to accept that you can't know such a thing for sure.. especially in the "information technology era".although It can be pinned down in the end.

  10. Seems like data processing vs fact combining will prevent lots of lies in our society….maybe…..hopefully

  11. You gotto understand that the claims you make are quite extrordinary. Such claims are not thrown around carelessly without evidence. If you're going to claim that the technology to increase human cognitive function ten-fold has been around since 1992, you got to give me more than just assertions.

  12. I'm just clarifying, but I have never said that I'm having any difficulty accepting that I don't know things, I have been wrong many times and I have accepted those incidences, but what I did do that defines my understanding, is to seek the correct answer…
    for example: I thought it was George Washington that said "The United States is in no sense founded on the christian religion" when it was actually unanimously voted by the entire congress into the Treaty of Tripoli in the late 1700s

  13. you got me wrong.. I didnt say you have difficulty accepting a mistake or something..
    I meant a different thing:
    when i tracked the quote I was expecting a clear cut answer in a fast way.. which I didnt.. and I realized that these day we take it for granted; to search(google) and find a sure answer

  14. well I was reading "lol. I guess its hard to accept that you can't know such a thing for sure…" and you have to admit that it can be interpreted to mean that you thought i would've had a hard time accepting being wrong…
    still, no harm, no foul, I understand, we're all happy πŸ™‚

  15. Humans with calculators and spell checkers.. DUH! Stay out of my body and stop tracking me.

  16. "Yes, quotes on the internet are always credited to the right people." – Leonardo da Vinci

  17. I get the idea, it is a good one and I agree somewhat.
    Computers need to get closer and quicker to our brains.
    But this is not really "co-operation" at all ….
    Computers do not "operate", let alone "co-operate", they only execute instructions that we line up for it – frozen for the most part.
    Someday we may find that we to only execute a program to, but until then,
    does your car cooperate with you when you drive to work?
    If you say yes, then what would be your car's choice in that? πŸ˜‰

  18. "Poor people think they have to give a comment to everything… even poorer people think they have to give a comment on youtube" –

    Sir Dr.Dr. Isaac Newton

  19. in the example of protein folding, we just do not know yet how to code what it is that the humans are doing.
    if we had computation that would augment people's brains it would fit into their brains and amplify their intelligence in as many ways as possible – at the infinite limit making all of us the same.
    after we accomplish this we now have the political problem that everyone is essentially equal, which is why unless we are careful we will get one guy on top who plays god. πŸ˜‰

  20. "eventually" those "one little steps" get figured out and enable the system to be abstracted. i am not sure there is any indication that the whole human being cannot be automated, not that this is a goal. What we seem to do with machines and computers is to abstract out the best … but with computers and thinking
    1. we can not know what is the best
    2. the power this gives to any individual goes in diametric opposition to the political order that the people on top do anything to maintain.

  21. Yeah, computing pi is unsolvable, but we have a quick solution for that … we just go to the necessary precision. It's just a pattern recognition thing if I read your comment right.

    The thing is that information is power, and once it is abstracted into a program it is cheap as hell, so in order to maintain our political system all of this stuff goes against the status quo. that was my main point there.

    Not sure it would be a good idea to simulate the human brain. understand it first.

  22. "All right stop Collaborate and listen Ice is back with my brand new invention" -Thomas Jefferson

  23. A casual stroll through the lunatic asylum shows that faith does not prove anything. ~
    Frank Zappa

  24. i just cant believe how backward thinking this lecture is…? you're saying that artificial intelligence doesnt work because it doesnt work? that is your evidence?

  25. The traveling salesman problem … yet sales men travel and get their jobs done every day.

  26. it would be nice if understood what i said, and you would reply to my post or have a point of your own.

  27. > Coding what the humans are doing is impossible because the human brain does not process data in binary bits.

    That has not been established yet since there is no model, although you might want to look at the human connectome project, and maybe stop trolling all my posts.

  28. I agree most of this talk, but he didn't need to attack artificial intelligence (AI) to make his points. Because we often don't need to interact directly with AI, it sometimes fades into the background, and we forget it's there. He says β€œTwenty years later, the greatest change to how we relate to computers was the iPad, not HAL.”

    Is the iPad a greater change than the internet search engine, which is AI?

  29. This grad sucks at giving presentations. No effective communication skills. (effective).

  30. How to kill a presentation; mention terrorism, show a cartoon of an Arab when you do.

  31. fuck off with this chuck norris shit. The guy is a homophobic creationist christian, who's against science. He deserves no fucking respect.

  32. People use quotes as a way to strengthen their own position. If I can quote someone you respect, it adds credibility to whatever argument I’m making. Over time, the truth gets further and further away. The biggest names have always attracted people who are more than willing to put words into their mouths for their own gain – Confucious, Jesus, Aristotle, Shakespere, Einstein, and the biggest, most misquoted, of them all – God.

  33. in the time of einstein, a computer as powerful as your calculator filled a room so i think the quote is outdated πŸ˜›

  34. Fascinating Technology – same old War on Terror Propaganda.

    If these guys are so clever why did it take 10 years to find the worlds biggest scapegoat Osama? Almost dead from kidney failure and living next door to a military compound. Yet this human computer alliance can't figure out the ties with CIA and psychopaths like Dick Cheyney profiteering from war through Halliburton. My only question, if you're that bright who are you really working for?

  35. i can already make toast … who needs a toaster or a computer or an ai/ia … fire is sufficient as the enabler … all here are already in gestalt accord

  36. LoL @ using, the cheating done by IBM to beat Kasparov as some sort of feat.
    Jesus, why do nerds never research before telling a story…

  37. "The problem with the Internet is you can never tell if a quote is genuine or not." -Abraham Lincoln.

  38. 'If you're flammable and have legs, you are never blocking a fire exit.' – Mitch Hedberg

  39. Great choice of fictional author. Abraham Lincoln talking about the internet, most awesome thing i've read all day.

  40. OMFG, this guy is basically outlining the foundations of my company. Hey Shyam Sankar, shhhhh. You're giving away my bread and butter. JsJK

  41. @10:02 The proof of the presenter's bigotry and ignorance is astounding. Showing a representation of a stereotypical Arab when mentioning terrorism and blindly butchering a flag to associate it with these acts. Shame on you Shyam Sankar!

  42. This is the part Computational Intelligence plays in Star Trek.

  43. The Zeitgeist Movement (TZM) has been showing that we have the ability to generate energy, food, housing, health and education for everyone in the world, while, at the same time, we are able to ELIMINATE OUR ECONOMICAL, SOCIAL & ECOLOGICAL CRISIS along with corruption, poverty, abuse of power, unemployment, social stratification, and other factors that hinder the life and sustainability of the planet and its inhabitants.To know more about it, watch: "Zeitgeist Addendum" – It is on youtube

  44. Actually, he has been on TED


  45. why Arab? The cartoon you mention could be associated with a typical [that's typical or stereotypical… take your pick] Afghan or Pakistani.

  46. Yes untill they are times faster in 40 years then they will enslave us.

Leave a Reply

Your email address will not be published. Required fields are marked *