Today's headlines are full of technological advances that promise an optimized future, from artificial intelligence to diagnose illnesses to driverless cars that revolutionize transportation. One day, everything will be easier, faster and better, they tell us.
It's an attractive vision, but there's a downside to all this efficiency, says academic and writer Edward Tenner, author of The Efficiency Paradox: What Big Data Can not Do (Knopf's next week). "Trying to be ultimately efficient at all times will be successful in the short term," he says. "But in the long run, you would be harming your efficiency." Tenner is not a Luddite, and his book does not suggest giving up on efficiency and Big Data. It simply advises us to use common sense and keep in mind that there are always concessions.
The Verge spoke with Tenner about how the paradox of efficiency occurs, its costs and how to balance intuition and technology.
This interview has been slightly edited for clarity.
What is the paradox of efficiency? And how did you get interested in the subject?
I saw that there was something really new and very exciting on the web. The increase in mobile computing and the growing interest in artificial intelligence and big data were actually having such a big impact, in some way even greater, than the initial web of the nineties. This story kept growing in me, and gradually, I saw that there was a kind of involuntary consequence of this: trying to be ultimately efficient at all times will be successful in the short term. But in the long run, you would be harming your efficiency.
You define the efficiency in the preface of the book as capable of producing goods or providing a service with a minimum of waste. Then, you talk about "continuous process efficiency" versus "platform efficiency". What is the difference between these two?
People in Elizabethan times and even in the Middle Ages did not have the concept of efficiency that we have today. That really depended on the increase of thermodynamics in the nineteenth century and the need to obtain as much energy as possible from water turbines and steam engines. That efficiency of the 19th century is what I call "continuous process efficiency", and that is when the things that were made piece by piece could now be done in a sequence. For example, when paper was made in the eighteenth century, it was always in sheets. In the nineteenth century, entrepreneurs found a way for paper to come out continuously from a mill, and that was what made mass literacy, newspapers and expensive books possible. In its way it was as important as the Gutenberg revolution of the fifteenth century.
Now, the efficiency of the platform is really a completely different type. It's something that's really in the cloud, and it's about getting buyers and sellers together at minimal and extremely fast cost. Therefore, it is about obtaining transportation or buying a ticket or paying rent or banking.
The efficiency of the platform is wonderful, and I am not condemning it at all, but one of the unfortunate consequences is that it has tended to attract investment capital away from much more difficult things. It is much easier to make a small fortune with a start-up based on a platform that, for example, develop a more efficient battery. I came to believe that because these physical and chemical companies take much longer, are much more expensive, are much more disorganized and, therefore, are less attractive to investors. That is a negative side of the efficiency of the platform.
Was there a moment in American culture when we did not care so much about efficiency? To be clear, I mean the general culture that does not care, not subcultures or specific movements like Luddites.
One of the interesting things about American culture is that even subcultures that sought to belittle efficiency-like southern planters-were based on the principle of trying to get as much profit as possible from slave labor and land. Then, there was this industrial regulation both in the south and in the factories of the north.
The United States, I believe, has always been a pioneer of efficiency. They were admired by Europeans for their rigorous efficiency in doing everything, and the criticism of the Americans was that they were so worried about earning money and efficiently that they were losing the best things in life. On the other hand, European observers always came and tried to copy American methods!
The great industrial complexes of the Soviet era were based on the steelworks of Gary, Indiana, and Lenin and the other Soviet leaders greatly admired Henry Ford.
Author Edward Tenner.Photo by Michael Lionstar
Let's talk about some of the examples of the disadvantages of efficiency. In one of the chapters, he talks about the effects on the arts and culture.
By eliminating so many trial and error errors, the efficiency of the platform can block us in existing patterns. For example, publishers or film producers can analyze the data to see which genres have been most popular, which will attract viewers from a certain demographic group, and this could make the publication more predictable or more profitable.
But many of the great successes have been real surprises that have broken many of the rules. AI is really good to find hidden rules and apply them and optimize everything according to the hidden rules, but in reality it is the events that break the rules that have made our life more exciting.
I am also interested in a study that you mention about how popularity works and the cost of getting rid of the guardians of popular culture.
People have presented gatekeepers as a burden. They are a level between consumers and producers. Then, if you do not have them, you are reducing transaction costs and making things more efficient. You can find things yourself. In the mid-1990s, Bill Gates and his co-authors wrote The Road Ahead about the economy of the friction-free future, where these intermediaries would not exist.
But these guardians did have a useful role. They might recognize a talent that was not quite ready to be popular, but that had something interesting and exciting that was worth developing. If you eliminate goalies, it's a bit like sports without coaches.
For example, there was a Princeton study that showed that when you statistically study which people (ordinary consumers, not an elite panel of critics) think about the quality of various jobs offered on the web, those that become very popular only have one Small advantage in quality. It's not really random, but it's small. When you observe popularity patterns on the web, there is a small central interest that expands rapidly. Without guardians, much of the popularity depends on what happens to be popular first.
In your chapter on education, you talk about the "value of the inefficient medium," such as paper, for example. What are some examples where inefficiency makes us learn better?
I have read reading and comprehension studies that psychologists have done over the years. Electronic reading and reading paper have their own advantage. The electronic medium is better for recognizing details, but that reading on paper gives a better holistic sense of what an author is trying to say. That is a compensation.
This is similar to what I say in my chapter on geography. The map on paper is uncomfortable in many circumstances and inferior to the electronic map, which I use all the time. But, on the other hand, the paper map gives you an idea of the broader terrain, and is very useful for orientation.
Medicine is an area with high hopes for AI and big data: precision medicine, AI diagnosis. What are some of the disadvantages here?
In medicine, there are warning signs, and these warnings, in turn, must be addressed or discarded by further testing. As more diagnoses develop, there is a high possibility of false positives that cause people to go through more tests, and some of the additional tests may actually have side effects of their own.
Recently, in The New York Times, there was a review of the new book by Barbara Ehrenreich, who is swearing completely to the medical system. On the other hand, there are people who pay large sums of money for the so-called concierge medicine with doctors who always control them. There are different styles, and I am not underestimating the life extension project, but I think that quite a few critics of medicine have pointed out the advantage of a holistic approach to people's health and the kind of understanding that the best outdated doctors have had. .
You should not completely trust that because sometimes those wonderful old fashioned doctors had outdated ideas that have been contradicted by the research data. Therefore, you need big data, but there are many difficulties in analyzing big data, and there is some tension between academic statisticians and data analysts in the commercial sector about what constitutes good practice in the use of this data.
"There are many things that even young children can appreciate that the most advanced machine learning technologies can not."
How should we think about these concessions? Who should help us determine what concessions are important enough for us to do so?
It really is for each person to use electronic and similar materials in a way that suits their own lives. This is not a book about politics. It is a book that tells people: "Do not be afraid of your common sense". I think everyone can recognize what works for them, and people will have very different styles.
The last chapter of his book talks about strategies to balance algorithms and common sense. How did these strategies come about?
I tried to see which of the ideas was applied in the chapters. For example, people are familiar with the idea of chance, so they did not need much presentation. The important thing of chance is that if you eliminate the errors, you will depend too much on the immediate and recent experience and you will not be sufficiently open to productive surprises. But the concept of "desirable difficulty", on the other hand, where we can learn better if things are more difficult, is less familiar to people because it occurs in studies, for example, of reading comprehension that show that something less readable could in actually encourage people to concentrate more.
What else is missing?
There are two factors that people underestimate and that are serious problems in the application of efficient technology. One of them is what is called "local knowledge". We all know that there is a route that can look really good on a map, but we know it is a problem because we have covered it. For example, there is an intersection that seems to be the shortest, but I know that traffic is tied there, and it is faster to take a longer road, and [traffic app] Waze has not done so. From time to time, Waze points out a really crazy address, and if people do not have common sense, sooner or later they will be very disappointed. As I have come to recognize that Waze is not infallible, I use it, and if I see that something is not right, I try to stop and take a look at a printed map and find out what is going wrong.
The other is tacit knowledge. The idea is that no matter how much information is fed into an intelligent system, there are many, many things that are tacit, meaning that they are not explicitly stated anywhere. You can not find that information in an encyclopedia.
An example is how young children can understand the meaning of a proverb, such as "a point in time saves nine" or "a rolling stone does not pick up moss," in a way that a computer can not. There are many things that even young children can appreciate that the most advanced machine learning technologies can not, and I think that for me it is one of the most exciting things about the mind and about being human.