3 Breakthrough Technologies 2017

•  
   
   

3 Breakthrough Technologies2017these technologies all have staying power. They will affect the economy and our politics, improve medicine, or influence our culture. Some are unfolding now; others will take a decade or more to develop. But you should know about all of them right now.

reversing paralysis

Scientists are making remarkable progress at using brain implants to restore the freedom of movement that spinal cord injuries take away.

The French neuroscientist was watching a macaque monkey as it hunched aggressively at one end of a treadmill. His team had used a blade to slice halfway through the animal’s spinal cord, paralyzing its right leg. Now Courtine wanted to prove he could get the monkey walking again. To do it, he and colleagues had installed a recording device beneath its skull, touching its motor cortex, and sutured a pad of flexible electrodes around the animal’s spinal cord, below the injury. A wireless connection joined the two electronic devices.

The result: a system that read the monkey’s intention to move and then transmitted it immediately in the form of bursts of electrical stimulation to its spine. Soon enough, the monkey’s right leg began to move. Extend and flex. Extend and flex. It hobbled forward. “The monkey was thinking, and then boom, it was walking,” recalls an exultant Courtine, a professor with Switzerland’s École Polytechnique Fédérale de Lausanne.

In recent years, lab animals and a few people have controlled computer cursors or robotic arms with their thoughts, thanks to a brain implant wired to machines. Now researchers are taking a significant next step toward reversing paralysis once and for all. They are wirelessly connecting the brain-reading technology directly to electrical stimulators on the body, creating what Courtine calls a “neural bypass” so that people’s thoughts can again move their limbs. 

h

At Case Western Reserve University, in Cleveland, a middle-aged quadriplegic—he can’t move anything but his head and shoulder—agreed to let doctors place two recording implants in his brain, of the same type Courtine used in the monkeys. Made of silicon, and smaller than a postage stamp, they bristle with a hundred hair-size metal probes that can “listen” as neurons fire off commands.

Reversing Paralysis

• BreakthroughWireless brain-body electronic interfaces to bypass damage to the nervous system.

• Why It MattersThousands of people suffer paralyzing injuries every year.

• Key Players- École Polytechnique Fédérale de Lausanne 

- Wyss Center for Bio and Neuroengineering 

- University of Pittsburgh 

- Case Western Reserve University

• Availability10 to 15 years

To complete the bypass, the Case team, led by Robert Kirsch and Bolu Ajiboye, also slid more than 16 fine electrodes into the muscles of the man’s arm and hand. In videos of the experiment, the volunteer can be seen slowly raising his arm with the help of a spring-loaded arm rest, and willing his hand to open and close. He even raises a cup with a straw to his lips. Without the system, he can’t do any of that.

Just try sitting on your hands for a day. That will give you an idea of the shattering consequences of spinal cord injury. You can’t scratch your nose or tousle a child’s hair. “But if you have this,” says Courtine, reaching for a red espresso cup and raising it to his mouth with an actor’s exaggerated motion, “it changes your life.”

Grégoire Courtine holds the two main parts of the brain-spine interface.

PHOTOGRAPH BY HILLARY SANCTUARY | EPFL

The Case results, pending publication in a medical journal, are a part of a broader effort to use implanted electronics to restore various senses and abilities. Besides treating paralysis, scientists hope to use so-called neural prosthetics to reverse blindness with chips placed in the eye, and maybe restore memories lost to Alzheimer’s disease (see “10 Breakthrough Technologies 2013: Memory Implants”).

And they know it could work. Consider cochlear implants, which use a microphone to relay signals directly to the auditory nerve, routing around non-working parts of the inner ear. Videos of wide-eyed deaf children hearing their mothers for the first time go viral on the Internet every month. More than 250,000 cases of deafness have been treated.

In this video made by EPFL researchers, a monkey with a spinal cord injury that paralyzed its right leg is able to walk again.

But it’s been harder to turn neural prosthetics into something that helps paralyzed people. A patient first used a brain probe to move a computer cursor across a screen back in 1998. That and several other spectacular brain-control feats haven’t had any broader practical use. The technology remains too radical and too complex to get out of the lab. “Twenty years of work and nothing in the clinic!” Courtine exclaims, brushing his hair back. “We keep pushing the limits, but it is an important question if this entire field will ever have a product.”

Courtine’s laboratory is located in a vertiginous glass-and-steel building in Geneva that also houses a $100 million center that the Swiss billionaire Hansjörg Wyss funded specifically to solve the remaining technical obstacles to neurotechnologies like the spinal cord bypass. It’s hiring experts from medical-device makers and Swiss watch companies and has outfitted clean rooms where gold wires are printed onto rubbery electrodes that can stretch as our bodies do.

A close-up of a brain-reading chip, bristling with electrodes. 

Flexible electrodes developed to simulate the spinal cord.

The head of the center is John Donoghue, an American who led the early development of brain implants in the U.S. (see “Implanting Hope”) and who moved to Geneva two years ago. He is now trying to assemble in one place the enormous technical resources and talent—skilled neuroscientists, technologists, clinicians—needed to create commercially viable systems.

Among Donoghue’s top priorities is a “neurocomm,” an ultra-compact wireless device that can collect data from the brain at Internet speed. “A radio inside your head,” Donoghue calls it, and “the most sophisticated brain communicator in the world.” The matchbox-size prototypes are made of biocompatible titanium with a sapphire window. Courtine used an earlier, bulkier version in his monkey tests.

As complex as they are, and as slow as progress has been, neural bypasses are worth pursuing because patients desire them, Donoghue says. “Ask someone if they would like to move their own arm,” he says. “People would prefer to be restored to their everyday self. They want to be reanimated.”

Self-Driving Trucks

Tractor-trailers without a human at the wheel will soon barrel onto highways near you. What will this mean for the nation’s 1.7 million truck drivers?

Roman Mugriyev was driving his long-haul 18-wheeler down a two-lane Texas highway when he saw an oncoming car drift into his lane just a few hundred feet ahead. There was a ditch to his right and more oncoming cars to his left, so there was little for him to do but hit his horn and brake. “I could hear the man who taught me to drive telling me what he always said was rule number one: ‘Don’t hurt anybody,’” Mugriyev recalls.
But it wasn’t going to work out that way. The errant car collided with the front of Mugriyev’s truck. It shattered his front axle, and he struggled to keep his truck and the wrecked car now fused to it from hitting anyone else as it barreled down the road. After Mugriyev finally came to a stop, he learned that the woman driving the car had been killed in the collision.
Could a computer have done better at the wheel? Or would it have done worse
We will probably find out in the next few years, because multiple companies are now testing self-driving trucks. Although many technical problems are still unresolved, proponents claim that self-driving trucks will be safer and less costly. “This system often drives better than I do,” says Greg Murphy, who’s been a professional truck driver for 40 years. He now serves as a safety backup driver during tests of self-driving trucks by Otto, a San Francisco company that outfits trucks with the equipment needed to drive themselves.At first glance, the opportunities and challenges posed by self-driving trucks might seem to merely echo those associated with self-driving cars. But trucks aren’t just long cars. For one thing, the economic rationale for self-driving trucks might be even stronger than the one for driverless cars. Autonomous trucks can coördinate their movements to platoon closely together over long stretches of highway, cutting down on wind drag and saving on fuel. And letting the truck drive itself part of the time figures to help truckers complete their routes sooner.
But the technological obstacles facing autonomous trucks are higher than the ones for self-driving cars. Otto and other companies will need to demonstrate that sensors and code can match the situational awareness of a professional trucker—skills honed by years of experience and training in piloting an easily destabilized juggernaut, with the momentum of 25 Honda Accords, in the face of confusing road hazards, poor surface conditions, and unpredictable car drivers.
And perhaps most important, if self-driving trucks do take hold, they figure to be more controversial than self-driving cars. At a time when our politics and economy are already being upended by the threats that automation poses to jobs (see “The Relentless Pace of Automation”), self-driving trucks will affect an enormous number of blue-collar workers. There are 1.7 million trucking jobs in the U.S., according to the Bureau of Labor Statistics. Technology is unlikely to replace truckers entirely anytime soon. But it will almost certainly alter the nature of the job, and not necessarily in ways that all would welcome.
Self-Driving Trucks
BreakthroughLong-haul trucks that drive themselves for extended stretches on highways.Why It MattersThe technology might free truck drivers to complete routes more efficiently, but it could also erode their pay and eventually replace many of them altogether.Key Players- Otto - Volvo - Daimler - PeterbiltAvailability5 to 10 years“We’re not waiting”
Otto’s headquarters, in the once-seedy South of Market section of San Francisco, isn’t much like many of the other tech startups that have transformed the area. Proudly oblivious to that neighborhood upgrade, it’s a barely renovated former furniture warehouse converted to a garage and machine shop, with semi trucks in various states of dismantlement hulking over benches of tools and computers. “No fancy, shiny offices here,” brags Eric Berdinis, Otto’s young and clean-cut-looking product manager.
Berdinis shows off the latest generation of the company’s fast-evolving technology, which is currently installed on Volvo semis. Unlike the bolted-on, kludgy-looking hardware that’s been on testing runs for the past year, the newer versions of the company’s sensor and processing arrays are more sleekly integrated throughout the Volvo cab. The equipment includes four forward-facing video cameras, radar, and a box of accelerometers that Berdinis boasts is “as close as the government allows you to get to missile-guidance quality.”
Particularly key to Otto’s technology is a lidar system, which uses a pulsed laser to amass detailed data about the truck’s surroundings. The current third-party lidar box costs Otto in the vicinity of $100,000 each. But the company has a team designing a proprietary version that could cost less than $10,000.


Inside the cab is a custom-built, liquid-cooled, breadbox-size micro-supercomputer that, Berdinis claims, provides the most computing muscle ever crammed into so small a package. It is needed to crunch the vast stream of sensor data and shepherd it through the guidance algorithms that adjust braking and steering commands to compensate for the truck’s load weight. Rounding out the hardware lineup is a drive-by-wire box to turn the computer’s output into physical truck-control signals. It does this through electromechanical actuators mounted to the truck’s mechanical steering, throttling, and braking systems. Two big red buttons in the cab—Otto calls them the Big Red Buttons—can cut off all self-driving activity. But even without them, the system is designed to yield to any urgent tugs on the steering wheel or heavy pumps of the pedals from anyone in the driver’s seat.
Even if drivers stay on in the cab, it’s not clear the economics will work out in their favor.
Otto was founded early in 2016 by Anthony Levandowski, who had been with Google’s self-driving-car effort, and Lior Ron, who headed up Google Maps, along with two others. It was a natural move to build on Google’s vast experience with its autonomous cars, which have driven more than two million miles on U.S. roads in several states, with an eye toward the four million trucks in the U.S. alone. Volvo Trucks, Daimler Trucks, and Peterbilt have been working on their own autonomous-truck technology.

The 360-Degree Selfie

Inexpensive cameras that make spherical images are opening a new era in photography and changing the way people share stories.

Seasonal changes to vegetation fascinate Koen Hufkens. So last fall Hufkens, an ecological researcher at Harvard, devised a system to continuously broadcast images from a Massachusetts forest to a website called VirtualForest.io. And because he used a camera that creates 360° pictures, visitors can do more than just watch the feed; they can use their mouse cursor (on a computer) or finger (on a smartphone or tablet) to pan around the image in a circle or scroll up to view the forest canopy and down to see the ground. If they look at the image through a virtual-reality headset they can rotate the photo by moving their head, intensifying the illusion that they are in the woods.


Hufkens says the project will allow him to document how climate change is affecting leaf development in New England. The total cost? About $550, including $350 for the Ricoh Theta S camera that takes the photos.

We experience the world in 360 degrees, surrounded by sights and sounds. Until recently, there were two main options for shooting photos and video that captured that context: use a rig to position multiple cameras at different angles with overlapping fields of view or pay at least $10,000 for a special camera. The production process was just as cumbersome and generally took multiple days to complete. Once you shot your footage, you had to transfer the images to a computer; wrestle with complex, pricey software to fuse them into a seamless picture; and then convert the file into a format that other people could view easily.
 ALLie CameraIt uses technology originally developed for the surveillance industry and can capture images in low light.
Today, anyone can buy a decent 360° camera for less than $500, record a video within minutes, and upload it to Facebook or YouTube. Much of this amateur 360° content is blurry; some of it captures 360 degrees horizontally but not vertically; and most of it is mundane. (Watching footage of a stranger’s vacation is almost as boring in spherical view as it is in regular mode.) But the best user-generated 360° photos and videos—such as the Virtual Forest—deepen the viewer’s appreciation of a place or an event.
Journalists from the New York Times and Reuters are using $350 Samsung Gear 360 cameras to produce spherical photos and videos that document anything from hurricane damage in Haiti to a refugee camp in Gaza. One New York Times video that depicts people in Niger fleeing the militant group Boko Haram puts you in the center of a crowd receiving food from aid groups. You start by watching a man heaving sacks off a pickup truck and hearing them thud onto the ground. When you turn your head, you see the throngs that have gathered to claim the food and the makeshift carts they will use to transport it. The 360° format is so compelling that it could become a new standard for raw footage of news events—something that Twitter is trying to encourage by enabling live spherical videos in its Periscope app.
 Kodak Pixpro SP360 4kIt can be mounted on a drone to produce aerial 360° videos.
Or consider the spherical videos of medical procedures that the Los Angeles startup Giblib makes to teach students about surgery. The company films the operations by attaching a $500 360fly 4K camera, which is the size of a baseball, to surgical lights above the patient. The 360° view enables students to see not just the surgeon and surgical site, but also the way the operating room is organized and how the operating room staff interacts.
Meanwhile, inexpensive 360° cameras such as Kodak’s $450 Pixpro SP360 4K are popping up on basketball backboards, football fields, and hockey nets during practice for professional and collegiate teams. Coaches say the resulting videos help players visualize the action and prepare for games in ways that conventional sideline and end-zone videos can’t.
Component innovations
These applications are feasible because of the smartphone boom and innovations in several technologies that combine images from multiple lenses and sensors. For instance, 360° cameras require more horsepower than regular cameras and generate more heat, but that is handled by the energy-efficient chips that power smartphones. Both the 360fly and the $499 ALLie camera use Qualcomm Snapdragon processors similar to those that run Samsung’s high-end handsets.
 360Fly 4KDustproof and water-resistant, it’s often used to record extreme sports.
Camera companies also benefited in recent years from smartphone vendors’ continuous quest to integrate higher-quality imaging into their gadgets. The competition forced component makers like Sony to shrink image sensors and ensure that they offered both high resolution and good performance in low light. As the huge smartphone market helped bring down component prices, 360°-camera makers found it possible to price their devices accessibly, often at less than $500. “There are sensors that now cost $1 instead of $1,000 because they’re used in smartphones, which have incredible economies of scale,” says Jeffrey Martin, the CEO of a 360°-camera startup called Sphericam. Advances in optics played a part as well. Unlike traditional cameras, which have fairly narrow fields of view, 360° cameras sport exaggerated fish-eye lenses that require special optics to align and focus images across multiple points.
The 360-Degree Selfie
BreakthroughConsumer cameras that produce 360° images, providing a realistic sense of events or places.Why It MattersPhotos and videos with this perspective could become the new standard for everything from news coverage to vacation shots.Key Players- Ricoh - Samsung - 360fly - JK Imaging (maker of Kodak Pixpro digital cameras) - IC Real Tech (maker of the ALLie camera) - Humaneyes TechnologiesAvailabilityNowMost 360° cameras lack displays and viewfinders. To compensate, camera makers developed apps that you can download to your phone to compose shots and review the resulting images. The cameras connect to the apps wirelessly, and many of them allow you to upload photos and video directly from your phone to Facebook and YouTube. In turn, those sites have made it possible over the past year for people not just to post recorded 360° content but to live-stream 360° videos as well.
Because creating 360° content requires stitching together multiple images, doing it on the fly for live streaming represents an impressive technical achievement. Computer-vision algorithms have simplified the process so that it can  be done on the camera itself, which in turn allows people to live-stream video with minimal delays. (It helps that most consumer-grade cameras have only two lenses and thus one stitch line. Professional versions can have six to 24 lenses.) The ALLie camera supports fast stitching and live-streaming, as do Ricoh’s upcoming Ricoh R development kit camera and Kodak’s Orbit360 4K, which will be available later this year for $500.
 Ricoh Theta SRicoh put the image sensors on the camera’s sides instead of behind its lenses, making its thin shape possible.
Spherical cameras represented 1 percent of worldwide consuby Elizabeth Woyke
4Seasonal changes to vegetation fascinate Koen Hufkens. So last fall Hufkens, an ecological researcher at Harvard, devised a system to continuously broadcast images from a Massachusetts forest to a website called VirtualForest.io. And because he used a camera that creates 360° pictures, visitors can do more than just watch the feed; they can use their mouse cursor (on a computer) or finger (on a smartphone or tablet) to pan around the image in a circle or scroll up to view the forest canopy and down to see the ground. If they look at the image through a virtual-reality headset they can rotate the photo by moving their head, intensifying the illusion that they are in the woods.

hHufkens says the project will allow him to document how climate change is affecting leaf development in New England. The total cost? About $550, including $350 for the Ricoh Theta S camera that takes the photos.
 This story is part of our March/April 2017 IssueSee the rest of the issue
SubscribeWe experience the world in 360 degrees, surrounded by sights and sounds. Until recently, there were two main options for shooting photos and video that captured that context: use a rig to position multiple cameras at different angles with overlapping fields of view or pay at least $10,000 for a special camera. The production process was just as cumbersome and generally took multiple days to complete. Once you shot your footage, you had to transfer the images to a computer; wrestle with complex, pricey software to fuse them into a seamless picture; and then convert the file into a format that other people could view easily. ALLie CameraIt uses technology originally developed for the surveillance industry and can capture images in low light.
Today, anyone can buy a decent 360° camera for less than $500, record a video within minutes, and upload it to Facebook or YouTube. Much of this amateur 360° content is blurry; some of it captures 360 degrees horizontally but not vertically; and most of it is mundane. (Watching footage of a stranger’s vacation is almost as boring in spherical view as it is in regular mode.) But the best user-generated 360° photos and videos—such as the Virtual Forest—deepen the viewer’s appreciation of a place or an event.
Journalists from the New York Times and Reuters are using $350 Samsung Gear 360 cameras to produce spherical photos and videos that document anything from hurricane damage in Haiti to a refugee camp in Gaza. One New York Times video that depicts people in Niger fleeing the militant group Boko Haram puts you in the center of a crowd receiving food from aid groups. You start by watching a man heaving sacks off a pickup truck and hearing them thud onto the ground. When you turn your head, you see the throngs that have gathered to claim the food and the makeshift carts they will use to transport it. The 360° format is so compelling that it could become a new standard for raw footage of news events—something that Twitter is trying to encourage by enabling live spherical videos in its Periscope app.
 Kodak Pixpro SP360 4kIt can be mounted on a drone to produce aerial 360° videos.
Or consider the spherical videos of medical procedures that the Los Angeles startup Giblib makes to teach students about surgery. The company films the operations by attaching a $500 360fly 4K camera, which is the size of a baseball, to surgical lights above the patient. The 360° view enables students to see not just the surgeon and surgical site, but also the way the operating room is organized and how the operating room staff interacts.
Meanwhile, inexpensive 360° cameras such as Kodak’s $450 Pixpro SP360 4K are popping up on basketball backboards, football fields, and hockey nets during practice for professional and collegiate teams. Coaches say the resulting videos help players visualize the action and prepare for games in ways that conventional sideline and end-zone videos can’t.
Component innovations
These applications are feasible because of the smartphone boom and innovations in several technologies that combine images from multiple lenses and sensors. For instance, 360° cameras require more horsepower than regular cameras and generate more heat, but that is handled by the energy-efficient chips that power smartphones. Both the 360fly and the $499 ALLie camera use Qualcomm Snapdragon processors similar to those that run Samsung’s high-end handsets.
 360Fly 4KDustproof and water-resistant, it’s often used to record extreme sports.
Camera companies also benefited in recent years from smartphone vendors’ continuous quest to integrate higher-quality imaging into their gadgets. The competition forced component makers like Sony to shrink image sensors and ensure that they offered both high resolution and good performance in low light. As the huge smartphone market helped bring down component prices, 360°-camera makers found it possible to price their devices accessibly, often at less than $500. “There are sensors that now cost $1 instead of $1,000 because they’re used in smartphones, which have incredible economies of scale,” says Jeffrey Martin, the CEO of a 360°-camera startup called Sphericam. Advances in optics played a part as well. Unlike traditional cameras, which have fairly narrow fields of view, 360° cameras sport exaggerated fish-eye lenses that require special optics to align and focus images across multiple points.
The 360-Degree Selfie
BreakthroughConsumer cameras that produce 360° images, providing a realistic sense of events or places.Why It MattersPhotos and videos with this perspective could become the new standard for everything from news coverage to vacation shots.Key Players- Ricoh - Samsung - 360fly - JK Imaging (maker of Kodak Pixpro digital cameras) - IC Real Tech (maker of the ALLie camera) - Humaneyes TechnologiesAvailabilityNowMost 360° cameras lack displays and viewfinders. To compensate, camera makers developed apps that you can download to your phone to compose shots and review the resulting images. The cameras connect to the apps wirelessly, and many of them allow you to upload photos and video directly from your phone to Facebook and YouTube. In turn, those sites have made it possible over the past year for people not just to post recorded 360° content but to live-stream 360° videos as well.
Because creating 360° content requires stitching together multiple images, doing it on the fly for live streaming represents an impressive technical achievement. Computer-vision algorithms have simplified the process so that it can  be done on the camera itself, which in turn allows people to live-stream video with minimal delays. (It helps that most consumer-grade cameras have only two lenses and thus one stitch line. Professional versions can have six to 24 lenses.) The ALLie camera supports fast stitching and live-streaming, as do Ricoh’s upcoming Ricoh R development kit camera and Kodak’s Orbit360 4K, which will be available later this year for $500.
 Ricoh Theta SRicoh put the image sensors on the camera’s sides instead of behind its lenses, making its thin shape possible.
Spherical cameras represented 1 percent of worldwide consumer camera shipments in 2016 and are set to reach 4 percent in 2017, according to the research firm Futuresource Consulting. The popularity of these devices will benefit the virtual-reality industry as well as camera makers. You don’t need special VR gear to view spherical videos, but YouTube says many people look at them on smartphones slipped into VR headsets, such as Google’s Cardboard and Daydream devices. And more people experimenting with 360° cameras means more content for other people to watch in VR.
In fact, John Carmack, the chief technology officer of Facebook’s Oculus VR subsidiary, has predicted that people will spend less than 50 percent of their VR time playing games. Instead, they may don VR headsets to do things like virtually attend a wedding.
 Samsung Gear 360Samsung has given these cameras to New York Times and Reuters journalists who are producing 360° news coverage.
Once people discover spherical videos, research suggests, they shift their viewing behavior quickly. The company Humaneyes, which is developing an $800 camera that can produce 3-D spherical images, says people need to watch only about 10 hours of 360° content before they instinctively start trying to interact with all videos. When you see 360° imagery that truly transports you somewhere else, you want it more and more.
Hear more amer camera shipments in 2016 and are set to reach 4 percent in 2017, according to the research firm Futuresource Consulting. The popularity of these devices will benefit the virtual-reality industry as well as camera makers. You don’t need special VR gear to view spherical videos, but YouTube says many people look at them on smartphones slipped into VR headsets, such as Google’s Cardboard and Daydream devices. And more people experimenting with 360° cameras means more content for other people to watch in VR.
In fact, John Carmack, the chief technology officer of Facebook’s Oculus VR subsidiary, has predicted that people will spend less than 50 percent of their VR time playing games. Instead, they may don VR headsets to do things like virtually attend a wedding.
 Samsung Gear 360Samsung has given these cameras to New York Times and Reuters journalists who are producing 360° news coverage.
Once people discover spherical videos, research suggests, they shift their viewing behavior quickly. The company Humaneyes, which is developing an $800 camera that can produce 3-D spherical images, says people need to watch only about 10 hours of 360° content before they instinctively start trying to interact with all videos. When you see 360° imagery that truly transports you somewhere else, you want it more and more.