Illustration by Erik Carter

Carnival Cruises, Delta, and 70 Countries Use a Facial Recognition Company You’ve Never Heard Of

NEC has 1,000+ biometrics contracts with agencies around the world

InIn July 2018, the mayor of Irving, Texas, signed a contract that would dramatically expand how the city’s police department could investigate crimes using facial recognition.

The police department agreed to port its mug shot database into the servers of NEC, the 121-year-old technology giant that built the department’s facial recognition program. With Irving’s 100,000 mug shots uploaded to an NEC service called WideNet, officers and detectives were now bought into a system that also included mug shots from nearby communities, like Grand Prairie and DeSoto, and could conduct searches across departments. According to emails obtained by OneZero through a public records request, the Irving Police Department now uses NEC’s facial recognition six to 10 times per week on average, though just 21% of those searches result in “strong leads.”

For the police department of Irving, a midsize city of 235,000 that sits between Dallas and Fort Worth, NEC was a natural partner. The company advertises itself as the vendor of one of the most accurate facial recognition systems in the world and readily touts its law enforcement bonafides. And as chance would have it, NEC America’s headquarters was located just 10 miles down the road from the Irving police station.

NEC and the Irving Police Department have enjoyed a close relationship. When an Irving police sergeant wanted talking points to persuade the department that they needed NEC’s WideNet facial recognition program, he turned to an NEC employee for help describing its accuracy, writing, “I don’t know what a good error rate is.” And when NEC researchers needed data to train a new, secretive algorithm that could detect concealed firearms like AR-15s in schools, the Irving Police Department offered up more than 700 firearms that the department had recently confiscated. (Irving Police was unable to comment about its relationship with NEC prior to the deadline for this article.)

For hundreds of government agencies around the world, NEC has become a gateway to facial recognition technology. In January, London police inked a deal with NEC to deploy a live facial recognition dragnet across the city. Among dozens of other projects, NEC’s facial recognition systems will be used to verify every athlete, organizer, volunteer, and member of the press at the Tokyo 2020 Olympics. The company offers agencies free facial recognition trials, according to over 1,000 pages of documents reviewed by OneZero. Municipalities can load mug shots and personal data into NEC’s software and try it out for a few months.

While much of the current debate around facial recognition centers around American tech giants like Amazon, Microsoft, Google, and, more recently, the smaller firm Clearview AI, facial recognition companies out of the spotlight such as Idemia, MorphoTrust, Gemalto, and NEC, continue to sign massive contracts with law enforcement agencies around the world.

Over the past 10 years, NEC has quietly emerged as perhaps the world’s largest purveyor of the technology.

“NEC is not a household name yet despite their prevalence in this space,” says Clare Garvie, senior associate at the Georgetown Center on Privacy and Technology. “Facial recognition is garnering so much attention now, but the NECs and MorphoTrusts of the world aren’t often associated with this technology. We see a lot of advocacy and reporting focusing on Amazon and other household names, which has meant that large facial recognition companies have largely been able to actually fly under the radar.”

NEC has more than 1,000 biometrics deployments for public safety across the world, including operations in 20 U.S. states.

Though the company keeps a relatively low profile in the United States, NEC is attempting to influence the U.S. government’s facial recognition policy. The company spent more than $600,000 on lobbying related to facial recognition and biometrics bills and appropriations in 2019 alone, according to lobbying records analyzed by OneZero.

Benji Hutchinson, vice president for federal operations at NEC America, told OneZero that NEC has more than 1,000 biometrics deployments for public safety across the world, including operations in 20 U.S. states. NEC operates in more than 70 countries, with hundreds of biometric contracts that encompass facial recognition, fingerprints, iris scans, and voice recognition. Despite its global ambitions, the company’s biometric empire has not been profiled until now, and its expansive facial recognition contracts have never been tallied in one place.

TThe 1970 World Expo in Osaka, Japan, offered a glimpse of the future. Huge domes and geometric towers housed early demonstrations of wireless mobile phones, levitating trains, and a screening of the first IMAX movie. It was the first World Expo in Asia, and more than 64 million people attended.

Those who visited the pavilion sponsored by NEC saw a unique demonstration called computer physiognomy. Over the next few decades, that term would evolve into the field of biometrics.

At the Osaka expo, guests were invited to sit in front of a large video camera to have their faces digitized and analyzed by a computer algorithm. Each of the 800 guests who complied had their faces matched to one of seven celebrities of the era.

Decades before the technology would be deployed in airports and police forensic units — and before the words “machine learning” would cross the lips of a venture capitalist — NEC was previewing facial recognition.

A timeline of NEC’s involvement in biometrics. Source: NEC

“It would have been naive to believe that the program actually produced scientific results, since physiognomic analysis had long been discredited as pseudoscience,” Kelly A. Gates wrote in her book Our Biometric Future. “But the experience of having one’s face analyzed by a computer program foretold a future in which intelligence machines would see the human face and make sense of it in a technologically advanced way.”

At the time, NEC was still called the Nippon Electric Company. Founded in 1899 as a telecommunications company and seeded with funds from U.S. communications firm Western Electric, Nippon Electric made a name for itself manufacturing telecommunications switchboards.

Before World War II, Japan’s economy depended on assembly lines and manufacturing—perfect for the precise telecommunications equipment that NEC built. The country’s manufacturing economy shifted toward consumer electronics such as televisions, computers, and telephones after the war, says Michael A. Cusumano, a distinguished professor at MIT Sloan School of Management who has written extensively on Japanese technology firms.

NEC was well positioned for the explosion of communication technology. It built radio broadcast technology and electronic telephone switchboards. By 1958, NEC had completed its first true, transistor-based computer.

Over the next three decades, NEC pushed its communications expertise into personal computers. By the early 1990s, NEC was the only company in the world to be among the top five producers of computers, semiconductors, and telecommunications equipment, according to Martin Fransman, a professor of economics at the University of Edinburgh who has researched Japan’s computer industry. It dominated 50% of Japan’s personal computer market. NEC was one of the most powerful companies in the world.

But NEC stumbled in the U.S. personal computer market. In the late 1990s, the firm entered into a drawn-out, ill-fated acquisition of Packard Bell, which resulted in nearly $2 billion in losses.

Over the course of the late 1990s and into the 2000s, NEC spun off its businesses by signing them into partnerships with companies like Samsung for computer and TV displays, or creating new companies like NEC Electronics, a standalone semiconductor company. NEC was still enormous, but it was phasing out of being a household electronics name in the United States.

In 1989, the company started research and development of commercial facial recognition by extending pattern-matching software it developed to identify characters like numbers and letters.

In the early 2000s, NEC began to shift its focus toward a division of the company that had, for decades, remained in the wings: biometrics, or the practice of verifying someone’s identity using biological features.

NEC’s first foray into biometrics was fingerprint analysis. As early as 1969, the company had begun working with the Japanese National Police Agency to build a system that would automatically identify fingerprints. In 1982, the National Police Agency installed the Automated Fingerprint Identification System in Japan; in 1983, it installed a version of the program in San Francisco.

Throughout the 1980s, NEC signed contracts everywhere from Washington state to Australia, cementing it as an early leader in biometric technology. In 1989, the company started research and development of commercial facial recognition by extending pattern-matching software it developed to identify characters like numbers and letters. According to a 2018 company presentation, NEC rolled out NeoFace, its first mass-market facial recognition product, in 2002.

Since its start with fingerprints, NEC has been an industry leader in the accuracy of its biometric technologies. In 2004, the National Institute of Standards and Technology tested 34 automated fingerprint matching devices, and called out NEC as a top performer. As recently as September 2019, the National Institute of Standards and Technology (NIST) weighed NEC’s facial recognition algorithms against nearly 200 other programs and indicated that the company was among the most accurate.

“NEC, which had produced broadly the most accurate algorithms in 2010 and 2013, submitted algorithms that are substantially more accurate than their June 2018 versions, and on many measures are now the most accurate,” the 2019 NIST report said.

Though accuracy has been a major selling point for NEC’s facial recognition systems, it’s unclear just how accurate they really are, especially as the field of facial recognition as a whole has proven unreliable outside of the lab. A 2018 analysis of commercial facial recognition systems found that the algorithms were more than 30% less accurate when attempting to identify women of color compared to white men, making systems little more accurate than a coin toss. Privacy advocates say that if the algorithm were to mistake one person whose image was taken at a crime scene for another, that innocent person could be unfairly implicated or investigated for committing a crime, putting them under undue police scrutiny.

“We have a technology that was created and designed by one demographic, that is only mostly effective on that one demographic, and they’re trying to sell it and impose it on the entirety of the country,” New York congresswoman Alexandria Ocasio-Cortez said in a 2019 congressional hearing on facial recognition and law enforcement.

A 2019 slide showing NEC’s biometric contracts in the United States. Source: NEC

NIST’s tests have come under some scrutiny as well. “You can think of NIST as being the easiest possible case to pass,” says Deborah Raji, an A.I. researcher who works with the AI Now Institute and the Algorithmic Justice League on auditing facial recognition systems. “The images are well lit and in a constrained environment with a consistent background. If the model does not do well on this test, that’s significant, but doing well on NIST does not at all mean the model can handle real-world conditions.”

There are few ways to independently audit NEC’s technology outside of NIST testing. In trials of the NEC technology in London, one of the only independent analyses of NEC’s algorithm found that 81% of 42 people flagged by the facial recognition algorithm were not actually on a watchlist.

To Raji, who worked on the audits that inspired congressional hearings on racial disparity in facial recognition, the results of these real-world trials are crucial.

“The only way to get a truly meaningful reading of a model’s accuracy in deployment is to pilot the technology and measure those metrics in the real world. And even after doing so, it takes so much more than accuracy to understand a model’s performance and likelihood for success in deployment,” she says.

CConcerns about the accuracy of NEC’s technologies, and that of facial recognition as a whole, have not impeded the company’s sales pitch around the world.

Today, in Surat, India, live video feeds from more than 500 cameras play across a 280-square-foot wall of televisions inside the police’s digital command center.

As the footage comes in, NEC’s facial recognition algorithms find the faces scattered in each frame of video. Those faces are matched against a database of 30,000 mug shots managed by the city. When the program recognizes a match, it alerts officers in real time.

Media reports suggest that Surat’s real-time facial recognition system was used to convict 150 people in 2015, the first year of operation. The city police credit the system with a 27% reduction in crime in the city of 5.5 million people.

“We would like to add a few more cameras, maybe 1,000 or 1,500 in the next phase,” said K.L.N. Rao, joint commissioner of police in Surat, in a promotional NEC video.

An NEC presentation slide detailing its work in Surat. Source: NEC

NEC sells its facial recognition technology under the brand NeoFace, which has a number of sub-brands: Express, for use in large security lines; Reveal, for low-quality images and police use; Smart ID, for using mobile devices to verify identity; and Watch, for live facial recognition from video. NeoFace Watch can even be used to identify individuals through car windshields, according to the company.

Hutchinson, of NEC America, stresses that the company does not sell live facial recognition — the type employed in Surat — to law enforcement or federal government agencies in the United States. But the company does sell the service to private companies in the United States for security and loss prevention, and it offers such products in countries “where the privacy laws are different.”

Within the United States, NEC sells its standard, non-live facial recognition to governments, universities, and companies in California, Arizona, Texas, Michigan, Maryland, South Carolina, Georgia, Pennsylvania, New York, Alaska, Wyoming, Montana, Idaho, Nevada, Oregon, Washington, Mississippi, Alabama, and Illinois, according to a February 2019 presentation from NEC’s Mitsuhiro Murooka, who is now the CEO of NEC Australia. Eight of those states adopted NEC’s facial recognition in just the past four years.

NEC pitches itself as a proven, trusted provider of everything law enforcement may need to get started in biometrics, according to more than a half-dozen NEC presentations given by NEC executives and surfaced by OneZero. The company boasts that more than 50% of fingerprint searches in the United States are done on NEC systems. It manages the Los Angeles County Sheriff’s Department’s 10 million identities, with support for facial recognition, iris scans, and fingerprint identification. The DHS said the department has scanned nearly 44 million people with facial recognition technology at border crossings. That technology is supplied to DHS by NEC.

Some of these contracts date back almost a decade: In a 2013 facial recognition software training guide sent to Fort Wayne, Indiana, NEC used an image of a boy passing through a turnstile in the New York subway system. New York’s police department had a five-year contract with NEC, as reported by BuzzFeed News.

Images from a 2013 training brochure for NeoFace given to Fort Wayne, Indiana police. Source: NEC: Fort Wayne Police Department

The scope of NEC’s contracts around the world is incredibly vast. NEC’s facial recognition is used by the federal police in Australia and in the boarding process of some Disney and Carnival cruises, as well as on some international Delta flights. Star Alliance, the world’s largest airline conglomerate, is developing a facial recognition boarding system with NEC for its 26 airlines. NEC manages India’s national ID database, with more than a billion identities. Lawson, a Japanese convenience store with more locations in Asia than 7-Eleven has in the United States, is trialing cashierless stores powered by NEC’s facial recognition. (7-Eleven Japan is also using NEC to trial its own cashierless stores.) The technology is also used in a California burger chain to pull up your loyalty account. By March, passengers on 90% of international flights into Japan will have their identities verified by the technology.

NEC’s facial recognition software sales don’t end with the company’s own software. Other companies also license and resell NEC’s technology. Emails obtained by OneZero show that DataWorks Plus, which specializes in law enforcement software, distributes NeoFace as one of the three facial recognition algorithms that it offers to customers. DataWorks Plus upgraded California’s San Bernardino and Riverside counties’ 3.2 million mug shots to be compatible with the NEC NeoFace system. (NEC’s Hutchinson says the company is moving away from the licensing model to provide services directly.)

The company spent $672,500 lobbying specifically for facial recognition and biometrics policy and spending in 2019 alone.

And NEC isn’t just in the software business. The company is building ATMs in Japan that use facial recognition, as well as security gates that recognize a person’s face and gait to verify their identity.

To influence policy in the United States, NEC has drastically increased its federal lobbying. The company spent $672,500 lobbying specifically for facial recognition and biometrics policy and spending in 2019 alone, according to records from the Center for Responsive Politics analyzed by OneZero.

Of that total, NEC spent $305,000 lobbying the Commercial Facial Recognition Privacy Act after it was introduced with bipartisan support in March 2019. Hutchinson says the lobbying effort was to bring “balance” to the legislation.

“The goal was to make sure that the debate that started to unfold was balanced,” he said. “From day one, we have always pushed for if there’s going to be legislation, let’s make sure it’s balanced legislation that deals equally with the highest levels of privacy but also doesn’t constrain the private sector from innovation.”

The bill was also supported by Microsoft. Privacy advocates who are pushing for a full facial recognition ban said the bill as announced fell short of what was needed to protect the public.

NEC spent an additional $250,000 lobbying Congress on Department of Defense and Department of Homeland Security facial recognition, according to federal lobbying forms, and $117,500 was spent lobbying for biometrics and border security programs for the DHS.

These figures amount to a sizable increase from 2018, when the company spent $100,000 lobbying Congress on facial recognition issues. Hutchinson attributes this to NEC America’s efforts to increase its presence in Washington, D.C.

Behind closed doors, NEC is pushing back against the debate about the ethics of facial recognition technology. In a slide from an October 2019 presentation, Masahiro Ikeno, CEO of NEC America, said “misinformation” about privacy and facial recognition was a barrier to more widespread adoption of the technology, citing work by the American Civil Liberties Union, Electronic Frontiers Foundation, and Georgetown Center on Privacy and Technology as an impediment.

NEC America CEO Masahiro Ikeno presents a slide to the NYC Japan Society calling out the ACLU and others as misinformation. Source: Japan Society NYC

“City bans are not the answer,” his slide read (emphasis NEC’s), alluding to the seven U.S. cities that have passed bans forbidding local police and governments from using facial recognition. However, the same slide welcomes some privacy regulation. Hutchinson confirms NEC’s stance against bans, which he says stifle innovation.

For NEC, this work is just the beginning: Modern facial recognition relies on A.I. breakthroughs in “deep neural networks” less than 10 years old. As the technology quickly becomes pervasive, companies that show early dominance are more attractive to governments that acquire technology on a slower time frame.

NNEC’s quiet influence, from America’s small-town police forces, to the halls of Congress, to cities around the world, suggests that the company has benefited from its relative obscurity by side-stepping public scrutiny.

“This technology is biased and error-prone and gives government the dangerous power to track our faces everywhere we go,” said Neema Singh Guliani, senior legislative counsel at ACLU, in a statement to OneZero. And this is why transparency about how the government is adopting this technology is crucial.

“I worry that the focus on Amazon and others allows these major companies like NEC and Idemia to fly under the radar.”

Facial recognition is seen as a fast, noninvasive method of security that allows for an unprecedented amount of control over public spaces. It is quickly becoming the standard technology for identification, from the nearly 44 million people scanned by DHS, to anyone who has passed in front of a CCTV camera in Surat, India.

Global demand for the technology, from every level of government, has meant that there are dozens of companies now working in this space. Silicon Valley tech giants like Amazon and Microsoft make for easy targets for criticism because they boast artificial intelligence chops and compete for some of the Department of Defense’s largest military contracts. But these corporations represent just a fraction of the biometrics and surveillance industry.

“I worry that the focus on Amazon and others allows these major companies like NEC and Idemia to fly under the radar, even though they’re the primary providers of this technology,” says Garvie, from the Georgetown Center on Privacy and Technology.

Because the technology that matches your face against a local watchlist, or an FBI database, is more likely to be built by a company like NEC than a tech giant from Silicon Valley.

Senior Writer at OneZero covering surveillance, facial recognition, DIY tech, and artificial intelligence. Previously: Qz, PopSci, and NYTimes.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store