Means of permitting or barring access to a particular location, these errors can have far reaching consequences for people trying to go about their daily lives.

For example, a black teenager in Detroit was barred from entering a skating rink after facial recognition, misidentified her as a different patron.

Similarly, in the UK, a woman was misidentified as a shoplifter, subsequently back searched, asked to leave the store, and banned from all other stores using the same technology until the company acknowledges mistake.

These biases in facial recognition risk excluding marginalized communities from everyday life.

More and more stores are deploying biometric surveillance to monitor customers, screen against watch lists, and share information with other businesses, security firms, and law enforcement without people's knowledge and often resulting in misidentifications, leading to false accusations, invasive searches, or access restrictions.

The Federal Trade Commission has recognized the dangers of facial recognition technology.

In December 2023, the FTC banned banned the grocery chain Ride from using facial recognition surveillance for five years.

Ride aid's facial recognition technology misidentified thousands of customers, disproportionately people of color and women as previous shoplifters.

Suspected individuals were then followed around stores and searched, ordered to leave, or publicly accused of shoplifting.

Facial recognition technology is also ripe for abuse.

Medicine Square garden owner James Dolan has used facial recognition as a retaliatory tool to bar those against whom he has a personal vendetta from access to his facilities.

In 2022, Dolan ejected a mother trying to watch the Roquettes show with her daughter's Girl Scouts group simply because she was identified as an employee of a law firm engaged in litigation against MSG, even though she never directly worked on a lawsuit.

He later ejected someone for creating a t-shirt that made fun of him.

This sort of retaliation sets a chilling precedent for how the rich and powerful can silence their opponents, critics, and those helping them fight for their day in court.

The risks are even higher for immigrant New Yorkers.

As the Department of Homeland Security continues to rely on biometric surveillance technologies to target immigrant communities, immigrants live in fear as they go about their daily lives.

Once biometric recognition systems collect data, companies and third party providers amass huge databases of highly personal information that could be exploited by ice.

All these examples point to Orwellian use cases where it will be impossible to move, purchase goods and associate freely.

The need for stronger privacy protections has never been clearer.

People should be should not be required to have their faces scanned and biometric data collected simply to buy groceries and other necessities of daily life.

The NYCLU thanks the committee for the opportunity to provide testimony today.

Nobody wants to live in a world where pervasive surveillance identifies them, tracks their movements and associations, and impacts which places they can visit, which services they can access, with whom they meet, or how they exercise their free speech rights.

Thank you very much.

Thank you.

And Daniel, I think.

In your intro, you mentioned targeting and discrimination against individuals.

And I think that those are the primary concerns of myself and Councillor Monto, Councillor Majoke, who are co-sponsors on this piece of legislation.

Again, can you before your testimony, I talked about the public good.

And can you give your opinion and perspective on how a ban of this nature would be for the public good?

Absolutely.

I think it it touches on different aspects.

So these technologies have much higher error rates for black and brown people, for women, for young people, and for older people.

People are much more likely to trust the output by a computer.

And when this discrimination runs wild and is amplified by a system at scale, that's when we see those discriminatory impacts, like in the case with Riotate, where thousands of people were misidentified.

The system is deployed at scale, and then discrimination is running wild.

When it comes to places of public accommodation, people are expecting to go into a store and being able to buy their groceries without being identified, tracked, or accused of a crime, mismatched to a shoplifter from another area, and then being locked out without the ability to appeal, to correct the record oftentimes, to have dangerous interactions.

And depending on access in their neighborhoods, their ability to move to different stores, that could be a lifeline.

If if you're unable to go anymore to the store that is in your vicinity in your neighborhood, and it creates additional barriers for you, and that that could be it could have dramatic ramifications for that person depending on where they live, their social and economic class, their ability to move somewhere else for purchasing those goods, etc.

Thank you.

And Daniel, another one of my primary concerns is around data privacy and data storage.

What do we know about data protection in the private sector?

I think it's really hard to give a comprehensive overview of that landscape because it is the Wild West.

We're lacking comprehensive privacy protections at the state level and even more so at the federal level.

We do have some limited cybersecurity provisions that were enacted under the Cybersecurity Shield Law around 2018-2019 that require um businesses to safeguard sensitive data, including biometric data, but it just doesn't go far enough.

And it does not protect data in the way that many other states do, whether that is Illinois, Washington, Texas, they all have long ago enacted strong biometric privacy protections that also cover the collection and the use, the deletion of biometric data.

That is all not present here.

So we we really leave businesses to their own devices what to do with the data and really take it from people without their knowledge or consent.

And that is a massive issue that we're seeing right now.

I think it comes to the fore now also with uh wearables, where we see meta glasses and the ability to um surreptitiously collect video footage that could be again aggregated and compared against um other forms of um biometric recognition databases, um, identify people without their their knowledge, and um to come back to the cybersecurity risk, we see breaches left and right.

Umce your biometric data is leaked or accessed by hackers, you can't take it back.

It is not like your credit card number that you can't can just change once it has been accessed by someone else, but rather your your biometrics are yours and they're immutable.

Um so uh it really runs the risks um that that could harm you for a lifetime.

Great, thank you.

And Daniel, I've got more, but I'll turn it over to my colleagues if they have any questions at this point.

I want to touch basis on what Daniel said about Ride Aid.

So when this data is collected, so does this how far and how wide does this data go?

It really depends.

Um, so we don't know exactly what kind of biometric surveillance system Ride 8 was utilizing in this in their stores, whether that was um a third-party provider, whether they worked specifically on their own in-house system.

But there's many surveillance vendors that um make it very cheaply available for any small store or for your home security, let's say, um, and it is being entered into wide data sharing networks.

So uh beyond the entity that is deploying the system in their four corners of their store, it gets hosted externally on the vendor's cloud platform.

And oftentimes that is then being shared to other customers, where the pitch here is uh look, we we collect already this this biometric data from your store, but we can make it even more valuable to you by connecting it with all the other stores that we partner with across the country, and we we could identify other known, let's say shoplifters, and that is also where a lot of that harm is coming in because the the larger the database, the higher the risk again for misidentification, um for comparing someone to who they have absolutely no resemblance and misidentifying them as that person and the FTC case in the settlement and also in the subsequent uh press releases uh provided a number of examples.

And if I if I may, I can I actually have um former Commissioner Alvaro Bedoya's statement pulled up, and I can just read two or three of those examples one second.

Just having to switch.

Yeah, so from his um statement in response of uh to the settlement.

In its complaint, the commission alleges a right-aid employee stopped and searched an 11-year-old girl because of a false match.

The girl's mother reported that she missed work because her daughter was so distraught about the incident.

Another example.

Right aid employees called the police on a customer because the technology generated an alert against an image that was later described as depicting a white lady with blonde hair.

The customer was black.

Many c many other customers, people shopping for food, medicine, and other basics were wrongly searched, accused, and expelled from stores.

Sometimes they were humiliated in front of their bosses, co-workers, or families.

And one more question for you.

So if I'm misidentified, how is that corrected?

How do you correct misidentification?

I think that's the big problem.

It really depends on a case-by-case basis how the store has policies in place, oftentimes not, how the training occurs for for the local security firm or the attendance in the store.

I think the spectrum runs wide, and what was found in in Ride 8 incident, that the training was insufficient and there was no clear process, and staff wasn't appropriately trained and was really left to their own judgment what to do, and that resulted in a lot of harm.

Um and problems here.

But I think we uh I think what what is also clear from that statement from Alvaro Bedoya, former commissioner, is that that was within the toolbox of the FTC.

But I think they were very clear that they very much uh also agree that this is not a technology that should be allowed to just run wild, and while they used the tools within their toolbox, they are fully uh calling for much stricter regulations and not get distracted with the minutiae of how these systems are being implemented and what perhaps training uh regulations could look like, but rather uh ensuring that um these technologies um create so much harm and danger that um I think we as a society should also make a clear line and draw a boundary of what what we want in in our neighborhoods and and say no to to certain tools because of the danger they represent.

So, Daniel, full disclosure before I ask this question.

I I despise this technology and I don't think it has any place anywhere.

I think it's garbage.

So when I ask this, what I want to know is on the other side of the argument, when people argue in favor of it, what are you hearing?

Is it just theft prevention coming out of these retail establishments, or is there something more to it?

Are they saying that there's a long-term benefit to having the data I think you you you hear the argument for for safety and security?

And I think we can all agree that we we understand that uh wanting to create as much security and safety in the stores as possible, but we also have to understand that uh surveillance is not safety, and uh especially with those tools that have shown uh biases and are flawed and result in in these outcomes that harm some communities more than others and result in dangerous interactions, that is not uh uh an efficient uh and functioning piece of technology for those purposes.

There are other ways how you can mitigate theft.

There's other ways um how you can create safe environments, and I think what we're seeing here is very much the opposite that this technology is is creating the opposite of safety.

It creates new dangers, new harms, and uh violates people's rights.

And then I think the other aspect to that uh is that there's other use cases for it.

So we have heard about stores that use biometric surveillance to track customers very much uh to understand and leverage intelligence on people, whether that is to push up prices, whether it is to target employees and watch how employees are walking and moving in stores, um, productivity ratings, evaluation of employees.

Um I think the algorithmic pricing component has a lot of um attention right now in Albany.

Just yesterday, uh multiple bills were passed to prevent algorithmic pricing, um, where live real-time tracking and information feeds into how much a store can ask for a particular good to one customer but not the other.

So I think it's it's very much a hot topic right now to prevent these unfair practices.

So, do you track or keep track of the error?

So, in line with what Counselor Hudson was saying, the errors that happen, do we have a percentage of how many of those errors are non-white individuals when it actually does happen?

And explain to us a little bit about um how the technology was built out to actually using a Caucasian mail to build that technology.

Yeah, so there's numerous studies from academia, from independent researchers, uh from the government that showcases time and can time again that the systems are less accurate for darker skin tones and for women, for young people and for older people.

Um I think the argument is always that the systems are getting better, and we have we've been hearing this for the last 15 years.

Um the problem is that even if you have um correct identifications for over 99% of uh comparisons you're making, that number is still too high, especially if you run real-time facial recognition, like in those stores, because you you have thousands, tens of thousands of people coming through, and so um, even if it is in the the lowest percentile and you have misidentification there, it will result in harm.

And if that is disproportionately falling on marginalized communities, um that is that is what we're seeing.

And um this the databases that were that are used for training, they're also oftentimes used um from very advantageous angles, like portraits, photographs, or other photographs where the camera is in a in a good angle to depict all the facial features, the geometries of your face, good lighting, um, basically lab um environments.

Similarly, the photos that they're comparing against.

Same scenario there.

In stores, those are uncontrolled environments.

The camera might be at a bad angle, the position, uh, the person might be wearing appliances, glasses, hats that might also further obscure the resolution might be lower, lighting might be bad.

And so, in those real world scenarios, the error rates are far, far higher, and that is also where those uh massive um um discrepancies are are are coming to play.

I have one more question for you.

Is the surveillance primarily put in minority and immigrant communities where they shop?

Yes.

Yes, so that that was the finding of the FTC Ride aid case.

Um in most other instances, we just don't know.

Um as we heard in the opening, we we do have in New York City a biometric disclosure law, but it it just doesn't go far enough.

It just uh requires stores to put a sign up at the entrance um that announces that biometric surveillance is taking place.

That is how we learned about the biometric surveillance by Wagmans earlier this year.

I think that was reported widely.

Um, but it it doesn't give people any power to decide whether the store is collecting their biometric data and comparing them to um who knows what databases.

Um and so we we don't really know which stores deployed outside of localities that that have passed these laws, and also there's no teeth.

So even in New York, we know that the law is being um ignored because there's no enforcement, there's no oversight uh and um I think um it is impossible to know which stores are running facial recognition because you can't tell.

It might just be a regular CC TV camera.

You have no idea if that data is just being collected on the old school system in a room behind the camera, if it is being shared out to a data center in another state, if it is being analyzed, if it is being shared, if it is being compared to to some watch lists, it is uh it is just impossible to know for everyday customers.

So, Daniel, so the I'm gonna go down a bigger rabbit hole here, so we might not have time to do this.

But so the the biggest companies that are using this technology, are we finding that they have contracts with the federal government?

Because I think about traveling and in the last couple years traveling both domestically and internationally.

Now you go to the airport or you get on a cruise ship, you show your passport at the beginning of the morning and then never again for the rest of the week because they're capturing your face.

So are we finding that they're integrating the data with the you know TSA or ICE where they're they have the data from these grocery stores?

Or we don't know because it's the wild west.

Yeah, I'm I'm not aware of any agreements or contracts to that end.

Um I think any data that is uh large enough data set is an attractive target, also for let's say immigration enforcement to tap into.

Once you amass enough data, we know they come for it.

And even if there's no existing data sharing agreement, uh affirmative sharing happening, um ice can just get it with a warrant, and there's nothing a company can do it against that.

So the the best uh practice is just not collected in the first place or delete it as soon as uh feasible for whatever purpose it was collected for.

So data minimization is the best practice here.

Um I have a question, and it it bothers me.

I worked for Ride Aid for 10 years, including its predecessors.

Right aid is now out of business.

So, what do they do with all the personal data that they might have on me and all the many thousands of people who work in pharmacy retail?

Have they sold my personal data to other companies?

Do they what they know about our my personal background, medications I've taken, any notations on me, which I have never seen.

So could you know speak to that and have those who have worked for right here or other pharmaceutical companies uh will have the same questions?

Unfortunately, I can't answer that.

I haven't seen the direct data and evidence that was used by the FTC.

So I'm I'm I'm not familiar enough to talk about the specific data collection practices, data sharing practices by Ride IT, unfortunately.

Okay.

Daniel, thank you for joining us and answering our question.

I guess along along uh what councillor Nave just asked, are you seeing a lot of these surveillances in the retail, mostly in retail areas, or are they mostly in other areas such as car dealers as well?

I think we just don't know enough of how widespread the technology is.

We know it is getting cheaper, and a lot of the vendors that um sell security cameras are just adopting it very quickly with just a mere software update.

So even if you may have purchased for your store a camera that didn't have that capability at the outset, um, oftentimes a new software update on the back end where you watch the footage, whether that's on your phone or your computer, online, uh, may in the future incorporate that capability.

And I think because of uh AI is getting cheaper, those those features are what sets a vendor apart from another vendor.

And so we we we we see this being adopted very very widely.

Um and then I think it is just some like speaking of the areas that are requiring um disclosures.

We know that beyond RideAid, um also um Wagmans is utilizing it, Macy's is using it, Whole Foods is using it, and there may there may be a long list of of other stores uh that have been using it.

Obviously, MSG and all the entertainment venues have been very aggressively using it.

There was just recent reporting by Wired magazine that went very much in depth about the alleged um tracking that occurred there on a trans woman, uh allegedly tracking her by the minutes when she went to the bathroom and how she moved around in the in the sports venue.

And so I think it is just an ongoing uh area where we keep learning more, and uh oftentimes we're we're just behind the deployment because uh, like with the the example I just gave, we the information here came from a court case.

So years after the fact of when that particular surveillance uh took place, and uh so we we're playing catch up, and that is why also I'm so encouraged by this piece of legislation, because it is technology agnostic.

It doesn't limit in on like a one particular technology, but rather it is future-proof by covering biometric surveillance broadly, because we know while it is facial recognition today, it might be if we think back uh a few years ago, former governor Cuomo experimented with ear recognition on our uh highways, bridges and tunnels.

Uh there might be just eye or iris detection based on uh the advancements in camera lens technology.

There might be other forms of uh biometric recognition that that will be more prevalent in the future, whether that's gate recognition or some sort of behavior recognition.

And so um being as tech agnostic as possible is I think the way to go and uh is the right approach.

Now, thank you for that.

Uh other question I have is how do we how do we fight the the perception that with this effort that it is a move against doing pre-business, especially in the business world?

How do we how do we change that perception?

I I think the perception is clear with uh that this is not against business.

I think customers don't want to be surveilled when they buy their goods.

Like I think that backlash that was uh happening in January after the the news that Wagmans was utilizing biometric recognition in their stores, facial recognition and customers rightfully were outraged because I don't want to be surveilled and monitor when I go buy my carrots in the store uh there's there's no need for that and while I think we're all amicable and understand the desire to have uh secure stores and be safe in the stores I don't think that is the the right approach for that and so that there's there's other proven ways to mitigate crime to mitigate uh or ensure the these stores are safe but it's uh likewise like we wouldn't want to show our IDs when we when we go grocery shopping and uh have then all the data's uh data locked and and shown to us and that's that's essentially what is what is happening here um it is just happening invisible to us and last last question uh for me is um what does the enforcement look like is is it is it's does the enforcement come from audit or does it come from self-reporting how do we make sure that enforcement is happening to the perpetrators at least yeah I think that's a that's an important question and um I uh I'm pleased to see a private right of action included in um the legislation um I think a good um comparison study is when we look at the biometric identifier law in Illinois and Texas both states passed strong biometric privacy protections almost at the same time um so it's been in effect over I think 15 years now in Illinois there's a private right of action where people can have their day in court in Texas there isn't and so what we're seeing over the last one and a half decades companies have time and time again violated the biometric privacy of residents in both states but there was no action in Texas and people couldn't do anything about it.

There was a enforcement provision for the state AG in Texas to bring actions but we all know the AG's offices are strapped for resources they're limited in their capacities um on what they focus on and so uh they never brought any actions against the companies whereas in Illinois people when they found out about violations of the law could go to court and um and fight for their rights there and um against the violations and so it is uh been a much more active state for uh upholding the biometric privacy protections and it led then to through some of the wins that occurred in Illinois that then the state AG in in Texas used the legal theory developed in Illinois to go after I believe it was Meta a few years ago um and uh achieved a massive settlement there um but the overall number of violations have never been curtailed there and so illinois I think is uh the the golden standard uh where people can can have the day in court and hold companies accountable and I think it's it's important here to to highlight it uh although we do hear a lot of um pushback against private right of actions like these it is so important because we see that all these tech companies are violating privacy protections all the time and if they were to follow the law they wouldn't have to be worried about um cases being being brought against them or there wouldn't be any cases brought against them so I think the prior right of action is is key here.

Counselor that's section 2-69 in your packet yeah and and la I said last one but you um so in terms of people at least that the demographic that is affected what do you do you know the numbers around it as to what is the demographic makeup of people that are being surveillance I missed the first part of your question.

What what I got was whether I know the number of people that are being surveilled by this or did I basically maybe like like race gender or stuff like that.

What's what's the makeup of that?

With the in the case of Ride, uh the FTC found that the facial recognition systems were uh deployed in much the deployment was in uh primarily black and brown neighborhoods uh and poorer neighborhoods.

Um by other stores, we just don't know where it is being installed, where it is being used.

Um I think um a very shocking number is when we look at the misidentifications of facial recognition by law enforcement.

I think most people will never know whether facial recognition was being used in um when they're being accused, but there are around a dozen nationwide cases, and the vast majority, I think 11 of the 12 uh people that have been misidentified there are um black men.

Um it might be I I think it is 11 out of the 12.

Um, so I think that tells you everything.

I am one more question on a personal level.

I've had medical procedures done in recent years.

Immediately I get AIDS from law firms dealing with medical malpractice.

How do they know when I had this procedure, this procedure?

Why am I getting ads there?

And they knew what surgery I had, and and they're saying, here, call us.

Maybe your doctor made a mistake, maybe your surgeon did it wrong.

Call us for a consultation about a possible lawsuit.

How do they know my personal business?

They get they would send me an A on my phone.

That is wild.

Yeah, so I I think that there's there's the the entire landscape of data brokers is just a wild west, and we're lacking the consumer privacy laws in in general about data collection.

And with all the data that is out there from your location information to your browsing information, what websites you're visiting, where you spend your time online, uh, companies can make inferences.

And so maybe you have Googled um keywords pertaining to the procedure, uh, and that was then utilized by marketers uh to target you.

Um there's a related bill on that.

It's the New York Health Information Privacy Act.

It did pass Albany last year, but unfortunately was vetoed by the governor.

Um it is again um we we we are strongly supporting it.

It hopefully we'll pass again this session, and uh I I can send you materials with regards to that legislation.

Right, and we don't publicize our personal business, but somehow they know.

And I have another question for you.

You talked about Texas, and you talked about some of the actions they took as far as lawsuits, I guess.

So would those turn out to be class action lawsuits?

And would the people that have been misidentified be compensated?

I'm not familiar with the details of the state AGs uh actions.

As far as I know, they were not class actions.

Um I don't know if individuals were being compensated.

It is definitely the case in Illinois where the private right of action exists, and and people go to court all the time all the time and are being compensated for violations of the law.

So in Illinois, absolutely.

So, Daniel, as I'm looking at the time here, um, and kind of want to wrap it up.

As the council weighs their vote on Monday, and the discourse is occurring within the community.

What do you think are the most important and salient points for our city to know about both the legislation and the technology?

I think we're with we're seeing awakening up by communities across the country, across the state, across Syracuse, becoming more aware of how surveillance technologies are impacting them on an everyday basis.

Biometric surveillance really stands out here as a particularly dangerous technology because it is deployed on them without their knowledge.

Um it is dangerous when it works, and it is even more dangerous when it doesn't work.

We have the data that showcases how inaccurate the technology is, how much higher the error rates are for black and brown people, for women, for young people, for older people.

That has been confirmed by studies across the board from academia to the government.

We have the FTC actions against the technology.

And I'm very encouraged by by the council here that you all focus on this issue and that you're moving this forward.

So thank you very much.

Thank you.

Anything to finish up with Tanya?

So, Daniel, what this um technology does, it doesn't target white men, right?

It it it it it what did I hear you correctly?

It doesn't target white men, you asked?

Yes.

It it it identifies it it it uh it worked, but it it targets everyone.

It just functions more accurately on white men.

So uh especially middle-aged white men like myself will probably have the the highest uh match rate or confidence rate uh because of uh how the accuracy performs of the vast majority of those systems.

That doesn't mean uh I shouldn't be worried.

I don't I don't want uh to be tracked in the store when I go for for my shopping.

Uh and uh and I think uh people should still be uh also uh uh have worries there about being misidentified.

I think we saw that with the example in the UK of a of a white woman being misidentified as a shoplifter as well.

So I think the the concerns should be across the board.

The concerns are much more heightened and the discrimination is much stronger when it comes to identifying or targeting black or brown people.

Thank you.

Right.

Well, Daniel, we very much appreciate you joining us today.

Thank you for your time.

Thank you very much.

All right, and with that, can I get a motion to adjourn?

Right.

Thank you very much.