The Logic interviews Niraj Bhargava, CEO, NuEnergy.ai

Niraj Bhargava, CEO and faculty member, NuEnergy.ai, was interviewed by The Logic – Canadian news outlet focused on the “innovation economy,”. Niraj shared his views on the importance of ethical AI and the need for building guardrails. 

Read the article titled ‘To set rules for AI in government, Ontario has to decide what ‘government’ means‘ below:

OTTAWA — The Ontario government is planning to lay out rules this fall for the use of artificial intelligence in government work, which could have sweeping implications for private-sector companies that do business with the province. 

One key question: what “government work” even is, when so many services are contracted out or provided by arm’s-length bodies. The province is promising it will use no AI in secret, and to apply it only in ways the public can trust and for the benefit of all. With transparency and explainability widely seen as essential to building public faith in AI and algorithmic tools, those commitments will force the government to decide whether police forces, medical practices and schools should be covered by new rules, and how much proprietary information vendors of high tech services might have to make public to get public business. 

“Machine learning and AI is not the future, it’s today, and we need to have the guardrails up in advance,” said Niraj Bhargava, the CEO of Ottawa AI consultancy NuEnergy.ai, whose business is helping firms use AI and machine learning algorithms responsibly. “In transportation, you don’t wait for the accidents to happen to realize you need guardrails on the superhighways.” 

The discovery in 2020 that numerous Canadian police forces were using facial-recognition software from Clearview AI was one speeding truck over the side. 

The federal privacy commissioner ultimately reported to Parliament that the U.S.-based company had violated Canadian privacy laws by scraping billions of images of people and compiling profiles on them without their consent. Though Clearview didn’t agree, it chose to leave Canada rather than press its case. 

The commissioner also found that the RCMP had broken the law by using Clearview’s technology to determine the identities of people the police force had photographs of. Some were victims of crimes whom the force was trying to identify, but most weren’t, the investigation concluded. The RCMP didn’t even account for many of its uses of Clearview that appeared in the company’s records. 

The national police force at first told the commissioner it hadn’t used the company’s services; it turned out that Mounties in five divisions had used Clearview hundreds of times between them. The Ontario Provincial Police had used it, too. And the Toronto Police Service, to the exasperation of Ontario’s privacy commissioner. “We have made it clear in the past that my office should be consulted before this type of technology is used,” Brian Beamish scolded. 

At best, the senior officers who might have reported the use of Clearview’s technology just didn’t know their own forces were into it, University of Ottawa law professor Teresa Scassa wrote in a submission to the province as it consulted on the impending rules in the spring. 

“This case raises important questions regarding the scope of the proposed commitment to transparency and AI. The first is whether police services will be included under government AI governance commitments—and if they are not, why not, and what measures will be put in place to govern AI used in the law enforcement context,” wrote Scassa, who specializes in intellectual property, privacy and technology. 

And then what about other government departments, and contractors? “For example, will new rules apply to the decision by a department to use the services of a human-resources firm that makes use of AI in its recruitment processes?” 

As with the Clearview case, this isn’t a hypothetical. 

A directive on artificial intelligence in the federal bureaucracy is limited to decisions made by civil servants that affect the public. That produced a dispute early this year over the Department of National Defence’s hiring of recruiters who use AI tools: the former official who wrote the directive told The Globe and Mail it covers that activity; DND disagreed. 

So does Scassa. “I look at the terms of the directive, and I have a hard time seeing how they could apply to a private-sector hiring company that’s been retained to run a hiring process,” she said in an interview. 

The associate minister responsible for Ontario’s impending rules, Kaleed Rasheed, declined an interview, but a government summary of the consultation results acknowledges that participants “want the rules that apply to the government to also apply to companies when they are providing services to the government.” 

This gets even more complicated if they do, Scassa said. How public must the internal workings of a private service provider’s algorithmic tools be? What about the data that feeds them? Who gets to see that data to validate it? 

“You start to peel the onion, you think you think you know what you’re dealing with, but there’s so many layers to it and it keeps changing,” Scassa said. “It’s even more of a challenge when you’re moving at light speed through the development of a lot of these policies.” 

She pointed to the aborted Sidewalk Labs development in Toronto, where a Google-affiliated company sought to develop a piece of prime waterfront land and use it as a test bed for data driven infrastructure products—traffic controls, energy-efficient building systems and so on. Concerns about privacy bled into questions about data ownership and the rights of marginalized people in public spaces. 

Testing “training data” for biases is essential, Bhargava said: that’s where the best intentions can go haywire. 

“There’s many, many noble pursuits for AI. Whether it’s identifying pedophiles and criminals and terrorists, you want public safety [authorities] to be using advanced technologies, because it’s such a brave new world now,” Bhargava said. “But [you have to] have that understanding that that training data can cause a concern.” 

You can’t use an AI assistant to help with medical treatment decisions for seniors if it’s drawing on data only from youths, he said. Facial-recognition technology, if it has legitimate uses, has to work equally well on people of all races. 

The risk is sharp where provincial services are concerned: those include health, justice, education and child protection, and social welfare. 

There’s a “tendency to to focus attention first on systems for people who were already marginalized and so they become the guinea pigs,” Scassa said. “It can very quickly just multiply and exacerbate hardship for people who already have limited resources to react or to respond or to seek redress.” 

Besides trying to set rules for “trustworthy AI,” the Ontario government has separate work underway on data governance and privacy. Bob Fay of the Centre for International Governance Innovation, who also submitted thoughts and cautions in the spring consultation—he’s CIGI’s managing director of digital economy and a former senior director at the Bank of Canada— said they’re all facets of the same thing. 

“The point to me is transparency,” Fay said in an interview. “That’s something that governments are struggling to deal with, and put rules in place around. But that’s where they should start…. The entire government is doing these seemingly independent consultations. My hope is, at the end, they’re put together as a coherent package.” 

Insisting on transparency is certainly possible—just write it into request-for-proposal documents—though it could make the Ontario government an unattractive customer for some suppliers, Bhargava said. 

“I’m a private-sector business and recognize that there’s times when we may not want to share certain IP and competitive differentiations we have,” he said. “But having that transparency of understanding how our model was trained is really important. We know that people are biased, and machines are biased. The decision on how much bias is acceptable or not could be a policy decision. But from a rules point of view, I think having transparency on these topics is critical.” 

Whatever the rules Ontario sets, Scassa said it’s a pity each level of government is making up its own. 

“As Canadians, we make some things harder on ourselves,” she said. “There are a lot of areas where it would be good to have more interprovincial cooperation or federal-provincial cooperation. It’s hard to go fast if you’re going to do that. But there are risks also in [pretending] that these are issues that are easily confined within provincial boundaries.” 

ABOUT THE AUTHOR