HomeEverythingEducation
Equities & FundsCrypto & Digital AssetsAI & TechnologyBusiness & CorporateUS Politics & PolicyGeopolitics & Global RiskMacro, Rates & FXCommodities & EnergyEuropean Politics & MarketsAsia-PacificReal Estate & Property
Story archiveAll categories
← All Stories

Few banks formally evaluate GenAI human-in-the-loop controls

Created at 10 Jul · 3:40 AM1 source↑ Market-relevant
IN SHORT

A new study reveals that only one in five banks utilize tools to formally monitor the effectiveness of human-in-the-loop controls for generative AI and large language models. The majority of institutions rely on judgment rather than automated testing.

✉Newsletter

PiQ Daily

Pick your topics. Get only what matters, on your cadence.

Key Numbers

20%banks formally monitor GenAI HITL controls

Who's Involved

Risk Benchmarking
conducted the inaugural Model Risk Management study

↳ Why This Matters

The limited formal evaluation of human-in-the-loop controls for generative AI by banks suggests a potential vulnerability in managing the risks associated with these powerful technologies, which could lead to errors, biases, or security breaches.

Key facts

  • Only 20% of banks use tools to formally monitor human-in-the-loop controls for generative AI and large language models.
  • The majority of financial institutions rely on judgment rather than automated tools to assess control effectiveness.
  • This finding comes from Risk Benchmarking's inaugural Model Risk Management study.

A recent study by Risk Benchmarking indicates that a significant majority of banks are not formally evaluating the effectiveness of human-in-the-loop (HITL) controls for generative artificial intelligence (GenAI) and large language models (LLMs). The inaugural Model Risk Management study found that only one in five banks employ tools to monitor these controls, with most institutions relying on subjective judgment instead of quantitative testing.

This approach highlights a potential gap in the robust management of risks associated with the rapidly evolving GenAI technologies being adopted by financial institutions. The study, which benchmarks model risk management practices, suggests that a more standardized and tool-driven approach to control efficacy is needed.

Frequently asked questions

A human-in-the-loop control involves human oversight and intervention in the AI's decision-making or output process to ensure accuracy, safety, and compliance.

Formal evaluation helps ensure that these controls are effective in mitigating risks such as inaccurate outputs, biases, data privacy breaches, and regulatory non-compliance associated with GenAI.

The study highlights a current gap in the industry's approach to managing GenAI risks, suggesting a need for more robust and standardized control monitoring practices among financial institutions.

What Happens Next

01Banks are expected to increase their adoption of formal monitoring tools for GenAI controls.

Get the newsletter.

Pick the topics you actually care about. We'll email when there's news worth your time, on the cadence you choose. Cancel any time from your account.

Cadence

How It Developed

A study found only 20% of banks formally monitor generative AI human-in-the-loop controls.
Most banks rely on judgment rather than tools to test control efficacy.

Sources

T1
Few banks formally evaluate GenAI human-in-the-loop controlsRisk.net

Related Stories

Chinese AI labs pursue custom chips to lower costs but heavy upfront investment a risk
9 Jul · 8:06 AM
Ethereum Foundation Uses AI Agents to Find Network Vulnerabilities
9 Jul · 6:30 PM
UN digital tech agency launches initiative to improve trust in AI agents
9 Jul · 1:38 PM
Ollama raises $65M Series B to help developers run open-weight AI models
9 Jul · 1:20 PM
Former Fed Chair Ben Bernanke Joins Anthropic AI Oversight Board
9 Jul · 5:35 PM