Key facts
- Only 20% of banks use tools to formally monitor human-in-the-loop controls for generative AI and large language models.
- The majority of financial institutions rely on judgment rather than automated tools to assess control effectiveness.
- This finding comes from Risk Benchmarking's inaugural Model Risk Management study.
A recent study by Risk Benchmarking indicates that a significant majority of banks are not formally evaluating the effectiveness of human-in-the-loop (HITL) controls for generative artificial intelligence (GenAI) and large language models (LLMs). The inaugural Model Risk Management study found that only one in five banks employ tools to monitor these controls, with most institutions relying on subjective judgment instead of quantitative testing.
This approach highlights a potential gap in the robust management of risks associated with the rapidly evolving GenAI technologies being adopted by financial institutions. The study, which benchmarks model risk management practices, suggests that a more standardized and tool-driven approach to control efficacy is needed.