‍The Challenges of Pervasive AI
Generative AI has enabled magical new capabilities and captured the imagination of the technology world and beyond.  Since late 2022, companies have spent billions of dollars exploring how these capabilities can transform their businesses.  Tens of thousands of enterprise apps have been built, many of which are quite amazing and inspiring

Complex Technology Issues
With this potential opportunity, a core problem has emerged – how does an organization or individual ensure that the AI app they have built is compliant with existing laws and regulations?  For example, if an enterprise software company has built an AI-based productivity app, how can the company ensure that the app does not violate GDPR in various locations – a requirement for world-wide deployment? Unlike “traditional” IT applications (bugs notwithstanding), it is not yet possible to assert that the AI app’s behavior is compliant. This is a direct result of the app being built on a non-deterministic infrastructure. The “QA” process of the AI app cannot be the same as the QA process for a traditionally coded app. The current testing process for AI apps depends primarily on human interaction, which makes it expensive, slow, non-repeatable, and subjective. 

The Current Approach: Human-Capital Heavy
Other key capabilities include: the ability to swap out rule sets without requiring retraining; an audit function that provides an audit trail for each regulator decision; and the ability to work with any LLM.Verific.ai has built a prototype using Llama 3 that demonstrates viability of the regulator engine. Development of the adjudicator engine is currently underway.
Time-Intensive: Manual processes take a significant amount of time, slowing down AI deployment and business operations.
Difficult to Automate: Human-powered systems are hard to scale, automate, and reproduce consistently.
Inconsistent and Incomplete: Relying on human intervention leads to gaps in compliance coverage and inconsistent results.
Reinforcement Learning from Humans (RLHF): Even in cases where AI technologies are employed, they depend on reinforcement learning from humans, further emphasizing the reliance on human oversight.

The Future of AI Governance
Verific.ai has developed new AI-based technology to address this challenge.  Verific has four main components: a pair of engines (adjudicator and regulator) to evaluate AI apps; a reasoning engine that ingests curated rules/regulations; a relevance engine to pick applicable rules; and a contextual adapter to apply a vertical set of rules to a use case.  The adjudicator engine generates a large collection of tests and prompts; evaluates the AI app against these tests; and collects, collates and analyzes the results.  This is analogous to today’s traditional test automation, which, as noted, is insufficient for AI apps given their non-deterministic underlying platform.  The regulator engine operates in real-time in-line with the app, evaluating each app output to ensure compliance against the rule set.  Together, the adjudicator and regulator engines allow the development of compliant apps while also providing guard rails against non-compliance.