Incorporating and Implementing AI-Relevant Controls into SOC Reporting
Key Takeaways:
- Identify AI-Specific Risks: Address risks related to data governance, model development, bias, explainability, and performance monitoring.
- Establish Strong AI Controls: Implement controls for data accuracy, model validation, bias detection, access management, and ongoing monitoring.
- Align Controls with SOC 2 Criteria: Map AI-specific controls to security, availability, processing integrity, confidentiality, and privacy.
- Build Trust and Credibility: Proactively incorporate AI controls into SOC reporting to strengthen the control environment and stakeholder confidence.
As artificial intelligence (AI) becomes more prevalent within organizational operations, the traditional or standard scope of SOC reporting is expanding. AI models introduce unique risks that are not traditionally covered in ordinary IT control frameworks. To gain and keep trust from stakeholders as AI integration continues, organizations have to transform their SOC reporting to cover AI-related risks.
SOC reporting, specifically SOC 2 reporting, provides service organizations with an evaluation of the suitability of their controls related to Trust Services Criteria, which includes security, availability, processing integrity, confidentiality, and privacy. AI systems touch on each of the five criteria, and as a result, these systems create different risks.
AI: The Key Risk Areas
Prior to developing and implementing AI-relevant controls, it is important for organizations to identify the main areas of risk associated with AI as it relates to their environments. A few of the key risk areas and some example control objectives are identified below.
Data Governance
AI models are only as reliable as the data they are trained on. Ensuring there are strong governance standards surrounding the data used to train AI models allows for more reliable outputs.
Control Objectives:
- The data used to train AI models is complete, accurate, and validated.
- The datasets used to train AI models are maintained as records.
- Data privacy and consent requirements are enforced for all data used to train AI models.
Development and Validation of AI Models
The process for developing and building models must be managed, standardized, and reproducible.
Control Objectives:
- Test, validate, and approve AI models prior to deploying them.
- Unauthorized changes to decision-making logic within the model is prevented.
Monitoring Models for Biases
Biases going undetected and uncorrected in AI models can put organizations at risk from a reputational and regulatory standpoint.
Control Objectives:
- Detect systematic errors in AI models that produce unfair outcomes (biases).
- Correct the identified biases by retraining models and using diverse data sets.
- Ensure equitable decisions across various demographic groups, where applicable.
Explainability and Clarity of Model Outputs
Relevant parties want to be able to understand how AI model logic is established, how decisions are made, and the meaning of the outputs.
Control Objectives:
- Model outputs are interpretable where necessary.
- AI model logic and any limits of the model are documented.
Model Decay Monitoring and Management
As real-world data changes over time, AI model performance degrades because models are running on information or logic that is no longer relevant.
Control Objectives:
- Monitor and detect AI model performance decline over time.
- Retrain or rollback AI models when performance declines past an established threshold.
Access and Change Management
AI models must be protected from unauthorized access and unauthorized changes (inappropriate manipulation).
Control Objectives:
- Access to AI models, all training data, and development areas are restricted to authorized users only.
- All changes are tracked and approved prior to deployment.
Integrating and Mapping AI Controls into SOC 2 Reporting
While identifying control objectives and defining AI-specific controls covers SOC 1 reports, SOC 2 reporting for organizations must map the AI-specific controls to one or more of the SOC 2 criteria. This helps ensure that controls become part of the overall environment rather than being treated simply as “add-ons.” For example:
- Security – User access controls specific to AI models and data used to train models.
- Availability – Monitoring controls to track uptime or downtime of AI services.
- Processing Integrity – Validation and accuracy controls to ensure the AI model outputs are as expected.
- Confidentiality – Controls to protect any sensitive data used to train AI models and systems.
- Privacy – Controls to ensure the organization remains compliant with data protection regulations.
As the adoption of AI continues, regulatory scrutiny will increase. Implementing AI-specific controls within organizational SOC frameworks will become a necessity. Acting early will strengthen the control environment and build credibility with clients and stakeholders.
If your entity is interested in obtaining any additional information on SOC reports, or if there are any other questions related to SOC, please contact us. For more information on these services and more, be sure to visit our firm’s SOC & Cybersecurity industry page, and don’t hesitate to contact Dave Hammarberg, CPA, CFE, CISSP, GSEC, MCSE, CISA, CCSFP, CHQP, CCA regarding our services.
About the Author
Cecily Carl joined McKonly & Asbury in December of 2023 and is an Advisory Supervisor with the firm. She is a member of the firm’s Advisory and Business Consulting
Segment, serving clients as an internal auditor and consultant on Sarbanes-Oxley (SOX) engagements, QAR engagements, and Governmental audits. She is also a member of the firm’s System and Organization Controls (SOC) practice. Cecily serves a wide range of industries in both public and private sectors.
