⚖️Trustworthy AI assessment
In short:
Allowing participants of the data space to provide trust in their profiling algorithms and users to trust the providers.
Easily audit profiling algorithms for the data space and compare them, ensuing a trust & ethical notation.
Clear communication of results and descriptions of algorithms to people.
Timeline
Start date: T0 (expected: Q1 2024)
End date : T0 + 12 months
Duration (in months): 12
Where we are right now
LORIA and Affectlog already have functioning versions of these platforms
Development to enhance their functionalities and connect them to the data spaces
Want to join the effort? See the Working Groups.
Objectives:
The objective is to deploy two operational platforms, each providing a different and complementary approach.
They are presented below.
General Framework
For the past 3 years, LORIA, in collaboration with the French Ministry of Education, has been working on a proof of concept for a platform that allows both the secure sharing of educational data and the testing of AI algorithms on these data. Several use cases are proposed by the platform, and it is designed in order to be easily extended with nes use cases. This platform has been named LOLA for Open Laboratory for Learning Analytics.
Each algorithm submitted to the platform receives a report in the form of an audit based on its own performance and a comparison with reference algorithms integrated into the platform, with regard to indicators related to the targeted use case.
Thus, compared to other data sharing platforms, the LOLA platform offers the following advantages:
Security: Any data deposited in the platform is never accessible externally, neither for downloading nor for copying.
Control: Any data provider can configure the access rights to whoever he wants and keep an eye on the experiments that are done on it (especially the audit reports of new algorithms).
Transparency: Thanks to the edition of an associated audit report, each algorithm contributor can certify its performance according to use cases based on validated and sectioned indicators.
Extensibility: Each use case is declined in the form of a "scenario" which represents a central concept of LOLA.
A use case provider always has the option of creating a new scenario that incorporates a description of the data, a specification of the algorithm's outputs, a set of indicators, and an experimental protocol for measuring performance.
Trustworthy Artificial Intelligence
In the short term, the platform aims to offer the EdTech community a space that would allow them to validate the choices of algorithms embedded in their applications and services.
In the context of the measurement mentioned in the introduction of this document, the criteria concerned are obviously the following
- (2) transparency offered by an open and shared platform,
- (5) Privacy and data governance
- (6) Technical robustness and safety
- and (7) Accountability
Infrastructure and Learning Analytics
This platform is also a way to offer an infrastructure and a common standard for the output related to the providers of digital traces as well as the output related to use cases.
Proposed Contributions
Scalability (LORIA’s contribution)
The LOLA platform is currently implemented in LORIA's LHS (high security laboratory), a private data center at the scale of a university.
The current dedicated infrastructure is sized for a proof of concept with a few use cases, but it is essential to connect it to a common larger cloud of Prometheus-X for a scaling up.
Complementary tools
To facilitate its use, it is essential to add to the platform complementary tools, some of which will be developed by Loria within the framework of the present project, namely :
LORIA: The characterization of the population referenced by the data made available. This characterization could also be integrated into the performance indicators of the use cases
LORIA: The implementation of offline tuning tools (including the generation of public artificial data for testing purposes)
Fraunhofer ISST / Univ. Koblenz: The implementation of automated analysis of AI trustworthiness indicators (including explainability, data security, privacy, and fairness)
Deliverables
Audit platform and associated tools
Methodological Framework
AffectLog's risk assessment platform uses AI to extend the enterprise-grade Application Security Verification Standard (ASVS) to assess the security and privacy of applications both upstream and downstream of an application development lifecycle. It can help organizations identify areas of weakness in their AI models and prioritize efforts to improve their security. AffectLog's scoring matrix assesses the specific needs and risks of the AI models, and includes a dynamic rating scale with a numerical score, giving a snapshot of the overall security of the application. It also provides contextually relevant resources and a tool for tracking progress over time and identifying trends in security. It offers a way to create a Secure Coding Checklist specific to the application, platform or organization, aligned with industry security and privacy standards.
AffectLog Basic (aligned with ASVS Level 1) - No access to source codebase/development lifecycle
AffectLog Basic assessment determines compliance with the ASVS Level 1, by evaluating the application's protection against common vulnerabilities that are easy to discover, including those on OWASP's Top 10 list. This is the minimum level of security that should be achieved by all applications, especially those not handling sensitive data. The controls can be checked either automatically or manually without source code access. It is designed to evaluate an application's capabilities against simple, low-effort attacks and not against more determined attackers. However, if the application handles high value data, it is not advisable to rely solely on AffectLog Basic review.
AffectLog Enhanced (aligned with ASVS Level 2) - Requires source code access
An application that meets the requirements of AffectLog Enhanced audit (ASVS Level 2) is considered to have adequate protection against most common risks. It should have effective security controls in place and be using them appropriately. It is typically appropriate for applications that handle important business-to-business transactions, such as those that process healthcare (non-PII) information or sensitive assets like minors’ data, or those that implement business-critical or sensitive functions. This level also applies to industries where integrity is crucial to their business, to protect against nefarious agents. These applications may be targeted by skilled and motivated attackers using highly practiced and effective tools and techniques.
AffectLog Ultimate (aligned with ASVS Level 3) - Requires full source code and development resource access
AffectLog Ultimate assessment, aligned with ASVS Level 3, is the highest level of verification and is typically reserved for applications that require significant security measures, such as those in military, health and safety and critical infrastructure. Organizations may require AffectLog assessment for applications that perform critical functions, where failure could significantly impact the organization's operations and even its survivability. The assessment checks for advanced application security vulnerabilities and good security design. It requires in-depth analysis of architecture, coding and testing, and the application should be modularized in a meaningful way, with each module taking care of its own security responsibilities. AffectLog platform uses Machine Learning to bring attention to threats and vulnerabilities that may have been missed by manual checks, especially when dealing with large code bases.
Relevant functionalities
AffectLog risk assessment platform offers assessments bespoke not only to mobile apps, web apps and APIs but also specific to the emerging technologies like Machine Learning (ML) based on industry-standard methodologies. AffectLog leverages semi-supervised machine learning models (AffectLog ML360°) to build on industry-grade application risk assessment standards to assess the security of applications by:
Automating the assessment process: AffectLog ML360° is trained on the ASVS requirements and is used to automatically evaluate the security of a web application against those requirements. This helps organizations save time and resources by automating the assessment process.
Identifying security vulnerabilities: AffectLog ML360° is used to analyze the code and configuration of a web application and identify potential vulnerabilities or weaknesses. This helps AI driven organizations identify and address security issues before they become a problem.
Providing recommendations for improvement: AffectLog ML360° is used to analyze the security of an application and provide recommendations for improvement based on the ASVS requirements. This could help organizations identify areas where their security profile could be strengthened.
Continuous monitoring: AffectLog ML360° is used to continuously monitor the security of an application and alert organizations to any changes or issues that may arise. This helps organizations stay on top of their security posture and quickly address any issues that arise.
Standards:
For LORIA’s platform
The standards used are essentially:
Nextflow: for the description of the scenarios
Docker, docker-compose : for the integration of the algorithms
xAPi, xAPI Schema : for data and their documentation
For Affectlog’s platform
AffectLog extends widely adopted industry standards for risk assessment, such as OWASP, UNESCO Recommendation on Ethics of AI, EU AI Act draft text, and NIST, to build its functionalities. AffectLog’s audit checklist encompasses Mobile Security Testing Guide, the Web Application Security Consortium (WASC) Threat Classification, Privacy Risk Assessment Methodology (PRAM) and the GDPR Data Protection Impact Assessment (DPIA) to assess the relevant security and data privacy risks of mobile and web applications. The platform aims to provide an auditable, explainable and transparent system to document the assessment checklist and the assessment scores.
Roles between partners:
Both LORIA (with the LOLA project) and Affectlog (with a maturing platform) have experience in designing, developing and implementing secure and specialized platforms for AI.
Within the framework of this project, the deliverables will be two distinct platforms carried on the common infrastructure operated by Prometheus-X.
The nature and objectives of these platforms are quite different and their development could be done quite independently.
The Fraunhofer Institute contribution is quite naturally integrated in the first platform as a complementary tool.
This requires a close collaboration especially in the integration phase
Last updated