ISTQB® Certified Tester AI Testing
The ISTQB Certified Tester AI Testing is a follow on to the ISTQB Certified Tester: Foundation Level v4.0 and is valuable to a wide range of professionals involved in the development, testing, and management of AI-based systems. This certification helps professionals stay updated with the latest AI testing methodologies, improve their skills, and enhance their career opportunities in the rapidly evolving field of AI.
Description
The ISTQB® CT‑AI course provides a concise introduction to AI, ML, and the methods used to test AI‑based systems. Participants learn to assess data quality, test ML models, evaluate GenAI and LLMs, and apply AI‑specific testing aligned with ISO/IEC 25059.
It covers the full AI/ML lifecycle, with hands‑on exercises in model creation, performance evaluation, data testing, metamorphic testing, red teaming, and LLM exploratory testing.
The course contents will include detailed explanations, practical exercises, and hands-on activities for each topic covered in the course outline.
Description
The ISTQB® CT‑AI course provides a concise introduction to AI, ML, and the methods used to test AI‑based systems. Participants learn to assess data quality, test ML models, evaluate GenAI and LLMs, and apply AI‑specific testing aligned with ISO/IEC 25059.
It covers the full AI/ML lifecycle, with hands‑on exercises in model creation, performance evaluation, data testing, metamorphic testing, red teaming, and LLM exploratory testing.
The course contents will include detailed explanations, practical exercises, and hands-on activities for each topic covered in the course outline.
Course Objectives
By the end of this course, participants will:
- Understand fundamental concepts, capabilities, and limitations of Generative AI in testing.
- Develop practical skills in prompt engineering for software test tasks.
- Identify and mitigate risks of using GenAI, including hallucinations, reasoning errors, and data privacy issues.
- Explore LLM-powered test infrastructure and operational practices (LLMOps, fine-tuning).
Course Objectives
By the end of this course, participants will:
- Understand fundamental concepts, capabilities, and limitations of Generative AI in testing.
- Develop practical skills in prompt engineering for software test tasks.
- Identify and mitigate risks of using GenAI, including hallucinations, reasoning errors, and data privacy issues.
- Explore LLM-powered test infrastructure and operational practices (LLMOps, fine-tuning).
Course Outline
Chapter 1 – Introduction to AI (120 minutes)
- AI‑based vs conventional systems
- Narrow, general, and super AI
- Types of AI technologies
- Generative AI
- Hardware for ML systems
- Development & hosting of AI models
- ML development frameworks
- Regulations & standards for AI
Chapter 2 – Quality Characteristics for AI‑Based Systems (45 minutes)
- ISO/IEC 25059 AI quality characteristics
- AI functional correctness
- AI robustness
- Transparency, user controllability, intervenability
- Societal & ethical risk mitigation
- Acceptance criteria for AI‑based systems
Chapter 3 – Machine Learning (375 minutes)
- ML forms: supervised, unsupervised, reinforcement learning
- ML workflow: data prep - model training - tuning - testing - deployment - monitoring
- Data acquisition, preprocessing, feature engineering
- Training/validation/test datasets; cross‑validation
- ML functional performance metrics (accuracy, precision, recall, F1)
- Neural networks and coverage measures
- Hands‑on exercises (H1/H2):
- Build an ML model
- Perform data preparation
- Evaluate metrics
- Explore effects of different models/dataset combinations
Chapter 4 – Testing AI‑Based Systems (195 minutes)
- Locked vs adaptive AI systems
- Statistical approaches to testing
- The test oracle problem
- Testing GenAI systems
- Red teaming (K3 level)
- Exploratory testing of LLMs (H2)
- Test levels for ML systems
- Risk‑based testing
Chapter 5 – Input Data Testing (180 minutes)
- Input data risks and mitigations
- Testing for bias
- Data pipeline testing
- Data representativeness testing
- Dataset constraint testing
- Label correctness testing
- Hands‑on exercise: input data testing (H2)
Chapter 6 – Model Testing for ML Systems (225 minutes)
- ML model risks
- Reviewing ML model documentation
- ML functional performance testing (probabilistic systems)
- Adversarial testing
- Metamorphic testing (H2)
- Drift testing
- Overfitting & underfitting
- A/B testing
- Back‑to‑back testing
Chapter 7 – ML Development Testing (30 minutes)
- ML development risks
- API testing
- Deployment testing: installability, rollback, canary, shadow, cross‑device
- Model conversion testing
Course Outline
Chapter 1 – Introduction to AI (120 minutes)
- AI‑based vs conventional systems
- Narrow, general, and super AI
- Types of AI technologies
- Generative AI
- Hardware for ML systems
- Development & hosting of AI models
- ML development frameworks
- Regulations & standards for AI
Chapter 2 – Quality Characteristics for AI‑Based Systems (45 minutes)
- ISO/IEC 25059 AI quality characteristics
- AI functional correctness
- AI robustness
- Transparency, user controllability, intervenability
- Societal & ethical risk mitigation
- Acceptance criteria for AI‑based systems
Chapter 3 – Machine Learning (375 minutes)
- ML forms: supervised, unsupervised, reinforcement learning
- ML workflow: data prep - model training - tuning - testing - deployment - monitoring
- Data acquisition, preprocessing, feature engineering
- Training/validation/test datasets; cross‑validation
- ML functional performance metrics (accuracy, precision, recall, F1)
- Neural networks and coverage measures
- Hands‑on exercises (H1/H2):
- Build an ML model
- Perform data preparation
- Evaluate metrics
- Explore effects of different models/dataset combinations
Chapter 4 – Testing AI‑Based Systems (195 minutes)
- Locked vs adaptive AI systems
- Statistical approaches to testing
- The test oracle problem
- Testing GenAI systems
- Red teaming (K3 level)
- Exploratory testing of LLMs (H2)
- Test levels for ML systems
- Risk‑based testing
Chapter 5 – Input Data Testing (180 minutes)
- Input data risks and mitigations
- Testing for bias
- Data pipeline testing
- Data representativeness testing
- Dataset constraint testing
- Label correctness testing
- Hands‑on exercise: input data testing (H2)
Chapter 6 – Model Testing for ML Systems (225 minutes)
- ML model risks
- Reviewing ML model documentation
- ML functional performance testing (probabilistic systems)
- Adversarial testing
- Metamorphic testing (H2)
- Drift testing
- Overfitting & underfitting
- A/B testing
- Back‑to‑back testing
Chapter 7 – ML Development Testing (30 minutes)
- ML development risks
- API testing
- Deployment testing: installability, rollback, canary, shadow, cross‑device
- Model conversion testing
Prerequisites
- ISTQB® Foundation Level (CTFL) certification (mandatory)
- Recommended: 6+ months in software testing, software development, or data‑related roles
Prerequisites
- ISTQB® Foundation Level (CTFL) certification (mandatory)
- Recommended: 6+ months in software testing, software development, or data‑related roles
Who should attend
This course is ideal for professionals involved in the testing or development of AI‑based systems, including:
- Testers, test analysts, test engineers, test consultants
- Test managers and QA professionals
- Data analysts and data scientists
- Software developers working on AI
- User acceptance testers
- Technical managers, product owners, and project managers
- Quality managers and IT leaders seeking an understanding of AI‑testing challenges
It is also appropriate for anyone seeking foundational knowledge of how AI‑based and ML‑based systems are tested.
Who should attend
This course is ideal for professionals involved in the testing or development of AI‑based systems, including:
- Testers, test analysts, test engineers, test consultants
- Test managers and QA professionals
- Data analysts and data scientists
- Software developers working on AI
- User acceptance testers
- Technical managers, product owners, and project managers
- Quality managers and IT leaders seeking an understanding of AI‑testing challenges
It is also appropriate for anyone seeking foundational knowledge of how AI‑based and ML‑based systems are tested.
What do our Clients say?
“One aspect of the ISTQB Certified Tester –AI Testing Course that stands out as particularly valuable is its focus on testing machine learning (ML) models beyond traditional software testing approaches.What makes this especially appealing is how the course highlights the shift from deterministic systems to probabilistic behavior. Unlike conventional applications where outputs are predictable, AI systems can produce different results for the same input depending on training data and model adjustments. The course does a great job explaining how to validate model quality using metrics like accuracy, precision, recall, and F1-score. How to test training and test data for bias, representativeness, and completeness and how to design test strategies for non-deterministic systems, where expected results aren’t”
Declan Jordan, Software QA Analyst, Technology & Business Consulting
What do our Clients say?
“One aspect of the ISTQB Certified Tester –AI Testing Course that stands out as particularly valuable is its focus on testing machine learning (ML) models beyond traditional software testing approaches.What makes this especially appealing is how the course highlights the shift from deterministic systems to probabilistic behavior. Unlike conventional applications where outputs are predictable, AI systems can produce different results for the same input depending on training data and model adjustments. The course does a great job explaining how to validate model quality using metrics like accuracy, precision, recall, and F1-score. How to test training and test data for bias, representativeness, and completeness and how to design test strategies for non-deterministic systems, where expected results aren’t”
Declan Jordan, Software QA Analyst, Technology & Business Consulting
Related Certifications
- ISTQB® Foundation Level
- ISTQB® Advanced Level – Test Analyst
- ISTQB® Advanced Level – Technical Test Analyst
- ISTQB® Specialist Level – Test Automation Engineer
- ISTQB® Specialist Level – Testing with Generative AI
Related Certifications
- ISTQB® Foundation Level
- ISTQB® Advanced Level – Test Analyst
- ISTQB® Advanced Level – Technical Test Analyst
- ISTQB® Specialist Level – Test Automation Engineer
- ISTQB® Specialist Level – Testing with Generative AI
Contact us to train your team
Looking to upskill your team? We offer tailored training designed around your needs. Flexible formats, expert instructors, real impact. Get in touch — your next training starts here.