CT-AI Certified Tester AI Testing Exam Questions and Answers

 Technology Most Typically Used to Implement AI:Genetic algorithms are a well-known technique used in AI. They are inspired by the process of natural selection and are used to find approximate solutions to optimization and search problems. Unlike search engines, procedural programming, or case control structures, genetic algorithms are specifically designed for evolving solutions and are commonly employed in AI implementations.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 1.4 AI Technologies, which identifies different technologies used to implement AI​​.

Setting model and algorithm hyperparameters is an essential step in the machine learning workflow, primarily occurring during the tuning phase.

Evaluating the model (A): This stage involves assessing the model's performance using metrics and does not typically include the setting of hyperparameters.

Deploying the model (B): Deployment is the stage where the model is put into production and used in real-world applications. Hyperparameters should already be set before this stage.

Tuning the model (C): This is the correct stage where hyperparameters are set. Tuning involves adjusting the hyperparameters to optimize the model's performance.

Data testing (D): Data testing involves ensuring the quality and integrity of the data used for training and testing the model. It does not include setting hyperparameters.

Hence, the most appropriate stage of the ML workflow to set model and algorithm hyperparameters isC. Tuning the model.

References:

ISTQB CT-AI Syllabus Section 3.2 on the ML Workflow outlines the different stages of the ML process, including the tuning phase where hyperparameters are set.

Sample Exam Questions document, Question #31 specifically addresses the stage in the ML workflow where hyperparameters are configured.

Back-to-back testing isused to compare two different versions of an ML model, which is precisely what is needed in this scenario.

The model initiallymisclassified dogs as wolvesdue to feature similarities.

Thetest team retrains the modelwith additional images of dogs and wolves.

The best way to verify whether this additional trainingimproved classification accuracyis to compare theoriginal model's output with the newly trained model's output using the same test dataset​.

A (Metamorphic Testing):Metamorphic testing is useful forgenerating new test casesbased on existing ones but does not directly compare different model versions​.

B (Adversarial Testing):Adversarial testing is used to check how robust a model is againstmaliciously perturbed inputs, not to verify training effectiveness​.

C (Pairwise Testing):Pairwise testing is a combinatorial technique for reducing the number of test casesby focusing on key variable interactions, not for validating model improvements​.

ISTQB CT-AI Syllabus (Section 9.3: Back-to-Back Testing)

"Back-to-back testing is used when an updated ML model needs to be compared against a previous version to confirm that it performs better or as expected"​.

"The results of the newly trained model are compared with those of the prior version to ensure that changes did not negatively impact performance"​.

Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:To verify that the model's performance improved after retraining,back-to-back testing is the most appropriate methodas it compares both model versions. Hence, thecorrect answer is D.

When verifyingautonomous AI-based systems, a critical aspect is ensuring that they maintain an appropriate level of autonomy whileonly requesting human intervention when necessary. If an AI system unnecessarily asks for human input, it defeats the purpose of autonomy and can:

Slow down operations.

Reduce trust in the system.

Indicate improper confidence thresholds in decision-making.

This is particularly crucial inautonomous vehicles, AI-driven financial trading, and robotic process automation, where excessive human intervention would hinder performance.

A. Test cases to verify that the system automatically confirms the correct classification of training data→ This is relevant for verifying training consistency but not for autonomy validation.

B. Test cases to detect the system appropriately automating its data input→ While relevant, data automation does not directly address the verification of autonomy.

D. Test cases to verify that the system automatically suppresses invalid output data→ This focuses on output filtering rather than decision-making autonomy.

Why are the other options incorrect?Thus, the mostcritical test casefor verifyingautonomous AI-based systemsis ensuring that itdoes not unnecessarily request human intervention.

Section 8.2 - Testing Autonomous AI-Based Systemsstates that it is crucial to testwhether the system requests human intervention only when necessaryand does not disrupt autonomy​.

Reference from ISTQB Certified Tester AI Testing Study Guide:

Adeep neural network (DNN)is a type of artificial neural network that consists of multiple layers between the input and output layers. TheISTQB Certified Tester AI Testing (CT-AI) Syllabusoutlines the following characteristics of a DNN:

Structure of a Deep Neural Network:

ADNN comprises at least three types of layers:

Input layer: Receives the input data.

Hidden layers: Perform complex feature extraction and transformations.

Output layer: Produces the final prediction or classification​.

Analysis of Answer Choices:

A (Only input and output layers)→ Incorrect, as a DNN must haveat least one hidden layer.

B (At least one internal hidden layer)→ Correct, as a neural network must havehidden layersto be considered deep.

C (Minimum of five layers required)→ Incorrect, asthere is no strict definitionthat requires at least five layers.

D (Output layer is not connected to other layers)→ Incorrect, asthe output layer must be connectedto the hidden layers​.

Thus,Option B is the correct answer, asa deep neural network must have at least one hidden layer.

Certified Tester AI Testing Study Guide References:

ISTQB CT-AI Syllabus v1.0, Section 6.1 (Neural Networks and Deep Neural Networks)

ISTQB CT-AI Syllabus v1.0, Section 6.2 (Structure of Deep Neural Networks)​.

Pre-processing is an essential step in data preparation that ensures data is clean, formatted correctly, and structured for effective machine learning (ML) model training. One common issue that can be resolved during pre-processing isnumbers stored as strings.

Explanation of Answer Choices:

Option A: Insufficient data

Incorrect. Pre-processing cannot resolve insufficient data. If data is lacking, techniques like data augmentation or external data collection are needed​.

Option B: Invalid data

Incorrect. While pre-processing can identify and handle some forms of invalid data (e.g., missing values, duplicate entries), it does not resolve all invalid data issues. Some cases may require domain expertise to determine validity​.

Option C: Wanted outliers

Incorrect. Pre-processing usually focuses on handling unwanted outliers. Wanted outliers may need to be preserved, which is more of a data selection decision rather than pre-processing​.

Option D: Numbers stored as strings

Correct. One of the key functions of data pre-processing isdata transformation, which includes converting incorrectly formatted data types, such as numbers stored as strings, into their correct numerical format​.

ISTQB CT-AI Syllabus References:

Data Pre-Processing Steps:"Transformation: The format of the given data is changed (e.g., breaking an address held as a string into its constituent parts, dropping a field holding a random identifier, converting categorical data into numerical data, changing image formats)"​.

Exploratory Data Analysis (EDA) is an essential technique for examining datasets to uncover patterns, trends, and anomalies, including outliers. In this case, the attacker manipulates the spam filter by injecting emails with red flags and masking them as internal company emails. The primary goal of EDA here is to detect these adversarial modifications.

Detecting Outliers:

EDA techniques such as statistical analysis, clustering, and visualization can reveal patterns in email metadata (e.g., sender details, email content, frequency).

Outlier detection methods like Z-score, IQR (Interquartile Range), or machine learning-based anomaly detection can identify emails that significantly deviate from typical internal communications.

Identifying Distribution Shifts:

By analyzing the frequency and characteristics of emails flagged as spam, testers can detect if the attack has introduced unusual patterns.

If a surge of internal emails is suddenly classified as spam, EDA can help verify whether these classifications are consistent with historical data.

Feature Analysis for Adversarial Patterns:

EDA enables visualization techniques such as scatter plots or histograms to distinguish normal emails from manipulated ones.

Examining email metadata (e.g., changes in headers, unusual wording in email bodies) can reveal adversarial tactics.

Counteracting Adversarial Attacks:

Once anomalies are identified, the spam filter’s detection rules can be improved by retraining the model on corrected datasets.

The adversarial examples can be added to the training data to enhance the robustness of the filter against future attacks.

Exploratory Data Analysis (EDA) is used to detect outliers and adversarial attacks."EDA is where data are examined for patterns, relationships, trends, and outliers. It involves the interactive, hypothesis-driven exploration of data."​

EDA can identify poisoned or manipulated data by detecting anomalies and distribution shifts."Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."​

EDA helps validate ML models and detect potential vulnerabilities."The use of exploratory techniques, primarily driven by data visualization, can help validate the ML algorithm being used, identify changes that result in efficient models, and leverage domain expertise."​

References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as EDA is specifically useful for detecting outliers, which can help identify manipulated spam emails.

Themachine learning (ML) workflowfollows a structured sequence of steps. Once stakeholders have agreed on theobjectives, business priorities, and the framework/model selection, the next logical step is toprepare and pre-process the databefore training the model.

Data Preparationis crucial becausemachine learning models rely heavily on the quality of input data. Poor data can result in biased, inaccurate, or unreliable models​.

The process involvesdata acquisition, cleaning, transformation, augmentation, and feature engineering​.

Preparing the dataensures it is in the right format, free from errors, and representative of the problem domain, leading to better generalization in training.

A (Tune the ML Algorithm):Hyperparameter tuning occursafter the model has been trainedand evaluated​.

C (Agree on Acceptance Criteria):Acceptance criteria should already have been defined in theinitial objective-setting phasebefore framework and model selection​.

D (Evaluate the Framework and Model):The selection of the framework and ML model has already been completed. The next step isdata preparation, not reevaluation​.

ISTQB CT-AI Syllabus (Section 3.2: ML Workflow - Data Preparation Phase)

"Data preparation comprises data acquisition, pre-processing, and feature engineering. Exploratory data analysis (EDA) may be performed alongside these activities"​.

"The data used to train, tune, and test the model must be representative of the operational data that will be used by the model"​.

Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Since the model selection is complete, thenext step in the ML workflow is to prepare and pre-process the datato ensure it is ready for training and testing. Thus, thecorrect answer is B.

Data mislabeling occurs for several reasons, which can significantly impact the performance of machine learning (ML) models, especially in supervised learning. According to the ISTQB Certified Tester AI Testing (CT-AI) syllabus, mislabeling of data can be caused by the following factors:

Random errors by annotators– Mistakes made due to accidental misclassification.

Systemic errors– Errors introduced by incorrect labeling instructions or poor training of annotators.

Deliberate errors– Errors introduced intentionally by malicious data annotators.

Translation errors– Occur when correctly labeled data in one language is incorrectly translated into another language.

Subjectivity in labeling– Some labeling tasks require subjective judgment, leading to inconsistencies between different annotators.

Lack of domain knowledge– If annotators do not have sufficient expertise in the domain, they may label data incorrectly due to misunderstanding the context.

Complex classification tasks– The more complex the task, the higher the probability of labeling mistakes​.

Among the answer choices provided, "Lack of domain knowledge" (Option A) is the best answer because expertise is essential to accurately labeling data in complex domains such as medical, legal, or engineering fields.

Certified Tester AI Testing Study Guide References:

ISTQB CT-AI Syllabus v1.0, Section 4.5.2 (Mislabeled Data in Datasets)

ISTQB CT-AI Syllabus v1.0, Section 4.3 (Dataset Quality Issues)

A. 64-bit CPUs.

While 64-bit CPUs are essential for handling large amounts of memory and performing complex computations, they are not specifically optimized for the types of operations commonly used in machine learning.

B. Hardware supporting fast matrix multiplication.

Matrix multiplication is a fundamental operation in many machine learning algorithms, especially in neural networks and deep learning. Hardware optimized for fast matrix multiplication, such as GPUs (Graphics Processing Units), is most suitable for implementing AI and ML because it can handle the parallel processing required for these operations efficiently.

C. High powered CPUs.

High powered CPUs are beneficial for general-purpose computing tasks and some aspects of ML, but they are not as efficient as specialized hardware like GPUs for matrix multiplication and other ML-specific tasks.

D. Hardware supporting high precision floating point operations.

High precision floating point operations are important for scientific computing and some specific AI tasks, but for many ML applications, fast matrix multiplication is more critical than high precision alone.

Therefore, the correct answer isBbecause hardware supporting fast matrix multiplication, such as GPUs, is most suitable for the parallel processing requirements of machine learning​​.

 Labelling Approaches:Among the options provided, pre-labeled datasets require the least time and effort because the data has already been labeled, eliminating the need for further manual or automated labeling efforts.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 4.5 Data Labelling for Supervised Learning, which discusses various approaches to data labeling, including pre-labeled datasets, and their associated time and effort requirements​​.

In a neural network, the activation value of a neuron is determined by a combination of inputs from the previous layer, the weights of the connections, and the bias at the neuron level. Here’s a detailed breakdown:

Inputs for Activation Value:

Activation Values of Neurons in the Previous Layer:These are the outputs from neurons in the preceding layer that serve as inputs to the current neuron.

Weights Assigned to the Connections:Each connection between neurons has an associated weight, which determines the strength and direction of the input signal.

Individual Bias at the Neuron Level:Each neuron has a bias value that adjusts the input sum, allowing the activation function to be shifted.

Calculation:

The activation value is computed by summing the weighted inputs from the previous layer and adding the bias.

Formula: z=∑(wi⋅ai)+bz = \sum (w_i \cdot a_i) + bz=∑(wi​⋅ai​)+b, where wiw_iwi​ are the weights, aia_iai​ are the activation values from the previous layer, and bbb is the bias.

The activation function (e.g., sigmoid, ReLU) is then applied to this sum to get the final activation value.

Why Option A is Correct:

Option A correctly identifies all components involved in computing the activation value: the individual bias, the activation values of the previous layer, and the weights of the connections.

Eliminating Other Options:

B. Activation values of neurons in the previous layer, and weights assigned to the connections between the neurons: This option misses the bias, which is crucial.

C. Individual bias at the neuron level, and weights assigned to the connections between the neurons: This option misses the activation values from the previous layer.

D. Individual bias at the neuron level, and activation values of neurons in the previous layer: This option misses the weights, which are essential.

References:

ISTQB CT-AI Syllabus, Section 6.1, Neural Networks, discusses the components and functioning of neurons in a neural network.

"Neural Network Activation Functions" (ISTQB CT-AI Syllabus, Section 6.1.1).

Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options isleast likelyto be a reason for the explosion in the number of parameters.

Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters.

Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered.

ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle.

Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters.

Hence, theleast likelyreason for the incredible growth in the number of parameters isC. ML model metrics to evaluate the functional performance.

References:

ISTQB CT-AI Syllabus Section 9.2 on Pairwise Testing discusses the application of this technique to manage the combinations of different variables in AI-based systems, including those used in self-driving cars.

Sample Exam Questions document, Question #29 provides context for the explosion in parameter combinations in self-driving cars and highlights the use of pairwise testing as a method to manage this complexity.

 A. This statement is incorrect. Current AI is true AI and there is no reason to believe that this fact will change over time.

AI is an evolving field, and the definition of what constitutes AI can change as technology advances.

 B. This statement is correct. In general, what is considered AI today may change over time.

The term AI is dynamic and has evolved over the years. What is considered AI today might be viewed as standard computing in the future. Historically, as technologies become mainstream, they often cease to be considered "AI".

 C. This statement is incorrect. What is considered AI today will continue to be AI even as technology evolves and changes.

This perspective does not account for the historical evolution of the definition of AI. As new technologies emerge, the boundaries of AI shift.

 D. This statement is correct. In general, today the term AI is utilized incorrectly.

While some may argue this, it is not a universal truth. The term AI encompasses a broad range of technologies and applications, and its usage is generally consistent with current technological capabilities.

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

References:The explanation is based on the definitions of different machine learning types as outlined in the ISTQB CT-AI syllabus, specifically under supervised learning and classification​​.

Object recognition issues in test automation often arise whendevelopers frequently change object identifiersin the GUI. AI can enhance the stability of GUI automation by:

Using multiple criteria for object identification

AI cantrack UI elements using multiple attributessuch asXPath, label, ID, class, and screen coordinatesrather than relying on a single identifier that may change over time.

This approach makes the automationless brittle and more adaptive to changes in the UI​.

Why other options are incorrect?

B (Ignore unrecognizable objects to avoid failures): Ignoring objects instead of identifying them properly wouldlead to incomplete or incorrect test execution.

C (Dynamically name objects and alter source code): AI-based testing tools donot modify application source code; they work byadjusting the recognition strategy.

D (Anticipate developer changes and pre-alter automation code): While AI can adapt,it does not predict future changes to the GUI, making this option unrealistic​.

Thus,Option A is the best answer, as AI tools enhance object recognitionby dynamically selecting the most stable and persistent identification methods, improving test automation reliability.

Certified Tester AI Testing Study Guide References:

ISTQB CT-AI Syllabus v1.0, Section 11.6.1 (Using AI to Test Through the Graphical User Interface (GUI))

ISTQB CT-AI Syllabus v1.0, Section 11.6.2 (Using AI to Test the GUI)​.

 Type of Testing for Stock-Price Prediction Models:Concept drift refers to the change in the statistical properties of the target variable over time. Severe weather storms causing massive losses in coffee production and affecting stock prices would require testing for concept drift to ensure that the model adapts to new patterns in data over time.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 7.6 Testing for Concept Drift, which explains the need to test for concept drift in models that might be affected by changing external factors​​.

 Testing Autonomy:Testing for human handover when it should not be relinquishing control is the least appropriate because it contradicts the very definition of autonomous systems. The other tests are relevant to ensuring smooth operation and transitions between human and AI control.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Testing Autonomous AI-Based Systems and Testing for Human-AI Interaction​​​​. ​

Understanding Supervised Learning - RegressionSupervised learning is a category of machine learning where the model is trained on labeled data. Within this category,regressionis used when the goal is to predict a continuous numeric value.

Regressiondeals with problems where the output variable is continuous in nature, meaning it can take any numerical value within a range.

Common examples include predicting prices, estimating demand, and analyzing production trends.

(A) Determining the optimal age for a chicken's egg-laying production using input data of the chicken's age and average daily egg production for one million chickens.✅(Correct)

This is a classicregression problembecause it involves predicting a continuous variable:daily egg productionbased on the input variablechicken's age.

The goal is to find a numerical relationship between age and egg production, which makesregression the appropriate supervised learning method.

(B) Recognizing a knife in carry-on luggage at a security checkpoint in an airport scanner.❌(Incorrect)

This is animage recognition task, which falls underclassification, not regression.

Classification problems involve assigning inputs to discrete categories (e.g., "knife detected" or "no knife detected").

(C) Determining if an animal is a pig or a cow based on image recognition.❌(Incorrect)

This is anotherclassification problemwhere the goal is to categorize an image into one of two labels (pig or cow).

(D) Predicting shopper purchasing behavior based on the category of shopper and the positioning of promotional displays within a store.❌(Incorrect)

This problem could involve a mix ofclassificationandassociation rule learning, but it does not explicitly predict a continuous variable in the way regression does.

Regression is used when predicting a numeric output."Predicting the age of a person based on input data about their habits or predicting the future prices of stocks are examples of problems that use regression."​

Supervised learning problems are divided into classification and regression."If the output is numeric and continuous in nature, it may be regression."​

Regression is commonly used for predicting numerical trends over time."Regression models result in a numerical or continuous output value for a given input."​

Analysis of Answer ChoicesReferences from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as it aligns with the principles of regression-based supervised learning.

Handlingtest datain AI-based systems presents numerous challenges, particularly in terms ofdata privacy and confidentiality. AI models often require vast amounts of training data, some of which may containpersonal, sensitive, or confidential information. Ensuringcompliance with data protection laws (e.g., GDPR, CCPA)and implementingsecure data-handling practicesis a major challenge in AI testing.

Data Privacy Regulations

AI-based systems frequently process personal data, such as images, names, and transaction details, leading toprivacy concerns.

Compliance with regulations such asGDPR (General Data Protection Regulation)andCCPA (California Consumer Privacy Act)requiresproper anonymization, encryption, or redactionof sensitive data before using it for testing.

Data Security Challenges

AI models mayleak confidential informationif proper security measures are not in place.

Protectingtraining and test data from unauthorized accessis crucial to maintainingtrust and compliance.

Legal and Ethical Considerations

Organizations mustobtain legal approvalbefore using certain datasets, especially those containinghealth records, financial data, or personally identifiable information (PII).

Testers may need toemploy synthetic dataordata maskingtechniques to minimize exposure risks.

(B) Output data or intermediate data❌

While analyzing output data is important, it does notpose a significant challengecompared to handlingpersonal or confidential test data.

(C) Video frame speed or aspect ratio❌

These aretechnical challengesin processing AI models but do not fall underdata privacy or ethical considerations.

(D) Data frameworks or machine learning frameworks❌

Choosing an appropriateML framework (e.g., TensorFlow, PyTorch)is important, but it is nota major challenge related to test data handling.

Handling personal or confidential data is a critical challenge in AI testing"Personal or otherwise confidential data may need special techniques for sanitization, encryption, or redaction.Legal approval for use may also be required."​

Why is Option A Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, asdata privacy and confidentiality are major challenges when handling test data for AI-based systems.

AI-based tools can significantly optimize regression test suites by analyzing historical data, past test results, associated defects, and changes made to the software. These tools prioritize and select the most relevant test cases based on previous defect patterns and frequently failing features, which helps in reducing the test execution time while maintaining effectiveness.

The optimization process involves:

Prioritizing test cases:AI-based tools rank test cases based on past defect detection trends, ensuring that the most relevant tests are executed first.

Reducing redundant test cases:The tool can eliminate test cases that do not contribute significantly to defect detection, reducing overall test execution time.

Augmenting test cases:The AI can also suggest new test cases if certain features are more prone to defects.

This approach has been proven to reduce regression test suite sizes by up to 50% while maintaining fault detection capabilities​.

Section 11.4 - Using AI for the Optimization of Regression Test Suitesstates that AI-based tools can optimize regression test suites by analyzing past test data and defect occurrences, leading to significant reductions in test execution time​.

Reference from ISTQB Certified Tester AI Testing Study Guide: