Databricks-Machine-Learning-Professional Databricks Certified Machine Learning Professional Questions and Answers

An HTTP Webhook is a mechanism that allows you to register a callback that is triggered by an event, such as a model registry event. The callback is an HTTP request that is sent to a specified URL, which can invoke an action or a notification on another platform or service. An HTTP Webhook is required for use cases that involve integrating the model registry with external tools or workflows that are not supported by Databricks1.

Sending a message to a Slack channel when a model version transitions stages is a use case that requires the use of an HTTP Webhook. This is because Slack is an external platform that is not natively integrated with Databricks, and the model registry events are not directly accessible by Slack. Therefore, to send a message to a Slack channel, you need to register an HTTP Webhook that is triggered by the model registry event of interest, such as MODEL_VERSION_TRANSITIONED_STAGE. The HTTP Webhook then sends a request to the Slack API endpoint that corresponds to the channel and the message content2.

The other options are incorrect because:

• Option A: Starting a testing job when a new model is registered does not require the use of an HTTP Webhook, but rather a job registry webhook. A job registry webhook is a type of webhook that triggers a job in a Databricks workspace when a model registry event occurs. A job registry webhook can be created using the Databricks REST API or the Python client databricks-registry-webhooks on PyPI3.
• Option B: Updating data in a source table for a Databricks SQL dashboard when a model version transitions to the Production stage does not require the use of an HTTP Webhook, but rather a Databricks SQL trigger. A Databricks SQL trigger is a mechanism that allows you to execute a SQL query or a notebook when a specified condition is met, such as a time interval or a file arrival. A Databricks SQL trigger can be created using the Databricks SQL UI or the Databricks REST API4.
• Option C: Sending an email alert when an automated testing job fails does not require the use of an HTTP Webhook, but rather a job alert. A job alert is a feature that allows you to send an email notification when a job run meets a specifiedcondition, such as a failure, a timeout, or a success. A job alert can be created using the Jobs UI or the Databricks REST API5.
• Option D: None of these use cases require the use of an HTTP Webhook is incorrect, as option E does require the use of an HTTP Webhook. References: MLflow Model Registry Webhooks on Databricks, Streamline MLOps With MLflow Model Registry Webhooks, Databricks SQL Triggers, Job Alerts, [Slack API]

Categorical feature drift is a change in the distribution of the input data over time for categorical features, which can affect the performance and accuracy of the model. Monitoring categorical feature drift is important to ensure that the model is still valid and reliable for the current data. One simple statistic to monitor for categorical feature drift is the combination of mode, number of unique values, and percentage of missing values for each categorical feature. These statistics can provide a quick overview of the changes in the data distribution, such as the most frequent category, the diversity of categories, and the quality of data. If these statistics deviate significantly from the baseline values, it may indicate a categorical feature drift. However, these statistics may not capture all the nuances of the data distribution, such as the relative frequencies of different categories, the similarity of categories, etc. Therefore, other methods, such as statistical tests or information-theoretic measures, may be needed to complement the simple statistics and provide a more comprehensive analysis of the categorical feature drift123 References:

• Monitoring Feature Drift - Databricks
• Drift Metrics: How to Select the Right Metric to Analyze Drift
• Detect data drift on datasets (preview) - Azure Machine Learning

The fs.read_table operation can be used to return a Spark DataFrame of a data set associated with a Feature Store table. This operation takes the name of the Feature Store table and an optional time travel specification as arguments. The fs.create_table operation is used to create a new Feature Store table from a Spark DataFrame. The fs.write_table operation is used to write data to an existing Feature Store table. The fs.get_table operation is used to get the metadata of a Feature Store table, not the data itself. There is a way to accomplish this task with fs, so option D is incorrect. References:

• Feature Store Client
• Feature Store Tables

The correct way to log a scikit-learn model using MLflow is to use the mlflow.sklearn.log_model function, which logs the model as an MLflow artifact for the current run1. This function takes the scikit-learn model object and an artifact path as required arguments, and optionally accepts other arguments such as conda environment, code paths, serialization format, etc2. The other options are incorrect because they either use the wrong flavor (spark instead of sklearn), the wrong function (load_model or track_model instead of log_model), or do not exist (track_model is not a valid MLflow function). References: MLflow Models, mlflow.sklearn

The Databricks REST APIs allow users to programmatically access Databricks features and services, such as clusters, jobs, workspaces, libraries, secrets, and more. The Jobs API enables users to create, edit, delete, run, and list jobs, as well as manage job runs. Users can use the Jobs API to programmatically create a new Databricks job whose schedule depends on the result of some automated tests in a machine learning pipeline. For example, users can use the create endpoint to create a new job with a cron_schedule parameter that specifies the cron expression for the job schedule. Users can also use the run_now endpoint to run an existing job with a run_name parameter that specifies the name of the run. Users can use the runs/get endpoint to get the run result state of a job run, and use conditional logic to trigger another job run based on the result. Users can also use the runs/list endpoint to list all the runs of a job, and use the runs/cancel endpoint to cancel a job run if needed. References:

• Databricks REST API — Databricks Documentation, p. 1
• Jobs API — Databricks Documentation, p. 1-2
• Create and run Databricks Jobs | Databricks on AWS, Create and run jobs using the CLI, API, or notebooks, p. 1

To enable MLflow Autologging for all supported machine learning libraries in a notebook, you can use the mlflow.autolog() function. This function enables autologging for each library by invoking the corresponding autologging function, such as mlflow.sklearn.autolog() or mlflow.spark.autolog(). This way, you do not need to call each autologging function separately. The mlflow.autolog() function also takes care of the compatibility and configuration issues that may arise from different versions of the Databricks Runtime for Machine Learning or workspace-wide settings in the Admin Console. The other options are incorrect because:

• Option A: mlflow.sklearn.autolog() will only enable autologging for scikit-learn, not for other libraries.
• Option B: mlflow.spark.autolog() will only enable autologging for Spark MLlib, not for other libraries.
• Option C: spark.conf.set(“autologging”, True) is not a valid way to enable autologging, as there is no such configuration option in Spark.
• Option D: It is possible to automatically log MLflow runs using the autologging functions provided by MLflow. References: Automatic Logging with MLflow Tracking, MLflow quickstart (Python)

The Model Registry is a centralized model store that allows you to manage the full lifecycle of MLflow Models. It provides model lineage, versioning, stage transitions, and annotations. You can use the Model Registry to discover and share models, collaborate on moving them from experimentation to production, integrate with approval and governance workflows, and monitor model deployments and performance1. The other options are incorrect because:

• Option A: Models is a component of MLflow that allows you to manage and deploy models from a variety of ML libraries to a variety of model serving and inference platforms. Models is not a centralized model store, but rather a convention for packaging machine learning models that offers self-contained code, environments, and models2.
• Option C: Model Serving is a feature of Databricks that allows you to host MLflow Models as REST endpoints. Model Serving is not a centralized model store, but rather a model deployment option that enables low-latency inference for real-time applications3.
• Option D: Feature Store is a feature of Databricks that allows you to store, discover, and share features for machine learning. Feature Store is not a centralized model store, but rather a centralized feature store that enables feature reuse, consistency, and governance across ML projects4.
• Option E: Experiments is a component of MLflow that allows you to track experiments to record and compare parameters and results. Experiments is not a centralized model store, but rather a logging service that stores runs, metrics,parameters, tags, and artifacts for each experiment. References: MLflow Model Registry, MLflow Models, MLflow Model Serving, Databricks Feature Store, [MLflow Tracking]

MLflow provides a utility function mlflow.models.signature.infer_signature that can infer the input schema and the output schema of a model from a sample dataset. The input schema and the output schema are represented as mlflow.types.schema.Schema objects, which define the names, data types, and shapes of the model inputs and outputs. The infer_signature function can be used for any model that implements a predict method that takes a pandas DataFrame or a numpy array as input and returns a pandas DataFrame, a pandas Series, a numpy array, or a list as output. The infer_signature function can also be used for models that implement a transform method that takes a pandas DataFrame as input and returns a pandas DataFrame as output. The infer_signature function can be used before logging the model with MLflow, as it does not require the model to be in MLflow format or registered with MLflow Model Registry. References:

• MLflow Models Documentation, Model Signature and Input Example, p. 6-7
• mlflow.types.schema Documentation, p. 1-2
• Machine Learning Engineering with MLflow, Managing Model Signatures and Schemas, p. 1

To replace all data in a feature table with newly computed data, you can use the fs.write_table method with the mode argument set to “overwrite”. This will overwrite the existing data in the feature table with the data in the provided DataFrame. The other options are incorrect because:

• Option A: fs.create_table will create a new feature table, not replace the data in an existing one. If the feature table already exists, it will raise an exception.
• Option B: fs.write_table without specifying the mode argument will use the default mode of “merge”, which will merge the new data with the existing data, not replace it.
• Option C: fs.write table is not a valid method name, as it is missing an underscore. Also, the mode argument of “merge” will not replace the data, but merge it with the existing data.
• Option E: fs.create_table with the mode argument of “merge” is not a valid combination, as the mode argument is only applicable for fs.write_table. Also, the mode argument of “merge” will not replace the data, but merge it with the existing data. References: Databricks FeatureStoreClient, Work with features in Workspace Feature Store

Concept drift in machine learning and data mining refers to the change in the relationships between input and output data in the underlying problem over time. In other domains, this change maybe called “covariate shift,” “dataset shift,” or “nonstationarity.” Concept drift can affect the performance and accuracy of predictive models that assume a static relationship between input and output variables. Concept drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc. Concept drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc References:

• Concept drift - Wikipedia
• A Gentle Introduction to Concept Drift in Machine Learning
• Model Drift & Machine Learning: Concept Drift, Feature Drift, Etc.
• Data Drift vs. Concept Drift: What Is the Difference?

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It allows users to express streaming computations the same way as batch computations on static data, using DataFrame and Dataset APIs. Structured Streaming can handle both unbounded and bounded data sources, and can process data in micro-batches or continuously. Structured Streaming can be used to provide continuous processing of data for machine learning applications, such as data ingestion, preprocessing, feature engineering, and model inference. StructuredStreaming can also integrate with MLflow and Delta Lake to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance123 References:

• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks
• [Continuous Machine Learning with Structured Streaming and MLflow - Databricks]

Z-Ordering is an optimization technique that can speed up the query by colocating similar records while considering values in multiple columns. Z-Ordering is a way of organizing data in storage based on the values of one or more columns. Z-Ordering maps multidimensional data to one dimension while preserving locality of the data points. This means that rows with similar values for the specified columns are stored close together in the same set of files. This improves the performance of queries that filter on those columns, as they can skip over irrelevant files or data blocks. Z-Ordering also enhances data skipping and caching, as it reduces the number of distinct values per file for the chosen columns1. The other options are incorrect because:

• Option B: Bin-packing is an optimization technique that compacts small files into larger ones, but does not colocate similar records based on multiple columns. Bin-packing can improve the performance of queries by reducing the number of files that need to be read, but it does not affect the data layout within the files2.
• Option C: Writing as a Parquet file is not an optimization technique, but a file format choice. Parquet is a columnar storage format that supports efficient compression and encoding schemes. Parquet can improve the performance of queries by reducing the storage footprint and the amount of data transferred, but it does not colocate similar records based on multiple columns3.
• Option D: Data skipping is an optimization technique that skips over files or data blocks that do not match the query predicates, but does not colocate similar records based on multiple columns. Data skipping can improve the performance of queries by avoiding unnecessary data scans, but it depends on the data layout and the metadata collected for each file4.
• Option E: Tuning the file size is an optimization technique that adjusts the size of the data files to a target value, but does not colocate similar records based on multiple columns. Tuning the file size can improve the performance of queries by balancing the trade-off between parallelism and overhead, but it does not affectthe data layout within the files5. References: Z-Ordering (multi-dimensional clustering), Compaction (bin-packing), Parquet, Data skipping, Tuning file sizes

To create nested runs in MLflow, you need to specify the nested=True argument in the parent run, and omit it in the child runs. This will set a special tag (mlflow.utils.mlflow_tags.MLFLOW_PARENT_RUN_ID) in the child runs that indicates the parent run ID. This way, the UI will show the child runs under the parent run in a collapsible hierarchy. The indentation of the code blocks is not relevant, as long as the child runs are created within the context of the parent run. The run name parameter is also not relevant, as it does not affect the nesting of the runs. References: Understanding Parent and Child Runs in MLflow, Nested runs using MLflowClient, creating-child-runs-in-mlflow

To move a model version from the Staging stage to the Production stage using the MLflow Client, you can use the client.transition_model_version_stage method. This method takes the name of the registered model, the version of the model, and the target stage as arguments. It transitions the model version to the specified stage and returns the updated model version object. The syntax of the method is as follows:

Python

client.transition_model_version_stage (name, version, stage)

AI-generated code. Review and use carefully. More info on FAQ.

The other options are incorrect because:

• Option A: client.transition_model_version_stage with the stage argument set to “Staging” will not move the model version from the Staging stage to the Production stage, but rather keep it in the same stage.
• Option B: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage.
• Option D: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage. Also, the method does not take from and to arguments, but only a stage argument.
• Option E: client.transition_model_version_stage does not take from and to arguments, but only a stage argument. The from argument is redundant, as the method will automatically transition the model version from its current stage tothe target stage. References: MLflow Model Registry, MlflowClient.transition_model_version_stage

To view the data visualizations that are logged with each run, you can go to the Artifacts section of the MLflow Run page in Databricks. The Artifacts section shows the files and directories that are logged as artifacts for a run. You can browse the artifact hierarchy and preview the files, such as images, text, or HTML1. You can also download the artifacts or copy their URIs for further use2. The other options are incorrect because:

• Option A: The MLflow Model Registry Model page shows the information and metadata of a registered model, such as its name, description, versions, stages, and lineage. It does not show the data visualizations that are logged with each run3.
• Option B: The Artifacts section of the MLflow Experiment page shows the artifacts that are logged for an experiment, not for a specific run. It does not allow you to preview the files or browse the artifact hierarchy4.
• Option C: Logged data visualizations can be viewed in Databricks using the Artifacts section of the MLflow Run page1.
• Option E: There is no Figures section of the MLflow Run page in Databricks. The Figures section is only available in the open source MLflow UI, which shows the plots that are logged as figures for a run5. References: View run artifacts, Log, list, and download artifacts, Manage models, View experiment artifacts, Logging Visualizations with MLflow