# Introduction ## 🌰 In a nutshell: What is MLStacks? [MLStacks](https://mlstacks.zenml.io) is a [Python package](https://pypi.org/project/mlstacks/) that allows you to quickly spin up MLOps infrastructure using Terraform. It is designed to be used with [ZenML](https://zenml.io), but can be used with any MLOps tool or platform. Simply write stack and component YAML specification files and deploy them using the MLStacks CLI. MLStacks will take care of the rest. We currently support modular MLOps stacks on AWS, GCP and K3D (for local use). ## 👷 Why We Built MLStacks [![maintained-by-zenml](https://user-images.githubusercontent.com/3348134/173032050-ad923313-f2ce-4583-b27a-afcaa8b355e2.png)](https://github.com/zenml-io/zenml) When we first created [ZenML](https://zenml.io) as an extensible MLOps framework for creating portable, production-ready MLOps pipelines, we saw many of our users having to deal with the pain of deploying infrastructure from scratch to run these pipelines. The community consistently asked questions like: * How do I deploy tool X with tool Y? * Does a combination of tool X with Y make sense? * Isn't there an easy way to just try these stacks out to make an informed decision? To address these questions, the ZenML team presents you a series of Terraform-based stacks to quickly provision popular combinations of MLOps tools. These stacks will be useful for you if: * You are at the start of your MLOps journey, and would like to explore different tools. * You are looking for guidelines for production-grade deployments. * You would like to run your MLOps pipelines on your chosen [ZenML Stack](https://docs.zenml.io/user-guide/starter-guide/understand-stacks). 🔥 **Do you use these tools or do you want to add one to your MLOps stack?** At ZenML, we are looking for design partnerships and collaboration to implement and develop these MLOps stacks in a real-world setting. If you'd like to learn more, please [join our Slack](https://zenml.io/slack-invite/) and leave us a message! ## 🤓 Learn More * Try the [Quickstart example below](/getting-started/quickstart) to get started with MLStacks. * Discover what you can configure with the different stacks in the [Stacks documentation](/stacks/stack-specification). * Learn about our CLI commands in the [CLI documentation](/reference/cli). ## 🙏🏻 Acknowledgements Thank you to the folks over at [Fuzzy Labs](https://www.fuzzylabs.ai/) for their support and contributions to this repository. Also many thanks to [Ali Abbas Jaffri](https://github.com/aliabbasjaffri) for several stimulating discussions around the architecture of this project. We'd also like to acknowledge some of the cool inspirations for this project: * [FuseML](https://fuseml.github.io/) * [Combinator.ML](https://combinator.ml/) * [Building a ML Platform From Scratch](https://www.aporia.com/blog/building-an-ml-platform-from-scratch/) # Installation MLStacks is a Python package that can be installed using `pip`. It is recommended that you install MLStacks in a virtual environment. You can install MLStacks using the following command: ```bash pip install mlstacks ``` ## Other requirements and dependencies MLStacks uses Terraform on the backend to manage infrastructure. You will need to have Terraform installed. Please visit the [Terraform docs](https://learn.hashicorp.com/tutorials/terraform/install-cli#install-terraform) for installation instructions. MLStacks also uses Helm to deploy Kubernetes resources. You will need to have Helm installed. Please visit the [Helm docs](https://helm.sh/docs/intro/install/#from-script) for installation instructions. If you're using a Mac, you will need to [install `jq`](https://jqlang.github.io/jq/) in order for some of the Terraform deployment commands and scripts to work. If you want to use the `mlstacks breakdown` command to get cost estimates for your MLOps stacks, you'll need to also have `infracost` installed as well as to be logged in. Please visit the [Infracost docs](https://www.infracost.io/docs/) for installation instructions. ## Cloud provider installation MLStacks currently supports the following stack providers: * AWS * GCP * K3D If you wish to deploy using these providers you'll need to have accounts (for AWS and GCP) and the relevant CLIs installed and authenticated. You will also need to have the relevant permissions to deploy, manage and destroy resources in these accounts. Please refer to the documentation for those providers for more information. # Quickstart # AWS Quickstart This quickstart will guide you through deploying a simple stack on AWS using `mlstacks`. We'll be deploying a simple S3 bucket. This is as simple and quick an example of how `mlstacks` works as it gets. ## Prerequisites First, install the `mlstacks` CLI: ```bash pip install mlstacks ``` You'll need an active AWS account to get started. You will also need sufficient permissions to be able to create and destroy resources. If you don't have [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli#install-terraform) or [Helm](https://helm.sh/docs/intro/install/#from-script) installed, you should also install them. ## Defining our stack Then, create a file called `quickstart_stack.yaml` wherever you have access to the `mlstacks` tool. In this file, add the following: ```yaml spec_version: 1 spec_type: stack name: "quickstart_stack" provider: aws default_region: "eu-north-1" default_tags: deployed-by: "mlstacks" components: - simple_component_s3.yaml ``` This defines our stack using the `mlstacks` specification. We'll now define the component that we want to deploy in a separate file called `simple_component_s3.yaml`: ```yaml spec_version: 1 spec_type: component component_type: "artifact_store" component_flavor: "s3" name: "quickstart_s3_bucket" provider: aws metadata: config: bucket_name: "quickstart-s3-bucket" tags: deployed-by: "mlstacks" region: "eu-north-1" ``` ## Deploying our stack Now, we can deploy our stack using the `mlstacks` CLI: ```bash mlstacks deploy -f quickstart_stack.yaml ``` This will deploy our stack to AWS. It will also deploy/provision an S3 bucket (beginning with `zenml-mlstacks-remote-state` by default) which will be used as a remote state store and backend for your Terraform assets. This will happen first before the deployment of your stack. You can now check your AWS console to see that the stack (and remote state bucket) has been deployed. ## Get stack outputs You can get the outputs of your stack using the `mlstacks` CLI: ```bash mlstacks output -f quickstart_stack.yaml ``` This will print out the outputs of your stack, which you can use in your pipelines. ## Destroying our stack Finally, we can destroy our stack (and the remote state S3 bucket) using the `mlstacks` CLI: ```bash mlstacks destroy -f quickstart_stack.yaml ``` ## What next? You can now try adding more components and deploying them to your cloud provider. You can also try deploying your stack to a different cloud provider. Good luck! And if you have any questions, feel free to [reach out to us on Slack](https://www.zenml.io/slack-invite) # GCP Quickstart This quickstart will guide you through deploying a simple stack on GCP using `mlstacks`. We'll be deploying a simple storage bucket. This is as simple and quick an example of how `mlstacks` works as it gets. ## Prerequisites First, install the `mlstacks` CLI: ```bash pip install mlstacks ``` You'll need an active GCP account and project to get started. (If you don't have one, you can create one [following these instructions](https://developers.google.com/workspace/guides/create-project). You will also need sufficient permissions to be able to create and destroy resources. If you don't have [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli#install-terraform) or [Helm](https://helm.sh/docs/intro/install/#from-script) installed, you should also install them. ## Defining our stack Then, create a file called `quickstart_stack.yaml` wherever you have access to the `mlstacks` tool. In this file, add the following: ```yaml spec_version: 1 spec_type: stack name: "quickstart_stack" provider: gcp default_region: "europe-north1" default_tags: deployed-by: "mlstacks" components: - simple_component_gcs.yaml ``` This defines our stack using the `mlstacks` specification. We'll now define the component that we want to deploy in a separate file called `simple_component_gcs.yaml`: ```yaml spec_version: 1 spec_type: component component_type: "artifact_store" component_flavor: "gcp" name: "quickstart_gcs_bucket" provider: gcp metadata: config: bucket_name: "quickstart_gcs_bucket" project_id: "" tags: deployed-by: "mlstacks" region: "eu-north1" ``` ## Deploying our stack Now, we can deploy our stack using the `mlstacks` CLI: ```bash mlstacks deploy -f quickstart_stack.yaml ``` This will deploy our stack to GCP. It will also deploy/provision a GCS bucket (beginning with `zenml-mlstacks-remote-state` by default) which will be used as a remote state store and backend for your Terraform assets. This will happen first before the deployment of your stack. You can now check your GCP console to see that the stack (and remote state bucket) has been deployed. ## Get stack outputs You can get the outputs of your stack using the `mlstacks` CLI: ```bash mlstacks output -f quickstart_stack.yaml ``` This will print out the outputs of your stack, which you can use in your pipelines. ## Destroying our stack Finally, we can destroy our stack (and the remote state GCS bucket) using the `mlstacks` CLI: ```bash mlstacks destroy -f quickstart_stack.yaml ``` ## What next? You can now try adding more components and deploying them to your cloud provider. You can also try deploying your stack to a different cloud provider. Good luck! And if you have any questions, feel free to [reach out to us on Slack](https://www.zenml.io/slack-invite) # K3D Quickstart This quickstart will guide you through deploying a simple stack using K3D to simulate a cloud provider using `mlstacks`. We'll be deploying a simple minio bucket. This is as simple and quick an example of how `mlstacks` works as it gets. ## Prerequisites First, install the `mlstacks` CLI: ```bash pip install mlstacks ``` You will need to have K3D installed. Please visit the [K3D docs](https://k3d.io/) for installation instructions. If you don't have [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli#install-terraform) or [Helm](https://helm.sh/docs/intro/install/#from-script) installed, you should also install them. ## Defining our stack Then, create a file called `quickstart_stack.yaml` wherever you have access to the `mlstacks` tool. In this file, add the following: ```yaml spec_version: 1 spec_type: stack name: "quickstart_stack" provider: k3d default_region: "local" default_tags: deployed-by: "mlstacks" components: - simple_component_minio.yaml ``` This defines our stack using the `mlstacks` specification. We'll now define the component that we want to deploy in a separate file called `simple_component_minio.yaml`: ```yaml spec_version: 1 spec_type: component component_type: "artifact_store" component_flavor: "minio" name: "quickstart_minio_bucket" provider: k3d metadata: config: bucket_name: "quickstart_minio_bucket" tags: deployed-by: "mlstacks" region: "local" ``` ## Deploying our stack Now, we can deploy our stack using the `mlstacks` CLI: ```bash mlstacks deploy -f quickstart_stack.yaml ``` This will deploy our stack to a local K3D cluster. You can now check your K3D console to see that the stack and the minio bucket has been deployed. ## Get stack outputs You can get the outputs of your stack using the `mlstacks` CLI: ```bash mlstacks output -f quickstart_stack.yaml ``` This will print out the outputs of your stack, which you can use in your pipelines. ## Destroying our stack Finally, we can destroy our stack using the `mlstacks` CLI: ```bash mlstacks destroy -f quickstart_stack.yaml ``` ## What next? You can now try adding more components and deploying them to this K3D provider. You can also try deploying your stack to an actual cloud provider instead of this local environment. Good luck! And if you have any questions, feel free to [reach out to us on Slack](https://www.zenml.io/slack-invite) # Core Concepts MLStacks is built around common concepts that are used to describe infrastructure for machine learning and MLOps. This section will introduce you to these concepts and how they are used in MLStacks. ## What's a stack? A Stack is a collection of stack components, where each component represents the respective configuration regarding a particular function in your MLOps pipeline such as orchestration systems, artifact repositories, and model deployment platforms. As a shorthand, you can think of a stack as a grouping of these components. ## What's a component? Components are the building-blocks of stacks. MLStacks currently supports the following stack components: * `artifact_store`: An [artifact store](https://docs.zenml.io/stacks-and-components/component-guide/artifact-stores) is a component that can be used to store artifacts. (e.g. S3 buckets on AWS) * `container_registry`: A [container registry](https://docs.zenml.io/stacks-and-components/component-guide/container-registries) is a component that can be used to store container images. (e.g. ECR on AWS) * `experiment_tracker`: An [experiment tracker](https://docs.zenml.io/stacks-and-components/component-guide/experiment-trackers) is a component that can be used to track experiments, including metrics, parameters, and artifacts. (e.g. MLFlow) * `orchestrator`: An [orchestrator](https://docs.zenml.io/stacks-and-components/component-guide/orchestrators) is a component that can be used to orchestrate machine learning pipelines. (e.g. Airflow) * `mlops_platform`: An MLOps platform is a component that can be used to deploy, monitor, and manage machine learning models. (e.g. ZenML) * `model_deployer`: A [model deployer](https://docs.zenml.io/stacks-and-components/component-guide/model-deployers) is a component that can be used to deploy machine learning models. (e.g. Seldon Core) * `step_operator`: A [step operator](https://docs.zenml.io/stacks-and-components/component-guide/step-operators) is a component that can be used to execute steps that require custom hardware. ## How does MLStacks work? MLStacks is built around the concept of a stack specification. A stack specification is a YAML file that describes the stack and includes references to component specification files. A component specification is a YAML file that describes a component. (Currently all deployments of components (in various combinations) must be defined within the context of a stack.) Once you write your stack specification, you can then use MLStacks' CLI to deploy your stack to your preferred cloud (or local K3d) provider. Terraform definitions are stored in your global configuration directory. MLStacks allows you to deploy or connect to a remote state store (e.g. S3, GCS, etc.) so that [you can collaborate](/reference/cli#using-remote-state-with-a-team) on your stacks and deployed infrastructure with your colleagues. Your global configuration directory could be in a number of different places depending on your operating system, but read more about it in the [Click docs](https://click.palletsprojects.com/en/8.1.x/api/#click.get_app_dir) to see which location applies to your situation. This is where the stack specs and the Terraform definition files are located. # Writing Spec Files ## Stack specification The core of a stack is the `stack.yaml` file. This file contains all the information needed to deploy a stack. It contains the following fields: ```yaml spec_version: 1 spec_type: stack name: provider: default_region: default_tags: - : components: - ``` Let's go through each of these fields in detail. ### `spec_version` This field defines the version of the `mlstacks` specification that this stack uses. This is currently `1` and is set as the default. ### `spec_type` This field defines the type of the specification. This is currently `stack`. ### `name` This field defines the name of the stack. This is used to identify the stack when deploying, destroying, or getting outputs from the stack. ### `provider` This field defines the provider that the stack will be deployed to. This is currently one of `k3d`, `gcp`, or `aws`. ### `default_region` This field defines the default region that the stack will be deployed to. If you specify a region that doesn't exist for your particular provider, the stack deployment will fail. If you don't specify a region in your stack specification, `mlstacks` will use whatever is set as the default region for your provider. Note that this will differ between providers. ### `default_tags` This field defines the default tags that will be applied to all resources created by the stack. This is useful for identifying resources created by the stack. This is an optional field. ### `components` This field defines the components that will be deployed by the stack. This is a list of component filenames. ## Component specification The core of a component is the `component.yaml` file. This file contains all the information needed to deploy a component. It contains the following fields: ```yaml spec_version: 1 spec_type: component component_type: component_flavor: name: provider: metadata: config: : tags: : region: ``` Let's go through each of these fields in detail. ### `spec_version` This field defines the version of the `mlstacks` specification that this component uses. This is currently `1`. This is set as the default. ### `spec_type` This field defines the type of the specification. This is currently `component`. ### `component_type` This field defines the type of the component. Available component types currently include: * `artifact_store`: An artifact store is a component that can be used to store artifacts. * `container_registry`: A container registry is a component that can be used to store container images. * `experiment_tracker`: An experiment tracker is a component that can be used to track experiments. * `orchestrator`: An orchestrator is a component that can be used to orchestrate pipelines. * `mlops_platform`: An MLOps platform is a component that can be used to orchestrate pipelines, track experiments, and manage the overall connection of MLOps components and tools together. * `model_deployer`: A model deployer is a component that can be used to deploy models. * `step_operator`: A step operator is a component that can be used to execute steps in a pipeline using custom hardware or platforms. ### `component_flavor` This field defines the flavor of the component. This is used to differentiate between different implementations of the same component type. For example, the `artifact_store` component type has the following flavors: * `minio`: A MinIO artifact store. * `s3`: An S3 artifact store. * `gcp`: A GCP/GCS artifact store. ### `name` This field defines the name of the component. This is used to identify the component when deploying, destroying, or getting outputs from the component. ### `provider` This field defines the provider that the component will be deployed to. This is currently one of `k3d`, `gcp`, or `aws`. ### `metadata` This field defines the metadata of the component. This is an (optional) dictionary with the following fields: #### `config` This field defines the configuration of the component. This is a dictionary you can pass in arbitrary fields to configure the component. For example for an artifact store, as shown [in the quickstart examples](/getting-started/quickstart/gcp), you can pass in a `bucket_name` field to configure the bucket name of the artifact store. Config is usually optional (except in the case of GCP deployments when you need to specify a `project_id`.) #### `environment_variables` This field defines the environment variables of the component. This is a dictionary you can pass in arbitrary fields to configure the component. For example you might want certain environment variables to be set and defined ahead of the deployment of certain components. Environment variables are optional. # AWS The AWS Modular recipe is available in the `mlstacks` repository and you can [view the raw Terraform files here](https://github.com/zenml-io/mlstacks/tree/main/src/mlstacks/terraform/aws-modular). A full list of supported components and flavors can be found in the [Supported Components and Flavors](#supported-components-and-flavors) section, as can a list of components that are coming soon. ## Supported components and flavors | Component | Flavor(s) | | ------------------ | ------------------------------------------------- | | Artifact Store | s3 | | Container Registry | aws | | Experiment Tracker | mlflow | | Orchestrator | kubeflow, kubernetes, sagemaker, skypilot, tekton | | MLOps Platform | zenml | | Model Deployer | seldon | | Step Operator | sagemaker | ## Coming Soon! * Airflow Orchestrator on AWS * Feast Feature Store on AWS * Label Studio Annotator on AWS * Model Registry components on AWS * Image Builder components on AWS # GCP The GCP Modular recipe is available in the `mlstacks` repository and you can [view the raw Terraform files here](https://github.com/zenml-io/mlstacks/tree/main/src/mlstacks/terraform/gcp-modular). A full list of supported components and flavors can be found in the [Supported Components and Flavors](#supported-components-and-flavors) section, as can a list of components that are coming soon. ## Important Notes for GCP Deployments All GCP deployments require the inclusion of a GCP Project ID in the metadata's config for each component. This is because GCP resources are tied to a project and cannot be created without one. ## Supported components and flavors | Component | Flavor(s) | | ------------------ | ---------------------------------------------- | | Artifact Store | gcp | | Container Registry | gcp | | Experiment Tracker | mlflow | | Orchestrator | kubeflow, kubernetes, skypilot, tekton, vertex | | MLOps Platform | zenml | | Model Deployer | seldon | | Step Operator | vertex | ## Coming Soon! * Airflow Orchestrator on GCP * Feast Feature Store on GCP * Label Studio Annotator on GCP * Model Registry components on GCP * Image Builder components on GCP # Azure MLStacks doesn't support Azure Cloud yet. We are working on adding support for modular stacks on Azure. In the meanwhile, be sure to check out [`matcha` from FuzzyLabs](https://github.com/fuzzylabs/matcha) which caters to deployments using Azure. If you'd like to contribute, please [join our Slack](https://zenml.io/slack-invite/) and leave us a message or [open an issue](https://github.com/zenml-io/mlstacks/issues/new) in the repository! # K3D The K3D Modular recipe is available in the `mlstacks` repository and you can [view the raw Terraform files here](https://github.com/zenml-io/mlstacks/tree/main/src/mlstacks/terraform/k3d-modular). A full list of supported components and flavors can be found in the [Supported Components and Flavors](#supported-components-and-flavors) section, as can a list of components that are coming soon. ## Supported components and flavors | Component | Flavor(s) | | ------------------ | --------------------------------------- | | Artifact Store | minio | | Container Registry | default | | Experiment Tracker | mlflow | | Orchestrator | kubeflow, kubernetes, sagemaker, tekton | | MLOps Platform | zenml | | Model Deployer | seldon | ## Coming Soon! * Airflow Orchestrator on AWS * Feast Feature Store on AWS * Label Studio Annotator on AWS * Model Registry components on AWS * Image Builder components on AWS # CLI MLStacks is a CLI tool that allows you to deploy and manage your ML infrastructure using the MLStacks specification. You can install the CLI using the following command: ```bash pip install mlstacks ``` ## Deploying a stack You can deploy a stack using the `mlstacks deploy` command. This command takes a path to a stack specification file as an argument. For example, if you have a stack specification file called `stack.yaml`, you can deploy it using the following command: ```bash mlstacks deploy -f stack.yaml ``` If you want to drop into the internal Terraform log messages and prompts, turn on debug mode with the `-d` or `--debug` flag: ```bash mlstacks deploy -f stack.yaml -d ``` ## Using remote state with a team MLStacks deploys a remote state bucket to the same cloud provider as you're using for your stack by default. This remote state backend has a default name that begins with `zenml-mlstacks-remote-state` and is deployed first before your stack gets deployed. If you'd like to connect to a pre-existing state bucket that you or a colleague have already created, you can do so by passing the bucket name to the `mlstacks deploy` command: ```bash mlstacks deploy -f stack.yaml -rb # e.g. mlstacks deploy -f stack.yaml -rb s3://zenml-mlstacks-remote-state-3d3r6 ``` This will then connect to the remote state bucket and use that as the backend for your stack deployment. ## Getting stack outputs Once you have a stack deployed, you can get the outputs of the stack using the `mlstacks output` command. This command takes a path to a stack specification file as an argument. For example, if you have a stack specification file called `stack.yaml`, you can get the outputs of the stack using the following command: ```bash mlstacks output -f stack.yaml ``` This will print out the outputs of the stack, which you can use in your pipelines. If you just want a single output you can add the `-k` or `--key` option and pass in the name of the output you want: ```bash mlstacks output -f stack.yaml -k my_key ``` ## Destroying a stack You can destroy a stack using the `mlstacks destroy` command. This command takes a path to a stack specification file as an argument. For example, if you have a stack specification file called `stack.yaml`, you can destroy it using the following command: ```bash mlstacks destroy -f stack.yaml ``` If you want to drop into the internal Terraform log messages and prompts, turn on debug mode with the `-d` or `--debug` flag: ```bash mlstacks destroy -f stack.yaml -d ``` ## Stack Cost Estimation with Infracost MLStacks integrates with [Infracost](https://www.infracost.io/) to provide cost estimates for your stacks. You can install Infracost by following the instructions in their documentation. Note that you'll need to be logged in to use it with `mlstacks`. Once you have Infracost installed, you can get a cost estimate for your stack using the `mlstacks breakdown` command. This command takes a path to a stack specification file as an argument. For example, if you have a stack specification file called `stack.yaml`, you can get a cost estimate for it using the following command: ```bash mlstacks breakdown -f stack.yaml ``` This will print out a cost estimate for your stack. ## Viewing Terraform definitions and Stack Specifications If you'd like to view the Terraform definitions that MLStacks generates for your stack, you can use the `mlstacks source` command. This command will print out the location of the Terraform definitions for your stack. ```bash mlstacks source ``` ## Cleaning Up If you want to clean up all the files and directories created by MLStacks, you can use the `mlstacks clean` command. This works at a global level (i.e. affecting all stacks), so you don't need to pass in a stack specification file. ```bash mlstacks clean ``` ## Check your mlstacks version To see what version of `mlstacks` package you're using, please use the following command: ```bash mlstacks version ``` # Terraform usage MLStacks uses Terraform under the hood to deploy and destroy the infrastructure that you specify in your stack specification files. We specifically designed the interface to conceal the Terraform implementation details from you, but if you want to use Terraform directly, you can do so. ## Where are the Terraform files stored? You can download our modular recipes by cloning our GitHub repository: ```bash git clone https://github.com/zenml-io/mlstacks.git ``` The specific directory you want to look at is `src/mlstacks/terraform`. ## Terraform next steps If you want to use Terraform directly, you can simply navigate to the root of one of the `xxx-modular` directories and run (for example) `terraform init` to initialize the Terraform directory. You can then run `terraform plan` to see what Terraform will do, and `terraform apply` to apply the changes. You are free to remix and use the Terraform modules and recipes as you see fit, but please note that this is not a core use case for MLStacks and you might no longer be able to use the MLStacks CLI to manage your stacks any more. # ZenML & MLStacks [![maintained-by-zenml](https://user-images.githubusercontent.com/3348134/173032050-ad923313-f2ce-4583-b27a-afcaa8b355e2.png)](https://github.com/zenml-io/zenml) It is not necessary to use the MLOps stacks recipes presented here alongside the [ZenML](https://github.com/zenml-io/zenml) framework, but it is highly recommended to do so. The ZenML framework is designed to be used with these recipes, and the recipes are designed to be used with ZenML. The ZenML CLI has an integration with this package that makes it really simple to use and deploy these recipes. For more information, [visit the ZenML documentation](https://docs.zenml.io/stacks-and-components/stack-deployment) for more but a quick example is shown below. ```shell # after installing ZenML zenml stack deploy -p gcp -a -n basic -r us-east1 -t env=dev -x bucket_name=my-new-bucket -x project_id=zenml ``` This command will deploy a GCP artifact store to `us-east1` region with a specific bucket name, project ID and tag, for example. To learn more about ZenML and how it empowers you to develop a stack-agnostic MLOps solution, head over to the [ZenML docs](https://docs.zenml.io). ## Importing `mlstacks` stacks into ZenML The ZenML CLI also has a command to import stacks created with `mlstacks` into ZenML. All stacks created with `mlstacks` generate a `.yaml` file that can be imported into ZenML with the following command: ```shell # after installing ZenML zenml stack import -f ``` The path of the stack file can be found by navigating to the directory containing all the Terraform source files. You can easily find this by running the following command: ```shell mlstacks source ``` This will print the path to the directory containing all the Terraform source and will ask you if you want to open the directory in your default file explorer. You can then navigate to the `.yaml` file and use that path to import it into ZenML as described above. # Debugging Tips These are some known problems that might arise out of running `mlstacks`. Errors for mlstacks deployments are usually related to changes that you might have made independently of the original recipes or they might also relate to network or permissions issues. Usually the quickest way to start afresh is to run `mlstacks clean`, but note that this will also delete deployments that you might have made using `mlstacks`. You can also try to debug the problem by running the terraform commands from within the `mlstacks` config directory where the Terraform definition files are stored. You can also run the `mlstacks` commands with the `--debug` flag to get more information and decision points along the way. ## Other known problems These are issues that sometimes get raised in the underlying Terraform implementation: * Running a Kubernetes-based deployment for the first time might result in an error with one of the resources - the Istio Ingressway. This is because of a limitation with the resource `kubectl_manifest` that needs the cluster to be set up before it installs its own resources. 💡 Fix - Run `terraform apply` again in a few minutes and this should get resolved. * When executing terraform commands, an error like this one: `timeout while waiting for plugin to start`\ 💡 Fix - If you encounter this error with `apply`, `plan` or `destroy`, do `terraform init` and run your command again. * While running `terraform init`, an error which says `Failed to query available provider packages... No available releases match the given constraint`\ 💡 Fix - First of all, you should create an issue so that we can take a look. Meanwhile, if you know Terraform, make sure all the modules that are being used are on their latest version. * While running a Terraform command, this error might appear too: `context deadline exceeded`\ 💡 Fix - This problem could arise due to strained system resources. Try running the command again after some time. # Analytics In order to help us better understand how the community uses ZenML, the pip package reports anonymized usage statistics. You can always opt out by setting the `MLSTACKS_ANALYTICS_OPT_IN` environment variable to `False`: ```bash export MLSTACKS_ANALYTICS_OPT_IN=False ``` ## Why does MLStacks collect analytics? In addition to the community at large, MLStacks is created and maintained by a startup based in Munich, Germany called [ZenML GmbH](https://zenml.io/). We're a team of techies that love MLOps and want to build tools that fellow developers would love to use in their daily work. [This is us](https://zenml.io/company#CompanyTeam) if you want to put faces to the names! However, in order to improve MLStacks and understand how it is being used, we use analytics to have an overview of how it is used 'in the wild'. This not only helps us find bugs but also helps us prioritize features and commands that might be useful in future releases. If we did not have this information, all we really get is `pip` download statistics and chatting with people directly, which while being valuable, is not enough to seriously better the tool as a whole. ## How does MLStacks and ZenML collect these statistics? MLStacks uses [Segment](https://segment.com/) as the data aggregation library for all our analytics. The entire code is entirely visible and can be seen at [client.py](https://github.com/zenml-io/mlstacks/blob/main/src/mlstacks/analytics/client.py). None of the data sent can identify you individually but allows us to understand how MLStacks is being used holistically. # FAQ ## What are the benefits of using mlstacks? MLStacks is a tool for deploying infrastructure to cloud providers. It is designed to make it easy to deploy and manage infrastructure for machine learning. It is built on top of [Terraform](https://www.terraform.io/), which means that it is cloud-agnostic and can be used to deploy infrastructure to any cloud provider that Terraform supports. MLStacks is designed and developed by a team that live and breathe MLOps. This means that it is designed to support the full range of infrastructure that you might need for your MLOps tooling. It is also designed to be modular, which means that you can easily mix and match different components to create the infrastructure that you need. ## What are the tradeoffs of using mlstacks? MLStacks is currently a project in beta. This means that it is still under active development and may have some rough edges. We are working hard to make it production-ready as soon as possible, but in the meantime, you may encounter some bugs or missing features. In particular, not all cloud providers and stack components are supported out of the box with the modular recipes that come with MLStacks. If you want to deploy to a cloud provider or use a stack component that is not supported, you will need to write your own recipe. We are working hard to add support for more cloud providers and stack components, but in the meantime, you can use the existing recipes as a starting point for writing your own. ## What are the alternatives to mlstacks? There are lots of ways to deploy infrastructure to cloud providers that span the full spectrum from manual to automated. MLStacks uses [Terraform](https://www.terraform.io/) as its backend for configuring and deploying infrastructure, but there are other tools that can help with this like [Pulumi](https://www.pulumi.com/) or cloud-specific tools like [AWS CloudFormation](https://aws.amazon.com/cloudformation). ## What's the connection between mlstacks and ZenML? MLStacks is developed and maintained by the core ZenML team. It is designed to work (well) with ZenML, but it can also be used independently of ZenML. ## Can I use mlstacks independently of ZenML? Yes! You can use MLStacks to deploy infrastructure for any MLOps tooling you like and it is designed to offer a range of components and flavors to support the full variety of MLOps tools. ## How do I use mlstacks within a team setting? MLStacks is designed to be used in a team setting. It is designed to support the full range of infrastructure that you might need for your MLOps tooling. We also spin up a remote state backend with every deployment (unless you're connecting to one that already exists) so that other team members can collaborate on your stacks and deployed infrastructure. Please see the section of the docs on [using remote state](/reference/cli#using-remote-state) for more information. # Contributing If you want to don the chef's hat and create a new recipe to cover your specific use case, we have just the ingredients you need! * Learn more about the design principles behind a recipe, and more information on testing in the [CONTRIBUTING.md guide](https://github.com/zenml-io/mlstacks/blob/develop/CONTRIBUTING.md)