Open source research and development in distributed systems

Installing SimKube #

This guide will walk you through installing the various SimKube components

Prerequisites #

The following prereqs are required for all components:

Rust (including Cargo) >= 1.79
Docker
kubectl >= 1.27
a Kubernetes cluster running at least v1.27

Additional prerequisites are necessary for your simulation cluster:

KWOK >= 0.4.0
CertManager for setting up mutating webhook certificates

Optional Prerequisites #

If you want to run SimKube on a local development cluster, kind >= 0.19 is the supported tooling for doing so.

If you want to test autoscaling, SimKube currently supports either the Kubernetes Cluster Autoscaler or Karpenter. You will need to install and configure these applications to use the corresponding KWOK provider. For the Kubernetes Cluster Autoscaler, a KWOK cloud provider is available, and for Karpenter, a basic KWOK provider is used. See Autoscaling for more information on configuring these tools.

Configuring your simulation cluster #

This section explains how to create a kind cluster on your local machine for running simulations. If you have a pre-existing Kubernetes cluster that you will be using for your simulation environment, you can skip this step.

Setting up kind #

From the kind website:

kind is a tool for running local Kubernetes clusters using Docker container “nodes”. kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.

You'll want to create a kind cluster with at least two nodes. Here's an example configuration:

# kind.yml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    labels:
      type: kind-control-plane
  - role: worker
    labels:
      type: kind-worker

If you are pushing the SimKube docker images to a local docker registry, you will additionally need to follow the steps in Create A Cluster and Registry to enable kind to access your images.

Create the cluster by running

> kind create cluster --name simkube --config kind.yml

Install Required Dependencies #

KWOK:

> KWOK_REPO=kubernetes-sigs/kwok
> KWOK_LATEST_RELEASE=$(curl "https://api.github.com/repos/${KWOK_REPO}/releases/latest" | jq -r '.tag_name')
> kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/kwok.yaml"
> kubectl apply -f "https://github.com/${KWOK_REPO}/releases/download/${KWOK_LATEST_RELEASE}/stage-fast.yaml"

Prometheus Operator:

> git clone https://github.com/prometheus-operator/kube-prometheus.git
> cd kube-prometheus
> kubectl create -f manifests/setup
> until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done
No resources found  # this message is expected
> kubectl create -f manifests/

cert-manager:

> kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.3/cert-manager.yaml
> kubectl wait --for=condition=Ready -l app=webhook -n cert-manager pod --timeout=60s

We're going to use self-signed certificates for this, so apply the following file to your cluster:

# self-signed.yml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned
  namespace: kube-system
spec:
  selfSigned: {}

> kubectl apply -f self-signed.yml

Installing SimKube using pre-built images #

SimKube images are hosted on quay.io; the easiest way to install and run SimKube in your cluster is to use these images along with the provided kustomize YAML files in k8s/kustomize:

git clone https://github.com/acrlabs/simkube && cd simkube

export PROD_CONTEXT=<your production cluster context name>
export SIM_CONTEXT=<your simulation environment context name>

# install sk-tracer in your production cluster
kubectl --context ${PROD_CONTEXT} apply -k k8s/kustomize/prod

# install sk-ctrl in your simulation environment
kubectl --context ${SIM_CONTEXT} apply -k k8s/kustomize/sim

You should now see the SimKube pods running in your cluster:

> kubectl --context ${PROD_CONTEXT} get pods -n simkube
NAMESPACE   NAME                              READY   STATUS      RESTARTS   AGE
simkube     sk-tracer-depl-74546ccb48-5gmbc   1/1     Running     0          11h
> kubectl --context ${SIM_CONTEXT} get pods -n simkube
NAMESPACE   NAME                              READY   STATUS      RESTARTS   AGE
simkube     sk-ctrl-depl-b6fbb7744-l8bwm      1/1     Running     0          11h

You'll need to also install skctl to start or interact with simulations; skctl is available on crates.io and you can install it with:

cargo install skctl

You can test if it worked by running skctl version (make sure that your Cargo bin directory is on your $PATH, e.g., echo "export ${CARGO_HOME}/bin:${PATH}" >> ~/.zshrc):

> skctl version
skctl 1.0.1

Now you should be able to run your first simulation!