Written by Jack Jia, Software Developer at IBM, dedicated to Zowe
This article originally ran on the Zowe Medium site.
If you have read our other blog about Open Mainframe Project’s Zowe Kubernetes Overview, you may have captured a general idea of what is the new Zowe Kubernetes deployment, and why Zowe provides it. In this blog, we would like to dive into the technical details of this new deployment and run a step-by-step exercise to deploy Zowe to Google Cloud Kubernetes Engine.
You will need a basic understanding of how Zowe and Kubernetes work before continuing. To learn more about Zowe, zowe.org is a recommended place to start with. If you want to know more about Kubernetes, kubernetes.io will be the best place.
Architecture view of Zowe Kubernetes deployment
Compared to deploying Zowe to z/OS system, as described in Zowe architecture, the following diagram depicts the difference in locations of Zowe components when deploying Zowe into Kubernetes cluster as opposed to running all components on a single z/OS system.
- Zowe System Services Server (aka
ZSS) and Cross Memory Server (aka
ZIS) are remaining on the z/OS side to communicate with z/OS native functions. Besides that, you have a choice to deploy all other components in Kubernetes.
- You can deploy Zowe to Kubernetes on any platform like,
- All Zowe Kubernetes workloads are running on a dedicated namespace (
zoweby default) to distinguish from other workloads in the same Kubernetes cluster.
- Zowe has its own
ServiceAccountto help with managing permissions.
- Server components use similar
zowe.yamlon z/OS, which are stored in
Secret, to configure and start.
- Server components can be configured using the same certificates used on z/OS components.
- Zowe claims its own
Persistent Volumeto share files across components.
- Each server component is running in separated containers and can scale independently based on its own resource consumption.
- Components may register themselves to APIML Discovery with their own
Podname within the cluster.
- Zowe workloads use
initContainersstep to prepare required runtime directories.
- Only necessary components ports are exposed outside of Kubernetes with
Service(or in combination with
When deploying Zowe server components into container orchestration software like Kubernetes, we follow standard Kubernetes practices and the cluster can be monitored and managed with common Kubernetes administration methods.
For each Zowe core component image, we provide 2×2 flavors to cover:
- base image of latest Ubuntu and Red Hat Universal Base Image,
- CPU architecture of
amd64(for x86 based CPU) and
s390x(for Linux on Z mainframe).
Currently, our images are shipped in
zowe-docker-release.jfrog.io registry, and you can find all images we released from https://zowe.jfrog.io/ui/repos/tree/General/docker-release%2Fompzowe.
For each image, we also release and update manifests to help you pick the right level of image based on your requirement. Typical image manifests we provide are:
For example, if you would like to automatically pick the most recent changes from v1 releases, you can use images like
These manifests covers both
s390x CPU architecture, so it will work no matter if you are running it in a distributed environment (typically x86 based CPU) or running on Linux on a Z mainframe.
In-depth details of Zowe Kubernetes deployment
You can deploy Zowe to Kubernetes v1.19+ or compatible OpenShift v4.6+.
Depending on your Kubernetes or OpenShift setup, some of the configurations you may need to customize for your environment, especially for Persistent Volume, Networking, and your domain name to access Kubernetes and z/OS.
Here is a list of example Kubernetes manifests we provide you can customize based upon.
Namespaceis a good start point to isolate your workload from others in the same cluster.
zowe-sais defined to help you customize the permission setup.
- Examples of defining
PersistentVolumeClaim. This persistent volume helps Zowe to share data between components. For example, one Desktop plugin may need to share its runtime to AppServer, or a static registered component may need to share its registration file to Discovery service. To define a
PersistentVolumeClaim, you need to find out the appropriate
StorageClassyou can use across all possible Kubernetes nodes and support
- Example of
Secret. These configurations are similar to the
zowe.yamlyou are using to start Zowe on z/OS. Zowe also ships a utility tools
<instance>/bin/utils/convert-fork8s.shto help you migrate the configurations and certificates from z/OS to Kubernetes.
- Example definitions of Zowe APIML API Catalog, Discovery and Gateway
gateway-serviceis the main service that must be exposed outside of Kubernetes and serves as entry point of Zowe. We also provide sample
Ingresswhich may be needed in some setup. These networking definitions are very attached to your Kubernetes flavor like how your load balancer, Ingress controller is set up, what are the domain and port you would like to use, etc.
Deploymentsmatch all other Zowe core components.
CronJobis predefined to help Zowe cleanup abandoned static definitions.
PodDisruptionBudgets can help you maintain the availability of services.
HorizontalPodAutoscalers can help you scale Zowe based on CPU consumption of your workload.
We try to decouple Zowe runtime scripts from component runtime code so they can build and release independently. Zowe components, as well as extension images, have no requirement to contain Zowe runtime scripts. The Zowe context will be prepared when we start the component inside a Zowe cluster.
This picture illustrates directory changes when a component container is running in a Zowe cluster.
Security is always a top priority of running Zowe on any platform. Zowe follows Kubernetes common best practices and also Kubernetes Hardening Guidance from National Security Agency and Cybersecurity and Infrastructure Security Agency.
- Zowe conformant images must define
zoweuser and group other than using
- Zowe prohibits starting component container with root user.
- Zowe example Deployments has predefined
securityContextto limit runtime user, group, and file system group to
- Zowe launch script is started as PID 1 and will handle the SIGTERM signal properly by propagating to all child processes.
- By default Zowe example Deployments also enabled
securityContextto protect the root file system from overwriting.
- Zowe core components are regularly scanned with security tools.
- Zowe workloads are isolated with Namespace and ServiceAccount.
- Zowe configurations and certificates can be securely stored in
Secret, or potentially Vault to protect from unexpected access.
- You can still enable the strict
VERIFY_CERTIFICATESoption to achieve the best security like running Zowe on z/OS.
Exercise deployment to Google Cloud
With the above knowledge, let’s try to deploy Zowe into Google Cloud Kubernetes Engine. Hopefully, after this section, you may have a better understanding of how Zowe can be deployed and what are necessary customization may be required.
You will need a Google Cloud account, proper billing setup, Google Cloud CLI, and
kubectl installed on your local computer. This step may cost you around $1 or $2 depends on how long you let your cluster runs.
- Visit https://console.cloud.google.com/ and create your own project.
- Choose Kubernetes Engine — Clusters from GCP menus. If you have never used this feature before, GCP may ask you to enable the feature before continuing.
- Click on “Create” from the interface.
- Select either “GKE Standard” or “GKE Autopilot”. I will choose “GKE Standard” for this demo.
- Provide a name for your cluster, customize a zone close to you, then click on “CREATE” button. Then let’s wait for your cluster to be ready. This may usually take 2–4 minutes.
- Select the new cluster you just created, click on “CONNECT” button. This interface provides commands you need to prepare Kubernetes config context on your local computer.
- Copy the command and run it on your local computer Terminal.
- Now we can start to follow Zowe documentation to deploy Zowe. First, let’s create Namespace and ServiceAccount:
$ kubectl apply -f common/ namespace/zowe created serviceaccount/zowe-sa created
- a) Visit https://console.cloud.google.com/filestore/instances to create an instance. In this example, I will call the
File share nameas
zowe_workspace. All other options I choose to default, but customized to the zone I picked for my cluster. Click on “CREATE” and wait for your Filestore to be ready. This may usually take several minutes.
- b) Check your newly created Filestore instance, pay attention to the information showing in this section “NFS mount point”:
The IP and share name will be needed in the next step.
$ kubectl apply -f samples/google-cloud/workspace-pvc.yaml persistentvolume/zowe-workspace-pv created persistentvolumeclaim/zowe-workspace-pvc created
To verify if your
PersistentVolumeClaim is configured properly, run
$ kubectl get pvc -n zoweNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEzowe-workspace-pvc Bound zowe-workspace-pv 10G RWX 3m4s
If you see
Bound, you can move to the next step.
$ kubectl apply -f samples/api-catalog-service.yaml -f samples/gateway-service-lb.yaml -f samples/discovery-service-lb.yaml service/api-catalog-service created service/gateway-service created
From the GKE console, check
Services & Ingress and wait for all services status are OK and Endpoints are available like this.
cd <instance-dir> ./bin/utils/convert-for-k8s.sh -x "22.214.171.124"
Please note the IP showing above is the
gateway-service external IP showing in previous step.
Follow the instructions shown on the screen and copy all the content of ConfigMaps and Secret and save it to your local computer. Now apply the configs to your cluster by running:
$ kubectl apply -f my-configs.yaml configmap/zowe-config created configmap/zowe-certificates-cm created secret/zowe-certificates-secret created $ kubectl apply -f workloads/instance-env/ deployment.apps/api-catalog created deployment.apps/app-server created deployment.apps/caching created cronjob.batch/cleanup-static-definitions created statefulset.apps/discovery created deployment.apps/explorer-jes created deployment.apps/explorer-mvs created deployment.apps/explorer-uss created deployment.apps/files-api created deployment.apps/gateway created deployment.apps/jobs-api created
Workloads menu from the GKE console, wait until all workloads’ statuses are OK. This may take about 3-5 minutes. .
- Congratulations, now you can access your Zowe instance running in the Kubernetes by accessing the
gateway-serviceexternal IP. In my case, it’s https://126.96.36.199:7554/. Once all APIMLservices are ready, you should see 3 green checkmarks. You can also access Zowe Desktop with URL https://188.8.131.52:7554/zlux/ui/v1.
- You can use your common Kubernetes operations practice. For example, examine logs of Zowe containers.
Once you have done your testing, you can delete the cluster you created, as well as the Filestore instance.
If you followed this introduction and successfully deployed your Zowe workload to your Kubernetes, now it’s time to explore the functions provided by Zowe and the high availability, high scalability, ease of maintenance inherited from Kubernetes.
If you enjoyed this blog checkout more Zowe blogs here. Or, ask a question and join the conversation on the Open Mainframe Project Slack Channel #Zowe-dev, #Zowe-user or #Zowe-onboarding. If this is your first time using the Open Mainframe slack channel register here.