Ray on OpenShift

Ray is a distributed computing framework for scaling Python applications. It is widely used for machine learning training, hyperparameter tuning, reinforcement learning, model serving, and general-purpose parallel computing. On OpenShift, Ray runs via the KubeRay operator, which manages Ray clusters as Kubernetes-native resources.

This module introduces how Ray runs on OpenShift, its core resources, and how it integrates with Kueue for batch scheduling.

Why Ray on OpenShift?

Ray provides a simple programming model for distributed computing: write Python functions and classes, and Ray handles distributing them across a cluster of workers. Running Ray on OpenShift brings several advantages:

Unified platform — Ray clusters share the same infrastructure, security, and monitoring as other OpenShift workloads.
Dynamic scaling — Ray autoscaler can request new worker Pods as workload increases, within the limits set by Kubernetes resource quotas.
Job isolation — Each RayJob or RayCluster runs in its own set of Pods with defined resource requests, preventing interference between workloads.
Kueue integration — Ray workloads can be admitted through Kueue’s queue layer for fair sharing and priority.

KubeRay is included in Red Hat OpenShift AI (RHOAI), providing a supported path for running Ray workloads on OpenShift.

Core Resources

The KubeRay operator manages three custom resources:

RayCluster

A RayCluster defines a long-running Ray cluster with a head node and worker nodes. It specifies:

Head node — Runs the Ray head process, GCS (Global Control Store), and the Ray dashboard.
Worker groups — One or more groups of worker Pods, each with its own resource configuration and replica count.
Autoscaling (optional) — Minimum and maximum replicas per worker group, with Ray’s autoscaler managing scale-up and scale-down.

RayClusters are useful when you want a persistent Ray cluster that multiple jobs can connect to.

RayJob

A RayJob creates a Ray cluster, submits a job to it, and optionally tears down the cluster when the job completes. It is ideal for batch workloads: you define the computation and the resources, and KubeRay handles the entire lifecycle.

Key fields:

entrypoint — The Python command or script to run.
runtimeEnvYAML — Python dependencies and environment configuration.
rayClusterSpec — Inline cluster specification (or reference to an existing RayCluster).
shutdownAfterJobFinishes — Whether to delete the Ray cluster after the job completes.

RayService

A RayService manages a Ray Serve deployment for model serving. It is not covered in this batch computing workshop, but it uses the same underlying RayCluster infrastructure.

Exercise 1: Creating a RayCluster

Prerequisites

Ensure KubeRay is installed on your cluster.

Verify the KubeRay operator is running:
```
oc get pods -n ray-system
```

Create a namespace for Ray workloads:

oc new-project ray-demo --skip-config-update || oc project ray-demo

Deploy a RayCluster

Create a RayCluster with 1 head node and 2 workers:

cat <<EOF | oc apply -f -
apiVersion: ray.io/v1
kind: RayCluster
metadata:
  name: ray-cluster-demo
  namespace: ray-demo
spec:
  rayVersion: '2.37.0'
  headGroupSpec:
    rayStartParams:
      dashboard-host: '0.0.0.0'
    template:
      spec:
        containers:
          - name: ray-head
            image: rayproject/ray:2.37.0
            resources:
              requests:
                cpu: "1"
                memory: "2Gi"
              limits:
                cpu: "1"
                memory: "2Gi"
            ports:
              - containerPort: 6379
                name: gcs-server
              - containerPort: 8265
                name: dashboard
              - containerPort: 10001
                name: client
  workerGroupSpecs:
    - groupName: default-worker
      replicas: 2
      minReplicas: 1
      maxReplicas: 4
      rayStartParams: {}
      template:
        spec:
          containers:
            - name: ray-worker
              image: rayproject/ray:2.37.0
              resources:
                requests:
                  cpu: "1"
                  memory: "2Gi"
                limits:
                  cpu: "1"
                  memory: "2Gi"
EOF

Watch the cluster come up:
```
oc get pods -n ray-demo -w
```
You should see 1 head Pod and 2 worker Pods start. Press Ctrl+C once all Pods are Running.
Check the RayCluster status:
```
oc get rayclusters -n ray-demo
```
The STATE column should show ready once the cluster is fully up.

Access the Ray dashboard (optional — if a route is available):

oc expose service ray-cluster-demo-head-svc --port=dashboard -n ray-demo
oc get route ray-cluster-demo-head-svc -n ray-demo

Exercise 2: Submitting a RayJob

A RayJob is the preferred way to run batch workloads on Ray. It creates a cluster, runs the job, and cleans up — ideal for batch computing.

First, create a ConfigMap with the Ray application code:

cat <<EOF | oc apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: ray-sample-code
  namespace: ray-demo
data:
  sample_code.py: |
    import ray
    import time
    import socket

    ray.init()

    @ray.remote
    def compute_task(task_id):
        """Simulate a compute-intensive task."""
        hostname = socket.gethostname()
        print(f"Task {task_id} running on {hostname}")
        time.sleep(5)
        result = sum(i * i for i in range(100000))
        return {"task_id": task_id, "host": hostname, "result": result}

    # Launch 20 tasks distributed across the Ray cluster
    futures = [compute_task.remote(i) for i in range(20)]
    results = ray.get(futures)

    print("\n=== Results ===")
    for r in sorted(results, key=lambda x: x["task_id"]):
        print(f"Task {r['task_id']} ran on {r['host']}")

    hosts = set(r["host"] for r in results)
    print(f"\nTasks distributed across {len(hosts)} nodes: {hosts}")
    print("Batch job completed successfully!")
EOF

Create a RayJob that runs the distributed computation:

cat <<EOF | oc apply -f -
apiVersion: ray.io/v1
kind: RayJob
metadata:
  name: ray-batch-job
  namespace: ray-demo
spec:
  shutdownAfterJobFinishes: true
  ttlSecondsAfterFinished: 300
  entrypoint: "python /home/ray/sample_code.py"
  runtimeEnvYAML: |
    working_dir: "."
  rayClusterSpec:
    rayVersion: '2.37.0'
    headGroupSpec:
      rayStartParams:
        dashboard-host: '0.0.0.0'
      template:
        spec:
          containers:
            - name: ray-head
              image: rayproject/ray:2.37.0
              resources:
                requests:
                  cpu: "1"
                  memory: "2Gi"
              volumeMounts:
                - name: code
                  mountPath: /home/ray
          volumes:
            - name: code
              configMap:
                name: ray-sample-code
    workerGroupSpecs:
      - groupName: default-worker
        replicas: 3
        rayStartParams: {}
        template:
          spec:
            containers:
              - name: ray-worker
                image: rayproject/ray:2.37.0
                resources:
                  requests:
                    cpu: "1"
                    memory: "1Gi"
                volumeMounts:
                  - name: code
                    mountPath: /home/ray
            volumes:
              - name: code
                configMap:
                  name: ray-sample-code
EOF

Verify the RayJob was created:
```
oc get rayjobs -n ray-demo
```
Watch the job progress:
```
oc get pods -n ray-demo -w
```
Press Ctrl+C once the head Pod shows Completed.
View the job results:
```
oc logs -n ray-demo -l ray.io/cluster=ray-batch-job-raycluster --tail=30
```
You should see tasks distributed across multiple worker nodes.
Check the RayJob status:
```
oc get rayjob ray-batch-job -n ray-demo -o yaml | grep -A10 "status:"
```
Because shutdownAfterJobFinishes: true, the Ray cluster will be automatically deleted after the job completes (after the TTL expires).

Exercise 3: Running a RayJob with Kueue

KubeRay integrates with Kueue for admission control. When Kueue manages a RayJob, the entire Ray cluster (head + all workers) is treated as a single workload. Kueue holds the job in a suspended state until the ClusterQueue has sufficient quota, then unsuspends it — providing gang-scheduling-like behavior where either the full cluster starts or nothing does.

How It Works

The integration relies on three mechanisms:

Queue selection — A label on the RayJob tells Kueue which LocalQueue to submit to: kueue.x-k8s.io/queue-name: user-queue.
Suspend control — Kueue manages the spec.suspend field. The RayJob starts suspended and Kueue sets it to false once admitted.
Gang admission — Kueue sums the resource requests across the head and all worker groups. The job is only admitted when the full amount is available.

There are a few constraints when using RayJobs with Kueue:

shutdownAfterJobFinishes must be true — Kueue needs the cluster to clean up so quota is released.
The RayJob must create its own cluster (you cannot reference an existing RayCluster).

Prerequisites

This exercise assumes Kueue is already installed and that you have a ClusterQueue configured. If you completed the Kueue module, the cluster-queue ClusterQueue is already available.

Create a LocalQueue in the ray-demo namespace that points to the existing ClusterQueue:

cat <<EOF | oc apply -f -
apiVersion: kueue.x-k8s.io/v1beta1
kind: LocalQueue
metadata:
  namespace: ray-demo
  name: ray-queue
spec:
  clusterQueue: cluster-queue
EOF

Verify the LocalQueue is active:
```
oc get localqueues -n ray-demo
```

Submit a RayJob Through Kueue

Create a ConfigMap with a sample Ray application:

cat <<EOF | oc apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: ray-kueue-code
  namespace: ray-demo
data:
  sample_code.py: |
    import ray
    import time
    import socket

    ray.init()

    @ray.remote
    def compute_task(task_id):
        hostname = socket.gethostname()
        print(f"Task {task_id} running on {hostname}")
        time.sleep(3)
        result = sum(i * i for i in range(50000))
        return {"task_id": task_id, "host": hostname, "result": result}

    futures = [compute_task.remote(i) for i in range(10)]
    results = ray.get(futures)

    print("\n=== Results ===")
    for r in sorted(results, key=lambda x: x["task_id"]):
        print(f"Task {r['task_id']} ran on {r['host']}")

    hosts = set(r["host"] for r in results)
    print(f"\nTasks distributed across {len(hosts)} nodes: {hosts}")
    print("Kueue-managed Ray job completed successfully!")
EOF

Submit a RayJob with the Kueue queue label:

cat <<EOF | oc apply -f -
apiVersion: ray.io/v1
kind: RayJob
metadata:
  name: ray-kueue-job
  namespace: ray-demo
  labels:
    kueue.x-k8s.io/queue-name: ray-queue  (1)
spec:
  suspend: true  (2)
  shutdownAfterJobFinishes: true  (3)
  ttlSecondsAfterFinished: 300
  entrypoint: "python /home/ray/sample_code.py"
  rayClusterSpec:
    rayVersion: '2.37.0'
    headGroupSpec:
      rayStartParams:
        dashboard-host: '0.0.0.0'
      template:
        spec:
          containers:
            - name: ray-head
              image: rayproject/ray:2.37.0
              resources:
                requests:
                  cpu: "1"
                  memory: "2Gi"
                limits:
                  cpu: "1"
                  memory: "2Gi"
              volumeMounts:
                - name: code
                  mountPath: /home/ray
          volumes:
            - name: code
              configMap:
                name: ray-kueue-code
    workerGroupSpecs:
      - groupName: default-worker
        replicas: 2
        minReplicas: 2
        maxReplicas: 2
        rayStartParams: {}
        template:
          spec:
            containers:
              - name: ray-worker
                image: rayproject/ray:2.37.0
                resources:
                  requests:
                    cpu: "1"
                    memory: "1Gi"
                  limits:
                    cpu: "1"
                    memory: "1Gi"
                volumeMounts:
                  - name: code
                    mountPath: /home/ray
            volumes:
              - name: code
                configMap:
                  name: ray-kueue-code
EOF

1	The `kueue.x-k8s.io/queue-name` label assigns this job to the `ray-queue` LocalQueue.
2	The job starts suspended. Kueue will set this to `false` when the job is admitted.
3	Required for Kueue integration — the cluster must shut down after completion so quota is released.

Observe the Admission Process

Check the Kueue workload that was created for the RayJob:
```
oc get workloads -n ray-demo
```
You should see a workload with ADMITTED showing True if the ClusterQueue has available quota. If the quota is exhausted (for example, from other running jobs), the workload will remain queued until capacity is available.
Watch the RayJob status transition from suspended to running:
```
oc get rayjobs -n ray-demo -w
```
Once Kueue admits the workload, the SUSPEND column changes to false and KubeRay begins creating the Ray cluster Pods. Press Ctrl+C once the status shows RUNNING.
Watch the Pods come up:
```
oc get pods -n ray-demo -w
```
Press Ctrl+C once you see the head and worker Pods running.

Once the job completes, view the results:

oc logs -n ray-demo -l ray.io/cluster=ray-kueue-job-raycluster --tail=20

Verify that Kueue released the quota after the job finished:
```
oc get workloads -n ray-demo
oc get clusterqueues
```
The workload should show as finished and the ClusterQueue’s used quota should reflect the released resources.

Clean Up

Remove all Ray and Kueue resources:

oc delete rayjobs --all -n ray-demo
oc delete rayclusters --all -n ray-demo
oc delete configmap ray-sample-code ray-kueue-code -n ray-demo 2>/dev/null
oc delete localqueue ray-queue -n ray-demo 2>/dev/null
oc delete route ray-cluster-demo-head-svc -n ray-demo 2>/dev/null

Summary

Ray runs on OpenShift as a distributed computing framework managed by the KubeRay operator.

In this module you:

Deployed a RayCluster with head and worker nodes.
Submitted a RayJob that distributed computation across the cluster and cleaned up automatically.
Submitted a RayJob through Kueue and observed the admission process — suspend, admit, run, and quota release.
Understood how Ray fits into the batch computing ecosystem alongside Kueue, Spark, and the Training Operator.

In the next section, you will learn about Slurm on OpenShift for running traditional HPC workload management on the platform.