Skip to content

Commit 22a664c

Browse files
committed
Add architecture and deployment info
1 parent 63a4648 commit 22a664c

File tree

8 files changed

+723
-5
lines changed

8 files changed

+723
-5
lines changed

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,8 @@ This project is realized in Go 1.19 and relies on Go workspaces to divide code i
2727

2828
| Directory | Contents |
2929
|--------------------------|----------|
30-
| [`docs`](./docs) | Documentation files (Work in progress) |
30+
| [`deployment`](./deployment) | Configuration files for deploying Polaris Scheduler and Polaris Cluster Agent |
31+
| [`docs`](./docs) | Documentation files (Work in progress) |
3132
| [`go`](./go) | Go workspace containing all Polaris Scheduler modules |
3233
| [`go/cluster-agent`](./go/cluster-agent) | The polaris-cluster-agent executable module |
3334
| [`go/context-awareness`](./go/context-awareness) | Context-aware scheduling plugins |

deployment/cluster-agent/kubernetes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../go/cluster-agent/manifests/deployment-k8s

deployment/scheduler

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../go/scheduler/manifests/polaris-scheduler

docs/README.md

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,49 @@
1-
# Polaris Distributed Scheduler Documentation
1+
# Polaris Scheduler Documentation
22

3-
## Components
3+
This documentation provides a quick overview of the Polaris Scheduler and its deployment.
44

5-
ToDo
65

6+
## Architecture
7+
8+
The high-level architectures of Polaris Scheduler and the Polaris Cluster Agent are shown in the diagram below.
9+
Both consist of the following three major layers:
10+
11+
1. The *Polaris Scheduler Framework* in the middle provides the object model and runtime services. It realizes the generic scheduling and sampling pipelines. To promote flexibility, sampling, filtering, and scoring policies need to be implemented as plugins.
12+
2. The *orchestrator-specific Connectors* at the bottom are used by the generic components in the upper layers.
13+
3. The *plugins* in the top layer realize scheduling and sampling policies.
14+
15+
![Polaris Distributed Scheduling Framework Architecture](./images/architecture.svg)
16+
17+
The central part of the architectures is the Polaris Scheduler Framework.
18+
All of its components are completely orchestrator-independent and only the Orchestrator Client needs to be realized by an orchestrator-specific connector.
19+
The Object Model provides abstractions, e.g., for nodes and jobs. The Submit Job API exposes an orchestrator-independent endpoint to submit new jobs to a scheduler instance. The Agent API of the Cluster Agent is used by the scheduler to communicate with the Cluster Agent and presents the only interface for the scheduler to a cluster.
20+
It provides endpoints for sampling nodes and submitting scheduling decisions and may be expanded in the future to provide further endpoints for retrieving additional cluster information.
21+
The Polaris Scheduler Runtime powers the scheduling process, sets up the API and pumps jobs into the scheduling pipeline.
22+
The Sampling Service serves an analogous purpose with respect to the sampling pipeline. The Nodes Cache is used by the Cluster Agent to cache information about the nodes in its cluster.
23+
It is updated at a high frequency, preferably through an event-based change notification mechanism if the underlying orchestrator supports it, to ensure that its information reflects the current state of the cluster.
24+
25+
26+
## Scheduling and Sampling Pipelines
27+
28+
![Scheduling and Sampling Pipelines](./images/scheduling-and-sampling-pipeline.svg)
29+
30+
The Scheduling Pipeline forms the centerpiece of the Polaris Distributed Scheduler and defines the scheduling and decision-making process.
31+
The pipeline consists of a sequence of stages, as shown in the above diagram, whose business logic is realized by plugins, which enables the realization of different scheduling polices.
32+
The Sort and Sample Nodes stages support only a single plugin each, while the other stages support multiple plugins.
33+
34+
35+
## Deployment
36+
37+
The [deployment](../deployment) directory of this repository contains configuration files to configure and deploy Polaris Scheduler and the Polaris Cluster Agent.
38+
Copy the contents of this directory to a working directory and adapt the configuration files as needed.
39+
40+
41+
### Polaris Cluster Agent
42+
43+
For the Cluster Agent we currently provide an implementation with the Kubernetes connector and [deployment files](../deployment/cluster-agent/kubernetes) for it.
44+
To deploy the agent in a cluster, adapt the `1-config-map.yaml` file and then run `kubectl apply -f ./` in the same folder.
45+
46+
### Polaris Scheduler
47+
48+
Since Polaris Scheduler is completely orchestrator-independent, we provide a docker-compose file for easy deployment of a single instance [here](../deployment/scheduler).
49+
Adapt the YAML configuration file in this directory to your needs - the default configuration will work fine in most cases, but you must configure the list of clusters in the `remoteClusters` object.

docs/images/architecture.svg

Lines changed: 233 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)