Single Scheduler — Multiple Worker Architecture with GRPC and Go — Part 3

Single scheduler (conductor) and multiple workers

Part 3 — The worker overview

Workers expose a GRPC-server to communicate with the scheduler. The main job of workers is to run specific jobs requested by the scheduler and report on those jobs. Jobs can be ruby/python/bash scripts or any executables available on the worker machine.

// jobsMutex is the lock to access jobs map.
// jobs is the map that holds current/past jobs.
// - key: job id
// - value: pointer to the created job object.
var (
jobsMutex = &sync.Mutex{}
jobs = make(map[string]*job)
)

// job holds information about the ongoing or past jobs,
// that were triggered by the scheduler.
// - id: UUID assigned by the worker and sent back to the scheduler.
// - command: command which the scheduler run the job with
// - path: path to the job file/executable sent by the scheduler.
// - outFilePath: file path to where the output of the job will be piped.
// - cmd: pointer to the cmd.Exec command to get job status etc.
// - done: whether if job is done (default false)
// - err: error while running the job (default nil)
type job struct {
id string
command string
path string
outFilePath string
cmd *exec.Cmd
done bool
err error
}

Configuration for the worker is done through the `config.toml` file. I wrote about how I approach configuration in Go projects. You can read more here:

  • addr: Address on which the GRPC server will be run.
  • use_tls: Whether the GRPC server should use TLS. If true, crt_file and key_file should be provided.
  • crt_file: Path to the certificate file for TLS.
  • key_file: Path to the key file for TLS.
  • addr: Address on which the GRPC server of the scheduler is run.
# worker/config.toml[grpc_server]
addr = "127.0.0.1:50000"
use_tls = false
crt_file = "server.pem"
key_file = "server.key"

[scheduler]
addr = "127.0.0.1:3000"

Only 3 GRPC requests required in the scheduler GRPC server: to start/stop/query jobs.

// Worker GRPC server definitionsservice Worker {
rpc StartJob(StartJobReq) returns (StartJobRes) {}
rpc StopJob(StopJobReq) returns (StopJobRes) {}
rpc QueryJob(QueryJobReq) returns (QueryJobRes) {}
}

message StartJobReq {
string command = 1;
string path = 2;
}

message StartJobRes {
string jobID = 1;
}

message StopJobReq {
string jobID = 1;
}

message StopJobRes {
}

message QueryJobReq {
string jobID = 1;
}

message QueryJobRes {
bool done = 1;
bool error = 2;
string errorText = 3;
}

These GRPC requests are translations of HTTP requests into GRPC requests. Read about the HTTP requests in part 2.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Koray Göçmen

Koray Göçmen

University of Toronto, Computer Engineering, architected and implemented reliable infrastructures and worked as the lead developer for multiple startups.