Single Scheduler — Multiple Worker Architecture with GRPC and Go — Part 3

Image for post
Image for post
Single scheduler (conductor) and multiple workers

Part 3 — The worker overview

// jobsMutex is the lock to access jobs map.
// jobs is the map that holds current/past jobs.
// - key: job id
// - value: pointer to the created job object.
var (
jobsMutex = &sync.Mutex{}
jobs = make(map[string]*job)
)

// job holds information about the ongoing or past jobs,
// that were triggered by the scheduler.
// - id: UUID assigned by the worker and sent back to the scheduler.
// - command: command which the scheduler run the job with
// - path: path to the job file/executable sent by the scheduler.
// - outFilePath: file path to where the output of the job will be piped.
// - cmd: pointer to the cmd.Exec command to get job status etc.
// - done: whether if job is done (default false)
// - err: error while running the job (default nil)
type job struct {
id string
command string
path string
outFilePath string
cmd *exec.Cmd
done bool
err error
}

GRPC Server

Scheduler

# worker/config.toml[grpc_server]
addr = "127.0.0.1:50000"
use_tls = false
crt_file = "server.pem"
key_file = "server.key"

[scheduler]
addr = "127.0.0.1:3000"

GRPC Server

// Worker GRPC server definitionsservice Worker {
rpc StartJob(StartJobReq) returns (StartJobRes) {}
rpc StopJob(StopJobReq) returns (StopJobRes) {}
rpc QueryJob(QueryJobReq) returns (QueryJobRes) {}
}

message StartJobReq {
string command = 1;
string path = 2;
}

message StartJobRes {
string jobID = 1;
}

message StopJobReq {
string jobID = 1;
}

message StopJobRes {
}

message QueryJobReq {
string jobID = 1;
}

message QueryJobRes {
bool done = 1;
bool error = 2;
string errorText = 3;
}

University of Toronto, Computer Engineering, architected and implemented reliable infrastructures and worked as the lead developer for multiple startups.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store