Single Scheduler — Multiple Worker Architecture with GRPC and Go — Part 3

Single scheduler (conductor) and multiple workers

Part 3 — The worker overview

Workers expose a GRPC-server to communicate with the scheduler. The main job of workers is to run specific jobs requested by the scheduler and report on those jobs. Jobs can be ruby/python/bash scripts or any executables available on the worker machine.

// jobsMutex is the lock to access jobs map.
// jobs is the map that holds current/past jobs.
//    - key: job id
//    - value: pointer to the created job object.
var (
  jobsMutex = &sync.Mutex{}
  jobs      = make(map[string]*job)
)

// job holds information about the ongoing or past jobs,
// that were triggered by the scheduler.
//    - id: UUID assigned by the worker and sent back to the scheduler.
//    - command: command which the scheduler run the job with
//    - path: path to the job file/executable sent by the scheduler.
//    - outFilePath: file path to where the output of the job will be piped.
//    - cmd: pointer to the cmd.Exec command to get job status etc.
//    - done: whether if job is done (default false)
//    - err: error while running the job (default nil)
type job struct {
  id          string
  command     string
  path        string
  outFilePath string
  cmd         *exec.Cmd
  done        bool
  err         error
}

Configuration for the worker is done through the `config.toml` file. I wrote about how I approach configuration in Go projects. You can read more here:

Handling configuration in Go

GRPC Server

• addr: Address on which the GRPC server will be run.
• use_tls: Whether the GRPC server should use TLS. If true, crt_file and key_file should be provided.
• crt_file: Path to the certificate file for TLS.
• key_file: Path to the key file for TLS.

Scheduler

• addr: Address on which the GRPC server of the scheduler is run.

# worker/config.toml
[grpc_server]
addr = "127.0.0.1:50000"
use_tls = false
crt_file = "server.pem"
key_file = "server.key"

[scheduler]
addr = "127.0.0.1:3000"

GRPC Server

Only 3 GRPC requests required in the scheduler GRPC server: to start/stop/query jobs.

// Worker GRPC server definitions
service Worker {
  rpc StartJob(StartJobReq) returns (StartJobRes) {}
  rpc StopJob(StopJobReq) returns (StopJobRes) {}
  rpc QueryJob(QueryJobReq) returns (QueryJobRes) {}
}

message StartJobReq {
  string command = 1;
  string path = 2;
}

message StartJobRes {
  string jobID = 1;
}

message StopJobReq {
  string jobID = 1;
}

message StopJobRes {
}

message QueryJobReq {
  string jobID = 1;
}

message QueryJobRes {
  bool done = 1;
  bool error = 2;
  string errorText = 3;
}

These GRPC requests are translations of HTTP requests into GRPC requests. Read about the HTTP requests in part 2.

Part 2 (Previous Part):

Single scheduler — multiple worker architecture with GRPC and Go — Part 2

Part 4 (Next Part):

Single scheduler — multiple worker architecture with GRPC and Go — Part 4

Same story, better code highlighting:

https://koraygocmen.com/blog/single-scheduler-multiple-worker-architecture-with-grpc-and-go-part-3

Full Code:

github.com/KorayGocmen/scheduler-worker-grpc