Single Scheduler — Multiple Worker Architecture with GRPC and Go — Part 4

Image for post
Image for post
Single scheduler (conductor) and multiple workers

Part 4 — The scheduler

The entry point of the scheduler is the scheduler/scheduler.go file which contains the init and the main functions. Main functions call 2 functions async, which are to start the HTTP server (API) and the GRPC server.

API uses Juline Schmidt’s HTTP router:

API exposes the 3 API requests to start/query/stop jobs on a worker. These functions are pretty standard and not important. The important parts are the 3 functions that do all the work: “startJobOnWorker”, “stopJobOnWorker”, “queryJobOnWorker”.

Those 3 functions are located at grpc_translator.go file and they translate HTTP requests to GRPC requests on specified workers.

GRPC translator calls worker functions to create and delete workers. These 2 functions just assign an id to a new worker and add the new worker to workers map and vice versa. Pretty standard. I am too bored to add them here.

Finally, the last important bit is the GRPC server of the scheduler. Which is used by the workers to register and deregister on the scheduler.

Alright, the worker is going to be slightly more complicated. But hopefully will be the last piece.

Written by

University of Toronto, Computer Engineering, architected and implemented reliable infrastructures and worked as the lead developer for multiple startups.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store