Auto Scaling Question
Closed this issue · 2 comments
Dear Team,
We are using AWS infra for edumeet services. we are working on autoscaling feature for which we need to load several media-node instances automatically based on connections or CPU usage.
Now challenge here is how can we provide medianode URLs on edumeet-roomserver with out disconnection.
Thanks
There is actually a very crude way to do that right now. This will be a lot simpler when the media nodes are managed through the management server, but for now, here goes:
- Connect to the room server interactive server by running
yarn connect
- Type
t
and pressEnter
. You should now be in terminal mode, and have raw access to the node process (BE CAREFUL!) - Do the following commands, and change the appropriate values:
const MediaNode = serverManager.mediaService.mediaNodes.items[0].constructor
const KDPoint = serverManager.mediaService.mediaNodes.items[0].kdPoint.constructor
let newNode = new MediaNode({ id: 'some-random-string', hostname: 'some.host.here', port: 3002, secret: 'lsdkfmlk', kdPoint: new KDPoint([ 10.394, 63.430 ]) })
serverManager.mediaService.mediaNodes.add(newNode)
serverManager.mediaService.loadBalancer.kdTree.addNode(new KDPoint([ 10.394, 63.430 ], { newNode }))
serverManager.mediaService.loadBalancer.kdTree.rebalance()
- Make sure you update the config file so that the media node is also initialized on the next restart of the server
The client now has graceful fail-over to a new media-node if the current one terminates/fails in some way.
A media-node can now be instructed to drain on a timer. Any existing rooms on the media-node will be allowed to remain on the node, but no new rooms are allowed. To drain, connect to the interactive console on the media-node and issue the drain
command with an optional number of seconds as a parameter, ex: drain 3600
for 1 hour (default is 1 hour). When the timer expires the media-node will terminate itself.
The client will gracefully fail-over to a new media-node if it is still on a media-node that terminates after draining.
The room-server now properly takes into account media-node health and health checking. This also includes getting information about when a media-node is draining. The room-server will also do an initial health check of all media-nodes when it starts. Whenever a media node comes back up after dying/draining/terminating, it will go back into use as usual.
The room-server now has a way to add a media-node at runtime by connecting to the interactive console, going to the terminal by typing t
and hitting enter. Then do serverManager.mediaService.addMediaNode({ hostname, port, secret, longitude, latitude })
with the appropriate settings.