The idea

Bringing serverless functionality and extendability to a websocket server.

But why??

Dynamically interchange "modules" at runtime without any downtime
no server config modifications required.. just add a function and you're done..
Hide the messy webscoket implementation and work with easy functions
wrap existing APIs into a websocket interface for legacy codebases (familiar interface)
Reduce websocket abuse (e.g. sending very large objects) by allowing for clever synchronization methods.

Installing

Requirements: Deno

To install run the command:

deno install -A -f -r --import-map=https://raw.githubusercontent.com/duart38/serverless-sockets/main/import_map.json -n ssocket https://raw.githubusercontent.com/duart38/serverless-sockets/main/src/mod.ts

After this you can use the server in any folder with the command:

ssocket

The server expects a folder called "plugs" to be present in the current directory.

it's also possible to pre-configure the server before installing it to make sure the configurations persist every time the command is run. To do this use the command to install and prepend the configuration flags to the command:

deno install -A -f -r --import-map=https://raw.githubusercontent.com/duart38/serverless-sockets/main/import_map.json -n ssocket https://raw.githubusercontent.com/duart38/serverless-sockets/main/src/mod.ts <flags-goes-here>

See configuration documentation (CLI) on how to configure the server.

For more information on installing scripts check this page out

Running locally

Requirements: Deno

clone the repo and go into the folder
go into the src folder

deno run -A mod.ts

🎉

How it works

Depending on your configuration your only concern (as a developer) will be the 'plugs' folder (folder name depends on the config). The default folder obtained with the cloning of this repository contains some example code in the 'plugs' folder. Here's an example of how the modular functions look like.

Simple return message(s)

export async function* test(message: SocketMessage, _from: WebSocket): ModuleGenerator { // _from is the id of the client
    for(let i = 0; i<message.payload.count; i++){ // message.payload.count is what the client sends us
      yield { // returning data back to the client
        event: "spam-mode", // this is optional. will default to the event name of this module
        payload: {
          name: `iteration ${i}`
        }
      }
    }
}

This returns the yielded object to the client.

export async function* test(message: SocketMessage<{count: number}>, _from: WebSocket): ModuleGenerator<{name: string}> {
    for(let i = 0; i<message.payload.count; i++){ // message.payload.count is what the client sends us
      yield { // returning data back to the client
        event: "spam-mode", // this is optional. will default to the event name of this module
        payload: {
          name: `iteration ${i}`
        }
      }
    }
}

Types are also supported.

Esentially you 'yield' back values to the client side every time you want to send them an update. If you're unfamiliar with generator functions take a look at this page

Broadcasting message to everyone

To broadcast a message to everyone you can either add type: EventType.BROADCAST, to your yields or you can use the staticly exposed broadcast() method from the Socket class.

NOTE: it is recommended to use the type property as it hides the implementation details and also prevents memory leaks.

yielding a broadcast (Recommended)

export async function* broadcast(message: SocketMessage, _from: WebSocket): ModuleGenerator {
  yield {
    type: EventType.BROADCAST, // instruction to broadcast this message
    event: "broadcast-1",
    payload: message.payload
  }
}

method-based broadcast (avoid if possible)

export async function* broadcast(message: SocketMessage, _from: WebSocket): ModuleGenerator {
  Socket.broadcast({
      type: EventType.BROADCAST,
      event: "broadcast-1",
      payload: message.payload
    }, _from
  );
  yield {event: '', payload: {}};
}

The reason this method is still available is because yielding does not work in certain scopes (I.E. inner lambdas and or functions).

Sending to specific client

Socket.sendMessage(/*client ID*/5, {event: "ev", payload: {}});

Synchronizing big objects with clients

NOTE: this method is only recommended if you have big objects that need to be synchronized between the server and the client!!. it also requires some extra logic on the client-side to make sure everything is in sync.

Oftentimes we tend to abuse the websocket protocol by sending really big objects with small changed to the client to keep things in sync.. this is .. well... bad.

To mitigate this the framework offers a method to synchronize big objects with the client by sending only the required instructions to the client which will then update it's local object.

To use this method you simply need to add type: EventType.SYNC, to your yields. example:

export async function* sync(message: SocketMessage, _from: WebSocket): ModuleGenerator {
  yield {
    type: EventType.SYNC,
    event: "sync-1",
    payload: {} // some big message here
  }
}

The eventType

/**
 * Global events that the socket server can send back to clients. depending on the value the client is to respond differently
 */
export enum EventType {
  /**
   * a regular message. oftentimes a reply to whoever sent the last message.
   */
  MESSAGE,
  /**
   * A broadcast event is sent to all clients.3
   */
  BROADCAST,
  /**
   * Reserved for authentication, cleans up code a bit.
   */
  AUTH,
  /**
   * Reserved for unknown errors
   */
  ERROR,
  /**
   * Sent back when the event was not found
   */
  NOT_FOUND,

  /**
   * Custom message, developers are free to do what they please here
   */
  CUSTOM_1,
  /**
   * Custom message, developers are free to do what they please here
   */
  CUSTOM_2,
  /**
   * Custom message, developers are free to do what they please here
   */
  CUSTOM_3,
  /**
   * Custom message, developers are free to do what they please here
   */
  CUSTOM_4,
  /**
   * Custom message, developers are free to do what they please here
   */
  CUSTOM_5,
}

CLI

Note: Configurations adapted on the command line are only active for the duration of the program. I.E. they will not persist.

This project ships with a command line interface that aims to help you configure the server entirely from the command line.

Inline configuration (will not persist)

To change configuration items:

deno run -A mod.ts --payloadLimit 10 --INSECURE.port 6969

The CLI is also capable of printing out all the configuration options by running:

deno run -A mod.ts -h

or if you want to read up the documentation of a specific item:

deno run -A mod.ts -h --payloadLimit

Generating modules

The CLI is able to generate modules:

deno run -A mod.ts --generate <name_of_event>

name_of_event here is the file name but also the name of the event that is to be called for on the client side to trigger the module.

Configuration

To configure the server see the config.js file within the 'src' folder of this project. The configuration class in this file should contain all the information and documentation needed.

Additionally, it is also possible to make (if installed as a script) a INIT.ts file with a function named INIT which, upon server start, will be called (see example below).

export async function INIT(_socket: Socket) {
  // Will be called when the server starts up (before the loading of the plugs)
}

This can be used for example when one needs to connect to some other server/database to retrieve data

If you want to shorten the import URLs you can use an import map. For example:

import { SocketMessage, ModuleGenerator } from "ssocket";

To do this (in VSCode) change the settings.json within .vscode to include:

"deno.importMap": "./import_map.json",

Also add this file inside your repository. A template of this file can be found here

Payload shape

This section is only needed if you are modifying the framework.

This is how a message is encoded when the client communicates with the server and vice versa.

SYNC payload shape

This section is only needed if you are modifying the framework.

The payload shape for the SYNC event will return the difference between the last sent or received object in the form of a multi-dimensional array.

[
  [number], // the difference between the 2 known arrays, will be negative if the size decreases and positive otherwise
  [indexToStart, number, number, ...], // subsequent arrays start with the index of where to start placing each item and this is then followed by all the items to be placed.
  ...
  ...
]

The

SocketMessage.syncIncoming(msg: Uint8Array)

message will take in a SYNC style payload and synchronize the already existing object with the new one.

Here's how this all looks:

Testing

deno test -A --unstable --coverage=cov_profile && deno coverage cov_profile

Considerations and limitations

When developing modules keep in mind that the idea of the module is for it to be spawned, to yield it's messages and then to be cleaned up aftwerwards. This means that you need to be careful when creating any floating references in the method that would prevent it from being cleaned up. These are things like refering to an object somewhere else that is never cleaned up (I.E. always in use by something else) or an interval timer.

Modules that are not cleaned up will continue to exist and for each client that calls the module (via it's event string) a new instance of that module is spawned and is also kept in memory indefinitely thus causing a memory leak.