Skip to content

Draft: Rework channel-source mapping and metadata backend

Rework how channels are mapped to sources, and ultimately endpoints. Decouple where metadata is stored from the data backends and standardize on a common channel specification for the configuration format. Also has some logic built in for specifying the time range of data the backend serves.

Description is WIP and will be ready when it has moved from draft stage.

Remaining items:

  • information-only servers need a way to map between channels and sources somehow
  • requests need to use that time range to properly delegate between endpoints
  • branch cleanup and documentation

How we implement the information-only server use case depends on how we're going to be thinking about channel source resolution and where the metadata will live. Should metadata be accessible directly by all servers, or will servers only serve metadata for the sources it provides?

Some observations:

  • Whether we store direct channel-endpoint information or indirect channel-source and source-endpoint information, the information-only server needs to store some amount of metadata on a per-channel basis.
  • In order to serve flight info requests for the stream endpoint, we need to know the data type for each channel to properly return the right schema back to the client.
  • From the first two points, the information server needs to be able a subset of metadata on a per-channel basis to serve info requests and route to other servers.
  • Only servers which serve publish requests need to perform updates to channel metadata, the rest will only need read access.

Closes #17. Closes #24.

Edited by Patrick Godwin

Merge request reports

Loading