Synento docs

Core concepts

Projects, sessions, stream-native recording, access control, and billing.

View as Markdown

An organisation owns projects, a project holds sessions, and each session is already its own recording.

Architecture overview

Synento is API-first and agent-native. Every feature is exposed through REST endpoints, and the dashboard is a client of those same APIs. A coding agent can create a project, publish media, and read it back through the same calls. Nothing requires a person in the dashboard.

┌──────────────┐    REST     ┌──────────────────┐
│  CLI / SDK   │ ◄─────────► │                  │
│  Dashboard   │             │  Synento server  │   append    ┌──────────────────┐
│  Your server │             │  (REST API +     │ ──────────► │  Durable streams │
└──────────────┘             │   WebSocket hub) │ ◄────────── │  (S2) = the      │
┌──────────────┐  WebSocket  │                  │    read     │  recording       │
│  Browsers    │ ◄─────────► │                  │             └──────────────────┘
│ (@synento/   │  binary     └──────────────────┘
│   client)    │  frames
└──────────────┘

Hierarchy: Organisation → Project → Session

Organisation

The top-level container representing a team, company, or account. All projects belong to an organisation.

  • Created automatically when you sign up, together with a first project
  • Billing applies at the organisation level (free tier: 10,000 participant-minutes a month, no card required; then pay-as-you-go with no monthly fee)

Project

A scoped workspace under an organisation. Each project has:

  • Environment: dev (the default), staging or prod; a label for your own organization. development and production are accepted and stored as dev and prod.
  • API keys (secret keys: sk_live_xxxxx): each key belongs to one project and creates sessions only in that project

Usage is recorded per project and billed per organisation. A Free organisation can hold up to 3 projects; pay-as-you-go has no limit.

Session

The core unit of live streaming and recording. Each session:

  • Has a unique ID (32 hexadecimal characters)
  • Can have multiple participants connected simultaneously via WebSocket
  • Is recorded as it happens, with no separate recording setup (stream-native)
  • Is replayable straight away, using the windowed playback API, with no processing step

As returned by @synento/node (the REST API uses the same fields in snake_case: project_id, created_by, is_public, started_at, ended_at, created_at):

type Session = {
  id: string;            // Unique session ID
  projectId: string;     // Parent project
  name: string;          // Display name (optional)
  createdBy: string;     // Creator identifier
  isPublic?: boolean;    // Whether anyone can access this session's replay
  startedAt: string | null; // When first participant joined
  endedAt: string | null;   // When last participant left
  createdAt: string;     // Creation timestamp
};

The stream-native recording model

Key principle: recording is the persisted stream

There is no separate recording pipeline. When participants stream via WebSocket, every message is appended to a durable stream in S2 as it is relayed. This same stored data powers both:

  1. Replay: iterating the stream in chronological (or variable-speed) order
  2. Export: reading the full stream and encoding it to MP4

Stream types

Synento persists a layout-agnostic recording: separated media tracks plus a typed event timeline. Presentation (grid, active-speaker, etc.) is resolved at playback time.

StreamPurposeExample
media/{userId}/camCamera video per participantWebCodecs EVC1 frames (older recordings: JPEG/WebP images)
media/{userId}/micMicrophone audio per participantOpus where the browser supports it, otherwise PCM s16le 16 kHz mono
media/{userId}/screenScreen share per participantSame video formats as cam
eventsStructured session timeline{ "v": 1, "type": "speaking.started", "actor": "alice", … }
directorLayout decisions, computed when the session ends{ "type": "director.cut", "data": { "focus": "bob" } }
chatText messages{ "text": "Hello!", "userId": "u1" } }
captions/{lang}Transcript cues, one stream per language{ "type": "caption.cue", "actor": "alice", "data": { "text": "…", "end": …, "lang": "en" } }
metaLegacy join/leave/controlJSON (superseded by events for new integrations)

Composition layer

The recording stores what happened; playback decides how to show it:

Tracks + events  →  Session state  →  Layout descriptor  →  Composed view
                         ↑                    ↑
                   director track      grid / active-speaker / presenter / …
  • Client: @synento/player (ReplayRenderer, Compositor, resolveLayout) decodes visible tracks and draws regions.
  • Server: FFmpeg export uses the same layout vocabulary (grid, presenter, orientation).
  • Manifest: GET /v1/sessions/:id/manifest indexes tracks, bounds, and capabilities for fast bootstrap.

What this means for developers

  • No encoding delay: replay reads the recording directly, with no processing step
  • Instant layout switching: change grid ↔ active-speaker without re-fetching
  • Selective track fetch: manifest-driven bandwidth savings (fetch only visible tracks)
  • Windowed reading: large sessions don't need to be loaded entirely
  • Variable speed replay: @synento/player plays back at any speed from 0.25x to 4x

Video and audio quality

@synento/client encodes in the browser, and the server relays and records those bytes unchanged, so what you set here is what every viewer, every replay and every MP4 export sees.

Default
Camera videoUp to 1280×720 (720p), 30 fps, 1.2 Mbps
ShapeThe camera's own: a 16:9 webcam is 1280×720, a phone held upright 720×1280; nothing is cropped
Smaller camerasSent at their own size, never scaled up, with a proportionally lower bitrate (at least 250 kbps)
CodecH.264, High profile where the browser can encode it, then Main, then Baseline (VP9 or VP8 if the browser has no H.264)
Screen shareThe same settings as the camera, unless you set screen separately
AudioOpus at 32 kbps, 16 kHz mono, suited to speech; uncompressed audio where a browser has no Opus
MP4 export1280×720 (landscape) or 720×1280 (vertical), each participant letterboxed in their own shape

If the camera stream you publish is below 720p and you did not choose a size, the SDK asks the camera for 720p. Constrain width, height or aspectRatio in getUserMedia yourself and your choice is kept.

Change the target per call. width and height are a bounding box, not a crop:

const room = new SynentoRoom({
  getToken,
  userId: "alice",
  video: { width: 1920, height: 1080, bitrate: 2_500_000 }, // 1080p
  screen: { bitrate: 2_000_000, fps: 15 },                  // sharper text, fewer frames
});

Bitrate sets how much a session stores and how much replay and export read, so it shows up in storage, replay and export usage: a participant-minute at the default 720p is about 9 MB.

What it does not do yet: every viewer receives the same stream, so there is no per-viewer quality switching (simulcast). On a congested connection the sender lowers its frame rate, to 80% and then 60%, while the resolution stays the same.

Access control model

Sessions support three levels of access:

LevelWho can accessHow
Private (default)Your project's API key, or a signed-in member of your organisationAPI key + session ID
PublicAnyone with the session ID can replay itGET /v1/public/sessions/{id}/…, no auth
Link-onlyHolders of an expiring replay tokenToken in the query string (TTL 1 min to 30 days)

Visibility governs replay only. Joining a live call always takes a connection token, minted with your API key.

See Access Control Guide for detailed API usage.

Connection tokens and WebSocket authentication

Every participant needs a connection token to join or replay a session:

  1. Request via POST /v1/sessions/{id}/connection with API key auth and a user_id
  2. Receive an access_token (JWT), valid for one hour by default (expires_in says how long, in seconds)
  3. Connect to wss://api.synento.com/ws?session={sessionId}&user_id=alice, sending the token as the WebSocket subprotocol synento.token.{token} (alongside synento.v1). @synento/client does this for you; the older &access_token={token} query parameter still works but puts the token in access logs.

A token is bound to one session, one user id and one mode (live, or replay with "replay": true, and optional from_ts, speed and focus). A replay connection adds &replay=true to the URL. A token used for a different session, user or mode is refused.

Usage tracking & billing

Usage is tracked per project and aggregated at the organisation level:

  • Participant-minutes: each participant's time connected to a live session
  • Storage: recordings held in GiB-months. 100 GiB kept for a whole month is 100 GiB-months; a recording deleted halfway through counts for half

The authoritative source for billing tiers is GET /v1/billing/plans. Both plans give the same free allowance; there is no monthly fee on either:

TierParticipant-minutesStorageRetentionCost
Free10,000 a monthup to 25 GiB heldup to 7 days$0, no card; new sessions stop at the allowance
Pay-as-you-go10,000 a month freeeach recording free for its first 30 daysup to 90 daysno monthly fee, then $0.003/participant-minute, and $0.10/GiB-month for recordings kept past 30 days

Retention is a whole number of days, from 1 up to that cap. New projects start at 7 days on Free and 30 on pay-as-you-go. Pay-as-you-go is a card on file billed monthly in arrears, only for usage past the free allowance.

Replay and export are included up to an allowance, because every replay reads the recording back out of storage:

TierReplay (replayed participant-minutes)MP4 export (exported participant-minutes)
Free1,500 per period, then refused until the period resets300 per period, then refused
Pay-as-you-go1,500 per period free, then $0.003 each300 per period free, then $0.004 each

Transcripts are billed per minute of speech: silence and muted time are removed first, and only what is transcribed counts. Both plans include 60 minutes per period; past that, Free is refused and pay-as-you-go is billed $0.006 a minute.

An exported participant-minute is one participant's camera and microphone in the MP4 for one minute. Exporting a 4-person, 30-minute call is 120; past the allowance that costs $0.24. Repeat requests for an identical export reuse it and do not count.

A replayed participant-minute is one participant's track played back for one minute, the same unit as recording. Watching a 4-person call for 10 minutes is 40. It covers dashboard playback, the playback API, public sessions and replay links alike. Past the allowance, playback and replay tokens are refused with 403 PLAN_LIMIT (and a WebSocket replay closes with 4003).

Call size is also set by plan. Free calls hold up to 4 people, and a Free organisation can have up to 8 people in calls at once across all its projects. Pay-as-you-go calls hold up to 16. GET /v1/billing/limits reports both (maxParticipantsPerCall, maxConcurrentParticipants). A join past either limit is refused when the connection token is requested (403 PLAN_LIMIT); if a token is used after the call has filled, the WebSocket closes with 4001 (call full) or 4002 (account limit) and @synento/client reports the reason instead of reconnecting. Someone rejoining a call they are already in is always let back in.

Two things worth being precise about, because both have caused real errors:

  • Usage is metered in participant-minutes, not room-minutes: a four-person call for ten minutes is 40, not 10.
  • Storage is GiB (2³⁰ bytes), the unit S2 bills in, and the API fields say so (includedStorageGib, storageUsedGib). Only storageBytesUsed is bytes. Scaling a GiB figure by 1e9 renders a 50 GiB allowance as "46.57 GB".

Replay API and windowed playback

The replay system uses window-based iteration to handle sessions of any size:

  1. GET /v1/sessions/:id/manifest: reconstruction contract (participants, tracks, bounds, capabilities)
  2. GET /v1/sessions/:id/playback/timeline: compact metadata (participant count, stream ids, minTs/maxTs)
  3. GET /v1/sessions/:id/playback?from_ts={minTs}&window_size=30000: read a 30-second window of all tracks + events
  4. Request the next window with from_ts set to the previous response's end_ts, until you pass maxTs

from_ts is an absolute timestamp in epoch milliseconds, on the same clock as minTs/maxTs (and the manifest's start_ts/end_ts). Start from minTs. from_ts=0 does not mean "the start of the session". That window ends 30 seconds after 1970 and comes back empty.

Each window returns events from separated media tracks, the events timeline, director track, and chat. The Player SDK composes them into configurable layouts (grid, active-speaker, presenter, screen-share).

See Recordings & Replay Guide for layout descriptors, export parameters, and participant redaction.

On this page