Core concepts
Projects, sessions, stream-native recording, access control, and billing.
An organisation owns projects, a project holds sessions, and each session is already its own recording.
Architecture overview
Synento is API-first and agent-native. Every feature is exposed through REST endpoints, and the dashboard is a client of those same APIs. A coding agent can create a project, publish media, and read it back through the same calls. Nothing requires a person in the dashboard.
┌──────────────┐ REST ┌──────────────────┐
│ CLI / SDK │ ◄─────────► │ │
│ Dashboard │ │ Synento server │ append ┌──────────────────┐
│ Your server │ │ (REST API + │ ──────────► │ Durable streams │
└──────────────┘ │ WebSocket hub) │ ◄────────── │ (S2) = the │
┌──────────────┐ WebSocket │ │ read │ recording │
│ Browsers │ ◄─────────► │ │ └──────────────────┘
│ (@synento/ │ binary └──────────────────┘
│ client) │ frames
└──────────────┘Hierarchy: Organisation → Project → Session
Organisation
The top-level container representing a team, company, or account. All projects belong to an organisation.
- Created automatically when you sign up, together with a first project
- Billing applies at the organisation level (free tier: 10,000 participant-minutes a month, no card required; then pay-as-you-go with no monthly fee)
Project
A scoped workspace under an organisation. Each project has:
- Environment:
dev(the default),stagingorprod; a label for your own organization.developmentandproductionare accepted and stored asdevandprod. - API keys (secret keys:
sk_live_xxxxx): each key belongs to one project and creates sessions only in that project
Usage is recorded per project and billed per organisation. A Free organisation can hold up to 3 projects; pay-as-you-go has no limit.
Session
The core unit of live streaming and recording. Each session:
- Has a unique ID (32 hexadecimal characters)
- Can have multiple participants connected simultaneously via WebSocket
- Is recorded as it happens, with no separate recording setup (stream-native)
- Is replayable straight away, using the windowed playback API, with no processing step
As returned by @synento/node (the REST API uses the same fields in snake_case:
project_id, created_by, is_public, started_at, ended_at, created_at):
type Session = {
id: string; // Unique session ID
projectId: string; // Parent project
name: string; // Display name (optional)
createdBy: string; // Creator identifier
isPublic?: boolean; // Whether anyone can access this session's replay
startedAt: string | null; // When first participant joined
endedAt: string | null; // When last participant left
createdAt: string; // Creation timestamp
};The stream-native recording model
Key principle: recording is the persisted stream
There is no separate recording pipeline. When participants stream via WebSocket, every message is appended to a durable stream in S2 as it is relayed. This same stored data powers both:
- Replay: iterating the stream in chronological (or variable-speed) order
- Export: reading the full stream and encoding it to MP4
Stream types
Synento persists a layout-agnostic recording: separated media tracks plus a typed event timeline. Presentation (grid, active-speaker, etc.) is resolved at playback time.
| Stream | Purpose | Example |
|---|---|---|
media/{userId}/cam | Camera video per participant | WebCodecs EVC1 frames (older recordings: JPEG/WebP images) |
media/{userId}/mic | Microphone audio per participant | Opus where the browser supports it, otherwise PCM s16le 16 kHz mono |
media/{userId}/screen | Screen share per participant | Same video formats as cam |
events | Structured session timeline | { "v": 1, "type": "speaking.started", "actor": "alice", … } |
director | Layout decisions, computed when the session ends | { "type": "director.cut", "data": { "focus": "bob" } } |
chat | Text messages | { "text": "Hello!", "userId": "u1" } } |
captions/{lang} | Transcript cues, one stream per language | { "type": "caption.cue", "actor": "alice", "data": { "text": "…", "end": …, "lang": "en" } } |
meta | Legacy join/leave/control | JSON (superseded by events for new integrations) |
Composition layer
The recording stores what happened; playback decides how to show it:
Tracks + events → Session state → Layout descriptor → Composed view
↑ ↑
director track grid / active-speaker / presenter / …- Client:
@synento/player(ReplayRenderer,Compositor,resolveLayout) decodes visible tracks and draws regions. - Server: FFmpeg export uses the same layout vocabulary (
grid,presenter, orientation). - Manifest:
GET /v1/sessions/:id/manifestindexes tracks, bounds, and capabilities for fast bootstrap.
What this means for developers
- No encoding delay: replay reads the recording directly, with no processing step
- Instant layout switching: change grid ↔ active-speaker without re-fetching
- Selective track fetch: manifest-driven bandwidth savings (fetch only visible tracks)
- Windowed reading: large sessions don't need to be loaded entirely
- Variable speed replay:
@synento/playerplays back at any speed from 0.25x to 4x
Video and audio quality
@synento/client encodes in the browser, and the server relays and records
those bytes unchanged, so what you set here is what every viewer, every replay
and every MP4 export sees.
| Default | |
|---|---|
| Camera video | Up to 1280×720 (720p), 30 fps, 1.2 Mbps |
| Shape | The camera's own: a 16:9 webcam is 1280×720, a phone held upright 720×1280; nothing is cropped |
| Smaller cameras | Sent at their own size, never scaled up, with a proportionally lower bitrate (at least 250 kbps) |
| Codec | H.264, High profile where the browser can encode it, then Main, then Baseline (VP9 or VP8 if the browser has no H.264) |
| Screen share | The same settings as the camera, unless you set screen separately |
| Audio | Opus at 32 kbps, 16 kHz mono, suited to speech; uncompressed audio where a browser has no Opus |
| MP4 export | 1280×720 (landscape) or 720×1280 (vertical), each participant letterboxed in their own shape |
If the camera stream you publish is below 720p and you did not choose a size,
the SDK asks the camera for 720p. Constrain width, height or aspectRatio
in getUserMedia yourself and your choice is kept.
Change the target per call. width and height are a bounding box, not a
crop:
const room = new SynentoRoom({
getToken,
userId: "alice",
video: { width: 1920, height: 1080, bitrate: 2_500_000 }, // 1080p
screen: { bitrate: 2_000_000, fps: 15 }, // sharper text, fewer frames
});Bitrate sets how much a session stores and how much replay and export read, so it shows up in storage, replay and export usage: a participant-minute at the default 720p is about 9 MB.
What it does not do yet: every viewer receives the same stream, so there is no per-viewer quality switching (simulcast). On a congested connection the sender lowers its frame rate, to 80% and then 60%, while the resolution stays the same.
Access control model
Sessions support three levels of access:
| Level | Who can access | How |
|---|---|---|
| Private (default) | Your project's API key, or a signed-in member of your organisation | API key + session ID |
| Public | Anyone with the session ID can replay it | GET /v1/public/sessions/{id}/…, no auth |
| Link-only | Holders of an expiring replay token | Token in the query string (TTL 1 min to 30 days) |
Visibility governs replay only. Joining a live call always takes a connection token, minted with your API key.
See Access Control Guide for detailed API usage.
Connection tokens and WebSocket authentication
Every participant needs a connection token to join or replay a session:
- Request via
POST /v1/sessions/{id}/connectionwith API key auth and auser_id - Receive an
access_token(JWT), valid for one hour by default (expires_insays how long, in seconds) - Connect to
wss://api.synento.com/ws?session={sessionId}&user_id=alice, sending the token as the WebSocket subprotocolsynento.token.{token}(alongsidesynento.v1).@synento/clientdoes this for you; the older&access_token={token}query parameter still works but puts the token in access logs.
A token is bound to one session, one user id and one mode (live, or replay
with "replay": true, and optional from_ts, speed and focus). A replay
connection adds &replay=true to the URL. A token used for a different
session, user or mode is refused.
Usage tracking & billing
Usage is tracked per project and aggregated at the organisation level:
- Participant-minutes: each participant's time connected to a live session
- Storage: recordings held in GiB-months. 100 GiB kept for a whole month is 100 GiB-months; a recording deleted halfway through counts for half
The authoritative source for billing tiers is GET /v1/billing/plans. Both
plans give the same free allowance; there is no monthly fee on either:
| Tier | Participant-minutes | Storage | Retention | Cost |
|---|---|---|---|---|
| Free | 10,000 a month | up to 25 GiB held | up to 7 days | $0, no card; new sessions stop at the allowance |
| Pay-as-you-go | 10,000 a month free | each recording free for its first 30 days | up to 90 days | no monthly fee, then $0.003/participant-minute, and $0.10/GiB-month for recordings kept past 30 days |
Retention is a whole number of days, from 1 up to that cap. New projects start at 7 days on Free and 30 on pay-as-you-go. Pay-as-you-go is a card on file billed monthly in arrears, only for usage past the free allowance.
Replay and export are included up to an allowance, because every replay reads the recording back out of storage:
| Tier | Replay (replayed participant-minutes) | MP4 export (exported participant-minutes) |
|---|---|---|
| Free | 1,500 per period, then refused until the period resets | 300 per period, then refused |
| Pay-as-you-go | 1,500 per period free, then $0.003 each | 300 per period free, then $0.004 each |
Transcripts are billed per minute of speech: silence and muted time are removed first, and only what is transcribed counts. Both plans include 60 minutes per period; past that, Free is refused and pay-as-you-go is billed $0.006 a minute.
An exported participant-minute is one participant's camera and microphone in the MP4 for one minute. Exporting a 4-person, 30-minute call is 120; past the allowance that costs $0.24. Repeat requests for an identical export reuse it and do not count.
A replayed participant-minute is one participant's track played back for one
minute, the same unit as recording. Watching a 4-person call for 10 minutes is
40. It covers dashboard playback, the playback API, public sessions and replay
links alike. Past the allowance, playback and replay tokens are refused with
403 PLAN_LIMIT (and a WebSocket replay closes with 4003).
Call size is also set by plan. Free calls hold up to 4 people, and a Free
organisation can have up to 8 people in calls at once across all its
projects. Pay-as-you-go calls hold up to 16. GET /v1/billing/limits reports both
(maxParticipantsPerCall, maxConcurrentParticipants). A join past either
limit is refused when the connection token is requested (403 PLAN_LIMIT);
if a token is used after the call has filled, the WebSocket closes with 4001
(call full) or 4002 (account limit) and @synento/client reports the reason
instead of reconnecting. Someone rejoining a call they are already in is
always let back in.
Two things worth being precise about, because both have caused real errors:
- Usage is metered in participant-minutes, not room-minutes: a four-person call for ten minutes is 40, not 10.
- Storage is GiB (2³⁰ bytes), the unit S2 bills in, and the API fields say
so (
includedStorageGib,storageUsedGib). OnlystorageBytesUsedis bytes. Scaling a GiB figure by 1e9 renders a 50 GiB allowance as "46.57 GB".
Replay API and windowed playback
The replay system uses window-based iteration to handle sessions of any size:
GET /v1/sessions/:id/manifest: reconstruction contract (participants, tracks, bounds, capabilities)GET /v1/sessions/:id/playback/timeline: compact metadata (participant count, stream ids,minTs/maxTs)GET /v1/sessions/:id/playback?from_ts={minTs}&window_size=30000: read a 30-second window of all tracks + events- Request the next window with
from_tsset to the previous response'send_ts, until you passmaxTs
from_ts is an absolute timestamp in epoch milliseconds, on the same clock as
minTs/maxTs (and the manifest's start_ts/end_ts). Start from minTs.
from_ts=0 does not mean "the start of the session". That window ends 30
seconds after 1970 and comes back empty.
Each window returns events from separated media tracks, the events timeline, director track, and chat. The Player SDK composes them into configurable layouts (grid, active-speaker, presenter, screen-share).
See Recordings & Replay Guide for layout descriptors, export parameters, and participant redaction.