Skip to main content

Admin Panel

The Admin Panel provides system-wide management capabilities for OpenTranscribe administrators. It is accessible through the Settings modal and includes tools for user management, authentication configuration, search indexing, data integrity, and system monitoring.

Access Requirements​

The Admin Panel is role-gated. The dividing rule is:

Anything that changes how the deployment runs, or that stores infrastructure credentials, is super_admin. Anything that manages users and their content is admin.

RoleAccess
UserProfile & Security (own MFA and active sessions), transcription and speaker-identification preferences, chat, AI prompts, watch sources, recording, System Statistics, File status
AdminAll user sections plus User Management, Quarantine, Takedown notices, the Recovery tab of File status (task health and retry settings), and Data & Storage (retention, media cache, data integrity, search index, speaker embeddings)
Super AdminAll admin sections plus Authentication, Audit Logs, role changes, and the controls that change how the deployment runs: the Backup tab of Data & Storage (with media mirror), the Speaker Engine and Maintenance tabs of Speaker Identification, pinning the local transcription model, the watch-source global settings and email configurations, and the Redaction Policy (redaction floor) tab of Privacy & Redaction
Changed in v0.5.0

Six panels moved from admin to super_admin: ASR provider, Speech processing, Backups, Media Mirror, Watch sources, and the Redaction policy floor. If a plain admin administers any of them today, promote that account (Settings → User Management → Role) before upgrading, or hand the work to an existing super admin.

To access admin features, open the user menu (top right of the navigation bar), choose Settings, and use the Administration and System groups in the sidebar. Rows above your role stay visible but are greyed out with a tooltip naming the role they need.

System Statistics​

The System Statistics section provides a real-time dashboard of your OpenTranscribe deployment.

Metrics Displayed​

  • Users -- Total registered users and recent signups
  • Files -- Total media files, recent uploads, total audio duration, and transcript segment count
  • Tasks -- Pending, running, completed, and failed task counts with success rate and average processing time
  • Speakers -- Total detected speakers and average speakers per file
  • Models -- Active Whisper and diarization model names
  • System Resources -- CPU usage (per-core and aggregate), memory usage, disk usage
  • GPU -- GPU name, VRAM usage (used/total/free), GPU utilization percentage, and temperature

System statistics dashboard showing users, files, tasks, and GPU metrics

GPU Monitoring​

GPU statistics are collected from the Celery worker via Redis. The backend queries nvidia-smi on the worker and caches the results. If Redis has no cached stats, the backend attempts a direct nvidia-smi query or dispatches an on-demand collection task to the CPU queue.

In multi-GPU scaling mode, stats are reported per active GPU worker.

User Management​

The Users section manages all user accounts, pending invitations, and per-account authentication settings.

Inviting users​

Invite User is the supported way to onboard when self-registration is off. You name:

  • Email
  • Full name (optional)
  • Role -- user, admin, or super_admin
  • Auth type -- local, ldap, oidc, or pki
  • Expiry in hours

The invitee receives an emailed link, proves control of the address, and chooses their own credential. For an external auth type no password is set at all — the account is handed to the identity provider. Invitation tokens are hashed at rest, single-use, and expiring.

Pending invitations are listed with a pre-computed status (pending, accepted, revoked, expired) and can be revoked.

note

Inviting or creating a super_admin raises a confirmation dialog. Invitations require a working transactional-mail transport — see auth email designation below.

Pending approvals​

When a super admin turns on Authentication → Local Auth → Require administrator approval for new accounts, every newly provisioned account — self-registered or created just-in-time by an external identity provider — lands in a pending state instead of becoming usable. Administrators approve or reject them (GET/POST /api/admin/user-approvals).

Approval is not the same as activation: deactivation revokes an account that was once usable, approval gates one that never has been. Turning the setting off releases everything still pending; rejected accounts stay rejected, and rejecting never deletes the row.

Creating users directly​

Add User creates an account immediately. Fields: full name, email (unique), role, auth type, and — for auth_type: local only — a password. Choosing an external auth type omits the password field entirely rather than storing a credential the account will never accept.

User table​

Columns for name, email, role, auth type and creation date, with badges for inactive and unverified accounts. A search bar filters by name or email.

Available actions​

For each user (except yourself):

ActionDescription
Change roleuser / admin / super_admin — the super_admin option is visible only to a super admin, is audited, and raises a confirmation dialog
Reset passwordSuper admin only. Applies the password policy and the reuse history, revokes every session, and optionally flags the account for a forced change on next sign-in. Refused for an account whose identity lives in a directory
Lock account / Clear login lockoutLock deactivates the account and revokes its sessions; the unlock action (Clear login lockout) is its true inverse — it clears both the deactivation and any failed-login lockout
Force sign-outRevokes every session for the account
Reset MFASuper admin only. Clears the user's TOTP enrolment and backup codes so they can re-enrol
Toggle local fallbackSuper admin only — allows an external (pki / oidc) account to also sign in with a local password. Rejected for ldap accounts, which never have one
Recover stuck filesTriggers file recovery for the user
Delete userPermanently deletes the account (confirmation required)
note

You cannot change your own role or delete your own account here; your row shows "Current User". The last remaining super admin cannot be demoted or deleted.

Speaker Engine​

(Formerly Engine Configuration, then Speech Processing. It is now the Speaker Engine tab of Settings → Speaker Identification: the tab is visible to admins but locked, and only a super admin can change it. Nothing on it affects the transcribed words.)

Admin-tunable, runtime-safe transcription engine settings. All changes apply live -- no worker restart required. Settings are DB-backed with environment-variable fallback.

The tab opens with the sidecar status and one line saying what the switch is; How it works expands the longer explanation. A badge and a Reset button appear beside a setting only when its value is overridden (in the database or by an environment variable); a value left at the server default shows neither.

Diarization Boundary Correction​

  • Boundary smoothing (default on): collapses 1--3 word "wrong-speaker islands" at speaker turn boundaries (pure-CPU, runs at finalize)
  • Acoustic backchannel re-check (default off, experimental, GPU): re-embeds short disputed/overlap words and reassigns them by voiceprint cosine
    • Under Advanced re-check tuning (closed by default):
      • Re-check Cosine Margin: minimum cosine advantage required to relabel a word (default 0.05)
      • Re-check Max Word Duration (s): only words shorter than this are eligible for re-check (default 1.0 s)

Speaker separation engine​

These choose which local engine detects speakers when speakers are detected on this server. They do not decide where speakers are detected: each user chooses that under Settings → Speaker Identification → Speaker Detection (this server, the transcription provider, the pyannote.ai cloud service, or off).

  • Diarizer Backend: Native engine (default) runs the diar-native sidecar as its own service; PyAnnote fallback engine runs PyAnnote inside the backend and is being phased out. Stored as the engine.diarizer_backend setting, with ENGINE_DIARIZER_BACKEND as the server default. The choice applies from the next file processed.
  • Require Diarization Sidecar (only matters for the native engine): off (default), a file whose sidecar request fails is processed with PyAnnote instead; on, the file fails so you notice the broken sidecar.

The native engine counts speakers itself and ignores the minimum and maximum speaker counts; the PyAnnote engine honours them. With the native engine, the read-only processing pipeline summary in each user's Settings reports this as a warning.

Lite deployments

The lite (cloud-only) image has no local speaker detection, so saving Diarizer Backend or Require Diarization Sidecar there is refused with "This deployment has no local speaker detection. Choose pyannote.ai or a provider that returns speakers." Boundary correction stays editable there.

Redaction Policy​

The admin enforcement floor for content redaction, on the Redaction Policy tab of Settings → Privacy & Redaction (super admin; the tab is locked for admins). While per-user redaction preferences live on the Content Redaction tab, this admin policy is the floor that overrides those preferences for all users.

  • Forced categories: Force PII redaction for all users, Force toxic-speech redaction for all users (with a Toxicity threshold) and Force profanity redaction for all users (users cannot turn a forced category off, only add more)
  • Mandate censored exports: no user can export the unredacted transcript
  • Mandate masking before every LLM provider (including self-hosted): by default only remote providers receive masked text and a local model receives the transcript unmasked; this switch masks it for local models too
  • Re-scan all files: recompute detection across existing transcripts, for example after a detection-model upgrade

Authentication Configuration​

Super Admin only. The Authentication section configures every identity source at runtime, without restarting services. Precedence is database → environment variable → coded default, so once you save a panel the stored value wins.

Tabs​

TabPurpose
Local AuthLocal password login, self-registration, password policy, MFA enforcement, account lockout, login banner
LDAP/ADDirectory connection, search base, bind credentials, attribute mapping, admission and admin groups
OIDCAny OpenID Connect provider — discovery URL, client ID/secret, callback, roles claim, token-validation controls
PKI/CertificateX.509 settings, CA path, trusted proxies, mode, revocation checking
Trusted HeaderIdentity asserted by a trusted reverse proxy (see Trusted-header (proxy))
SAML 2.0SAML 2.0 service provider — IdP metadata/entity ID, ACS/SLS endpoint settings, attribute mapping, admission groups (same group-list syntax as OIDC)
Group mappingsBind LDAP/OIDC group claims to an in-app group and/or a role grant, with a dry-run tester (IdP group mapping)
SCIM provisioningBearer tokens for SCIM 2.0 provisioning from an IdP — create, view status, and revoke (see SCIM Provisioning below)
Directory syncPeriodic reconciliation of LDAP-backed accounts against the directory: disables accounts the directory no longer reports and re-applies group mappings (off by default; has a dry-run mode and a cap on disables per run)
SessionsToken lifetimes, idle and absolute session timeouts, concurrent-session limit and policy
Auth EmailWhich email configuration carries authentication mail (see Auth email designation)
Change historyWho changed which auth setting, and when

Each tab has Save, and LDAP and OIDC have Test Connection to verify before committing.

Renamed in v0.5.0

The Keycloak tab is now OIDC and works with any conforming provider. Existing KEYCLOAK_* environment variables keep working permanently; stored database keys were renamed automatically by migration v379.

Two behaviours worth knowing:

  • Secrets are never sent back to the browser. A sensitive field shows "configured — leave blank to keep it"; the API returns null plus an is_set flag rather than a placeholder.
  • A typo'd key is rejected. Writes are validated against a per-category schema, and incoherent combinations (self-registration on with local login off) are refused with a message naming which switch to change first.

The Change history tab reads auth_config_audit in PostgreSQL and shows configuration changes with the account that made each one. It is a different source from the Audit Logs section below, which streams security events from OpenSearch and carries no configuration changes.

Auth email designation​

Password resets, invitations and email-verification links need a mail transport. The provider rows themselves live in Settings → Watch Sources → Email configurations; one of them is designated to carry authentication mail at Settings → Authentication → Auth Email (super admin only). Clearing the designation falls back to the SMTP_* environment transport. When neither exists, a warning appears on that tab, on the Invite form and in the first-run wizard summary; see Email delivery. A designation naming a missing or disabled configuration is rejected when you save it, and deleting or disabling the designated row is refused while it holds the designation.

SCIM Provisioning​

The SCIM tab manages the bearer tokens that authenticate an identity provider's SCIM 2.0 push provisioning against /scim/v2 — this is the only place tokens are created, viewed, or revoked. See SCIM 2.0 provisioning for the protocol-level detail.

FieldDescription
NameA label to tell tokens apart (e.g. "Okta production")
ExpiresOptional expiry date; leave blank for a token that never expires
StatusActive or Revoked
Last usedTimestamp of the most recent SCIM request authenticated with this token

Click Add Token to create one. The plaintext token is shown exactly once, immediately after creation — only its SHA-256 digest is stored, so if it's lost there is no way to retrieve it, only to revoke it and create a replacement.

Rotating a token means creating a new one, updating the directory's SCIM connector to use it, and then revoking the old one — there is no in-place "regenerate" that keeps the same token row. Revocation is one-way and immediate: every SCIM request bearing that token is refused from that point on.

A SCIM connector authenticated this way can create, update, and deactivate user accounts, but it can never create or modify a super_admin, and it can never write a role directly — group membership it creates is tagged source='scim', kept separate from LDAP/OIDC directory-sync reconciliation. Deactivating a user (active: false or DELETE) disables the account and revokes its sessions without deleting it or its transcripts.

Security Settings​

The Security section in the user settings area manages per-user MFA. Deployment-wide security policy lives in the Authentication section above.

MFA setup flow​

  1. A super admin enables MFA at Authentication → Local Auth → Multi-Factor Authentication (MFA) (Enable TOTP-based MFA; Require MFA for administrators and Require MFA for all users make it mandatory)
  2. Users see the MFA setup option in their Security settings
  3. Click Enable MFA to generate a QR code
  4. Scan with any TOTP authenticator app (Google Authenticator, Microsoft Authenticator, Authy)
  5. Enter the 6-digit code to confirm
  6. Save the generated backup codes — they are shown only once

MFA details​

  • Standard: RFC 6238 TOTP, 6-digit codes, single-use (a code cannot be replayed inside its step window)
  • Backup codes: one-time-use recovery codes; a code used to disable MFA is consumed
  • Disable: requires a valid TOTP code or backup code
  • Required MFA is enforced at the server. With mfa_required on, a user who has not enrolled receives an enrolment-scoped half-token that authorizes only the two setup endpoints — an API client that ignores the hint gets no session
  • External IdP users: PKI and OIDC users bypass local MFA only when they authenticated with their native method. If such an account falls back to a local password, local MFA applies
  • Admin reset: Settings → User Management → Reset MFA clears a user's enrolment so they can re-enrol

Active sessions​

Every user sees their own sessions (device, IP, last activity) in Settings → Profile & Security and can revoke any of them. Admins can list and revoke another account's sessions from the Users section; changing a credential or a privilege revokes sessions automatically.

Audit Log Viewer​

Super Admin only. The Audit Logs section provides a searchable, filterable view of all authentication and administrative events.

Tracked Event Types​

CategoryEvents
Authenticationauth.login.success, auth.login.failure, auth.logout, auth.logout.all
MFAauth.mfa.setup, auth.mfa.verify, auth.mfa.disable, auth.mfa.backup_used
Passwordauth.password.change, auth.password.reset_request, auth.password.reset_complete, auth.password.expired
Account lifecycleauth.account.lockout, auth.account.unlock, auth.account.disabled, auth.account.expired
Tokensauth.token.refresh, auth.token.revoke, auth.token.verify
Sessionsauth.session.created, auth.session.expired, auth.session.terminated, auth.session.limit_exceeded
Bannerauth.banner.acknowledged
Adminadmin.user.create, admin.user.update, admin.user.delete, admin.role.change, admin.settings.change
Content moderationadmin.file.quarantine, admin.file.release (also written when an organization administrator acts on their own organization's files; the event records the actor's role and which authority held the file; outcome partial with details.media_revocation when storage could not revoke, or restore, the file's already-issued media links), admin.takedown_notice.create, admin.takedown_notice.update, admin.takedown_notice.notify, admin.takedown_notice.export (notice records: ids, the notice type and counts only, never the basis or a reason)
Data integrityadmin.data_integrity.purge (a forced orphan purge)
Legal holdscontent.hold.place, content.hold.release (scope and file count; never the reason or matter reference), content.hold.refused (the action attempted and who holds it), content.transcript.edit (a transcript segment was edited, or its speaker reassigned, while the file was under a hold: the segment id, the fields changed and who holds the file, never the text. Editing a held file is allowed; this is its trail)
Auditaudit.export (an audit log was exported: platform or organization scope, format, row count, whether it was truncated)
Collectionscollection.visibility.change (a collection was restricted or opened up: the new mode and how many members lost sight of it, never its name), collection.membership.change (files were added to or removed from a collection: the counts, up to 100 file ids, and details.source: manual, auto_label or bulk_group; the file or collection name is never recorded), resource.share and resource.unshare (a collection or tag was shared: the typed resource_type, resource_uuid and tenant_scope fields identify it)
Content accesscontent.file.view, content.media.access (details.kind: stream_url, download_prepare, download_stream), content.export (format, surface and file count; details.surface is transcript (also a Markdown, Word or PDF export, named by details.format), subtitles, summary, chat or bulk, and a bulk export writes one event per file). Opening a recording is surface: detail, and its thumbnail, captions and transcript pages are part of that one view. Recorded only while content access auditing is on
Transcriptstranscript.view_unredacted (a transcript was revealed past the owner's redaction policy)
Prompt sharingprompt.share, prompt.unshare, prompt.clone
User credentialsuser.credential.set, user.credential.delete (a user saved or removed their own pyannote.ai key; the key itself is never recorded)

A few of these are worth knowing about specifically:

  • auth.login.failure with error_code: ACCOUNT_LINK_REFUSED is an external identity that was refused a takeover of an existing account by email match — see Account linking.
  • auth.session.limit_exceeded records the concurrent-session cap (FedRAMP AC-10), whether the policy evicted the oldest session or rejected the new one.
  • admin.role.change with an actor of idp_login or directory_sync is a privilege change driven by a directory group mapping rather than by a person.
  • Login-banner refusals are deliberately not audited per request — they would fire on every request of every pre-acknowledgment session. The acknowledgment itself is the artefact.

Content access auditing​

The content.* events record who opened a recording, asked for its media or exported a transcript. They are written in large numbers, so they are off by default: a super admin turns them on with Content access auditing at the top of the Audit Logs section, with no environment variable and no restart (it takes effect within about 30 seconds). Collection visibility and membership changes (and legal holds) are always recorded, whatever this switch says.

  • A repeat view of the same recording, or a repeat request for its stream URL, by the same person within 15 minutes is recorded once. Downloads and exports are never collapsed: each is a copy leaving the system.
  • No event ever contains transcript text, a filename or title, a collection name or a hold's reason. Events carry ids, kinds, counts and formats; the file or collection is identified by resource_type and resource_uuid, and tenant_scope says whether it belongs to an organization or to one person's personal workspace.
  • An event about a file in an organization carries that organization's id, so its organization admins see it. An event about a file in someone's personal workspace is never shown to the administrators of the organizations that person belongs to.
  • Under a support-access grant the event records the grant's id.
  • Storage: roughly 0.8 KB per event. With the de-duplication above an active user writes about 34 KB a day, so about 1 GB a month for 1,000 active users (one index shard, no replicas, one index per month, kept for the audit retention you configure). The switch shows this estimate.
  • A deployment edition can withhold the feature; the switch then says so, and nothing is recorded even when it is on.

Filtering​

Filter logs using any combination of:

  • Start Date / End Date -- Date range picker
  • Event Type -- Dropdown of all tracked event types
  • Outcome -- Success or Failure

Click Apply to reload with the selected filters.

Log Table Columns​

ColumnDescription
TimeTimestamp in compact locale format
EventEvent type code (e.g., auth.login.success)
UserUsername associated with the event
StatusOK or FAIL badge
IPSource IP address
DetailsClick ... to view full JSON event details in a modal

Exporting​

Export filtered logs in CSV or JSON format using the export buttons. The downloaded file is named audit-logs-YYYY-MM-DD.{format}.

The export holds up to 10,000 of the newest matching events, with the same columns in both views (time, event, outcome, user, target user, organization, tenant scope, resource type and id, address, user agent, error code, details). A larger result is cut at that cap: narrow the dates or the filters to get the rest. CSV cells that start like a spreadsheet formula are neutralised. Every export is itself an audit event (audit.export: the format and the row count, never the rows).

Organization audit log​

An organization administrator has their own screen, Settings > Audit log, shown when the deployment edition enables it (the org.audit_logs capability; it does not appear in the community edition, which has no organizations). It reads only that organization's events, and never the events of anyone's personal workspace, even a member's.

  • Filters: dates, event, outcome, a member (searched from your organization's members) and a file or collection ID. Paste a file's ID to see everything that happened to it: who opened it, requested or downloaded its media, exported it, added it to a collection or shared it.
  • Names: the log stores ids only. The screen resolves a person's name and a file's or collection's title when you open it, and only inside your organization: a person who left, or a file that was deleted, shows as an id.
  • Content access events (content.*) appear only while a super admin has turned on content access auditing; the event list offers them only on deployments that can record them.
  • Export (CSV or JSON, the same 10,000-row cap) adds the resolved names as extra columns. It is behind its own capability, so a deployment can offer the screen without the download; without it the buttons are hidden and the API answers 404. A platform administrator without an organization role cannot use either.

API: GET /api/org-admin/audit-logs (filters user_uuid, target_user_uuid, resource_type, resource_uuid; a uuid outside your organization is a 403) and GET /api/org-admin/audit-logs/export?format=csv|json.

An organization administrator can place a legal hold on files, or on a whole collection, from Settings > Legal holds (shown when the deployment edition enables the org.legal_hold capability; the community edition has no organizations). While a hold is in force, nobody can delete the held files (their owner included), retention will not purge them, an account or erasure request cannot remove them, and they cannot be reprocessed or retried, because regenerating a transcript replaces the evidence. Editing a transcript is still allowed, and the held files stay readable, searchable and playable.

  • Place a hold: choose a collection or paste file IDs (one per line, up to 500), give a reason (10 to 2,000 characters) and optionally a matter reference. Only files and collections of your own organization can be held: anything else answers "not found", the same as a missing one.
  • A collection hold covers files added later, and a file under it cannot be removed from the collection, nor can the collection be deleted, until the hold is released.
  • Release a hold with a reason. The record stays (Show released holds). A file that another hold, or the service operator's own hold, still covers stays held. Any account deletion or erasure request that was waiting on a file is retried when its last hold is released.
  • The reason is for your organization's administrators only. It is never copied into the audit log, a notification or a file response. Everyone who can see a held file or collection sees a Legal hold badge and, when they try to delete or reprocess it, a message saying whether the service operator, your organization or both hold it; only an organization administrator is given the hold ids.
  • Platform staff cannot place or release an organization's hold through the product, even a super admin: the screen and its API are for that organization's administrators, with no platform bypass.
  • Audit: placing and releasing a hold are always recorded (content.hold.place, content.hold.release), and so is a refused delete, cancel-upload, reprocess, retry, collection delete or removal from a held collection (content.hold.refused: the action and who holds it). See Audit Log Viewer.
  • Erasing an organization keeps the organization (and defers the rest) while any of its holds is in force.

API: GET/POST /api/org-admin/holds, GET /api/org-admin/holds/{uuid}, POST /api/org-admin/holds/{uuid}/release. The difference between a quarantine, the operator's hold and an organization hold is set out in Abuse and takedown.

Search Settings​

The Search & Indexing tab of Settings → Data & Storage manages OpenSearch neural search and the document index.

Status Dashboard​

Status chips display at-a-glance metrics:

  • Indexed -- Files indexed vs total (e.g., 142/142)
  • Model -- Current embedding model name
  • Health -- Overall index health (OK or Needs Repair)
  • Pending -- Files awaiting indexing (if any)

Embedding Model Selection​

Choose from several pre-configured sentence-transformer models:

Every model below is verified: it registers, deploys, and returns its declared dimension from a real prediction. Multilingual models are additionally checked for cross-lingual behaviour — the same sentence in two languages must land close together. Measured 2026-08-18 on opensearch:3.4.0.

ModelTierDimSizeLanguagesCross-lingual (cosine vs English)
all-MiniLM-L6-v2 (default)Fast38487.5 MBEnglish—
all-MiniLM-L12-v2Fast384128.3 MBEnglish—
multi-qa-MiniLM-L6-cos-v1Fast38480 MBEnglish—
paraphrase-multilingual-MiniLM-L12-v2Fast384465.5 MB50+es 0.98, zh 0.95, ar 0.94, ru 0.94, de 0.90
all-mpnet-base-v2Balanced768418.7 MBEnglish—
all-distilroberta-v1Best768315.5 MBEnglish—
distiluse-base-multilingual-cased-v1Best512480 MB15ru 0.93, es 0.90, de 0.87, zh 0.87, ar 0.85

For reference, the English-only models score 0.006–0.313 on those same translations — so the multilingual pair genuinely aligns languages rather than merely returning numbers.

Dimension matters when switching. Moving between models of the same dimension is a re-embed; changing dimension recreates the index. all-MiniLM-L12-v2, multi-qa-MiniLM-L6-cos-v1 and paraphrase-multilingual-MiniLM-L12-v2 are all 384, matching the default — all-MiniLM-L12-v2 in particular is the default with all twelve layers instead of six, so it is the lowest-risk quality upgrade available: a re-embed, no index recreation, and a trivial rollback. The cost is roughly half the encode throughput.

Changing the model triggers a full re-index of all documents. A confirmation modal warns about this before applying.

Adding a model

Do not add a model to the list without running python3 scripts/verify-embedding-models.py --url <throwaway-cluster> against it. Two classes of failure are invisible otherwise: a model OpenSearch does not actually provide (registration fails at every version), and a model trained for dot-product scoring — the chunks index is cosinesimil, so such a model is ranked by a metric it was never trained for and the scores look entirely plausible while being wrong.

Re-indexing Operations​

ButtonDescription
Re-index AllRebuilds the entire search index from scratch
Re-index PendingOnly indexes files that are not yet indexed
StopCancels a running re-index operation

Re-indexing progress is tracked in real time via WebSocket with a progress bar, file count, percentage, and ETA.

Data Integrity​

The Data Integrity tab of Settings → Data & Storage verifies consistency between the PostgreSQL database and OpenSearch indices.

Index Overview​

Displays a card grid showing each OpenSearch index with:

  • Index name and label
  • Document count breakdown (speakers, profiles, clusters, metadata, chunks)
  • Total document count
  • PostgreSQL reference counts (active files, completed files, speakers)

Data integrity panel showing OpenSearch index health

Integrity Check​

Click Run Check to scan all indices for orphaned documents -- records in OpenSearch that no longer have a corresponding database entry. The check:

  1. Scans each index sequentially with progress tracking
  2. Identifies orphaned documents
  3. Automatically cleans up (deletes) orphaned records
  4. Reports results in a summary table

Results show per-index totals: documents scanned, orphans found, and orphans cleaned.

Reconciling a large orphan set​

A sweep refuses to delete more than 10% of an index in one run (refused: ratio_guard in the result), because that is exactly what a database restored empty looks like from OpenSearch's side. When the orphans are real — documents left behind by a database restore or by rows removed directly in Postgres — reconcile them through the API:

  1. GET /api/admin/data-integrity/counts — a dry run. Each index reports orphaned_docs, refused, and orphan_keys: the file (or speaker) identifiers whose documents would be deleted (the first 200; orphan_keys_truncated says when there are more).
  2. Check those identifiers really are gone from the application.
  3. POST /api/admin/data-integrity?force=true&confirm=true — the forced run. It is recorded in the audit log as admin.data_integrity.purge.

A forced run still cannot delete the documents of a file or speaker that exists: every candidate is re-checked against PostgreSQL after the index is read and immediately before the delete, and a sweep whose PostgreSQL side is empty is refused even with force. Index documents are derived data — a mistaken purge is repaired by a reindex.

Orphaned data panel and storage sweep​

The top of the Data Integrity tab shows an Orphaned data panel: a read-only scan of what has been left behind for files that no longer exist, for both the search index (orphaned documents per index) and stored media files (files, objects and the space they take, with the first few file IDs). Scan again refreshes it.

Remove orphaned data deletes what the scan found, after a confirmation that states exactly how many search documents and stored files (objects and size) will go. It is disabled, with the reason shown, when nothing is orphaned or when everything found is held back by a safety limit. If a limit refused the sweep, the panel explains why and offers I understand, remove anyway, which asks for a second, final confirmation before it sends force=true. An empty database is never overridable.

Storage objects are orphaned when their media_file row is gone (a database restore or reset against a persisted object store, or rows removed with raw SQL). The app's own delete paths already remove a file's objects, so this only reclaims space after out-of-band removal. The same sweep is available through the API:

  1. GET /api/admin/data-integrity/storage/counts -- a dry run. It reports orphan_files, orphan_objects, orphan_bytes, the first 200 orphans (file ID, owner ID, objects, bytes) and refused (ratio_guard or empty_valid_set) when a real run would decline.
  2. POST /api/admin/data-integrity/storage?confirm=true -- the deletion. It runs as a background task and is refused without confirm=true. GET .../storage/status reports whether it is running and the result of the last run.
  3. POST /api/admin/data-integrity/storage?confirm=true&force=true -- overrides the ratio guard only.

Only keys of the shape user_<id>/file_<id>/... are considered: the original, thumbnails, the browser playback copy, derived and masked caches and the waveform. Bulk exports, avatars, media/ objects, other buckets and everything belonging to a file that still exists are never touched. Safeguards, matching the index sweep: the sweep refuses when PostgreSQL holds no files at all (even with force), refuses to delete more than 10% of the file objects in one run without force (above a floor of 25 objects), re-checks each file ID against PostgreSQL immediately before removing its objects, and skips any file with an object newer than one hour (so an upload in progress is safe). Each confirmed sweep is recorded in the audit log as admin.data_integrity.storage_purge (at request time and again on completion) with counters only, never object keys or error text. Deleted objects cannot be recovered; restore them from backup if a sweep removed something it should not have.

Embedding Consistency​

The Speaker Embedding System tab of Settings → Data & Storage shows Embedding Consistency first. It ensures all speakers in the PostgreSQL database have corresponding embeddings in the OpenSearch speaker indices.

Consistency Counts​

When you click Check, the system reports:

  • Total PG Speakers -- Speakers in the database
  • v3 Indexed / Missing -- Speaker embeddings in the v3 index (512-dim, pyannote)
  • v4 Indexed / Missing -- Speaker embeddings in the v4 index (256-dim, WeSpeaker), if it exists
  • Unrepairable -- Speakers that cannot be repaired (no audio segments available)
  • Orphans -- Embeddings in OpenSearch with no matching database speaker

Repair Operation​

Click Repair to re-extract missing embeddings. The system processes each file with missing speakers, extracts new embeddings from the audio, and indexes them. Progress is tracked in real time with a progress bar and ETA. You can Stop a running repair at any time.

Embedding Migration (v3 to v4)​

Below it, Embedding Migration manages the upgrade from v3 speaker embeddings (512-dim pyannote) to v4 (256-dim WeSpeaker).

Migration Benefits​

  • Improved speaker matching accuracy
  • Smaller embedding dimensions (256 vs 512) for faster search
  • Better cross-recording speaker identification

Migration Workflow​

Status Chips​

  • Mode -- Current active mode (V3 or V4)
  • V3 Docs -- Document count in the v3 index
  • V4 Docs -- Document count in the v4 index
  • Status -- Migrating indicator when active

Operations​

ActionDescription
Start MigrationBegins extracting v4 embeddings for all files. Progress tracked via WebSocket.
Stop MigrationPauses the migration. Can be resumed later.
Finalize MigrationPerforms an atomic OpenSearch alias swap from v3 to v4. Only available after all files are processed.
Retry FailedRe-processes files that failed during migration (available when migration is stalled).
Force CompleteSkips remaining failed files and finalizes anyway (use with caution).
Force Re-extractRe-runs v4 extraction for all files, even those already migrated.
warning

Finalization performs an atomic alias swap. Once finalized, the system uses v4 embeddings for all speaker operations. This is not easily reversible.

File Retention / Auto-Deletion​

The File Retention tab of Settings → Data & Storage configures automatic deletion of old transcription files.

Configuration Options​

SettingDescriptionDefault
Enable RetentionMaster toggle for auto-deletionOff
Retention DaysFiles older than this are eligible for deletion365
Run TimeDaily execution time (HH:MM format)02:00
TimezoneTimezone for the scheduled runUTC
Delete Error FilesAlso delete files stuck in error statusOff

Safety Features​

Enabling retention requires an explicit confirmation step -- you must check a confirmation checkbox acknowledging that files will be permanently deleted.

Preview and Manual Run​

  • Preview -- Shows a table of files that would be deleted under the current settings, including title, owner, age, size, and status
  • Run Now -- Triggers an immediate retention pass (requires a second confirmation)
  • Refresh Status -- Checks results after a manual run

Status Display​

Shows the last run timestamp, number of files deleted, and the next scheduled run time.

Retry Settings​

The Failed Task Retries panel, in the same Recovery tab, controls how failed transcription tasks are retried.

SettingDescriptionDefault
Limit retriesToggle to cap the number of retry attemptsOn
MaxMaximum number of retry attempts per task (1-10)3

When retry limits are disabled, failed tasks will continue retrying indefinitely. Click Save to apply changes or Reset to restore the default values.

Task Health​

Settings → System → File status → Recovery (admins) is the Task Health Monitor: it finds stuck work and restarts it. After Refresh Health Data it lists stuck tasks and inconsistent files and offers system-wide recovery actions, each behind a confirmation: Recover Stuck Tasks, Fix Inconsistent Files, Startup Recovery and Recover All User Files. These affect every user's files and tasks, not only yours.

For an operational overview of the pipeline, System Statistics shows pending, running, completed and failed task counts, the success rate, average processing time, queue depths, throughput and a Recent Tasks list, without needing the Flower dashboard.