Documentation

JobMetrics extends Message

in package

• Application

Generated from protobuf message <code>livekit.SimulationRun.JobMetrics</code>

Table of Contents

Properties

$accuracy_score  : mixed

Headline scores, 0-1, computed by the worker as weighted means over the members that ran (a missing member renormalizes the rest): accuracy = task_completion x3 + stt.keyterm_recall + (1 - stt.wer) + tts.enunciation_score experience = conversation.turn_taking_score x2 + tts.naturalness_score + llm.conciseness_score

$audio_judge_model  : mixed

audio (LALM) judge; "" if none ran

$conversation  : mixed

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>

$experience_score  : mixed

Generated from protobuf field <code>optional float experience_score = 2;</code>

$has_remote_session  : mixed

false = waveform-only capture (e.g. SIP)

$judge_model  : mixed

text judge for judged scores; "" if none ran

$llm  : mixed

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>

$simulator  : mixed

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Simulator simulator = 8;</code>

$stt  : mixed

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>

$t0  : mixed

call start (first speech edge); turn times relative to it

$task_completion  : mixed

The accuracy anchor: the scenario verdict as 1/0, or the fraction of the declared target state the call reached.

$tts  : mixed

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

$turns  : mixed

Conversation timeline: one entry per transcript turn, both speakers.

Methods

__construct()  : mixed

Constructor.

clearAccuracyScore()  : mixed

clearConversation()  : mixed

clearExperienceScore()  : mixed

clearLlm()  : mixed

clearSimulator()  : mixed

clearStt()  : mixed

clearT0()  : mixed

clearTaskCompletion()  : mixed

clearTts()  : mixed

getAccuracyScore()  : float

Headline scores, 0-1, computed by the worker as weighted means over the members that ran (a missing member renormalizes the rest): accuracy = task_completion x3 + stt.keyterm_recall + (1 - stt.wer) + tts.enunciation_score experience = conversation.turn_taking_score x2 + tts.naturalness_score + llm.conciseness_score

getAudioJudgeModel()  : string

audio (LALM) judge; "" if none ran

getConversation()  : Conversation|null

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>

getExperienceScore()  : float

Generated from protobuf field <code>optional float experience_score = 2;</code>

getHasRemoteSession()  : bool

false = waveform-only capture (e.g. SIP)

getJudgeModel()  : string

text judge for judged scores; "" if none ran

getLlm()  : LLM|null

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>

getSimulator()  : Simulator|null

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Simulator simulator = 8;</code>

getStt()  : STT|null

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>

getT0()  : Timestamp|null

call start (first speech edge); turn times relative to it

getTaskCompletion()  : float

The accuracy anchor: the scenario verdict as 1/0, or the fraction of the declared target state the call reached.

getTts()  : TTS|null

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

getTurns()  : RepeatedField

Conversation timeline: one entry per transcript turn, both speakers.

hasAccuracyScore()  : mixed

hasConversation()  : mixed

hasExperienceScore()  : mixed

hasLlm()  : mixed

hasSimulator()  : mixed

hasStt()  : mixed

hasT0()  : mixed

hasTaskCompletion()  : mixed

hasTts()  : mixed

setAccuracyScore()  : $this

Headline scores, 0-1, computed by the worker as weighted means over the members that ran (a missing member renormalizes the rest): accuracy = task_completion x3 + stt.keyterm_recall + (1 - stt.wer) + tts.enunciation_score experience = conversation.turn_taking_score x2 + tts.naturalness_score + llm.conciseness_score

setAudioJudgeModel()  : $this

audio (LALM) judge; "" if none ran

setConversation()  : $this

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>

setExperienceScore()  : $this

Generated from protobuf field <code>optional float experience_score = 2;</code>

setHasRemoteSession()  : $this

false = waveform-only capture (e.g. SIP)

setJudgeModel()  : $this

text judge for judged scores; "" if none ran

setLlm()  : $this

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>

setSimulator()  : $this

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Simulator simulator = 8;</code>

setStt()  : $this

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>

setT0()  : $this

call start (first speech edge); turn times relative to it

setTaskCompletion()  : $this

The accuracy anchor: the scenario verdict as 1/0, or the fraction of the declared target state the call reached.

setTts()  : $this

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

setTurns()  : $this

Conversation timeline: one entry per transcript turn, both speakers.