Documentation

RunMetrics extends Message
in package

Aggregates over this run's jobs. Reuses the JobMetrics group shapes.

Generated from protobuf message livekit.SimulationRun.RunMetrics

Table of Contents

Properties

$accuracy_score  : mixed
mean over scored jobs
$conversation  : mixed
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>
$experience_score  : mixed
Generated from protobuf field <code>optional float experience_score = 2;</code>
$jobs_failed  : mixed
infra failures, excluded from aggregates
$jobs_simulator_fault  : mixed
spoiled by the simulator: flagged, not scored
$jobs_total  : mixed
Generated from protobuf field <code>uint32 jobs_total = 9;</code>
$llm  : mixed
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>
$scenario_pass_rate  : mixed
share of jobs whose scenario verdict passed
$stt  : mixed
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>
$tts  : mixed
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

Methods

__construct()  : mixed
Constructor.
clearAccuracyScore()  : mixed
clearConversation()  : mixed
clearExperienceScore()  : mixed
clearLlm()  : mixed
clearScenarioPassRate()  : mixed
clearStt()  : mixed
clearTts()  : mixed
getAccuracyScore()  : float
mean over scored jobs
getConversation()  : Conversation|null
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>
getExperienceScore()  : float
Generated from protobuf field <code>optional float experience_score = 2;</code>
getJobsFailed()  : int
infra failures, excluded from aggregates
getJobsSimulatorFault()  : int
spoiled by the simulator: flagged, not scored
getJobsTotal()  : int
Generated from protobuf field <code>uint32 jobs_total = 9;</code>
getLlm()  : LLM|null
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>
getScenarioPassRate()  : float
share of jobs whose scenario verdict passed
getStt()  : STT|null
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>
getTts()  : TTS|null
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>
hasAccuracyScore()  : mixed
hasConversation()  : mixed
hasExperienceScore()  : mixed
hasLlm()  : mixed
hasScenarioPassRate()  : mixed
hasStt()  : mixed
hasTts()  : mixed
setAccuracyScore()  : $this
mean over scored jobs
setConversation()  : $this
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>
setExperienceScore()  : $this
Generated from protobuf field <code>optional float experience_score = 2;</code>
setJobsFailed()  : $this
infra failures, excluded from aggregates
setJobsSimulatorFault()  : $this
spoiled by the simulator: flagged, not scored
setJobsTotal()  : $this
Generated from protobuf field <code>uint32 jobs_total = 9;</code>
setLlm()  : $this
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>
setScenarioPassRate()  : $this
share of jobs whose scenario verdict passed
setStt()  : $this
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>
setTts()  : $this
Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

Properties

$accuracy_score

mean over scored jobs

protected mixed $accuracy_score = null

Generated from protobuf field optional float accuracy_score = 1;

$conversation

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>

protected mixed $conversation = null

$experience_score

Generated from protobuf field <code>optional float experience_score = 2;</code>

protected mixed $experience_score = null

$jobs_failed

infra failures, excluded from aggregates

protected mixed $jobs_failed = 0

Generated from protobuf field uint32 jobs_failed = 10;

$jobs_simulator_fault

spoiled by the simulator: flagged, not scored

protected mixed $jobs_simulator_fault = 0

Generated from protobuf field uint32 jobs_simulator_fault = 11;

$jobs_total

Generated from protobuf field <code>uint32 jobs_total = 9;</code>

protected mixed $jobs_total = 0

$llm

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>

protected mixed $llm = null

$scenario_pass_rate

share of jobs whose scenario verdict passed

protected mixed $scenario_pass_rate = null

Generated from protobuf field optional float scenario_pass_rate = 3;

$stt

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>

protected mixed $stt = null

$tts

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

protected mixed $tts = null

Methods

__construct()

Constructor.

public __construct([array<string|int, mixed> $data = null ]) : mixed
Parameters
$data : array<string|int, mixed> = null

{ Optional. Data for populating the Message object.

@type float $accuracy_score
      mean over scored jobs
@type float $experience_score
@type float $scenario_pass_rate
      share of jobs whose scenario verdict passed
@type \Livekit\SimulationRun\JobMetrics\STT $stt
@type \Livekit\SimulationRun\JobMetrics\LLM $llm
@type \Livekit\SimulationRun\JobMetrics\TTS $tts
@type \Livekit\SimulationRun\JobMetrics\Conversation $conversation
@type int $jobs_total
@type int $jobs_failed
      infra failures, excluded from aggregates
@type int $jobs_simulator_fault
      spoiled by the simulator: flagged, not scored

}

clearAccuracyScore()

public clearAccuracyScore() : mixed

clearConversation()

public clearConversation() : mixed

clearExperienceScore()

public clearExperienceScore() : mixed

clearScenarioPassRate()

public clearScenarioPassRate() : mixed

getAccuracyScore()

mean over scored jobs

public getAccuracyScore() : float

Generated from protobuf field optional float accuracy_score = 1;

Return values
float

getConversation()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>

public getConversation() : Conversation|null
Return values
Conversation|null

getExperienceScore()

Generated from protobuf field <code>optional float experience_score = 2;</code>

public getExperienceScore() : float
Return values
float

getJobsFailed()

infra failures, excluded from aggregates

public getJobsFailed() : int

Generated from protobuf field uint32 jobs_failed = 10;

Return values
int

getJobsSimulatorFault()

spoiled by the simulator: flagged, not scored

public getJobsSimulatorFault() : int

Generated from protobuf field uint32 jobs_simulator_fault = 11;

Return values
int

getJobsTotal()

Generated from protobuf field <code>uint32 jobs_total = 9;</code>

public getJobsTotal() : int
Return values
int

getLlm()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>

public getLlm() : LLM|null
Return values
LLM|null

getScenarioPassRate()

share of jobs whose scenario verdict passed

public getScenarioPassRate() : float

Generated from protobuf field optional float scenario_pass_rate = 3;

Return values
float

getStt()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>

public getStt() : STT|null
Return values
STT|null

getTts()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

public getTts() : TTS|null
Return values
TTS|null

hasAccuracyScore()

public hasAccuracyScore() : mixed

hasConversation()

public hasConversation() : mixed

hasExperienceScore()

public hasExperienceScore() : mixed

hasScenarioPassRate()

public hasScenarioPassRate() : mixed

setAccuracyScore()

mean over scored jobs

public setAccuracyScore(float $var) : $this

Generated from protobuf field optional float accuracy_score = 1;

Parameters
$var : float
Return values
$this

setConversation()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.Conversation conversation = 7;</code>

public setConversation(Conversation $var) : $this
Parameters
$var : Conversation
Return values
$this

setExperienceScore()

Generated from protobuf field <code>optional float experience_score = 2;</code>

public setExperienceScore(float $var) : $this
Parameters
$var : float
Return values
$this

setJobsFailed()

infra failures, excluded from aggregates

public setJobsFailed(int $var) : $this

Generated from protobuf field uint32 jobs_failed = 10;

Parameters
$var : int
Return values
$this

setJobsSimulatorFault()

spoiled by the simulator: flagged, not scored

public setJobsSimulatorFault(int $var) : $this

Generated from protobuf field uint32 jobs_simulator_fault = 11;

Parameters
$var : int
Return values
$this

setJobsTotal()

Generated from protobuf field <code>uint32 jobs_total = 9;</code>

public setJobsTotal(int $var) : $this
Parameters
$var : int
Return values
$this

setLlm()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.LLM llm = 5;</code>

public setLlm(LLM $var) : $this
Parameters
$var : LLM
Return values
$this

setScenarioPassRate()

share of jobs whose scenario verdict passed

public setScenarioPassRate(float $var) : $this

Generated from protobuf field optional float scenario_pass_rate = 3;

Parameters
$var : float
Return values
$this

setStt()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.STT stt = 4;</code>

public setStt(STT $var) : $this
Parameters
$var : STT
Return values
$this

setTts()

Generated from protobuf field <code>.livekit.SimulationRun.JobMetrics.TTS tts = 6;</code>

public setTts(TTS $var) : $this
Parameters
$var : TTS
Return values
$this
On this page

Search results