GPT‑Live System Card

In these newly developed evaluations, we use real audio examples from users who have chosen to share their voice interactions to help improve our models. Before these examples are used, they are processed through our privacy and eligibility safeguards, including checks for user permissions and deletion/opt-out status, filtering of ineligible data, and steps to reduce personal information through PII scrubbing and de-identification. We then transcribe the audio, generate the model's response, and evaluate that response for safety.

We compare the new GPT-Live models to their respective predecessors, the models that power Advanced Voice Mode (AVM).

These evaluations are not prevalence weighted, meaning they do not reflect rates of safety performance we see in real usage. Instead, these evaluations were meant to be difficult. For each category below, the evaluation is built around cases in which the existing AVM models were not yet giving ideal responses.

We observe that the GPT-Live models generally provide equal or better safety performance across these adversarially selected prompts than AVM models. GPT-Live-1 shows a slight regression on emotional reliance from 0.88 to 0.82, and GPT-Live-1 mini shows a slight regression on sexual content from 0.97 to 0.95. Note that neither of these are statistically significant.