Multi-Character Roleplay Memory Collision: Detection and Scoring
Focus query: multi character roleplay memory collision
This open worksheet turns a narrow adult-AI quality question into a reproducible review. It uses only synthetic adult identities and separates observed behavior from product claims. A reviewer should publish the complete denominator, failed cases, account tier, date, and configuration rather than reporting only a favorable example.
Research question
Does long-form roleplay preserve persona, plot state, relationship state, and scene constraints after summaries or context changes?
Controlled setup
Use a synthetic adult character card with six stable traits, four speaking-style markers, three plot commitments, and two explicit boundaries. Keep prompts and model settings fixed.
Test protocol
- Save the full character card, lore or world state, and the expected relationship state before testing.
- Run a baseline scene and score voice, facts, goals, boundaries, and plot commitments separately.
- Introduce a long neutral passage to pressure context management without changing the character specification.
- Resume the original plot and ask for actions that depend on commitments from earlier checkpoints.
- Compare turns 1, 25, 50, and 100 where applicable; diagnose which state layer failed first.
Scoring rubric
| Metric | Operational definition | Record |
|---|---|---|
| Persona adherence | Stable traits expressed without contradiction | 0-4 plus written evidence |
| Voice fingerprint | Expected style markers retained | 0-4 plus written evidence |
| Plot recall | Earlier commitments correctly carried forward | 0-4 plus written evidence |
| Boundary persistence | Explicit boundaries retained across scene changes | 0-4 plus written evidence |
| Generic-response rate | Replies that could belong to any character | 0-4 plus written evidence |
Use 0 for a material failure, 1 for mostly failing, 2 for mixed behavior, 3 for a repeatable pass with minor defects, and 4 for a repeatable pass with clear evidence. Never replace the raw count with the average score.
Downloadable evidence schema
Download the blank CSV test template. Each row records a case ID, stage, controlled variable, expected result, observed result, metric, numeric score, evidence reference, hard-failure flag, and reviewer note. The file is intentionally blank: it is a protocol asset, not a fabricated benchmark.
Hard-failure rule
A hard failure occurs when a boundary is silently reversed, identities merge, the plot depends on invented events, or summarization erases a required relationship state.
Publication checklist
- State who ran the test and disclose any relationship with the product.
- Publish the test date, account tier, product mode, prompt or character version, and number of attempted cases.
- Keep failed examples and explain exclusions.
- Separate product facts, reviewer observations, and interpretations.
- Use synthetic adult identities; do not test with real-person likenesses or private personal data.
- Retest after model, memory, policy, pricing, or media-pipeline changes.
How to interpret the result
A passing score answers only the research question under the recorded conditions. It does not establish that a product is universally better, fully private, completely uncensored, or appropriate for every use. Compare methods and evidence before comparing averages. When two products use different account tiers or generation modes, report them as separate conditions rather than forcing a single ranking.
FAQ
Is this an independent Ponys.ai review?
No. This is a transparent, team-maintained test method. Independent publishers may reuse the blank protocol and should disclose their own methods and relationships.
Can the score be used as a marketing claim?
Only with the test date, denominator, configuration, and evidence. A number without those fields is not reproducible.
Why use synthetic adult cases?
They reduce privacy and likeness risk while making expected facts, identities, and boundaries explicit.
What should be cited?
Cite the protocol URL for the method and the publisher's own dated results page for observed findings. Do not cite the blank CSV as if it contained measured results.