simple ai - a proposed inference-time framework for reducing ai slop.

introduction

at some point, you may have experienced the feeling of reading ai-generated content, doing a double take, rereading carefully, and realizing that very little was actually said. the verbose yet vacuous nature of ai-generated text makes the content needlessly hard to engage with.

tasteful writing and coding are central to knowledge work, yet “good taste” – relevance, clarity, and concision – seems to be notoriously hard for modern llms to grasp. “taste” is notably non-verifiable, making it poorly served by today’s training pipelines. what we end up with is the now-commonplace “ai slop”: text that is fluent but lacking in style and substance (shaib et al., 2025; chakrabarty et al., 2025).

to investigate what user’s think of the verbosity of chatbot responses, we ran a small-scale analysis of thoughttrace, a dataset of real-world multi-turn human–ai conversations with users’ self-reported reactions to chat–assistant responses. the dataset comprises 1,058 users and 2,155 conversational turns. through annotating a random subset of 200 conversations with a human and llm-judge (gemini-2.5-pro), the two prominent user complaints we found were both related to style and presentation:

1. unnecessary detail. users complain that too much unnecessary detail blurs the essence, or core idea, behind the response.
2. broad but shallow. users complained that the scope of responses is too expansive, but simultaneously reported that models fail to go deeply on what users actually needed.

how did we end up here? machine learning has historically framed model improvement as optimization against a low-dimensional reward, whether learned from human preferences (rlhf) or derived from verifiable outcomes (rlvr). we argue that this paradigm is poorly equipped to address the problem of taste and that it has instead contributed to ai-slop. there is a large body of works on precisely stylistic failures in human-written content: from poor stylistic coding choices (fowler, 1999; tufano et al., 2015) to poorly-written prose and how to fix it (williams, 1990; pinker, 2014; sword, 2012). models are trained on enormous quantities of mediocre human text with exactly such failures, and scalar preference rewards are too coarse to disentangle good style from bad. a marginally better reward function attached to the same single-pass autoregressive generation process will not get us to clean, “de-slopped” writing.

it would be silly for a human to produce a high-quality piece of prose by practicing under intense scalar feedback until they can churn out a masterpiece in a single pass. instead, humans write iteratively: they start with an outline; they brainstorm a first draft to get their ideas down; they discard drafts; they follow guidelines or best practices (williams, 2005; strunk, 2000); and they take multiple revision passes, each with a specific goal in mind to fix one principle at a time (e.g., fixing structure, then tone). we propose to put llms in the same setting that humans use to write tasteful prose and clean code. concretely, this requires proposing a new inference-time framework:

llm-generated output as an object to iterate for multiple iterations. rather than generating a response in a single autoregressive pass, the response is constructed and refined over multiple drafting, critique, and revision passes. this form of inference-time scaling is now standard for verifiable domains such as mathematical reasoning but remains largely unexplored for writing. under the current paradigm of autoregressive, one-shot writing, we could imagine having a model think really hard (chain-of-thought) for two hours and then write the final response in fifteen seconds. what we want instead is an agent that works the way writers actually work: sketch a bad outline and tinker with the draft; edit a specific section with respect to a specific best practice. then repeat.

self-scaffolding agents organized around codified best practices. in place of a low-dimensional reward, the agent scaffolds its own work around an expressive, human-interpretable rulebook of domain expertise – a library of writing (williams, 1990; pinker, 2014) or coding (fowler, 1999; tufano et al., 2015) principles.

this research proposal outlines an approach to operationalizing taste through this agentic, inference-time framework. in order, we propose, (i) a rigorous evaluation methodology for ai slop, (ii) the design of self-scaffolding writer–reader agents that audit their drafts against best-practice rubrics, and (iii) reinforcement learning over the agentic harness itself, so that open, small models can learn how to decompose and orchestrate their revision work. while frontier labs have historically folded everything into rlhf or rlvr, the framework proposed here is qualitatively different: the rubric is not collapsed into a scalar; it is kept expressive and guides the model in stages at inference time.

related work

inference-time iteration and self-refinement. a substantial body of work scales inference-time compute through fixed self-correction workflows. self-refine prompts a model to iteratively critique and revise its own outputs. reflexion converts environment feedback into verbal self-reflections that condition subsequent attempts. where has even been work proposed on learning correctors have been trained to map outputs to improved outputs (welleck et al., 2023); however, this work was done for more verifiable domains, such as math. test-time-compute scaling and process supervision have shown large gains in verifiable domains such as mathematics (snell et al., 2024; lightman et al., 2023). our proposal differs from this previous line of work in two ways. first, prior pipelines are typically fixed workflows; we instead propose that the agent should be free to decide – and should learn via rl to decide (stream 3) – how to break up its drafting and revision work. second, self-refine and its successors are largely structured around verifiable tasks; they are not organized around an explicit framework of taste. instead, we leverage the fact that agents are recently at the point where they are capable enough to follow expressive rulebooks directly. the aim of our proposal is to develop a framework for how adherence to best practices should be decomposed across focused passes.

rubric-guided rewards. a complementary line of work uses structured rubrics in place of preference signals. rubrics-as-rewards extends rlvr to non-verifiable domains by aggregating checklist-style rubric judgments into a reward for on-policy post-training (gunjal et al., 2025); and auto-rubric pipelines are typically constructed with a human in the loop. critically, these approaches remain post-training methods: the expressive rubric is ultimately reduced to a scalar reward used to update weights for single-pass autoregressive generation. a training-centric approach would bake complex rubrics straight into the weights via reinforcement learning. we propose instead to keep the rubric intact and break it down at inference time, so that good writing becomes something that is learned as a curriculum: through one principle and one section at a time.

pipelines to edit ai writing. closest to our project ideas is a small body of work that builds pipelines to edit ai writing. chakrabarty et al. (2025) define categories of idiosyncrasies common in ai writing (e.g., cliches, redundant exposition) and builds a pipeline that identifies problematic spans in a draft and rewrites each one using few-shot prompts authored with help from professional writers. a follow-up (chakrabarty et al., 2025) trains this behavior into the model: they fine-tune gpt-4o and llama-3.1-70b on professional writers’ edit traces and train writing-quality reward models to score edited drafts against their originals. our proposed framework differs in two key ways. first, these methods are rigid: a model identifies spans that violate a category and then makes local edits to each in a single pass, whereas we revise sequentially, letting the agent schedule its own passes and address one principle from the rubric at a time. second, their automated editing pipeline is trained on expert human edits, whereas ours is based on established style guides at inference time. in short, we spend inference-time compute on guided, iterative revising based on well-known style guides, such as the elements of style and refactoring: improving the design of existing code, rather than local edits based on an editor.

separating the reading mind from the writing mind

“quite often you will discover, on examining the completed work, that there are serious flaws in the arrangement of the material, calling for transpositions.”

— william strunk jr., the elements of style

for human writers, expressing ideas concisely requires careful editing and iterative drafting: a multi-step revision process that llms do not natively perform. revision requires shifting perspectives: generating text as the writer, then evaluating it through the eyes of a reader. as a feasibility study, we replicated this separation in an llm by splitting the task: a writer agent creates a first draft, and a reader agent critiques the draft against a style rubric; the writer then revises based on the critique.

we compare this approach to an alternative approach of supplying the rubric in the system prompt during composition. in that baseline, the writer holds the rubric as a guide while drafting: in principle it can plan an active-voice sentence in advance, or suppress a hedging qualifier before it appears. in the writer–reader framework, by contrast, the rubric only enters after the draft exists. the question our study asks: is it better to guide the writer with the rubric during composition, or to let a reader discover violations and pass structured feedback back to the writer?

methods

we compare the ability of an llm (gpt-5-mini) to conform to the elements of style under the two approaches.

• rubric in system prompt. a single forward pass with our distilled elements of style guide supplied verbatim as the system prompt (no wrapping, prefix, or additional formatting). the user prompt is then given and the model generates its response with the guide in context.
• agentic writing (writer–reader loop). the writer drafts with no rubric in its context. a separate reader call (the same underlying model, with the rubric in context) audits the draft line-by-line and returns a structured critique. the writer receives the critique and revises. critique and revision may repeat for up to two rounds.

we evaluated responses on a random subset of 10 prompts drawn from a corpus of 1,000 natural-language queries spanning everyday curiosity (cooking, climbing, biology, music, software, etc.).¹

reader critique format. the reader returns a json object conforming to the schema:

{
  "violations": [
    {"principle": "...", "span": "...", "reason": "..."},
    ...
  ]
}

each entry names the violated principle, quotes the offending span verbatim, and gives a one-line reason. the structured output is rendered into a bulleted feedback message for the writer.

revision instruction. the writer is then handed the following system prompt: “a style reviewer flagged the issues below. revise your response so each flagged issue is fixed. you may make small edits in service of fixing those issues, but do not rewrite content that wasn’t flagged.”

scoring. to score the two approaches, an independent audit call (same model) runs the same distilled elements of style rubric over each final output and returns the violations it observes. we report the total count of flagged violations per condition.

results

without any guidance (a generator model writes a single draft), the model accumulates 19 violations across 10 responses (1.9 violations per response). with a rubric in the system prompt, we see the number of violations barely decrease, with 18 violations remaining across 10 responses (1.8 per response). the writer–reader paradigm, by contrast, results in a sharp drop in the number of violation, with 5 violations (0.5 per response) across 10 responses, where 7 of 10 responses have 0 violations.

there were a few common violations that appear after the writing loop. most prominent examples are active-voice (3 of 5 remaining violations) and parallel-construction (1 of 5). converting a passive construction to an active one typically requires reshaping the clause around it, which may brush up against the conservative revise instruction. violations that can be repaired with local edits, avoid-qualifiers, omit-needless-words, positive-form, are essentially eliminated by the second pass. these results, while preliminary, highlight the substantial headroom available to a principled agentic framework.

research plan

stream 1: defining and evaluating “tasteful” ai writing

our first contribution will be to define and evaluate what it means for writing to be clean and tasteful. to start, we will build on previously-proposed taxonomies of ai slop: both shaib et al. (2025) and chakrabarty et al. (2025) identify failure modes of llm-outputted text, through interviews with a small group of experts in nlp, writing, and philosophy. we plan to draw from these identified hallmarks of ai-slop as a starting point to enforce what “good style” in llm-writing would look like. notably, two dimensions are important when it comes to measuring ai-slop: one is information density, the amount of substantive content relative to text length, and another is information relevance, how well the content addresses the specific nuances of the prompt (shaib et al., 2025). the objective then, at a high level, is to produce writing that is both relevant and concise.

the goal of this stream will be to create a benchmark for “taste” in ai models, with the two dimensions of relevance and concision acting as guiding principles. notably, no benchmark (to the best of our knowledge) currently exists that focuses on enterprise-specific writing tasks: settings where clear, concise writing are increasingly important.

we will propose a benchmark, stylebench, designed to evaluate a model’s ability to transform first-draft inputs into polished writing across various domains. we plan to include tasks including: producing cohesive marketing strategies from messy notes, revising preliminary drafts into rigorous technical documentation, and refactoring disorganized code to adhere to established refactoring practices. we propose to collect the “rough draft” data from several online forums: messy marketing notes can be found on the enron email corpus. we plan to obtain disorganized code from open source github repositories.

stream 2: self-scaffolding agents for anti-slop writing

the second stream designs self-reflecting agents for anti-slop writing. our first hypothesis concerns the degree to which slop detection and repair (in writing, code, and documentation) can be automated with good agent scaffolding around known best practices (strunk, 2000; williams, 1990). specifically, we hypothesize the following: models are not very good at outputting “tasteful” content when asked during generation, but particular harness choices will make them much better:

1. distilling taste from best-practice guides: we hypothesize that, for humans, much progress in taste and style can be achieved by abiding by a checklist-structured handbook (williams, 1990; tufano et al., 2015), and that agents are now capable of picking up on good taste through access to such handbooks.
2. decomposing the rubric: we hypothesize that it is better to have the agent focus on improving one rubric principle per focused pass, instead of attempting to master all principles at once.
3. decomposing the llm-generated content (e.g., code or text): we hypothesize that it is hard for a model to attend to all sections of a document at once, and that critics should narrow their concerns to focus on one section at a time.

another design goal we propose is legible edits. best practices may conflict: omitting needless words can fight clarity; active voice can fight emphasis; parallel construction can fight concision. rather than resolving these tensions without the user knowing, the agent should surface them. we will have the harness emit, alongside its final output, a report of the conscious trade-offs it chose to make between conflicting best practices. this report serves as an interpretability artifact for the human reader and supervision for the evaluation framework of stream 1.

stream 3: reinforcement learning over the agentic harness

finally, we believe that small, open models can also develop such capabilities within the proposed harness, and large models may not be great at this out-of-the-box. we will use reinforcement learning (rl) on the agent to learn how to better utilize expert handbooks. our plan here is for rl to operate at the level of the harness: learning how to decompose the work, which lenses to apply in which order, when to stop revising, and how to allocate a fixed inference budget across passes. we propose to evaluate whether small open models post-trained within this harness can match the writing quality of larger frontier models operating without it. this would help establish the harness itself as a meaningful enabler allowing the model to pick up on “taste.”

slow ai: ai that matches a human’s pace.

my mind digests information at a much slower pace than i’d like to believe. in college, i could breeze through a math lecture at 2x speed, convinced that i was following everything the professor was saying, only to stare blankly at a problem set not knowing where to begin.

math became very enjoyable for me once i slowed down and acknowledged that absorbing new information takes longer than i typically would anticipate. i would pick out a handful of high-quality problems, learn them inside and out, notice exactly where i got stuck, make a few logical leaps, and return to the same problem the very next day with a fresh pair of eyes. after a certain point, i realized that i didn’t actually need to consume that much information at all. understanding a new topic well¹ had more to do with engaging deeply with a few core concepts.

i believe our brains are wired to get stuck on simple problems for extended periods of time. we might toil over the same problem for months – thinking about it in the shower, on the bus, from multiple different angles and perspectives. great scientists and artists often do so for years.

when information is hurled at us at 200 miles an hour – packaged in a fluent, convincing voice – the path of least resistance becomes to accept that information at face value, without taking the time to question, critique, verify, and make sense of it at our own pace.

ai can be used for slow thinking.

just as we now have widespread access to a tool for offloading thinking, we have an equally capable one for facilitating deep thinking, the type necessary to reach states of understanding and creativity: ai can follow a feyman-esque trail of questions, generate concrete examples on-the-spot, pull in documents, rubber-duck, and play devil’s advocate.

ethicist and cognitive scientist josh may offers a helpful rule of thumb for using ai in intellectual tasks: “you should use llms to generate inputs to your thinking, not outputs for others to read.”

designing slow ai.

lately, i’ve been thinking about ways to design ai to be more compatible with slow thinking.

first, an ai system should encourage the user to wrestle with the sequence of decisions that need to be made along the way to producing a final, polished response. just as understanding a mathematical proof requires wrestling with the underlying maze of failed paths along the way - one eventually feels solid about the sequence of logical steps that make a correct path correct - someone who receives an ai-generated response should be familiar with the major conceptual decisions behind the final response. decomposing the path of conceptual decision nodes that take questions to answers allows one to engage with alternative responses that could have been generated, reason through why those responses may have been valid (or invalid), and generalize those insights into future questions.

to test out this new mode of human-ai interaction, we created an interface that lays bare the messy conceptual roadblocks – the conflicting assumptions, interpretations, and frameworks – that shape an ai’s final response. the interface was a first attempt at allowing users to work through a space of possible decisions and resulting outputs, in the form of an interactive decision tree. confronted with a multiplicity of decision points, participants of our study felt a stronger sense of ownership over the final llm-generated responses. when compared to a traditional linear chat interface, users were surprised to find how long it took them to work through just one response,² and how much they learned from the differing viewpoints along the way. our design draws inspiration from the concept of multiverse analysis, a scientific method that specifies and runs a set of data-analytical choices, reporting results for each.

second, instead of generating standalone responses, an ai system could present responses in the form of deliberations between different parties of generative agents. a discourse-style interface could surface the hidden assumptions and tradeoffs underlying complex, multiperspective problems. simply reframing the response as a debate may be enough to invite users to read as critics rather than recipients.

third, ai memory should be designed to prevent hidden assumptions from quietly accumulating in the context window. in recent work, we find that, as chat histories progress, models tend to get caught in old pieces of code (see figure 2) or vestiges of earlier responses that are no longer relevant. this problem of models becoming “more repetitive, and sometimes subtly wrong” as chat histories progress is a familiar headache. rather than linearly accumulating a full conversation transcript in context – tunnel-visioning the model with past lines of reasoning – we can design smarter, more structured ways to condition on the past. one way is to create a wide-angled view of chat history, representing past conversations as knowledge graphs. the model then conditions only on a high-level summary of the past, just enough to guide retrieval, while seeing the full conversation details when they become relevant.

finally, the system should respect that not every human problem deserves to be touched by an ai. in the mid-70’s, joseph weisenbaum’s computer power and human reason warned against consulting machines on tasks that require deeply human traits like empathy and wisdom. thus, we can design tools to encourage users to reflect on their boundaries with ai. to test this out, we developed a chrome extension which allows users of chat interfaces to define (by placing a pin in a quadrant graph) how much involvement they’d like an ai assistant to have in different areas of their work and life - from direct, concrete responses to reflective questions thrown back to the user. based on the user’s preferred boundaries, the tool produces a memory file that users can upload to a chat interface to guide the ai’s level of involvement.

dangers of the growing culture of fast ai.

i worry that the culture surrounding autonomous ai is self-reinforcing: the less we engage, the harder it is to find our ways back to engaging.

when information is handed to us already distilled and neatly packaged, the line between our own understanding and ideas introduced by an ai agent begins to blur. without giving ourselves the time to think critically about what we are receiving, we risk drowning out our own voices. to make matters worse, post-training pipelines have been shown to incentivize agents to steer user behavior toward states that are easier to satisfy. indeed, claude user trends show that disempowerment patterns in real-world llm usage are growing over time. to date, the human line project has documented almost 300 cases of ai psychosis.

interestingly, recent work on self-distillation has shown that llms learn better and forget less when they explain new concepts to themselves. rather than feeding the model content it cannot relate to (e.g., off-policy expert demonstrations), having the model explain the concept in its own words allows it to fold information into its pre-existing knowledge in a more sturdy way. just like machines, humans are better able to digest new knowledge when they carry a self-awareness about where their current understanding begins and ends.

so, while it is useful to spawn an agent to speed up work that we wouldn’t gain much from doing ourselves (e.g., writing plotting code), we should be more selective in choices to outsource our thinking during processes of knowledge creation. a key question here lies in when to speed up and slow down. while ai no doubt provides incredible “boosts” of speed when used in the right places at the right times, operating at such high speeds also makes steering the direction of knowledge work much more difficult. without the time to properly digest information at a human speed, it is easy to fall into the trap of spending weeks down unproductive rabbit holes - circling around and missing the right solution (or even the right questions).

amidst a culture of fast ai, it is worth leaning into the slow-thinking mind, the one that was wired to get caught up in simple problems over extended periods of time. indeed, at the current speed of ai progress, our capacities for slow, deliberate thinking may turn out to be our defining superpower.

this post took shape through productive discussions with andre ye, mitchell gordon, marwa abdulhai, andy liu, omar khattab, smitha milli, sana pandey, deb roy, philippe laban, tamara broderick, and other wonderful folks at iclr 2026.