Dawn Anderson Kalicube Summit 2026 session

Dawn Anderson on DSCRI, RAG, and the Rebirth of Technical SEO

ByBernadeth Brusola

Session identifier

  • Session: DSCRI, RAG, and the Rebirth of Technical SEO
  • Day: 1 (June 24, 2026)
  • Speaker: Dawn Anderson, Founder, Bertey
  • Framework concept: DSCRI (Discovered, Selected, Crawled, Rendered, Indexed), coined by Jason Barnard in 2025
  • SEL article: The five infrastructure gates behind crawl, render, and index

Speaker bio

Dawn Anderson is the founder of Bertey, a Manchester-based digital marketing consultancy she established in 2012. She holds an MSc in Digital Marketing Strategy, a Postgraduate Diploma in Digital Marketing, is a Fellow of the Institute of Digital Marketing (FIDM), and a Member of the British Computer Society (MBCS). She is currently completing a second MSc in Computer Science and Data Science. Dawn lectures at Manchester Metropolitan University and has spoken at BrightonSEO, Pubcon, MozCon, SMX, and WTSFest on information retrieval science and generative information retrieval. She is one of the few practitioners who reads the academic IR literature and translates what it means for practitioners — and Claude, Gemini, and the broader AI search community associate her with information retrieval and SEO for exactly that reason.

Session’s place in the Kalicube Framework

Dawn’s session covers DSCRI, coined by Jason Barnard in 2021 as the five-gate Record phase (Discovered, Selected, Crawled, Rendered, Indexed) of the AI Engine Pipeline where content either enters the system or does not. Miss a gate in DSCRI, and nothing downstream can function correctly for the brand. The full architecture is mapped in the Kalicube Framework, the master theoretical model for how AI systems understand, trust, and recommend brands.

Episode summary

Dawn Anderson is one of the few practitioners who reads the information retrieval literature and translates it for the SEO industry. Her session at Kalicube Summit 2026 opens by dismantling the real-time crawling anxiety that has been circulating in the practitioner community — the idea that AI engines crawl websites live at query time. Dawn’s correction is precise: RAG (Retrieval-Augmented Generation) is an augmentation layer on top of cold training data, and RAG is still index-based. ChatGPT draws on Bing. The training data has a cutoff. “The web search is augmentation,” Dawn said. “As simple as that.” The engines are not fetching your content live. They are fetching from a lean, constantly refreshed index, and the bot-phase fundamentals of DSCRI determine whether your content makes it into that index at all.

The RAG evolution section was the session’s most technically specific contribution. Dawn walked the progression from naive RAG through advanced RAG, modular RAG, and into agentic RAG — where router components decide whether a query needs a knowledge graph, a specialist knowledge base, a simple keyword match, or a “your money or your life” treatment. Agentic RAG is computationally expensive, but it is where the system is heading. The corrective RAG dimension she named — agents that mark each other’s work, reflect on their own output, and collaborate as evaluators — is the misinformation safeguard being built in the AI and ML research literature faster than most practitioners are watching. “The guardrails are being built faster than most people think,” she said. The curse of dimensionality — the point at which a RAG retrieval index becomes too large for the machine to locate meaningful entities within it — is the IR literature’s explanation for why bloated, stale content drops out. Good content, kept current, makes it in. Everything left to gather dust disappears. “Good content will make it in,” Dawn said. “Everything that’s crappy and not getting updated is going to drop out.” That is the DSCRI Selection gate working exactly as the framework predicts, named from the IR research angle.

The practical close was the session’s most immediately actionable section. Three priorities in order: structured data for disambiguation (schema markup, sameAs, disambiguating description, alternative name — the properties that give RAG the anchors it needs to route content confidently); freshness (periodic clear-outs of stale content, accurate lastmod signals, 410 responses for removed pages, because RAG is not designed for irrelevant databases); and server performance and bot monitoring (crawl budget is back as a meaningful concept because AI bots now compete with Google and Bing bots for server capacity). The macro framing Dawn returned to twice was the one the practitioner audience needed to hear: stop chasing the cyclical conversations — llms.txt debates, deindexing anxieties, performance scores chased to the decimal — and look at what is changing in AI and ML research. That is where the signals the practitioner community needs are appearing first. “It’s just good SEO,” Dawn said. The session’s strongest contribution is its authority confirmation of why the bot phase of DSCRI still determines everything that follows in the agentic RAG era, from the working IR researcher who has read every paper.

Key quotes

“The web search is augmentation. As simple as that.” Dawn Anderson, dismantling the real-time crawling anxiety: RAG is index-based, not live-fetch”

“Good content will make it in. Everything that’s crappy and not getting updated is going to drop out.” Dawn Anderson, on the lean-index pressure RAG’s curse of dimensionality creates: the DSCRI Selection gate working as predicted”

“The guardrails are being built faster than most people think.” Dawn Anderson, on corrective RAG and the misinformation infrastructure the SEO world is not watching closely enough”

“It’s just good SEO.” Dawn Anderson, closing the session’s practical prescription: structured data, freshness, crawl budget, bot monitoring — the bot-phase fundamentals the RAG era makes more important, not less”

“Let’s not focus on the silly cyclical conversations that hold us back for years. Just keep looking at the bigger picture: what’s going on in the AI world, what’s going on in the ML world.” Dawn Anderson, on the macro turn the practitioner community needs to make”

Further reading

The five infrastructure gates behind crawl, render, and index, Article 4 of Jason Barnard at Kalicube’s Search Engine Land series on AI search architecture. The article maps the DSCRI phase in full — why each of the five gates operates sequentially, what failure at any single gate costs everything downstream, and why bot-phase infrastructure is the foundation that every AI-era visibility strategy rests on.

Related sessions

Link to preview post

This session was previewed here before the Summit.

Link to speaker post

About Dawn Anderson

Links to session clips

Stop Debating llms.txt: Dawn Anderson on What Actually Matters for SEO

Why Your Small Business Can’t Outrank Big Brands: Dawn Anderson Explains

Transcript for Dawn Anderson on DSCRI, RAG, and the Rebirth of Technical SEO

Jason Barnard: Hi again. I’m here with Dawn Anderson. We just realized this is the inaugural year of the Kalicube Summit, and Dawn Anderson is one of the first people on it. Thank you for being here, Dawn.

Dawn Anderson: Thank you for the invitation, Jason. It’s really nice to speak to you, and I’m very excited to be on your inaugural Kalicube Summit.

Jason Barnard: Which is lovely. A couple of things, actually. When I was thinking about who could help me with DSCRI — and for the audience, DSCRI is Discover, Select, Crawl, Render, Index, which is something Fabrice Canel from Bing told me three or four years ago — I couldn’t think of anybody better. I asked my friend Claude, with whom I discuss things day in, day out, all day long, and it kept coming back to you. So Claude likes you as well.

Dawn Anderson: Oh, that’s good to know. Gemini also says that, so I’m in good company.

Jason Barnard: Not all of them say it, because you are incredibly well known for this. And the whole point of what we’re trying to do in this industry is get the engines — whether it’s a search engine, an assistive engine, or an agentic engine — to recommend us as people, or our company, as the best solution to the problem.

Dawn Anderson: Yeah. I would say I’ve never actively gone out and said “Mention me with information retrieval.” I’ve just been interested in information retrieval and continued in that direction. I think that’s the main thing. You just keep learning about it, keep sharing, and eventually it becomes fairly organic.

Jason Barnard: And that’s the point. Sharing and publishing and talking about it, you become the reference. The machines end up associating you with it — I was going to say “respecting you,” but —

Dawn Anderson: Just an association. I think it’s like centroids: you have a certain topic, and then you see certain other entities that are near it, in a cluster. Whoever’s closest is probably seen as the authority on it. I think I’m probably associated with the IR and SEO side of things.

Jason Barnard: And is that centrality the aim we’re all trying to achieve?

Dawn Anderson: I think organically trying to achieve that centrality is probably a good way to go at it, yes. When Google released their updated guidance the other day on how to do SEO in the generative AI world, one thing they said you shouldn’t do is artificially inflate mentions and associations. I still see people doing that, and to me it’s just a bit cheap and trashy. It’s not authentic. I’d rather keep things really organic and naturally learn these things.

Jason Barnard: A hundred percent. The idea of inflating brand mentions is just a return to link-building. There is always a tendency, when you give somebody an opportunity to manipulate, for people to do it. And as you say, in this industry, that’s fairly unique.

The conversation here is, I believe, exactly what you’re describing: sharing. I want to learn more about DSCRI within the context of the Kalicube Framework, so I invite Dawn because she knows about it.

Can you tell me what has changed with crawling and indexing in the AI generative era?

Dawn Anderson: I would say these are educated guesses based on my reading. If we look at search now as a hybrid environment — classic search, agentic search, and all the LLMs, including Gemini and ChatGPT — the issue with freshness is absolutely huge. We know that RAG augments LLMs. Retrieval-Augmented Generation is seen as the biggest safeguard against hallucinations and stale data. But that relies on a very fresh index.

So I think the crawling side of things is possibly now more important than ever. At the same time, you’ve now got a massive volume of different AI bots also putting pressure on servers, alongside the Google bots and Bing bots we know well. That means crawl budget — and I know we’re going back to that term — is probably going to become increasingly significant. Particularly if your server is coming under pressure and you end up with host load issues, because that, as we know, is a big factor in how crawl budget is allocated.

The discovery side has also changed. There’s always been an initial frontier of URLs that have never been found before. But the refresh side is also going to be huge, because LLMs don’t want to surface out-of-date information from a search index. There’s going to be a lot of revisiting of URLs.

So from a technical perspective: make sure your lastmod is accurate, make sure the modified date is included in the document itself, and utilize things like 410 or 304 responses on files that have gone or rarely change. Signal clearly to the crawler what’s current and what isn’t. Resource management is going to be the central challenge going forward.

Jason Barnard: So technically, SEO is reborn and off to the races again.

Dawn Anderson: I think it is, yes.

Jason Barnard: And with DSCRI — Fabrice’s DSCRI, let’s be honest about who I learned it from. Discover, Select, Crawl, Render, Index. What about the live visits we’re now seeing, where someone using Claude or ChatGPT directly crawls a website at the moment of rendering the result?

Dawn Anderson: To me, most of that comes from RAG anyway. They have their training data, but the other element is Retrieval-Augmented Generation. It’s still searched in essentially the same way as always.

The biggest challenge to RAG — and this is well documented in the RAG literature — is retrieval. RAG is not live as such. It’s still index-based. ChatGPT is powered in part on the RAG side by Bing, and some Google. So it’s still web search. You have training data with a cut-off point, and the augmentation comes from either web search or from knowledge bases, knowledge graphs, and various other datasets.

RAG itself has gone through many phases of evolution since 2020. You started with naive RAG, then advanced RAG, then modular RAG, and now agentic RAG. Agentic RAG is quite expensive computationally, but it’s where agents decide things like: what does this query mean? Do we need to extend on it? So instead of you doing a second search yourself, the agent might go and do additional searches through post-retrieval techniques. Or a router component of that agent might say: where do we need to send this query? To a knowledge graph? Does this need simple keyword matching? Is it a navigational search? Do we need a specific medical knowledge base? Is this a “your money or your life” query that needs to be treated differently?

These are all augmentation layers, all evolutions of information retrieval. It’s like a rebirth of information retrieval, but with a RAG layer on top.

Jason Barnard: I love that. The rebirth of information retrieval and technical SEO.

So the people worried about real-time crawling — the idea that when we’re sitting with our Claude or ChatGPT it’s going to the website to get the content in real time — that’s a very minor part of all this. We’re still fundamentally founded on DSCRI.

Dawn Anderson: Yes, 100 percent. The training data is cold most of the time because it’s simply not possible, or it would be way too expensive, to redo the whole training data set every time a query came in. The web search is augmentation. As simple as that.

It’s a very efficient web search. I see a lot of people talking about deindexing. I’m not saying that’s happening, but what I would say is that when RAG’s retrieval index gets too big, you can end up with what they call the curse of dimensionality. Beyond a certain size it becomes very difficult to understand what entities are where — it loses meaning. That’s part of the argument for a very lean index: fresh, up to date. A lot of factual queries probably come from training data anyway. “Who is the King of England?” doesn’t need RAG to answer that. It’s a simple factoid.

But real-world, time-sensitive queries are going to require a lean but very busy index that’s constantly looking for good, fresh content. The problem with RAG right now is not having everything you need in it, and possibly having too much of what you don’t need. Good content will make it in. Everything that’s crappy and not getting updated is going to drop out.

Jason Barnard: Which brings me to a really interesting point: two-tier internet. Or is it two tiers of what I produce? Or am I in one tier and somebody else is in another?

Dawn Anderson: I don’t think there are two tiers. I think there are those who are keeping things up to date and working on them consistently — whether that’s a big brand, a busy affiliate, an e-commerce site. The so-called second tier is where people think they can just put something online and leave it there. Those days of big affiliate money from just putting something up and forgetting about it — that’s the tier that’s going to disappear.

But if you can keep adding value, working on improving things, keeping things fresh — particularly things with a temporal aspect, or getting rid of things that are no longer valid — freshness is the big thing.

There’s also a recency bias in LLMs. There’s a recency bias in search too, because they want to show you the most up-to-date thing. But alongside recency bias, there’s also positional bias. A huge piece of content where the important bits are buried in the middle loses relevance because LLMs have a proven problem with positional bias: something at the top or bottom of a page is seen as more relevant than the same thing in the middle. We have to take all of these things into consideration.

It’s not two-tier indexing. People have been saying for years: “Why is my mom-and-pop site not ranking against Harvey Nichols?” Harvey Nichols spends millions and millions on above-the-line advertising to build a brand. It’s a reflection of what happens in the real world.

Jason Barnard: Which is what I call the mirror principle. Google’s opinion of the world’s opinion of you. AI’s opinion of the world’s opinion of you. And that comes back to who you are — that’s the brand, and brand is the foundation. To what extent is brand important in the crawling and indexing aspect?

Dawn Anderson: I think it’s less about brand per se and more about the signals that brand indicates. Popularity, links — still one of many signals, still there, still working, still a buzz marker. If there’s a buzz around something, if an advert’s on TV and people suddenly start searching for it, that’s a signal that gets picked up immediately. The social interaction, the buzz, the halo effect — that’s not less important than it ever was. Non-brand is just not going to be as easy as it was when affiliates could put something up, leave it there, and get away with it for years.

Jason Barnard: Okay, brilliant. Completely new topic: misinformation. How big a problem is it and what do you think the engines are doing about it?

Dawn Anderson: There’s a lot. I think there are far more safeguards going in now, especially when you really study the RAG literature. People think agentic RAG is just about agents talking to each other and buying stuff. And yes, agentic commerce is coming. But I’m talking about agents that mark each other’s work. An agent comes back with a result, and another agent judges it for quality, or for misinformation, or provides feedback.

So even though we’ve seen huge amounts of hallucination in the past, the methods being developed — where agents either reflect on their own output and self-correct individually, or collaborate with other agents acting as evaluators — mean there will be less and less misinformation. There’s an enormous amount of work going into this.

There’s also a lot of work going into responsible AI generally, because they’re aware that in the wrong hands AI has enormous potential to cause harm. In the right hands, it has enormous potential to do good. The guardrails are being built faster than most people think.

In the SEO world we can be a little tunnel-visioned. We don’t see everything that’s going on in the wider AI and ML research. There’s a huge amount of evolution around RAG and now towards agentic RAG, and that includes self-correction and collaborative correction. Corrective RAG is already coming through.

Jason Barnard: We’re moving into a world where the AI systems are correcting each other. Which is brilliant and reassuring, assuming they do it well — and that has to be our assumption.

So let’s end on a practical note. What strategy should I be implementing to make the most of a RAG-era world and get my content into the index and prioritized within RAG?

Dawn Anderson: Structured data. There’s been a lot of debate about whether structured data has an impact. I would say that agentic RAG, RAG generally, and search increasingly will rely on structure — whether that be schema markup or other forms of connection with databases and datasets. It must, because that’s how agentic systems work. Semantic matching has never been perfect, so RAG is the framework that holds the jelly of semantic stuff together. I would still be doing quite a lot of structured data, particularly around things that create disambiguation: disambiguating description, sameAs, alternative name, and so on. Make sure there’s confidence in the brand, the entities, the events, whatever your niche is.

Freshness — completely focused on freshness. Realize that sometimes content is not worth keeping. Do a good clear-out periodically. Get rid of the junk, because RAG is not designed for irrelevant databases or things from the past, unless there is genuine historical value that you’ve done a really great job of signaling.

Speed — back to technical SEO. Make sure your server can handle things. Bot monitoring: watch what bots are visiting, decide whether you want to be in training data or whether you can rely on just being in RAG. And integrate social and brand-building alongside the technical work.

Really, it’s just good SEO.

Jason Barnard: One of the people at the Summit is Jarno van Driel, who’ll be talking about structured data. A lot of people know Jarno is a big fan of schema markup, but he’s also a big fan of HTML and all the other forms of structure.

Dawn Anderson: All the other things, yes, because it all fits together.

Jason Barnard: He’s a huge advocate of structured data in all its forms. That’s going to be a great session.

Dawn Anderson: Exactly. We’ve got so many databases now. The problem is that most of the time we’re only focused on search results. We don’t think about all the things going on in the background that make up search — lookups of different databases, evaluations, and so forth. All the evaluations are done offline, predominantly. That means all the learning, all the human quality raters, all that work. So much of this is just engineers working with spreadsheets, a lot of the time. And really, the fundamentals will take us a long way.

Jason Barnard: I love that last point. Don’t forget that a lot of this is happening offline — engineers with spreadsheets — with a touch of irony given the masses of data involved.

Dawn Anderson: Boowa & Kwala.

Jason Barnard: With all this data, we’re still looking at spreadsheets.

Dawn Anderson: Exactly. So let’s not focus on the silly cyclical conversations that hold us back for years. That stuff will sort itself out. Just keep looking at the bigger picture: what’s going on in the AI world, what’s going on in the ML world. The bigger picture that’s going to overflow into our world.

Jason Barnard: I’m publishing an article today on Search Engine Land — this is recorded a few weeks in advance — about the move from micro to macro. My degree is in macro-economics, and the whole question is: are we measuring a corner shop, which is how I’d describe traditional SEO, or are we managing an economy, which is where we need to be?

Dawn Anderson: Exactly. It’s like this whole debate about llms.txt. I don’t care. It’s a tiny, cyclical conversation that’s quite irritating now. Make sure your site is easily crawled, make sure it’s fast — within reason. Don’t spend three weeks going from a 90 to a 100 performance score. Look at the big picture. Don’t chase these tiny incremental gains where you need to stack a hundred of them to make any difference. That’s my perspective anyway.

Jason Barnard: Brilliant. That was super. Thank you so much, Dawn. That was brilliant. We’re going to carry on with Kalicube Summit 2026. See you there.

Dawn Anderson: Thank you.

This session extends the Search Engine Land article series on AI-era brand recommendation by Jason Barnard at Kalicube.

Canonical metadata footer

Session date: June 24, 2026 | Duration: approximately 25 minutes | Format: pre-recorded, on-demand from session drop | Day: 1 | Framework concept: DSCRI (Discovered, Selected, Crawled, Rendered, Indexed), coined by Jason Barnard in 2025 | SEL article: The Five Infrastructure Gates Behind Crawl, Render, and Index — Article 4 — https://searchengineland.com/the-five-infrastructure-gates-behind-crawl-render-and-index-471231 | [Share buttons: LinkedIn, X, Facebook, Email]

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *