A scheduled embedding job keeps a WordPress site’s retrieval-augmented generation (RAG) index in step with its content without re-embedding the whole site on every run. The pattern is consistent across implementations: detect which pages and posts changed, fetch only those, strip and chunk their text, generate embeddings, replace their existing vectors, and record what was indexed so the next run can repeat the comparison. Two documented approaches exist today: an external n8n workflow that syncs WordPress and WooCommerce content into Pinecone, and an experimental native WordPress implementation that stores embeddings in MariaDB.
This guide is for site owners, developers, and automation builders who already have a WordPress site and want to keep a RAG index fresh rather than rebuild it by hand. It covers the pipeline stages, the two documented implementations, the change-detection logic that makes scheduled jobs safe, and the failure modes worth monitoring.
The seven stages of a scheduled embedding job
Recurring RAG syncs differ in tooling but not in structure. The documented n8n workflow and the native WordPress experiment both implement the same stages in some form:
- Trigger. A schedule or manual start begins the run. The n8n template runs on a manual trigger or a daily cron schedule.
- Enumerate. The job lists published content and its modified timestamps. The n8n workflow fetches up to 100 published WordPress pages and up to 100 published posts, capturing IDs and modified timestamps through the WooCommerce/WordPress REST API.
- Diff. Live IDs and modification dates are compared against a record of what was already indexed. The n8n workflow uses an n8n Data Table named
indexed_wc_blog_v1and selects up to a configurablestaticBatchSizeof pages and posts that are new or changed. - Fetch. Full content is retrieved only for the selected batch, from the WordPress pages and posts endpoints, and the results are combined.
- Normalize and chunk. HTML is stripped, metadata is prepared, and the text is split into chunks sized for the embedding model.
- Embed and upsert. Chunks are sent to an embedding provider and the resulting vectors are written to the vector store. The n8n workflow deletes any existing Pinecone vectors for the same
fileIdbefore inserting the new chunks. - Record and prune. The item’s identifier and
dateModifiedare written back to the tracking table. Entries that exist in the tracking table but no longer exist in WordPress have their vectors and rows deleted.
Stages 3 and 7 are what separate a scheduled job from a full re-index. Everything else is mechanical.
Path one: an external n8n workflow with Pinecone
The n8n workflow template is the more complete documented reference for the external approach. It targets sites that want the index to live outside WordPress, in a managed vector database.
What the workflow does
After the diff step, the workflow fetches full content for the selected batch, strips HTML, prepares metadata, deletes existing Pinecone vectors for the same fileId, splits the text, generates OpenAI embeddings, and inserts the chunks into Pinecone. It then upserts the page or post’s fileId and dateModified into the indexed_wc_blog_v1 Data Table. In the same run, it identifies tracking entries whose content no longer exists in WordPress and removes both the vectors and the tracking rows.
Setup requirements
- WooCommerce API credentials with access to the WordPress REST API endpoints for pages and posts.
- An OpenAI API credential for creating embeddings.
- A Pinecone API credential, a selected Pinecone index, and values for the index host and namespace.
- An n8n Data Table named
indexed_wc_blog_v1with text columnsproductId,indexedAt, anddateModified. - Configuration values for
wooBaseUrl,pineconeIndexName,pineconeIndexHost, andpineconeNamespace.
The delete-before-insert behavior is the part that makes reruns idempotent: if the same post is processed twice, the second run replaces its vectors rather than duplicating them.
Path two: native vector search inside WordPress
A draft pull request to the WordPress ai repository, “Add native vector search” (PR #683, opened 2026-06-08, updated 2026-07-15), proposes a leaner alternative that avoids an external vector database entirely. It is labeled as a draft and carries a [Status] Blocked label, so it should be treated as an experiment rather than a production feature.
How the native design works
The proposal adds an opt-in rag-search experiment backed by MariaDB native VECTOR(1536) storage and a cosine VECTOR INDEX, using text-embedding-3-small embeddings. Activation is gated on two conditions: MariaDB 11.8 or later and an authenticated OpenAI connector. A fallback path stores embeddings in post meta and searches them in memory, which the proposal describes as performant enough for smaller sites.
For automation, the relevant pieces are the indexing lifecycle hooks, a one-hour dirty-post cron schedule, short batch indexing, transactional replace-by-post behavior where possible, cleanup hooks, and wp ai rag WP-CLI utilities. The user-facing additions are RAG results merged into the public search page and a related-posts feature comparable to Jetpack’s.
Why the MariaDB dependency matters
The rationale in the pull request is that MariaDB 11.8 LTS ships a community-available vector index implementation, so WordPress can offer RAG search without adding another runtime service or PHP dependency. The same document notes that MySQL 9 does not have vector search capabilities despite having a vector store. That makes the host’s database engine, not the WordPress version, the deciding factor for the native path.
Why change detection decides whether the job is safe
The tracking table is the source of truth for what has already been embedded. Two design choices follow from that.
First, the job trusts dateModified. If content changes without that timestamp being updated — a direct database edit, an import script, or a plugin that writes content without touching the post’s modified date — the diff will not select the item and the index will serve stale text. Any automation that writes post content should be checked for this behavior.
Second, deletion handling must be explicit. The n8n workflow derives deletions by comparing the tracking table against live WordPress IDs, then removes the corresponding vectors and rows. Without that step, deleted posts continue to appear in retrieval results indefinitely.
A third consideration is the embedding model itself. Vectors from different models are not interchangeable, and the native experiment fixes the first schema at 1536 dimensions. Changing embedding models or dimensionality generally requires re-embedding the entire corpus, so the model choice should be treated as a durable configuration decision rather than a tunable.
An illustrative run: a 400-post site on a daily schedule
The following is an illustrative scenario, not a tested configuration or a measured result. It shows how the documented parameters interact.
Suppose a site has 400 published posts and pages, and editors change roughly five items per day. With the n8n template’s daily cron trigger, each run enumerates up to 100 pages and up to 100 posts. Because enumeration is capped, a site with more than 100 published items of a given type will not see all of them in a single run; over successive days the diff still converges, but the coverage window per run is bounded. The diff then selects only the new or changed items, up to staticBatchSize, so a typical daily run processes a handful of posts rather than the full corpus. Each processed item has its old vectors deleted, its chunks re-embedded, and its dateModified recorded.
The practical implication is that batch size and enumeration caps, not the schedule alone, determine how quickly the index catches up after a large content migration.
Failure modes and diagnostic checks
These are the conditions most likely to produce a silently stale or duplicated index.
- Duplicate chunks. If the delete-by-
fileIdstep fails while the insert succeeds, retrieval returns repeated passages. Check vector counts perfileIdagainst the number of chunks the item should produce. - Orphaned vectors. If the deletion pass fails, removed posts remain retrievable. Compare the tracking table against live WordPress IDs and verify that missing IDs have no vectors.
- Stale content with an unchanged timestamp. Confirm that the editing path actually updates
dateModifiedbefore assuming the diff logic is broken. - Partial batches. A run interrupted mid-batch can leave some items embedded and others recorded as pending. The upsert-then-record ordering in the n8n workflow means an interruption between those two steps causes the item to be reprocessed next run, which is the safer failure direction.
- Dimension or model mismatch. A vector store configured for one embedding dimension will reject or mis-handle vectors from a different model. Verify the index dimension matches the embedding model in use.
- Database version gate. On the native path, a host running MySQL 9 rather than MariaDB 11.8 or later cannot activate the
rag-searchexperiment at all.
Verification
For the n8n workflow, verification is a comparison exercise: count vectors in the Pinecone namespace, count rows in indexed_wc_blog_v1, and confirm that every live published page and post that should be indexed appears in the table with a current dateModified. A retrieval test against a recently edited post is the fastest end-to-end check.
For the native experiment, the pull request documents a manual wp-env smoke test using wp ai rag status as the first command, with an observed environment of WordPress at http://localhost:8888 and MariaDB 12.3.2. That status command is the intended starting point for confirming whether RAG is available on a given host.
Limitations and open questions
The native path is not a released feature. PR #683 is a draft with a blocked status, so its design, gating conditions, and CLI surface may change before or instead of shipping. Its schema is fixed at 1536 dimensions in the first version, and the proposal explicitly lists moving to embeddings supplied by the WP AI Client as unfinished work.
Neither source documents a local embedding model path for this workflow. The n8n template uses OpenAI embeddings and the native experiment gates on an authenticated OpenAI connector, so sites that require fully local embedding generation are outside what these two sources cover. Embedding cost and throughput are likewise not quantified in either source; any cost estimate should be built from the provider’s current published pricing and the actual chunk volume of the site.
Finally, the n8n template is a community-published workflow template rather than official WordPress or WooCommerce documentation. Treat its parameter names and defaults as a starting configuration to review, not as a specification.
Next steps
If the index already exists and only needs to stay current, start by confirming that the tracking table or Data Table is populated and that dateModified is reliable on your site. That single check determines whether the diff stage will do its job. If the index does not exist yet, run a full initial indexing pass first, then enable the schedule; a scheduled job assumes a baseline to compare against.
Readers who have run either approach are invited to share their environment, batch settings, and any errors encountered in the community, since the practical constraints of scheduled embedding jobs are better documented by operators than by templates.
FAQ
Does a scheduled job re-embed the entire site every time it runs?
No. The documented n8n workflow compares live page and post IDs and modified timestamps against the indexed_wc_blog_v1 Data Table and selects only items that are new or changed, up to a configurable staticBatchSize. The native WordPress experiment takes a similar approach with one-hour dirty-post cron scheduling and short batch indexing.
How are deleted WordPress posts removed from the vector index?
In the n8n workflow, entries that exist in indexed_wc_blog_v1 but no longer exist in WordPress have their vectors deleted from Pinecone and their rows removed from the Data Table. Without this step, deleted content would remain retrievable.
Can the native WordPress vector search be used today?
It is proposed in a draft pull request (WordPress/ai PR #683) that carries a [Status] Blocked label, so it should be treated as an experiment rather than an available feature. It also requires MariaDB 11.8 or later and an authenticated OpenAI connector.
Why does the database engine matter for native RAG search?
The proposal's rationale states that MariaDB 11.8 LTS ships a community-available vector index implementation, while MySQL 9 does not have vector search capabilities despite having a vector store. The feature is therefore gated on a supported MariaDB version.
What happens if the embedding model is changed later?
Vectors produced by different models are not interchangeable, and the native experiment's first schema is fixed at 1536 dimensions. Changing models or dimensionality generally requires re-embedding the corpus rather than mixing old and new vectors.