Automated Keyword Mapping & Local AI Semantic Discovery

​

Executive Overview (DBX v5.5.2)

In enterprise digital marketing, keyword research is often fragmented between isolated tools: web scrapers, third-party keyword generators, Google Ads Keyword Planner spreadsheets, and rank trackers. This disconnect results in keyword cannibalization, missed topical clusters, and uncoordinated URL targeting. YPYM DBX v5.5.2 introduces Automated Keyword Mapping (/keyword-discovery), a linear 4-stage pipeline that bridges product positioning to media planning and rank tracking:

1. Zero-Third-Party Landing Page Extraction: Captures product names, value propositions, key features, and competitor benchmarks directly from HTML markup.
2. Local Ollama AI Semantic Engine: Formulates foundational root seeds using self-hosted Qwen 2.5 models without sending confidential briefs to external commercial APIs.
3. Dual-Bid CPC Intelligence: Connects to Google Ads Keyword Planner historical data, providing exact search volumes and both Min CPC (low-range bid) and Max CPC (high-range bid) in Indonesian Rupiah (IDR).
4. Cannibalization Guardrails & URL Slug Mapping: Automatically maps queries to relative target slugs, distinguishing the Primary Target keyword from Supporting LSI semantic variants on shared URLs.

---

​

Technical Architecture

The module operates as a decoupled, multi-tier pipeline inside the YPYM DBX platform monorepo:

---

​

The 4-Stage Operational Workflow

The interface is structured into four sequential tabs, accessible directly above the main workspace canvas:

​

1. Initial Brief & Product Profile (Tab 1)

Users define the core product parameters:

• Product Name & Category: Core identifying label and industry taxonomy.
• Landing Page URL: The target URL crawled to extract semantic headings, copy, and positioning.
• Target User Segment: The primary buyer persona (e.g. UMKM, Enterprise, B2B Procurement).
• Competitor URLs: Between 1 and 10 competitor reference links used to expand topical coverage.
• Brand Terms & Aliases: Dedicated list of trademarked phrases to isolate branded from generic search queries.
• 1-Click Brief Loader: Includes a Load Example Brief shortcut for instant pre-population during initial onboarding and testing.

​

2. Live Logs & Raw Candidate Extraction (Tab 2)

Once analysis starts, the crawler executes asynchronously:

• Live Terminal Telemetry: Displays timestamped operational logs ([INFO], [SUCCESS], [WARN]) for HTTP fetching, HTML tag extraction, Ollama inference, and Google Ads metrics matching.
• Raw Candidates Table: Lists all candidate terms before curation with search volume, dual CPC benchmarks (Min CPC and Max CPC), competition level, search intent, and data origin (AI Seed vs Google Ads).
• Raw Keyword Management: Users can delete low-relevance raw terms directly before proceeding.

​

3. Review & Keyword Curation (Tab 3)

The curation workspace allows users to filter, inspect, and approve search queries:

• Search Intent Filters: Segment candidates by Commercial, Transactional, Informational, or Navigational intent.
• Topical Clusters: Group keywords by business vertical (e.g. Finance, Banking Cluster, Features & Operational, Brand & Core Solution).
• Suggested URL Slug: Shows the relative target path (e.g. /pinjaman/nobu-kur or /pinjaman/nobu-kur/pricing).
• Cannibalization Guardrails: When multiple keywords share the same URL slug, DBX automatically assigns roles:
  • Primary Target: The highest-volume query assigned as the main focus keyword for page title and H1.
  • Supporting LSI: Secondary semantic variants intended to be woven naturally into the body copy of the same page.
• Optimistic Checkbox Selection: Checkbox toggles update the local interface instantly (0ms latency) while persisting state changes to PostgreSQL in the background.

​

4. Approved Final Keywords & Export Suite (Tab 4)

Approved queries transition into the final repository:

• Executive Metrics Summary: Real-time totals for Approved Final Keywords, Aggregated Monthly Search Volume, and Portfolio Average CPC.
• Multi-Format Export Suite:
  • Download .txt: Line-delimited keyword list for instant import into Google Ads Campaign Editor or Google Sheets.
  • Download .csv: Full 9-column tabular data export (Keyword, Search Volume, Min CPC (IDR), Max CPC (IDR), Avg CPC (IDR), Competition, Search Intent, Topical Cluster, Suggested URL Slug).
  • View JSON: Formatted JSON modal for developer API consumption and programmatic ingest.

---

​

Production Deployment & Local AI Engine Architecture

​

Does AI Require a Separate Project on VPS?

No. The local AI engine does not require an independent Git repository or detached application deployment. DBX communicates with the AI engine over standard internal HTTP REST endpoints. In production environments (Coolify, Docker Compose, or Bare-Metal VPS):

1. Ollama Service Container: Ollama runs as a container service alongside PostgreSQL, Redis, and DBX Web:
   services:
     ollama:
       image: ollama/ollama:latest
       container_name: ypym-dbx-ollama
       restart: unless-stopped
       ports:
         - "11434:11434"
       volumes:
         - ollama_storage:/root/.ollama
       environment:
         - OLLAMA_KEEP_ALIVE=24h
   
2. Environment Variable Configuration: Inside apps/web/.env.production, configure the inference endpoint:
   # Local Ollama Inference Endpoint
   OLLAMA_BASE_URL=http://ollama:11434
   KEYWORD_DISCOVERY_MODEL=qwen2.5:32b
   
3. Recommended Model Sizing by Server Specifications:
   • GPU-Accelerated VPS (NVIDIA T4 / A10G / RTX 4090): Use qwen2.5:32b or qwen2.5:14b for deep semantic nuance and sub-2-second inference.
   • CPU-Only VPS (8 vCPU / 16 GB RAM): Use qwen2.5:7b-instruct-q4_K_M. Requires only ~4.5 GB RAM and completes inference in 3 to 4 seconds.
4. Zero-Downtime Deterministic Fallback: If the Ollama service is temporarily restarting or downloading a model update, the backend pipeline automatically activates its built-in deterministic heuristic engine. The user’s crawl and keyword generation workflow never fails with a 500 server error.

---

​

Database Persistence & Audit Schema

Keyword mapping operations are persisted across dedicated PostgreSQL 16 tables:

---

​

Summary of DBX v5.5.2 Enhancements

• Dual-Bid CPC: Low and high bid ranges calculated directly in IDR.
• Pure Numerical Search Volume: Clean digits formatted without arbitrary /mo labels.
• Segmented Control Tabs: Unified navigation matching standard DBX interfaces.
• Optimistic State Management: Zero-delay checkbox curation with background PostgreSQL synchronization.
• Direct Export Options: Clean .txt, full .csv, and developer-ready JSON.