AppWT — IndexNow Was Submitting 11 URLs Instead of 14,442 for 6 Weeks

Digital Agency (AppWT internal)

The Problem

AppWT was running a daily IndexNow ping script (/usr/local/bin/indexnow-ping.sh) that submitted appwt.com URLs to Bing, Yandex, Naver, and Seznam search engines for recrawl. Discovery: the script was grep'ing <loc> tags from /sitemap.xml — but appwt.com's /sitemap.xml is a sitemap-index (a list of 8 child sitemaps), not a list of URLs. The script was telling search engines to recrawl 11 sitemap files when the actual catalog was 14,442 page URLs split across the 8 child sitemaps. AppWT had been operating at 0.08% of available IndexNow signal strength for ~6 weeks.

The Approach

1. Identified the bug by reading the indexnow-ping.sh logic and noticing it stopped at the top-level sitemap
2. Rewrote the script to recursively walk sitemap-index → child sitemaps → extract page URLs
3. Added 10,000-URL batching (IndexNow protocol limit) with --data-binary @file via stdin (avoids ARG_MAX shell limit)
4. Re-ran the daily push manually to recover the lost 6 weeks of signal
5. Memory pin saved (feedback_universal_freshness_indexnow_method.md updated)

The Result

Single run pushed 14,442 appwt.com URLs to Bing across 3 batches (10,000 + 10,000 + 5,226), all returning HTTP 200. Combined with the same fix applied to other AppWT-managed sites (templates.appwt.com: 2,778 URLs; total network: 28,000+ URLs in one run). Search engines now have a complete recrawl signal for the first time in 6 weeks. Bing/Yandex recrawl expected within 24-48 hours; full AI-engine index propagation 2-6 weeks.

Measurable Outcomes