Index Check documentation
How the app works and what each result means. For quick setup steps and common questions, see support.
On this page: Overview · Plans · Setup · The storefront scan · Kinds of URL · What the results mean · Explaining a Search Console export · Noindex rules · Hide from search · The Search Console connection · Setup checklist · Password-protected storefronts · Troubleshooting · Limits · Data and uninstalling
Overview
Search Console's Page indexing report lists the URLs Google isn't indexing, under reasons like "Alternate page with proper canonical tag" or "Crawled - currently not indexed". On a Shopify store most of those URLs are filtered, sorted, tag and variant views of your real pages, and Google leaving them out is what you want. A few are real problems. Index Check tells them apart, in Shopify terms, and fixes the ones an app can fix.
What it doesn't do: it doesn't promise better rankings, change your theme's code, or edit anything in Search Console. It can't change a canonical tag your theme prints; when that's the fix, it says so.
Plans
| Free | Pro | |
|---|---|---|
| Storefront scan | Yes | Yes |
| Search Console export explainer | Yes | Yes |
| Noindex rules | No | Yes |
| Hide pages from search | No | Yes |
| Search Console connection with a weekly check | No | Yes |
Pro has a 14-day free trial. Development stores get every Pro feature at no charge. Current prices are on the Shopify App Store listing.
Undoing never needs Pro. If you move to Free, your noindex rules are kept but stop applying, and weekly Search Console checks stop. You can still turn rules off, unhide pages and disconnect Search Console.
Setup
- Install the app and approve the permissions. It reads products and content to know your store's URLs, and writes to products and content only to hide the items you choose from search.
- Scan your storefront. Click Scan storefront and keep the page open; it takes a minute or two.
- Turn on the app embed (for noindex rules). Click Open App embeds in the setup checklist, switch on Index Check and click Save. Scan again: the checklist shows the embed as On.
- Turn on the rules you need under Fix: noindex rules, then scan again to see them working.
- Connect Search Console (optional). Click Connect Search Console, sign in with Google, choose your store's property and click Check now.
The storefront scan
The scan reads your storefront the way Google does and checks one or two examples of each kind of URL your store has.
- robots.txt. It reads your robots.txt twice: once for Index Check, to know what it may fetch, and once as Googlebot, to know what Google may crawl. If robots.txt returns a server error or doesn't answer, the scan stops, because it can't tell what it's allowed to check.
- Sitemap. It reads your sitemap and up to 30 of its files, and sorts every URL it finds by kind.
- Pages. It fetches up to 45 URLs: one or two examples of each kind, starting with your home page, and follows the links your real pages show. For a kind your store has but doesn't link to (a vendor page, say), it builds one example URL from your store's own data, so every kind gets checked.
On each page it reads the HTTP status, the canonical tag, and any noindex (in a meta tag or the X-Robots-Tag header). It doesn't follow redirects; a redirect is itself a result. It identifies itself as GoodkeelIndexCheck/1.0 (+https://goodkeel.com/). The last 5 scans are kept.
Kinds of URL
| Kind | Example |
|---|---|
| Your real pages (should be in Google) | |
| Home page, products, collections, pages, blogs, blog posts, policies | /products/red-shirt |
| Variations of your real pages (usually best left out of Google) | |
| Products reached through a collection | /collections/shirts/products/red-shirt |
| Product variant URLs | ?variant=123 |
| Filtered collections | ?filter.v.availability=1 |
| Sorted collections | ?sort_by=price-ascending |
| Collection tag pages | /collections/shirts/cotton |
| Vendor and product type pages | /collections/vendors?q=Acme |
| Search results | /search?q=shirt |
| Blog tag pages | /blogs/news/tagged/sale |
| Recommendation links | ?pr_prod_strat=… |
| Tracking parameters | ?utm_source=…, ?gclid=… |
| Other query parameters | ?anything=… |
| Lists | |
| Collection pages 2 and up | ?page=2 |
| Counted, not checked | |
| Cart, account and checkout; other URLs | /cart |
URLs in a market or language subfolder (/fr/products/…) count as the same kind.
What the results mean
Each kind of URL gets the result of its worst example, the reason, the example URLs, and where the fix lives.
- Fine: a real page Google can index, with a canonical pointing to itself. Or a collection page 2 and up with its own canonical, as Google recommends for paginated lists.
- Harmless: a variation Google leaves out, or treats as a copy of the main page. That covers a canonical pointing to the main page, noindex, a robots.txt block, a redirect or a 404. Nothing to fix.
- Problem: a variation Google can crawl and index, with a canonical pointing to itself, so Google can index it as an extra copy of your real pages. For filtered, sorted, tag, vendor, product type and search pages, a noindex rule fixes it. For the others the fix is in your theme's canonical tag.
- Look at this: a real page that's noindex, redirects, returns "not found", is blocked by robots.txt, or has a canonical pointing elsewhere. That may be on purpose; check the page in your Shopify admin. Also used when a page returned a server error or didn't answer in time.
- Not checked: counted but not fetched (cart, account and checkout pages, and URLs the app can't place).
If your robots.txt lets Google crawl a page but not Index Check, the scan doesn't judge it.
Explaining a Search Console export
In Search Console, open Indexing > Pages, click a reason under "Why pages aren't indexed", then Export > Download CSV. Upload the zip in Explain a Search Console export. You can upload several at once (up to 12 files). The files are read and not stored.
Every URL is sorted by the kind of Shopify page it is and judged for that reason. The same reason can be harmless on a variation and worth a look on a real page: "Crawled - currently not indexed" on a sorted collection is what you want; on a product it isn't.
- Recognised reasons: server error (5xx), redirect error, blocked by robots.txt, excluded by noindex tag, soft 404, blocked by 401, 403 or another 4xx, not found (404), blocked by the page removal tool, crawled - currently not indexed, discovered - currently not indexed, alternate page with proper canonical tag, duplicate without user-selected canonical, duplicate where Google chose a different canonical, and page with redirect.
- The overview export (from the Pages report itself, not a reason) has counts but no URLs. Index Check shows the counts, with what each reason means.
- Other hosts. A Domain property includes every subdomain. URLs on a host other than your store's main one are grouped separately.
Noindex rules
Each rule adds <meta name="robots" content="noindex,follow"> to one kind of page, through the Index Check app embed. "follow" keeps the links on those pages crawlable. A rule can only add noindex; it never removes one your theme prints.
| Rule | Applies when |
|---|---|
| Collection tag pages | A collection is filtered by one or more tags |
| Vendor pages | The vendor collection page (/collections/vendors?q=) |
| Product type pages | The product type collection page (/collections/types?q=) |
| Search results | The search results page |
| Filtered collections | A collection or search page with an active filter, including a price range |
| Sorted collections | A collection sorted differently from its default order |
- Off by default. A rule is offered only when the latest scan found that kind of page as a problem. A rule that's on always shows, so you can turn it off.
- Not offered where a canonical already handles it. If those pages point Google to the main page with their canonical, adding noindex on top can send mixed signals.
- No rule for collection page 2 and up. Google recommends each page of a list keeps its own canonical and stays indexable.
- The app embed must be on. Rules are stored with the app and applied by the embed; with the embed off, nothing is applied. On the Free plan the embed writes a comment saying the rule wasn't applied.
To check a rule is working, scan again: that kind of page moves to Harmless. Or view the source of such a page and look for <!-- goodkeel-index-check: noindex rule=tag -->.
Hide from search
Hiding a product, collection, page or blog post sets Shopify's own seo.hidden setting on it. That adds noindex and nofollow, removes it from your sitemap, and hides it from your store's own search. Visitors can still open it.
- Choose products and collections with Shopify's picker, and pages and blog posts from the lists.
- Index Check won't hide an item that's already hidden by something else, so Unhide only removes what Index Check set.
- The list Hidden by Index Check shows what it hid, with Unhide for each.
- Uninstalling the app leaves hidden items hidden, because the setting is on your own products and pages. Unhide them first if you want them back in search.
The Search Console connection
Optional, and read-only. You sign in with Google in a new tab and grant Index Check permission to view your Search Console data. It can't change anything in Search Console.
- Property. You choose one of your Search Console properties. Only verified properties that cover your store's address are offered; Domain properties are listed first. Index Check reads data for that property only.
- The weekly check. Each week Index Check asks Google's URL Inspection about a sample of up to 50 of your URLs: up to 15 real pages that weren't indexed last time (to see whether that changed), up to 15 pages that get impressions in Google, then pages from your latest scan, your real pages first.
- Results. For each URL: indexed or not, Google's reason in its own words, the canonical Google chose when it differs, and Index Check's verdict for that reason on that kind of page. Changes since the previous check are listed at the top.
- Check now runs a check at once, at most once a day. The weekly check runs on its own.
- Limits. Google allows 2,000 inspections a day per property. Index Check uses at most 500 a day per store, and carries on the next day if it reaches that.
- Stored. Google's sign-in token, encrypted, and the results of the last 4 checks. Not your Google account's name or email address.
- Disconnect removes Index Check's access at Google and deletes the token and results. You can also remove the access in your Google Account (Security > Third-party apps and services).
The property must cover the address your store uses. If you haven't added your store to Search Console yet, add it there first (a Domain property covers every address). Search Console can't verify a store that's still password-protected, because Google can't read the verification. See how Index Check uses Google data.
Setup checklist
- Scan your storefront: Scanned, Scanning…, Storefront password needed, or The last scan stopped early.
- Turn on the app embed: On or Off, from the latest scan (it looks for the embed's marker on your home page). Scan again after switching it on.
- Turn on the noindex rules you need: how many are on, how many are suggested, or Nothing to fix.
- Connect Search Console (optional): Connected, Choose a property, or Not connected. Not counted in "steps done".
On the Free plan, the Pro steps read "Pro feature" and only the scan counts.
Password-protected storefronts
If your storefront has a password, the scan stops at the password page and a Storefront password section appears. Save the password there and the scan opens your store the way a visitor does. It's stored encrypted, sent only to your own storefront, never shown back or logged, and you can remove it at any time. Or remove the password in Online Store > Preferences.
Troubleshooting
A rule is on but the scan still shows a problem
View the source of one of those pages and look for a comment starting with goodkeel-index-check:
- No comment at all: the app embed is off, or the theme editor change wasn't saved.
goodkeel-index-checkwith nothing after it: the embed is on, but the page doesn't match the rule. Check you turned on the rule for that kind of page.rule=… not applied (Pro feature): the store is on the Free plan.
Shopify's storefront cache can take a few minutes to show a change. Then scan again.
The scan stopped early
The reason is shown. Usually robots.txt couldn't be read, or the storefront has a password (see above).
No Search Console property is offered
None of the properties your Google account can use covers your store's address. Add your store in Search Console (a Domain property covers every address), or disconnect and connect the Google account that has it.
Search Console asks me to connect again
Google stopped accepting Index Check's access: it was removed in your Google Account, or it expired. Connect again; your chosen property is kept if the account can still use it.
Limits
- The scan checks examples of each kind of URL (up to 45 URLs), not every page.
- It reads up to 30 sitemap files.
- The weekly Search Console check covers up to 50 URLs.
- Noindex rules cover the six kinds above. Other variations (variant URLs, tracking parameters, products reached through a collection) are handled by your theme's canonical tag.
- One store per installation.
Data and uninstalling
The app keeps your store's domain and access token, your scan results, what you hid through the app, your storefront password if you save it (encrypted), and, if you connect Search Console, an encrypted Google token and the results for your sampled pages. Nothing about your customers. When you uninstall, the app embed and its noindex tags go with the app, and we delete your store's data and revoke any Search Console connection. Pages you hid stay hidden (see Hide from search). See the privacy policy.