# **🔍 Overview**

Search Atlas needs to access your website in order to crawl pages, run audits, and power OTTO SEO. If your staging environment is protected by a **Vercel deployment passcode**, Search Atlas cannot crawl it — the passcode blocks all automated crawlers, including ours. This article explains how to set up a crawlable staging environment that stays hidden from Google.

# **⚠️ Why a Vercel Passcode Blocks Search Atlas**

Vercel's password protection (available on Pro and Enterprise plans) requires a browser-based authentication step. Because Search Atlas sends automated HTTP requests rather than using a browser session, it cannot pass this challenge. The crawler receives a **401 Unauthorized** or redirect response and cannot proceed past the login wall.

The solution is to remove the passcode gate for your staging subdomain and rely on **noindex directives** to prevent Google from indexing it instead.

# **🛠️ Recommended Setup: Open Staging + noindex**

Follow these steps to give Search Atlas full crawl access while keeping your staging content out of Google's index.

1. **Remove the Vercel passcode** from your staging deployment or create a dedicated staging subdomain (for example, **staging.yourdomain.com**) that has no password protection.
2. **Add a noindex meta tag** to every page on the staging environment. Place the following tag inside the `<head>` of your page template: **`<meta name="robots" content="noindex, nofollow">`**
3. **Block the staging subdomain in your staging robots.txt** by adding the following rules to **staging.yourdomain.com/robots.txt**:

   - User-agent: \*
   - Disallow: /

   Using both a noindex tag and a robots.txt Disallow provides two layers of protection against accidental indexing.
4. **Verify the staging URL is not submitted** to Google Search Console. Do not add your staging subdomain as a property in GSC, and do not include it in any production sitemap.
5. **Add your staging site to Search Atlas.** Go to **Left sidebar → OTTO SEO**, create a new project, and enter your staging subdomain URL. Search Atlas will now be able to crawl the site freely.

# **📊 Confirming Search Atlas Can Crawl Your Staging Site**

After completing the setup above, run a manual crawl to confirm everything is working:

1. Go to **Left sidebar → OTTO SEO → Overview**.
2. Select your staging project and click **Scan**.
3. Once the scan completes, navigate to **Left sidebar → OTTO SEO → All Audits** to review discovered pages and issues.
4. Use **Left sidebar → OTTO SEO → Page Explorer** to browse individual URLs and confirm your key staging pages are being indexed by Search Atlas (not by Google).

If the crawl returns zero pages or shows access errors, double-check that the Vercel passcode has been fully removed for that deployment and that no IP allowlist or firewall rule is blocking external crawlers.

# **💡 Additional Tips**

- **Use environment variables** in your Vercel project to conditionally inject the noindex meta tag only when **VERCEL\_ENV** equals **preview** or **development**. This prevents the noindex tag from accidentally appearing on your production build.
- **Separate projects in Search Atlas:** Keep your production site and your staging site as separate OTTO SEO projects so audits and recommendations do not mix between environments.
- **Vercel preview deployments** (the auto-generated **.vercel.app** URLs) are already set to noindex by Vercel by default on most plans, so you can point Search Atlas directly at those URLs without additional configuration — just make sure password protection is off.
- If you use **Vercel's GitHub integration** for continuous deployment, remember that each new preview deployment generates a new URL. Update your Search Atlas project URL whenever you want to scan a specific deployment.

# **🔒 Quick Reference: Staging Checklist**

- Vercel passcode removed from staging deployment
- noindex + nofollow meta tag added to all staging pages
- robots.txt on staging subdomain set to Disallow: /
- Staging subdomain not added to Google Search Console
- Staging subdomain not included in production sitemap
- Search Atlas OTTO SEO project created for staging URL
- Manual scan completed successfully in Search Atlas

If you need further assistance, open the chat widget in the bottom-right corner of the platform and type **human teammate** to be connected with a member of our team.