How do I create robots.txt?
Last checked 31 Aug 2026
Short answer
Ask Lovable to add a robots.txt file in the public folder that allows crawling of public pages, blocks private/admin routes, and points to your sitemap.xml.
What robots.txt does
robots.txt tells well-behaved crawlers which parts of your site they may or may not crawl, and where to find your sitemap. It's a plain text file, not enforcement — it won't stop a determined bot, but Google and other major crawlers respect it.
Asking Lovable to create it
Add a robots.txt file at the root allowing all crawlers to index public pages, disallowing /admin and /account, and referencing my sitemap.xml.
A minimal example:
User-agent: *
Allow: /
Disallow: /admin
Disallow: /account
Sitemap: https://yourdomain.com/sitemap.xml
Where it lives
robots.txt must be served at the root of your domain — https://yourdomain.com/robots.txt — usually by placing the file directly in public/.
What to block (and what not to)
- Block: admin panels, account/settings pages, checkout success pages with sensitive tokens, staging/duplicate routes.
- Don't block: your CSS/JS assets — blocking these can stop Google rendering your pages properly, which hurts rather than helps.
- Don't rely on it for security — anything genuinely sensitive needs actual auth/RLS, not just a Disallow line.
robots.txtis public and readable by anyone.
What usually goes wrong
- Accidentally blocking the whole site with
Disallow: /left over from a staging setup — always double check this before going live. - Forgetting to add the
Sitemap:line, missing an easy win for crawl discovery. - Blocking asset folders (
/assets,/_next, etc.), which can prevent Google from rendering pages correctly.
Related
Sources
Was this answer helpful?
Related questions
- AnswerHow do I connect Google Search Console?Use Lovable's Google Search Console integration to verify your domain and manage search data directly from chat, or verify manually by adding the DNS/HTML verification record Google provides.
- AnswerHow do I add schema markup?Ask Lovable to add JSON-LD structured data (schema.org markup) relevant to your page type — e.g. Organization, Product, Article or FAQ — inserted as a script tag in the page head.
- AnswerDoes Lovable automatically create a sitemap?No — a sitemap.xml isn't generated automatically for a typical project. You need to ask Lovable to add one, or it may be included as part of a dedicated SEO setup/review feature on newer projects.
- AnswerHow do I stop a Lovable page being indexed?Ask Lovable to add a noindex meta tag to that specific page, and optionally disallow it in robots.txt — don't block the whole domain by mistake.
You might also need
- GuideHow to Set Up SEO for a Lovable ProjectTitles, descriptions, canonical URLs, Open Graph and structured data — the technical SEO baseline every project needs.
- GuideHow to Generate a Dynamic SitemapServe an XML sitemap that updates itself as you publish new content, instead of a static file that goes stale.