Short story short. Add routes.IgnoreRoute(“robots.txt”); to your RouteConfig.cs and add a file called robots.txt to your project root folder.