User-agent: * Disallow: /admin/ Disallow: /data/ Disallow: /file/ Disallow: /img/ Disallow: /inc/ Disallow: /install/ Disallow: /mobile/ Disallow: /pages/ Disallow: /photo/ Disallow: /plugins/ Disallow: /whatever/folder/ Disallow: /whatever/else/ Disallow: /whatever/assets/ Disallow: /folder/ Disallow: /directory/foo.html Disallow: /folder2/whatever.html