# Stops robots crawling the directories User-agent: * Disallow:/Jobs/Juniors/* Disallow:/media/rss/template-news.htm Disallow:/search/* Disallow:latest-news.php Disallow:/health/* Disallow:/js/fancyBox/* Disallow:/about/ft/foundationform-old.php Disallow:/w3devcampus/* Disallow:/patients/patient_leaflets_box.php Disallow:/services/purchasing/index_box.php Disallow:/mobi/* Disallow:/documents/_Departments/Undergraduates/ Disallow:/media/events/* User-agent: Twitterbot Disallow: