Site Architecture and Search Engine Optimization

A well–designed site architecture—when paired with best practices for links and plain language keywords—will help customers find your content on search engines and on your website. There are four main architecture topics to address: protocols for crawlers, navigation, file and directory structure, and coding.

Protocols for Crawlers

Publish an XML sitemap—Search engines' crawlers look for an XML sitemap to help them quickly and easily access your site's pages for indexing.
Publish a robots.txt file—Search engine robots will check a special plain text file in the root of each server called robots.txt before indexing a site. Robots.txt implements the Robots Exclusion Protocol, which allows the site administrator to define what parts of the site are off–limits to specific robot user agent names.
Markup structured pages with schemas—Many Web pages are generated from structured data. When this data is formatted into HTML, it becomes very difficult for search engines’ crawlers to recover the original structured data. On–page schema markup enables search engines to understand the information on Web pages and provide richer search results.

Read about achieving good SEO for additional tips on optimizing your site's architecture for crawlers.

Navigation

Navigation toolbars guide users quickly to popular searches and top tasks. There are multiple possibilities for placement of navigation aids.

Be consistent and consolidate—"Help," "FAQ," and "Instructions" are related and can all be grouped together. Also, break information up inside the category if needed; list individual departments separately within a larger "Departments" category.
Use breadcrumbs—Breadcrumbs provide a "trail" to help people find their way through your site, and are used in conjunction with regular navigation toolbars. They're nearly always text links, but in a smaller font. Large sites should have top–of–the–page navigation pointing to the top–level pages and category navigation on the left with breadcrumb navigation on the page itself.
- Example: Home » Category 1» Bucket A » Page

Use Footer Navigation—Footer navigation should be placed at the bottom of the page. Include text–only links that are redundant to the top–level navigation, or in the header, so customers don't have to scroll back up to the top of the page to navigate. This practice allows you to enforce important keywords, and it helps users who have graphics turned off in their browser.
Use keyword phrases for links—These links may go to the same place as top–level navigation, but they're labeled with related keywords.

File and Directory Structure

Create a shallow directory structure—Most search engines only recognize three directory levels. They'll selectively index only some files in those top directories. It's crucial to place your most important pages at the first or second directory level, with no more than 50 files per directory.
Use real words for file and directory names—Name files, folders, images, videos, and attachments with keywords. Use hyphens, not underscores, to separate keywords. Make file and directory names keyword–rich, but not too long.
- Example: Good
  http://www.usa.gov/Citizen/Topics/Money-Taxes.shtml
- Examples: Not as Good

http://www.usa.gov/Citizen/Topics/Money_Taxes.shtml

http://www.usa.gov/Citizen/Topics/MoneyTaxes.shtml

http://www.usa.gov/Citizen/Topics/topic3.shtml

http://www.usa.gov/doc.cfm?index=12345

Put the most important things at the top of the page—One of the easiest ways to satisfy search engines and users is to quickly get to the point of a page by designing it like a pyramid. Put the most important information at the very top of the page, in text or text links that go to top–level pages. Content should be placed so that the most important, useful information is at or near the top of the page. The least important information and links should be lower on the page.
Optimize entry pages—Pages that bring you traffic are entry pages, and each should be optimized and submitted to search engines via their webmaster tools. Make the pages stand–alone, like your home page. When a visitor lands on one of your entry pages, the visitor needs to know where they are, who your organization is, and what the page is about. Include full navigation on all entry pages and make it obvious what the page and site is about. Don't assume visitors will find the index page first.

Coding

Title tags are key—Each page must have its own descriptive <title> tag that matches the topic of the page. This text appears whenever someone bookmarks the page, and it provides important information for the search engines. Remember that meta keyword tags are nearly useless these days but are known to be somewhat helpful when the content of the page strongly supports those keywords. Be selective with what you put in that tag. Don't waste time calculating density and meeting meta keyword character specifications. Just focus on backing up the actual content on the page, or using synonyms and misspellings.
Headings are key—Separate the content on your page in subsections and label them with headings. Use keyword rich headings so that visitors can scan your content. Also use HTML heading tags (<h1>, <h2>, <h3>) to ensure search engines’ crawlers recognize the text as headings.
Place style sheets and JavaScript in separate files—Place cascading style sheets and JavaScript into separate files rather than having the script on the page. Otherwise, it could interfere with the crawlers' ability to quickly find keywords within your content. Watch out for JavaScript that is used for navigation menus that special–needs users can never see and search engines can't follow.
Check output programs that generate HTML—Be extra careful with What You See Is What You Get (WYSIWYG) HTML editors or programs that save content as HTML. The generic code they create will often not meet the needs of all users or search engines.

Resources

Content Lead: Ammie Feijoo and Marilyn Kercher
Page Reviewed/Updated: September 4, 2012