Posts tagged how to

You can directly access your raw HTTP logs. The logs include valuable data on searches, clicks, and USASearch tag page loads.

With the raw logs, you can conduct in-depth, ad hoc analyses to answer any questions that you can’t answer using the standard analytics reports in the Admin Center. You can also automatically integrate your search logs with your other reports and business processes. 

To access your raw logs, you’ll first need to upload your public key on the Raw Logs Access page in the Admin Center. 

We’ll set up your public key on our secure FTP servers and send you an email when your logs are available for download. 

Once your key is set up, you’ll be able access your log files via secure FTP. We upload your logs each day, usually within several hours after midnight GMT. After a week, we purge the older files.

We recommend customers set up a cron script to automatically download the latest files each day.

Troubleshooting tip: On Unix systems, you can generate a RSA public key with no passphrase by typing in the following command. Once you’ve created a key, it’ll be located at /home/user/.ssh/id_rsa.pub.

% cd ~
% ssh-keygen -q -b 4096 -t rsa
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:

USASearch > Admin Center > YourSite > Raw Logs Access

One of the most common search tasks conducted by public is looking for current job openings. To help searchers quickly find relevant jobs we have created the federal Jobs GovBox.

When you turn on the federal Jobs GovBox, searchers looking for jobs on your site will see up to three relevant USAJobs listings on your results page.

• Federal Department Websites—If your website is affiliated with a specific federal department (e.g., Department of Commerce), the federal Jobs GovBox will return relevant, location- and keyword-matched job openings across all agencies within your department.
• Federal Agency or Office Websites—If your website is affiliated with a specific federal agency (e.g., NOAA) or office (e.g, National Ocean Service), the federal Jobs GovBox will return relevant, location-and keyword-matched job openings at your agency.
• State and Local Websites and Cross-Agency Portals—If your website is a state or local government site or a federal cross-agency portal (e.g., USA.gov), the federal Jobs GovBox will return relevant, location- and keyword-matched job openings across all federal agencies.

Searchers will also see a direct link to USAJobs.gov for all current job openings (specific to your department or agency, if applicable), as shown in the sample below for a search on jobs on the USDA.gov.

We offer our search results as a hosted results page or as an application programming interface (API). 

When to Use the Hosted Results

Most agencies use our hosted search results page. We recommend that you to use our hosted page for the typical use case of returning search results on your website. Using the hosted page has many benefits such as allowing you to:

• Devote minimal development resources for the initial set up.
• Benefit from new features automatically as they’re rolled out.
• Stay current with search best practices, such as type-ahead search and formatting of search results.

When to Use the API

You may need to access our search results to present them outside of a typical search results page. We offer our results as an API for these use cases.

You can access the API and its documentation at Admin Center > YourSite > Search API. 

The API returns results from each of our indexes—web, image, document (ODIE), RSS (news), video, best bets (boosted content), and related searches—in JSON, JSON-P, or XML.

We recommend you use our API for limited use cases such as:

• Displaying search results within a native mobile application (i.e., an Android or iPhone app).
• Integrating USASearch results within existing applications (such as your intranet or publications catalog).
• Creating data mashups.

Examples of how agencies are currently using the API follow.

Use Case 1. Display Results in Your Mobile App

USA.gov and the White House use the Search API to render USASearch results within their mobile apps. 

Screenshot of a USASearch results page in the USA.gov iPhone app

 
Screenshot of a USASearch results page in the White House iPhone app

Related Links

• USA.gov mobile app
• WhiteHouse mobile app
• White House Github account (information on and source code for the iOS and Android mobile apps)

Use Case 2. Integrate USASearch Results within existing applications

RI.gov manages hundreds of distinct websites. The state provides a consistent search experience across all of its websites. Some sites are centrally controlled by RI.gov. Others are not. Using the Search API, RI.gov programmatically tells USASearch the site’s domain from which the user searched (e.g., tmc.dot.ri.gov). RI.gov then highlights results from this specific domain within RI.gov’s common look and feel.

See the sample results page below that shows a set of results displayed on Rhode Island’s Transportation Management Center for a search on traffic.

 

Step 2. Searcher sees results on RI.gov with a common look and feel that highlights results from the original site

Did you know? The Search API is available for government websites only. Other, publicly available APIs (i.e., for use by anyone) are at http://www.usa.gov/About/developer-resources/developers.shtml.

USASearch > Admin Center > YourSite > Search API

Your pre-launch checklist will be unique to your agency’s workflow, requirements, and deadlines. A typical punch list to go live with USASearch follows.

Connect Your Site to USASearch

Most agencies add these two snippets of code to the template(s) in their content management systems (rather than adding them to individual pages).

 1. Does your search box point to USASearch? Update your search box form code point to USASearch so that, when users execute a search, they’ll see your USASearch-hosted results page.

<form accept-charset="UTF-8" action="http://search.usa.gov/search" id="search_form" method="get"><div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="&#x2713;" /></div>
    <input id="affiliate" name="affiliate" type="hidden" value="YourSiteHandle" />
    <label for="query">Enter Search Term(s):</label>
    <input autocomplete="off" class="usagov-search-autocomplete" id="query" name="query" type="text" />
    <input name="commit" type="submit" value="Search" />
</form>

 2. Do you have the USASearch Javascript tag on your webpages? This Javascript helps us find your new and updated pages. It also powers the type-ahead search.

<script type="text/javascript">
//<![CDATA[
      var usasearch_config = { siteHandle:"YourSiteHandle" };
      var script = document.createElement("script");
      script.type = "text/javascript";
      script.src = "http://search.usa.gov/javascripts/remote.loader.js";
      document.getElementsByTagName("head")[0].appendChild(script);
//]]>
</script>

Double Check Your Content Sources

 3. Have you told us which domain(s) you want to search? List one or more domains that you want to search on the first page of web results.  Most agencies opt to leave out the ‘www’ to include all subdomains (e.g., list usa.gov to include answers.usa.gov, blog.usa.gov, publications.usa.gov, etc). Some agencies—especially department-level portals—opt to also include their bureaus’ websites (e.g., Commerce.gov includes commerce.gov, noaa.gov, trade.gov, etc.)

 4. Have you told us about your RSS feeds? We love RSS feeds. Tell us about all of your feeds, even if you opt not to show them  (see #7 below). They’re the quickest way for you to tell us about your new and updated content. Feeds also have well-structured titles, URLs, descriptions, and dates that we use to improve your search results page. (Tip: Preview your feeds from the RSS page in the Admin Center to confirm that we’ve indexed your content.)

 5. Have you told us about your social media accounts? Flickr photostreams, tweets, and YouTube videos are a great way to highlight your recent content and to let searchers know about your social media channels. Tell us about all of your social media, even if you opt not to show them (see #7 below). 

Customize Your User Interface

 6. Have you made the search results page look like your website? Customize the look and feel (colors, fonts, favicon) and header and footer of your search results page to create a seamless experience for users as they search and browse your website.

 7. Have you told us what to show on your results page? Tell us which results modules (i.e., the stuff in the center of the page) and sidebar options (i.e., the navigation in the left-hand column) you want to appear on your search results.

Set Up Nice-to-Have Features (Optional)

 8. Have you masked your domain?  We offer DNS masking, which allows you to show searchers your search.YOURSITE.gov (instead of search.USA.gov). If you leverage DNS masking you will have to change your search box form action from search.usa.gov to search.YOURSITE.gov. 

 9. Have you added the Javascripts for your other third party web services to your results page? If you have a third party service running on your website, you likely want it to run on your results page. Agencies often include their 4Q, AddThis, Foresee, Google Analytics, Omniture, or WebTrends code, among others.

Don’t Forget SEO (Optional)

 10. Have you optimized your site so we can find your content? We follow search engine optimization (SEO) best practices. Verify that you have a robots.txt and an XML sitemap. These two files are typically located in the root directory of your website (i.e., agency.gov/robots.txt and agency.gov/sitemap.xml, respectively). Also verify that your robots.txt file allows our user agent (usasearch) to access your public content.

 11. Have you registered your site with the major commercial search engines? Register for both Bing Webmaster Tools and Google Webmaster Tools to maximize the coverage of your content in their search results.

The Low Click-Thru Rate (CTR) Queries page in the Admin Center reports on the top 10 searches with low click-thru rates. It shows the most popular search terms that people generally don’t act (click) on. 

You’ll see a list of up to 10 terms that returned search results, but resulted in a click less than 20 percent of the time over the past day or so. In other words, fewer than one in five people who searched for that topic clicked on a search result. (Terms must have been searched by at least 10 unique searchers per day to be included in this list.)

Use this report to identify issues with coverage. Create a Best Bet, add pages that may be missing from the results, or do both to improve recall. Incorporate language from these popular search terms into your titles and descriptions to improve the relevance of results. 

Click on any linked word or the  symbol in the list, and you’ll be taken to a Query Timeline chart. There you’ll be able to select a period of time for the term, and you can also add a second query for comparison.

Click on the  symbol, and you’ll be taken to a list of the top clicked URLs for that term.

Did you know? This low click-thru report lists only those terms that returned search results. Visit the Query Logs page to see the top queries that returned no results.

USASearch > Admin Center > YourSite > Low CTR Queries

The Trending Queries page in the Admin Center reports on the top 10 changing search terms. 

You’ll see a list of up to 10 terms with the greatest gain between yesterday and today. (Terms must have been searched by at least 10 unique searchers per day to be included in this list.)

Use this report to identify newly popular terms. Create new content or update existing content to ensure it’s current, accurate, and complete. 

See the sample report below that shows the 10 trending queries on USA.gov on Monday, November 5, 2012—the day before Election Day.

Click on any linked word or the symbol in the above list, and you’ll be taken to a Query Timeline chart. There you’ll be able to select a period of time for the term, and you can also add a second query for comparison.

Click on the symbol, and you’ll be taken to a list of the top clicked URLs for that term.

To investigate why the term may be trending, click on the Twitter, news, or trends icons to see what’s happening on other websites.

USASearch > Admin Center > YourSite > Trending Queries

The Click Stats page in the Admin Center reports on the URLs your searchers have clicked on. Data start from October 2012.

You’ll see a list, in frequency order, of the most clicked URLs for any time period. You may choose to display 10, 50, 100, 500, or 1,000 results in order of frequency.

See the sample report below that shows the 10 most clicked URLs on USA.gov in October 2012.

Click on the symbol in the list above, and you’ll see the top query terms that led to that URL over the same time period.

See the sample report below that shows the top queries that led to a click on http://www.usajobs.gov/ on USA.gov in October 2012.

From the list of queries, you will be able to click on the symbol again to see the top clicked URLs for that term over the same time period.

See the sample report below that shows the top clicked URLs for jobs on USA.gov in October 2012.

USASearch > Admin Center > YourSite > Click Stats

Do you want your search results page to run web services such as 4Q, AddThis, Foresee, Google Analytics, Omniture, or WebTrends?

Log in to the Admin Center and select the option, Third Party Tracking. Input the JavaScript code you’d like to call from your search results page. Click submit to send us your request. We’ll input your code for you and send you an email to confirm that we’ve done it.

Google Analytics tip: In your Google Analytics JavaScript, be sure to set your domain if you’ve requested domain masking and you want to include your search subdomain (e.g., search.commerce.gov) with your main domain (e.g., commerce.gov).

_gaq.push([‘_setDomainName’, ‘commerce.gov’]);

For more information, read Google’s tip, Tracking Multiple Domains.

ForeSee tip: Coordinate with your ForeSee representative and USASearch to implement your customer satisfaction survey on your results page. The four general steps follow.

1. Email USASearch to set up a CNAME for search.{yourdomain}.gov.
2. Update the files path in your Foresee code to use an absolute path instead of a relative path.
   
   Find => ‘files’: ‘/fsrscripts/’,
   
   and replace it with => ‘files’: ‘//{yourdomain}/fsrscripts/’,
   
   in the following five files.
   a. foresee-trigger.js
   b. foresee-tracker.js
   c. foresee-alive.js
   d. foresee-qualifier.js
   e. foresee-test.js
3. Ask ForeSee to send USASearch your foresee-trigger.js file.
4. USASearch will send you an email to confirm that we’ve set up your CNAME and installed your foresee-trigger.js file.

USASearch > Admin Center > YourSite > 3rd Party Tracking

The Monthly Reports page in the Admin Center reports on the total number of searches and click throughs. A click through is recorded each time a searcher clicks on a results link. Data start from the time your agency started using USASearch (but no further back than 2009).

Data are shown for the present month-to-date by default. Change the time period by selecting a different year or month.

View a detailed report of the number of times searchers have input specific terms and phrases by clicking on the CSV link to download the list as a comma delimited file. The CSV file contains three columns.

1. Query—The search term or phrase.
2. Raw count—Total number of times the query was submitted.
3. IP-deduped—Total number of times the query was submitted by any one IP address. This excludes bots and other traffic that send in a query multiple times from one IP address. It is often a more accurate representation of “real,” human traffic.

Did you know? On the first of each month, we email you a report with data on the previous month’s queries, clicks, and top search terms.

USASearch > Admin Center > YourSite > Monthly Reports

The Query Logs page in the Admin Center reports on the terms and phrases your searchers have input. Data start from the time your agency started using USASearch (but no further back than 2009).

Specific Search Queries

You can limit the search for any specific query term or phrase to a specified time period.

See the sample report below that shows the number of times housing (or its stemmed equivalents) was searched on USA.gov in October 2012.

Top Queries

You can also get a list, in frequency order, of (a) most popular queries and (b) top queries with no results for any time period. You may choose to display 10, 50, 100, 500, or 1,000 results in order of frequency.

Most Popular Queries

See the sample report below that shows the 10 most popular queries on USA.gov on October 15, 2012.

Top Queries with No Results

If there are any terms that were searched by 10 unique searchers within the period of a day and returned no results, they’ll appear here.

For example, on one agency’s website in October 2012, a dozen searches for frostline returned no results. The frost line—also known as depth of frost or freezing—is the depth to which the groundwater in soil is expected to freeze. The agency’s pages on the topic referred to this concept as depth of freezing. The agency can use this no results data to help searchers find its content by adding a best bet, updating its existing web pages, or both.

 Query Timeline

Click on any linked word or the  symbol in the above lists, and you will be taken to a Query Timeline chart. There you will be able to add a second query for comparison.  Note also that placing your cursor over any part of the timeline will give the search frequency for a particular date.

See the sample report below that shows a comparison of jobs and employment on USA.gov for the nine months between February and November 2012, with the cursor placed over the spike on jobs May 30.

 Clicked URLs

Click on the  symbol in any of the lists, and you’ll be taken to a list of the top clicked URLs for that term.

See the sample report below that shows the top clicked URLs for jobs on USA.gov in October 2012.

From the list of clicked URLs, you will be able to click on the  symbol again to see the top query terms that led to that URL over the same time period.  

See the sample report below that shows the top queries that led to a click on http://www.usajobs.gov/ on USA.gov in October 2012.

USASearch > Admin Center > YourSite > Query Logs