Scoring Criteria

Home/Awards/Scoring Criteria

The Ten Best Web Support Sites of 2019 Site Scoring System

Scoring Instructions for Judges

For each of the 25 criteria described on the following pages, please assign a numerical score that reflects your judgment about this item. Most of the questions deal with your subjective impression of the site, supplemented by information supplied by the entry essays for the last two sections, so please draw the best conclusion you can from the available information.

Each item is worth a specific number of points. They are not all worth the same number of points as they are not all of the same importance. We’ve offered brief text descriptions that correspond to scoring levels for some of the sections, but not all – read carefully. There is no single blueprint for a perfect support site. If the site developers have found an innovative way to deliver a quality support experience, feel free to assign a higher score than our text descriptions might suggest (not above the maximum number of points for that category, please.) Similarly, you should downgrade a poor implementation.

Note that we have three categories of entries—Small (under $100 million), Medium ($100 million to $999 million), and large ($1 billion and up) Company Divisions. Although the scoring questions are the same, each Company’s entry only competes against others in their Division. This approach is intended to “level the playing field” for companies that don’t have the resources to support complex, full-featured sites. This is important for the companies in the competition, but should;d not effect the judges scores in any way.

For each item, you may assign any score between 0 and the maximum for that category. Since a site’s overall standing will be based on a cumulative score (maximum 1000 points), it’s important that you assign a score for every item; answers like “n/a” or “how would I know?” are the same as giving a score of zero points. However, if you can’t locate a feature (because of password problems or difficult navigation) or if an entry essay doesn’t provide essential information, feel free to assign a low score for that scoring question. Note that we are encouraging companies that have onboarding or site overview videos to include links in their essays to help familiarize judges with their sites. Please view any video links provided and reflect them in their scores as appropriate.

Finally, the comments: companies that enter this competition value judges comments very highly. Accordingly we are expanding the comments we are asking from the judges. Judges must make at least one comment for each of the five broad categories being judged, and can enter a comment for each of the 25 scored areas if they choose.. In addition, please describe the three most important enhancements you’d recommend for each site. Length is up to you—anywhere from a few notes to a mini-essay, if that’s what you feel like writing. Comments are no longer optional, because we’ve found that most people who enter are very eager to hear feedback from the judges about how to improve their sites. Your comments are the single most important benefit of the competition to those who enter

Site Scoring System

■ Site User Experience (400 Total Points 40% of all Points)

Based on your first impressions of the site and your subsequent exploration of key features, please assign scores for the following criteria:

1. What’s your impression of the support site’s overall organization and navigation? (200 points total for this group, see below)

  • She is well organized and intuitive. Strong, consistent navigation model helps users quickly and easily find all post-sale support- and service-related resources. (200 points)
  • Tech support resources are easy to find, but other post-sale content (e.g., forums, training, downloads, phone support, videos, and other support channels) is not well integrated into the main support site. (100 points)
  • All elements of the site are poorly organized and hard to find (0 points)

2. Are Menus, Pathways, Links, Breadcrumbs and “You are here” evident, logical, and easy to use? (80 points total for this group, see below)

  • Link-based pathways are clearly defined; usually, less than three clicks are needed to reach popular services and high-traffic articles (80points)
  • Pathways are logical, but some obstacles (e.g., excessive clicks, multiple logins) interfere with access to content (40 points)
  • Redundant, confusing pathways to content (0 points)

3. What is your impression of the Site’s Dynamic Presentation? Is the site part of the users tool kit rather than a simple web site requiring a great deal of effort to use effectively? (60 points total for this group, see below)

  • The user interface is dynamic and the site helps to easily personalize the interface (60 points)
  • Yes, but personalization is mostly limited to product ownership, and takes more effort (30 points)
  • No personalization (0)

4. Entry Points? (60 points total for this group, see below)

  • Google search results take you to correct information. Any leaf node is your Home page. Logical entry points are intuitive to new users and veteran users. Site functionality is self-evident in design. (60 points)
  • Some links are vague or don’t lead to expected destinations (30)
  • Entry points are inconsistent, misleading, confusing (0 points)

■ Content Offering (200 Points Total, 20% of All Points

1. Content type (60 points total for this group, see below)

  • Site has at least ten of the following; knowledgebase articles, forums, software downloads, technical documentation, release notes, bugs, security notifications, technical support notifications and lifecycle notifications,  product specifications and data sheets, compliance statements with technical standards (IEEE, RFC, W3C. 503, etc.), and Environmental Standards (ROHS, etc.), training/education. (60 points)
  • Site has at least five of the above. (30 points)
  • Search engine deals primarily with knowledgebase content (0 points)

2. Minimum content platforms, formats (60 points total for this group, see below)

  • All content available in at least five of the following; HTML, PDF, Video, Wiki, Forums, Podcast/Audio, blind reader friendly, reads page to you. (60 points)
  • Content available in two or three of the above formats and/or not all content available in most formats. (30 points)
  • Few, if any of the above formats available or very little content in those formats. (0 points)

3. Multilingual Content Support (30 points total for this group, see below)

  • Dynamic translation that’s accurate and certified (30 points)
  • 75+ languages (18 Points)
  • Content offered in 30-50 languages, including non-western character sets (14 points),
  • Content offered in 15-20 languages (5 Points),
  • English plus 9 Euro languages (3 points)
  • One language only (0 points).

4. Disabilities access. Content supports Section 503/508web standards for disabilities (10 points total for this group)

  • Auditory (read with ears/screen readers) and/or tactile (read with fingers)

5. Personal Content Portability: Dynamic Books/Build your own content and take it with you from device to device even to an airport terminal/library/loaner device (40 points total for this group).

■ Content Engagement and Interaction (190 Points Total, 19% of all Points)
After exploring the site’s content and trying a few knowledgebase queries, please assign scores for the following criteria. Suggestion: to create a test query, select a question from the middle of the FAQ list and reword it.

1. Minimum Interaction Required for Feedback, Forums, Cases, Chat (20 points total for this group)

  • Search, Browse/Navigate, Feedback, Comment (publicly on content, visible to others to see/engage/respond to), Forum, Chat (Live Chat), Call (contact phone #s, click to call), Case

2. Can Escalate any Content into Tracked Interaction; Feedback, Chat, Click to Call/Chat/Schedule an Expert (20 points total for this group)

  • Schedule a call/chat/expert with escalation (chat/forum to case, case to management), Get Help with Content (search/browse in content and escalate from content with context for help with that specific content), unified interaction model (all escalations and interactions).

3. Seamless Cross Platform Engagement: Desktop, Mobile, Tablet, Airport Floater Terminal, Mobile Apps, Responsive Design, In-Product. (40 points total for this group)

  • Manage your website/app experience across multiple devices seamlessly: desktop AND mobile And tablet (pc, Mac, Unix, Windows/Apple, IOS/Android, cross browser MS Edge and IE11, Firefox, Chrome, Opera, Safari, Silk, mobile/responsive design, in-product and “phone home” support integration to support portal.
  • Unified Interaction Model, Cross-Channel Aggregation into CRM

4. Search –>Advanced Search –>Federated Search–>Embedded =Universal Search, + Features Export Search Results, Save Searches to Profile, Subscribe to Searches, Send/Share
Basic Search vs Good Search Results vs Successful Outcomes in Tasks/Activities for Why You Came to the Site
Experts
Engagement Level: –>dedicated staff for Feedback, Forums, Social Media, etc. (80 points total for this group)

  • Minimum search offering: federated search, clear filtering, clear advanced search options/guidance, search wizard, move from pull–>push search content delivery model.

5. Offline/Public Access to Content (Federal/Secure Site Use Cases) (40 points total for this group)

  • Public versus entitled content experience = clearly label content requiring login/additional permission to access content but still show snippet with lock icon; allow offline content access for secure site (download Content/take to with you, across devices/platforms = personal content library),

■ Site Improvement Processes – Essay (100 Points Total, 10% of all Points)
Please assign scores for the following criteria:

1. Strategy for site management and ongoing development. Does the site use metrics to determine the most important site needs. Does the site continue to develop and improve. (20 points total for this group)

2. How do site managers research customer needs, industry trends in self-support sites? (20 points total for this group)

3. Managing and maintaining site engagement with all stakeholders. (10 points total for this group)

  • Does the site perform alpha/beta previews/access, usability studies, customer design council (at exec level) and users.

4. How does the site do operational management of site processes, systems, and content? (20 points total for this group)

  • How do you implement and maintain your strategy day-to-day?
  • Do you use robust KM processes, standards-driven content management processes?

5. Lessons Learned. (30 points total for this group)

  • Weight score by realness of challenge, business model change vs simple incremental update to existing functionality

■ Site Measurement Processes – Essay (100 Points Total, 10% of all Points)
Based on your review of the site-challenge essay and your experience with the site, please assign scores for the following questions:

1. Transactional activity metrics. (10 points total for this group)

  • Traditional: CSAT, NPS, Customer Effort Scores, Google Analytics, Google Tag Manager

2. Content management metrics. (40 points total for this group)

  • How does the site measure  and manage content from initial development, publication, updates, into retirement/archives/deletion
  • How does the site measure content quality, content recency, ease of use?
  • How does the site measure content fundability/search quality/search results quality via search, app/tools quality

3. Overall metrics and analytics strategy to guide site management and ongoing development (50 points total for this group)

  • How does the site measure outcomes: case deflection, content update cycle time, publish cycle time, etc.?
  • Do site developers use formal usability studies, customer A/B testing, multivariate testing?
  • How do site developers balance holistic analytics versus organizational goals?
  • Do site developers messier demonstrated business value?
  • Do site developers measure demonstrated hard business ROI?

Eligibility Entry Procedures
Entries may be submitted by individuals, companies, or outside firms that participated in the site development process. Each entry must be accompanied by a completed application form, along with an essay that describes the major challenge the site developers faced and three key site features. (The essay should be submitted in electronic form.) We will accept video links to familiarize the judges with your site. An onboarding video might work well for that or an overview video. Do not attach video files as they are too large. Only links will be accepted and will probably help your scores.

Deadline for all entries is Tuesday, April 30, 2019.

Scoring
The judges will use a scoring system that addresses five general areas:

  • Site user experience
  • Content offering
  • Content engagement and interaction
  • Site improvement (Essay)
  • Site measurement processes (Essay)

Publications
The ten winning sites will be profiled in an annual (approximately 140 page) ASP report that will include screen shots, essays, and related material. All entrants will receive a complimentary pdf of this report after publication. An individual site assessment with average judges scores versus overall averages for your size company, along with specific judge comments is emailed to every entrant. This report is confidential to each entrant.

Administration
Entries should be submitted to The Association of Support Professionals in electronic form (see application for details). Entry fee is $1,100 for large company entries, $900 for medium company entries, and $700 for small company entries.

Click Here to Download PDF File of the Site Scoring Instruction for Judges