Scoring Criteria

The ASP Best Web Support Sites of 2023 Site Scoring System

Scoring Instructions for Judges

For each of the 22 criteria described on the following pages, please assign a numerical score that reflects your judgment about this item. Most of the questions deal with your subjective impression of the site, supplemented by information supplied by the entry essays for the last two sections, so please draw the best conclusion you can from the available information.

Each item is worth a specific number of points. They are not all worth the same number of points as they are not all of the same importance. We’ve offered brief text descriptions that correspond to scoring levels for some of the sections, but not all – read carefully. There is no single blueprint for a perfect support site. If the site developers have found an innovative way to deliver a quality support experience, feel free to assign a higher score than our text descriptions might suggest (not above the maximum number of points for that category, please.) Similarly, you should downgrade a poor implementation.

For each item, you may assign any score between 0 and the maximum for that category. Since a site’s overall standing will be based on a cumulative score (maximum 1000 points), it’s important that you assign a score for every item; answers like “n/a” or “how would I know?” are the same as giving a score of zero points. However, if you can’t locate a feature (because of password problems or difficult navigation) or if an entry or essay doesn’t provide essential information, feel free to assign a low score for that scoring question. Note that we are encouraging companies that have onboarding or site overview videos to include links in their essays to help familiarize judges with their sites. Please view any video links provided and reflect them in their scores as appropriate.

Finally, the comments: companies that enter this competition value judges comments very highly. Accordingly we are expanding the comments we are asking from the judges. Judges must make at least one comment for each of the 22 categories being judged. In addition, please describe the three most important enhancements you’d recommend for each site. Length is up to you—anywhere from a few notes to a mini-essay, if that’s what you feel like writing. Comments are no longer optional, because we’ve found that most people who enter are very eager to hear feedback from the judges about how to improve their sites. Your comments are the single most important benefit of the competition to those who enter.

Judge Nominations In addition to the highest scoring entries, ASP will honor several companies that have particular aspects to their support sites that are beyond the norm for the industry. While they may not have the highest overall scores, they may have a particularly active Community, or the best Search you have ever seen. It can be anything that you notice that you think is outstanding. Each judge is asked to nominate three companies for something outstanding about their site. You can nominate the same company for different site attributes, but we would like three nominations. If you think your sites were all poor, just say so along with your nominations. This way we will know that you just didn’t forget about it. Feel free to look outside the box as well. There may be some innovations out there that we do not yet recognize in our 22 scoring areas. We are no longer honoring the top ten highest scoring companies. In fact, we have renamed this competition the ASP Best Support Websites competition. We will still honor the very top scorers, but will also honor sites that do something  extraordinary. Kind of like the Oscars, when they honor the best cinematography, best sound track, or best screenplay. We are totally dependent on judges to help us find theses things, so please be helpful

Site Scoring System

■ Site User Experience (400 Total Points 40% of all Points)

Based on your first impressions of the site and your subsequent exploration of key features, please assign scores for the following criteria:

1. What’s your impression of the support site’s overall organization and navigation? (200 points total for this group, see below)

• Site is well organized and intuitive. Strong, consistent navigation model helps users quickly and easily find all post-sale support- and service-related resources. (200 points)
• Tech support resources are easy to find, but other post-sale content (e.g., forums, training, downloads, phone support, videos, and other support channels) is not well integrated into the main support site. (100 points)
• All elements of the site are poorly organized and hard to find (0 points)

2. Are Menus, Pathways, Links, Breadcrumbs and “You are here” evident, logical, and easy to use? (80 points total for this group, see below)

• Link-based pathways are clearly defined; usually, less than three clicks are needed to reach popular services and high-traffic articles (80points)
• Pathways are logical, but some obstacles (e.g., excessive clicks, multiple logins) interfere with access to content (40 points)
• Redundant, confusing pathways to content (0 points)

3. What is your impression of the Site’s Dynamic Presentation? Is the site part of the users tool kit rather than a simple web site requiring a great deal of effort to use effectively? (60 points total for this group, see below)

• The user interface is dynamic and the site helps to easily personalize the interface (60 points)
• Yes, but personalization is mostly limited to product ownership, and takes more effort (30 points)
• No personalization (0)

4. Entry Points? (60 points total for this group, see below)

• Google search results take you to correct information. Any leaf node is your Home page. Logical entry points are intuitive to new users and veteran users. Site functionality is self-evident in design. (60 points)
• Some links are vague or don’t lead to expected destinations (30)
• Entry points are inconsistent, misleading, confusing (0 points)

■ Content Offering (200 Points Total, 20% of All Points

1. Content type (60 points total for this group, see below)

• Site has at least ten of the following; knowledgebase articles, forums, software downloads, technical documentation, release notes, bugs, security notifications, technical support notifications and lifecycle notifications,  product specifications and data sheets, compliance statements with technical standards (IEEE, RFC, W3C. 503, etc.), and Environmental Standards (ROHS, etc.), training/education. (60 points)
• Site has at least five of the above. (30 points)
• Search engine deals primarily with knowledgebase content (0 points)

2. Minimum content platforms, formats (60 points total for this group, see below)

• All content available in at least five of the following; HTML, PDF, Video, Wiki, Forums, Podcast/Audio, blind reader friendly, reads page to you. (60 points)
• Content available in two or three of the above formats and/or not all content available in most formats. (30 points)
• Few, if any of the above formats available or very little content in those formats. (0 points)

3. Multilingual Content Support (30 points total for this group, see below)

• Dynamic translation that’s accurate and certified (30 points)
• 75+ languages (18 Points)
• Content offered in 30-50 languages, including non-western character sets (14 points),
• Content offered in 15-20 languages (5 Points),
• English plus 9 Euro languages (3 points)
• One language only (0 points).

4. Disabilities access. Content supports Section 503/508web standards for disabilities (10 points total for this group)

• Auditory (read with ears/screen readers) and/or tactile (read with fingers)

5. Personal Content Portability: Dynamic Books/Build your own content and take it with you from device to device even to an airport terminal/library/loaner device (40 points total for this group).

■ Content Engagement and Interaction (200 Points Total, 20% of all Points)
After exploring the site’s content and trying a few knowledgebase queries, please assign scores for the following criteria. Suggestion: to create a test query, select a question from the middle of the FAQ list and reword it.

1. Minimum Interaction Required for Feedback, Forums, Cases, Chat (20 points total for this group)

• Search, Browse/Navigate, Feedback, Comment (publicly on content, visible to others to see/engage/respond to), Forum, Chat (Live Chat), Call (contact phone #s, click to call), Case

2. Can Escalate any Content into Tracked Interaction; Feedback, Chat, Click to Call/Chat/Schedule an Expert (20 points total for this group)

• Schedule a call/chat/expert with escalation (chat/forum to case, case to management), Get Help with Content (search/browse in content and escalate from content with context for help with that specific content), unified interaction model (all escalations and interactions).

3. Seamless Cross Platform Engagement: Desktop, Mobile, Tablet, Airport Floater Terminal, Mobile Apps, Responsive Design, In-Product. (40 points total for this group)

• Manage your website/app experience across multiple devices seamlessly: desktop AND mobile And tablet (pc, Mac, Unix, Windows/Apple, IOS/Android, cross browser MS Edge and IE11, Firefox, Chrome, Opera, Safari, Silk, mobile/responsive design, in-product and “phone home” support integration to support portal.
• Unified Interaction Model, Cross-Channel Aggregation into CRM

4. Search –>Advanced Search –>Federated Search–>Embedded =Universal Search, + Features Export Search Results, Save Searches to Profile, Subscribe to Searches, Send/Share
Basic Search vs Good Search Results vs Successful Outcomes in Tasks/Activities for Why You Came to the Site
Experts
Engagement Level: –>dedicated staff for Feedback, Forums, Social Media, etc. (80 points total for this group)

• Minimum search offering: federated search, clear filtering, clear advanced search options/guidance, search wizard, move from pull–>push search content delivery model.

5. Offline/Public Access to Content (Federal/Secure Site Use Cases) (40 points total for this group)

• Public versus entitled content experience = clearly label content requiring login/additional permission to access content but still show snippet with lock icon; allow offline content access for secure site (download Content/take to with you, across devices/platforms = personal content library),

■ Site Improvement Processes – Entry or Essay (100 Points Total, 10% of all Points)
Please assign scores for the following criteria:

1. Strategy for site management and ongoing development. Does the site use metrics to determine the most important site needs. Does the site continue to develop and improve. (20 points total for this group)

2. How do site managers research customer needs, industry trends in self-support sites? (20 points total for this group)

3. Managing and maintaining site engagement with all stakeholders. (10 points total for this group)

• Does the site perform alpha/beta previews/access, usability studies, customer design council (at exec level) and users.

4. How does the site do operational management of site processes, systems, and content? (20 points total for this group)

• How do you implement and maintain your strategy day-to-day?
• Do you use robust KM processes, standards-driven content management processes?

5. Lessons Learned. (30 points total for this group)

• Weight score by realness of challenge, business model change vs simple incremental update to existing functionality

■ Site Measurement Processes – Entry or Essay (100 Points Total, 10% of all Points)
Based on your review of the site-challenge questions and your experience with the site, please assign scores for the following questions:

1. Transactional activity metrics. (10 points total for this group)

• Traditional: CSAT, NPS, Customer Effort Scores, Google Analytics, Google Tag Manager

2. Content management metrics. (40 points total for this group)

• How does the site measure  and manage content from initial development, publication, updates, into retirement/archives/deletion
• How does the site measure content quality, content recency, ease of use?
• How does the site measure content findability/search quality/search results quality via search, app/tools quality

3. Overall metrics and analytics strategy to guide site management and ongoing development (50 points total for this group)

• How does the site measure outcomes: case deflection, content update cycle time, publish cycle time, etc.?
• Do site developers use formal usability studies, customer A/B testing, multivariate testing?
• How do site developers balance holistic analytics versus organizational goals?
• Do site developers measure demonstrated business value?
• Do site developers measure demonstrated hard business ROI?