Tue 19 Sep 2017
Better Connected accessibility testing is resource intensive, but extremely important to the programme for two reasons. Firstly, we need to support the fact that the law requires websites to be accessible to all users. Secondly, we need ensure that the sites we identify as providing a ‘very good’ user experience are inclusive and accessible to people with disabilities. So, we need to know that all our four-star sites have cleared this hurdle.
As part of our improvement programme for Better Connected, we have been working to make our accessibility reports easier to understand - and therefore act upon - for the large numbers of people who manage websites but are not coders familiar with the details of web accessibility standards.
To make things clearer, we now publish the 14 assessment criteria we use to evaluate accessibility on the public part of the website, and have also simplified the way we present councils’ individual results.
This year, we are going a stage further by presenting simplified scoring rules that will make it easier for web managers to interpret results and understand where their sites present real barriers, as opposed to inconveniences, for people with disabilities.
Each of our 14 assessment criteria identify issues on a scale of seriousness from 0-10. A score of 0 means the task or pages being tested are accessible for that criterion. A score of 8 means there are significant difficulties and a score of 10 is classified as a ‘showstopper’ meaning a site user will be unable to progress beyond that point because of the issue identified.
A score of 10 on any criterion for any task or pages tested will automatically lead of a task score of 0, and a score of 8 on any criterion for any task or pages tested will automatically lead of a task score of 1. Both scores denote a task ‘fail’ with consequences for overall scores at both stage one and stage two of our testing.
Not all of our 14 criteria present problems serious enough to warrant scores of 8 or 10. For example, criterion 1 requires sites to have unique and informative page titles. Page titles impact blind users who use screen reading software by providing a form of navigation.
The worst a task can be scored on this criterion is 6 (where page titles are missing or inaccurate). Although this shortcoming would fail the WCAG 2.0 criteria, we do not deem it ‘difficult’ in terms of completing a journey and consequently we would not mark it down more severely in the Better Connected test.
We have made some revisions to the 14 criteria, making scores more explicit for some issues where they were not before (eg that forms presented on mobiles must be responsive) and downgrading some issues from 8 to 7 (eg absence of skip links and lack of accessible controls to stop movement). You can access a document with the revised criteria for accessibility testing in 2017-18 from the library of the Web Improvement Group in the Knowledge Hub (login required).
We are also clarifying overall scoring as follows:
Stage one examines home pages for the 14 testing criteria used in our full test. Sites fail if they score 8 or 10 on one or more criteria or where there are any issues in 7 or more of the 14 criteria
Stage two examines three tasks and the site’s top page for the 14 testing criteria. If a task scores a 10 on one or more criteria, it will fail automatically with a 0 score for the task. If a task scores an 8 on one or more criteria, it will fail automatically with a 1 score for the task. Stage two cannot be passed if more than one of the four tasks tested is scored 1 or 0. If one task is scored 0, a pass can only be achieved if all other tasks are passed with the top mark of 3. Sites will also fail tasks where there is accumulation of less serious issues. The severity of the issue (on our scale of 0-10) is multiplied by its frequency within the task, with the fail threshold set at 100.
The following issues will be given a score of 10, leading to an automatic stage one fail or a fail marked at 0 for a stage two task:
• A keyboard trap is present (criterion 4)
• A trap is present for screenreader users (criterion 4)
• Captcha is used without an accessible alternative (This should include an audio version of the CAPTCHA) (criterion 8)
• Contents auto play after loading page (criterion 12)
• Content flashes at a rate of 3 times per second or more (criterion 13)
• No form or email option is available for contacting the council to complete a specific service (eg order bulky waste collection) (criterion 14)
The following issues will be given a score of 8, leading to an automatic stage one fail or a fail marked at 1 for a stage two task:
• No Headings on the page (criterion 2)
• Swipe gesture required for navigation to off-screen content (criterion 3)
• The page’s sequential order is not logical or consistent (criterion 4)
• Bleedthrough: content not visually present is being picked up within the sequential order of the page (criterion 4)
• Page elements do not change to show they are in focus (ie ‘links, tabs and other key elements are not sufficiently highlighted’) (criterion 5)
• Page elements do not change to deliver a mobile specific experience – (ie site is not purposed for mobile and presents desktop version only) (criterion 5)
• Email option only for contacting the council to complete a specific service (eg order bulky waste collection) (criterion 14)
• Non-responsive form (sites accessed from mobile) (criterion 14)
• Inaccessible content (forms and essential info like how to claim benefits) (criterion 14)
Councils that take advertising on their sites, and are worried about the impact of advertisments with moving content appearing during Better Connected testing, should note that in our published criteria we have downgraded ‘No accessible controls present to stop movement’ from an issue scoring 8 to an issue scoring 7. In fact, this issue was already scored as a 7 for the purpose of 2016-17 testing.
Please let us have any comments or questions on these developments.
A forthcoming post will set out this year’s arrangements for stage one and stage two testing.