Do Automated Solutions like #AccessiBe Make the Web More Accessible?

Update March 9th 2021: I have added several sections to this post, including commentary on the accuracy of my findings, in response to feedback I received on Twitter.

---

If you have an interest in web accessibility, you’ve doubtless found yourself at some point involved in a discussion about the best way to fix inaccessible web content. We can all agree that the vast majority of websites aren’t accessible, but what we can’t seem to agree on is what we should be doing about it. Do accessibility toolbars work well? Should you describe all of your images, or is it ok to hide decorative content so screen reader users don’t know it’s there? What even is an inaccessible website? Although the Web Content Accessibility Guidelines (WCAG) exist for a reason and do a good job of conveying what accessible means, you’ll still find many people who argue that they don’t go far enough. As the Internet adapts to meet the changing needs of our society, so do the WCAG.

Automated Accessibility Solutions

One conversation which is happening more and more surrounds whether automated solutions can solve the problem of an inaccessible Internet. We’re moving towards AI-based solutions in many areas of our lives, so can it help us make web-based content more accessible to disabled people? Companies are testing this out in different ways. In 2016, Facebook started using AI to generate alt text for images, a solution which was universally praised by non-disabled people but got many raised eyebrows from the blind community. Whilst auto-generated alt text sounds like a great idea in theory, the reality was underwhelming. Many blind users report that it hasn’t made the social network much more accessible to them. One blind user stated that the auto-generated descriptions didn’t allow her to comment any more meaningfully on photos posted by her friends than she otherwise would have.

Facebook isn’t the only company with the goal of using artificial intelligence as a means of building accessibility solutions. Microsoft funds a whole range of projects under the banner of AI for Accessibility, which try and make the world more inclusive in areas such as employment, education and community. It is clear that AI has the potential to make certain aspects of our lives more accessible, negating the need for human support. However, as demonstrated by Facebook, these projects vary in usefulness.

I have a personal interest in all of these projects. I’m a blind person who relies on the Internet being accessible if I’m going to participate anywhere close to equally in society. I often find myself fighting with systems which were not built with a blind person in mind. Sometimes, I’m able to reach out to the web developer and they are receptive, at other times, I have to cut my losses and move on. I have very serious concerns about the Web not being accessible to me. What happens if my bank changes their website and I can no longer access it? How will I study if I can’t read the materials that are uploaded onto the virtual learning system? Will I ever get a job, when so many of the online tests and even application forms are not accessible to me?

AccessiBe and the Promise of an Accessible Internet

I have also found myself heavily involved in the debate around a fairly new tech company who have the goal of making the Internet accessible by 2025. That company is AccessiBe. They market a product to businesses which, with a single line of code, will make a website WCAG 2.1 compliant. In theory this sounds great. I definitely want the Internet to be more accessible. I have no reason to want AccessiBe to fail, in fact, if their product can do what they say it can, it would without a doubt transform the way I interact with the Web. Yet when I heard about the product I had reservations. Could it really do all that they say it can? Surely fixing accessibility is more nuanced than only one line of code and some AI?

I am not the only person who has concerns. Adrian Roselli, an Accessibility and User Experience Consultant, has written a very detailed article outlining the history of AccessiBe and its questionable performance when implemented on a website. The #AccessiBe hashtag on Twitter is filled with concerned web developers and end users, some of whom are quite angry by now.

AccessiBe continues to defend their product, and why wouldn’t they. They are a business, and it’s quite possible that they believe that their solution really is the best one out there. After presumably seeing my previous concerned tweets, this morning they tagged me in a tweet linking to an article posted on their website where they share how they feel AccessiBe fits into the future of web accessibility.

They invite people to read the article with an open mind, and so I set aside a couple of other projects to really take some time with it. My first observation was that they seem to view themselves as victims of unfair attacks by people online. I will concede that some of us are reaching the end of our tether, and so are probably being more abrupt than we otherwise might be. But there seems to be very little consideration on their end that as a disabled person, I’m tired and frustrated with the Net not being accessible and solutions which don’t actually fix the problem. They also outline what they deem to be common arguments against AccessiBe, and why those arguments are invalid. I encourage you to read the article for yourself and make up your own mind.

Does AccessiBe Work?

I would like to preface this section of the post by stating that I am not an accessibility tester. I’m a blind person who has been using a screen reader for the best part of 22 years, and thus I can comment on the usability of sites with and without AccessiBe. These are my observations and should be taken as such.

I chose to test AccessiBe on a blog post on the Asda mobile site. It wasn’t an overly complex page, it didn’t include complex charts or graphs, but it did include multiple tables, blocks of text and menus, making it ideal for such a test. You can view the blog post, which will help you decide Which Xiaomi phone is right for you?

My first observation was that neither version of the page, with AccessiBe enabled in screen reader mode, or disabled, was fully accessible. I noticed issues with both versions, however those issues changed depending on whether I had enabled AccessiBe’s screen reader mode.

In both versions of the page, the title of the blog post is at heading level 1. This is what I would expect, and so I imagine this has been included in the HTML code. It’s important to create a logical heading level structure so that screen reader users can navigate more efficiently.

When screen reader mode is disabled, and the page is presumably in its default setting, the name of each device appears as a heading level 2. This is also to be expected, and it falls within that logical structure I rely on. I could press H to move through all the headings on the page, or 2 to move to the next heading at level 2. Under the heading for the device is a block of text, followed by a table with information about the device. The problem here is that the table isn’t marked up that well, Ideally, HTML tags should be used so when I’m moving through it, I know what header the data corresponds to. By using the correct HTML tags, I would be able to more easily know the structure of the table.

I expected that, when AccessiBe’s screen reader mode was enabled, these issues would be resolved. What I actually got was a far more chaotic experience. The names of the devices appeared at heading level 2, but so did every single header in the table. I could no longer navigate by heading level 2 to move between devices, because I would land on all of the information in the table instead. It made effective navigation of the page almost impossible. It also seemed to think that Cost, which was the first row in the table, was the actual header, so every time I moved between different pieces of information in the table it would read out the cost first.

In the end, it was far easier to navigate the blog post without AccessiBe. Although I’d have liked for better table markup, I could understand the information perfectly well without it, and I could at least navigate the page quickly. With AccessiBe enabled, this became impossible.

Other concerns

I also have privacy concerns about the kind of information AccessiBe might give companies. Presumably, in order to continually notify me that there is the option of a screen reader mode, AccessiBe detects that I am a user of assistive technology. Do they tell the company this information? What kind of impact might that have, if I were to apply for a job on a website which was using AccessiBe.

We can argue about whether this is AccessiBe’s responsibility or not. The problem of data tracking goes far beyond them, and they certainly aren’t the only company I question in that regard. But I do think it is a very valid concern and one which should be clearly addressed. Ultimately, if web developers built an accessible website in the first place, I wouldn’t even have to worry whether my choice to use an accessibility plugin might indirectly result in discrimination against me.

I think it is really important for us to have these conversations in an open and honest manner, which is why I have chosen to write this post. I welcome input from those who feel like AccessiBe and other AI solutions are enhancing their life, and those who, like me, have concerns. Having concerns does not mean that I dismiss the impact of AI entirely. I have previously written about how AI describes so-called “adult content” to blind people, and why we may be able to use tech as a solution for limited access in this area. My end goal is, and will always be, a society which is more accessible.

Accuracy of Test

It was suggested to me by a Twitter user that the results I found were due to a bug with the software, rather than the overlay itself. It is quite possible that AccessiBe was faulty at the time of testing, however I find the implication that as such, my user experience is invalid to be a frustrating assertion. I had no reason to believe the software was faulty. I was able to replicate my results every time I visited the website. Bug or not, AccessiBe was not providing me with a more accessible user experience.

I have since tested the overlay again, on the same Asda Mobile page. Whilst I no longer get the issue of every header in the table being at heading level 2, table markup is still poor. Data does not correlate to a header, which is manageable in such a small table with only two columns, but would quickly become impossible to navigate in a larger table. Therefor, either AccessiBe makes the user experience worse, or gives me an equally poor experience to the one I have when screen reader mode is not enabled. Either way, it is hardly a glowing endorsement. Furthermore, as WCAG 1.3.1: Info and Relationships states that “The intent of this Success Criterion is to ensure that “information and relationships that are implied by visual or auditory formatting are preserved when the presentation format changes,” the failure of the AccessiBe overlay to create accurate table markup means that AccessiBe does not to my best knowledge, in fact, make the site WCAG 2.1 compliant.

The fact that with the plugin enabled the site is no more or less accessible becomes somewhat irrelevant when you consider that the entire product is built on the claim that AccessiBe will make websites comply with WCAG 2.1. If they can’t do that, they are selling people a product built upon false advertising. I am only making these comments in line with my own experiences and understanding of WCAG compliance. As such, they should not be taken to be legally binding or infallible. In fact, I welcome feedback and alternative perspectives.

I have included a video of my original test in order to highlight the issues I was experiencing. I hope that this gives context that words cannot. The video is captioned and a transcript is provided, however if you require the transcript in an alternative format please reach out to me and I can get that to you.

Thanks to Adrian Roselli for helping caption this video.

Where do we go from here?

I was contacted by members of the AccessiBe team after publishing this post. You can read their comment below the post. They have also repeatedly asked me on Twitter to tell them how their product doesn’t work, and engaged in debates around their own use of Twitter and how the community has engaged with them. I find it interesting that AccessiBe are swift to respond to tweets which concern our feelings on the product, and that they repeatedly ask for examples and “facts” as they put it. Yet when such examples are given, they either do not comment, or invite disabled people to speak with them in private.

I am more than happy to speak with organisations privately, however, typically I am paid for my consultancy work. AccessiBe is essentially asking us to give them free usability testing, and refusing to discuss our comments publicly. They are running an event where they will demonstrate the product live, and respond to questions from the community. It will be interesting to see how they justify their lacklustre response and their business model going forward. I would encourage anyone with an interest in their product to attend the live demonstration and to submit questions for their CEO to answer. Ultimately, we deserve accountability and honesty. Whilst AccessiBe are inviting disabled people to speak with them in private, they should be paying for us to act as usability testers.

I said earlier in this post that I have no reason to want AccessiBe to fail and I still stand by my remarks. I would like nothing more than the Web to be accessible to me. However, I seriously question whether AccessiBe are the right people to be doing this when they insist on shouting over the very community their product is supposed to benefit. I personally feel that this is a matter which should be handled by the courts at this stage. If my theory is correct regarding their product and WCAG compliance, this will be something that must be handled by disabled people engaging in legal advocacy. This is incredibly frustrating as it means disabled people pulling on already limited resources, however I see no other way for the matter to be resolved.

Sustainability

There are arguments to be made that both support the use of automated tools like AccessiBe, and which discourage it. Both certainly have merit.

AccessiBe argues that their product is more sustainable because the business only has to pay for one solution, rather than spending money on employing developers who have knowledge of accessibility. There is something to be said for this, however, when you consider that businesses must pay monthly for AccessiBe, is it actually any less expensive? Most principles of accessibility are quite simple and certainly well documented. In fact, a basic knowledge of HTML will get you surprisingly far when you’re only creating simple content.

Furthermore, we know that many small businesses use content creation tools like WordPress. Surely, considering that WordPress has their own accessibility team, money would be better spent ensuring that these authoring tools comply with ATAG and WCAG.

What happens if, in five years time, AccessiBe shuts down? The business is left with a website that is no more accessible than it was when they built it, and they’ve been spending money every month for an accessibility solution that isn’t actually tied to their website. Surely in this situation, the money would have been better spent employing someone to do a small amount of web development work, or for the person responsible for the website at the business to take a basic web accessibility course?

I don’t have the answer to all of these questions, and I think that businesses need to consider their own finances and make a decision that’s right for them. However, I do encourage people to seriously consider where their money would be used most effectively. AccessiBe quotes a figure of $10,000 to make a website accessible, however they haven’t provided any justification for this figure. Just because something is posted online doesn’t make it true. I encourage people to fact check my blog post, equally, check the figures quoted by AccessiBe and question what data substantiates them.