I just set up a new Native Instruments Kontrol S61 MK3 keyboard. After installing the Hardware Connection Service as outlined in the set up instructions, the next thing you’ll need to do is update the firmware on the device. Unfortunately, I ran into significant problems here. The firmware updater could not detect my keyboard, displaying the message:

Please connect your KONTROL S MK3 and click RESCAN.

The problem is that Windows couldn’t find a driver for the “KONTROL S61 MK3 DFU” device. Native Instruments provide some troubleshooting instructions for this scenario, but these didn’t help either. In step 18 of that guide, when manually choosing the driver, the troubleshooting instructions tell you:

WhatsApp

If you use WhatsApp on Windows, you are likely aware by now that WhatsApp have discontinued the UWP app and replaced it with an app that effectively just wraps the WhatsApp web app. Unfortunately, while the accessibility of the UWP app was quite decent, the web app has some major accessibility problems which have not yet been resolved. Fortunately, one of the nice things about the web is that it’s possible to tweak accessibility a great deal with scripting. This can only be done if you run the web app in a web browser rather than the desktop app, but since the desktop app basically just wraps the web app anyway, there’s no real disadvantage to doing this.

Before settling on the project to implement a full accessibility cache in Firefox, I investigated several other alternatives. In my last post, I discussed asynchronous accessibility APIs. Another alternative I considered is switching to UI Automation and having the UIA tree accessed directly in web content processes, rather than communicating via the main UI process. While this is not possible with IAccessible2 because of browser sandboxes, it could theoretically be possible with UIA because it has an intermediary Windows component called UIAutomationCore which sits between the client and the server. As an operating system component, there are less concerns with regard to allowing it in the browser sandbox. Unfortunately, this proved to be infeasible then and I think it is still infeasible now:

Accessibility API queries are generally synchronous: each query blocks both the client and the server until the query completes and returns its result. This causes significant challenges for modern, multi-process web browsers, requiring them to cache the accessibility trees from all other processes in the main UI process. This raises the question: why not make accessibility APIs asynchronous? As usual, there is a theoretical/principle answer and a pragmatic answer.

Let’s start with theory. For a browser, synchronous accessibility APIs are a real problem. So much of the complexity in multi-process browser accessibility architecture comes back to the need to support synchronous APIs. If they were async, we might not need to maintain a cache in the main process at all, which would save us a lot of pain, complexity and performance problems. From that perspective, async APIs would be great. Async APIs would likely also help assistive technology products to avoid hangs due to queries taking a long time or apps which stop responding, which is a real source of pain for users.

UI Automation (UIA) is Microsoft’s recommended accessibility framework for Windows, replacing the earlier Microsoft Active Accessibility (MSAA) framework. Despite this, to access web content, screen readers such as NVDA and JAWS continue to use IAccessible2, an open source API based on MSAA. This is not just because of the significant effort involved in switching to a different API, although that is certainly a factor. More importantly, it is because UIA is currently insufficient as an accessibility API for the web. Here are some of the reasons:

Sometimes, it is necessary to move a row up or down in a Google Sheet. For example, I might be maintaining a list of tasks ordered from highest to lowest priority and realise that a task later in the sheet is actually higher priority than earlier tasks. This Google support article says you can do this by selecting the row and then choosing Move row up (or down) from the Edit menu. However, when I selected the cells in the row, these Move row items still didn’t appear in the Edit menu.

Everyone loves paying bills, right? One of the best parts about paying bills is surely that they usually provide all the amounts, billing codes, reference numbers, etc. with punctuation and spaces; e.g. amount $1,234, reference code 123 456-789. But banking and payment portals - sometimes even the payment portal for the organisation issuing the bill! - won’t accept spaces and punctuation! This means you can’t simply copy and paste the numbers, which is really annoying. Wouldn’t it be awesom if there were some easy, automatic way to strip out everything except numbers and the decimal point before pasting?

The Firefox accessibility engine is responsible for providing assistive technologies like screen readers with the information they need to access web page content. For the past couple of years, the Firefox accessibility team have been working on a major re-architecture of the accessibility engine to significantly improve its speed, reliability and maintainability. We call this project “Cache the World”. In this post, I explain the reasons for such a massive undertaking and describe how the new architecture solves these issues.

I recently got a new laptop: Dell XPS15 9510. While this is a pretty nice machine overall, its audio drivers are an abomination. Among other things, the Waves MaxxAudio software it ships with eventually leaks all of your system memory if you use audio constantly for hours, which is the case for screen reader users. I eventually got fed up and disabled the Waves crap, but this makes it impossible for me to use the headset mic on my Earpods.

I’m finally learning to cook some decent food, so I need to be able to read recipes. For a while, I was reading them out of Simplenote on my iPhone. However, I encountered several frustrations with this approach (and this applies to any notes or text app really):

• When you’re not editing, Simplenote shows the note such that each line is an item for VoiceOver; i.e. you flick right to read the next line. However, if you bump the screen or perform the wrong gesture accidentally, you can easily lose your position. If the screen locks or you have to switch apps, you lose your position completely, since VoiceOver doesn’t restore focus to the last focused item in apps.
• When editing, you can review the note line by line using the rotor. The advantage here is that the cursor doesn’t get lost when you switch apps or the screen locks. However, lines can be smaller than is ideal due to the screen size, so one recipe instruction might get split across multiple lines. Also, moving the editing cursor with VoiceOver is notoriously buggy, often getting stuck, etc. Finally, again, if you bump the screen or perform the wrong gesture, you can lose your position (or worse, accidentally type text into the document).
• The screen lock problem could be solved by disabling auto lock, but that obviously has an impact on battery.
• Having to repeatedly take my phone out of my pocket to read the next instruction was impractical, especially given the risk of losing my spot in the recipe. Leaving it on a bench somewhere meant I had to keep walking back to wherever my phone was located, which was similarly tedious. This might seem simple enough, but when you’re moving around a lot, using your hands for other things, getting your hands dirty, etc., it just isn’t efficient.

I considered a couple of solutions: