[Accessibility-testing] 2018 goals review
dan at fabulich.com
Tue Feb 6 01:56:24 EST 2018
I'm not sure it's necessary to develop a single game playable on parser platforms and hypertext or choice-based platforms.
Furkle has already developed a "happy path testing guide" for Twine https://furkleindustries.com/iftf-a11y/twine-testing/twine-testing.html <https://furkleindustries.com/iftf-a11y/twine-testing/twine-testing.html>
In September, we said that I would come up with a similar guide for testing "Cloak of Darkness" in WinFrotz.
Here's the passage from our meeting notes in September:
> A "happy path" introduction to testing parser IF, for users of screen readers. It would document how to download and install WinFrotz, and how to use it to open "Cloak of Darkness." (Or, perhaps another small one-puzzle game. Cloak of Darkness uses a light/dark puzzle, which is unfortunate/ironic for VI testing.) This document would also describe the expected anatomy of parser UI, so the tester would know what kinds of things should be there. There's a status line; each room has a description; you can enter commands; the responses need to be distinguishable from the commands you type; and after several commands there should be a way to scroll back through the transcript.
I didn't write that doc yet, but I think that's still the obvious next step, despite the light/dark puzzle.
After that, all we need are lists of possible/expected bugs to watch out for in parser-based games and in hypertext games (separate lists), and a list of games to test.
> On Feb 5, 2018, at 5:35 PM, Jason McIntosh <jmac at jmac.org> wrote:
> Hi folks,
> I did have a Skype chat with Deborah last week (hi Deborah!) to help update my notions of what the testathon actually is, and how this team’s discussions had evolved the definition over the course of 2017.
> My current take on testing method goes like this — and for simplicity’s sake, allow me to focus just on testing IF gameplay platforms here. (I realize this gets a bit long, and I appreciate your patience!) This merges together my talk with Deb, my understanding of the group’s discussion so far (up through the October 2017 meeting), and a couple of initial suggestions from me. So:
> The test-coverage matrix will be akin to a two-dimensional grid whose Y-axis is specific assistive techs: Magnifiers, screen readers, dictation software, and so on — not at all an exhaustive list of every make and model, but a representative sample. This can also have “not-techs” relevant to a11y testing, e.g. play without the use of a mouse.
> The X-axis is IF play platforms: Quixe, Parchment, Gargoyle, Lectrote, Sugarcube, Harlowe, ChoiceScript, et cetera, noting the versions of all software involved.
> Having established this matrix, we the IFTF accessibility testing team would then prepare an original work of IF that would exercise the accessibility features of its platform. This, admittedly, is me rolling in a new idea, but I humbly report it’s based on y’all’s conversation so far. I envision this game as similar to “Cloak of Darkness” (http://www.firthworks.com/roger/cloak/), except tuned for accessibility: to get to the end of the (very short!) game, the player must interact with the game in a variety of ways, including ways that might present an unintended challenge to players with disabilities.
> For example, one “puzzle” might involve opening a lock by looking at an image of a note and typing in a code found in that image. The game will include “alt text” for that image, and it’s up to the play platform to display it correctly, and under the correct conditions. Similarly, there may be a puzzle that involves ASCII artwork, or listening to an audio clip, and so on.
> We will adapt this game for as many IF systems as needed (Inform, Twine, ChoiceScript…) in order to cover all the play-platforms we wish to cover. This adaptation will involve making platform-appropriate changes, such as adding a challenge based on "cycle-links” in a Twine game. Furthermore, if a certain platform cannot accommodate a certain disability at all — for example, it allows an image to be displayed, but not with alt-text — then we mark this down in the appropriate spots of the matrix as a systemic failure.
> Having established all this, we invite the testers to play! We coordinate with testers to get as much coverage of the matrix as possible, and the IFTF team can itself fill in gaps where needed. On completing the game, testers fill in a survey detailing which technologies they used to play, and allowing the, to make additional notes and comments. We plug their results and notes into the matrix as appropriate.
> Does all the above sound on-point so far?
>> On Jan 31, 2018, at 11:30 PM, Jason McIntosh <zendonut at gmail.com> wrote:
>> Yeah, I suspect there’s a disconnect in my own understanding here.
>> I’m going to contact Deborah in a side-channel presently and get a review of what, exactly, “Testathon” means, if it doesn’t mean the thing I just (shakily) described.
>> In the meantime, if any of the rest of y’all have examples (even notional or hypothetical ones) of what some individual, crowd-sourcable tests might look like in an IF testathon, I’d love to see them!
>>> On Jan 31, 2018, at 11:16 PM, Zachary Kline <zkline at speedpost.net> wrote:
>>> This seems… Fundamentally different from what my understanding of AT testing, and a reasonably helpful goal, is. Does anybody else have input? <confused face>
>>>> On Jan 31, 2018, at 8:14 PM, Jason McIntosh <jmac at jmac.org> wrote:
>>>> While I would agree with this statement, my understanding of the Testathon (based on conversations I had with Deborah years ago, pre-IFTF, when she first suggested this as a project) is that knowledge or possession of AT is not required to run the tests.
>>>> Rather, testers would be instructed on what the expected, standards-compliant results of various test inputs look like, and they confirm whether these results appear or not for each test. In a sense, the testers themselves would pretend to be AT devices, making sure that the output they see matches something that a standards-compliant AT device would be happy with.
>>>> Granted, this is all based on memory, and I’m having trouble finding online articles about past examples of this online. (When I google for “accessibility testathon”, I find… IFTF’s own webpages.)
>>>>> On Jan 31, 2018, at 10:09 PM, Andrew Plotkin <erkyrath at eblong.com> wrote:
>>>>> On Tue, 30 Jan 2018, Jason McIntosh wrote:
>>>>>> • Testers can be anyone with user-level access to the software being tested. They needn't be assistive technology users, or even people with disabilities -- though of course we'll welcome and encourage participation from AT users & PwDs.
>>>>> In our last big meeting, we were tending towards the idea that it would be easier to recruit assistive technology testers and teach them about IF than to recruit IF players and teach them about testing assistive technology.
>>>>> "And Aholibamah bare Jeush, and Jaalam, and Korah: these were the borogoves..."
>>>>> Accessibility-testing mailing list
>>>>> Accessibility-testing at iftechfoundation.org
>>>> Accessibility-testing mailing list
>>>> Accessibility-testing at iftechfoundation.org
>>> Accessibility-testing mailing list
>>> Accessibility-testing at iftechfoundation.org
> Accessibility-testing mailing list
> Accessibility-testing at iftechfoundation.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Accessibility-testing