[Accessibility-testing] Next call / next actions?

[Jason McIntosh]

Feb 25 meeting debrief:

Present: Claire, Deborah, Jmac, Zack, Zarf

(Acknowledging that I was late, for no good reason. My thanks to Zack for making the call in my absence, and Claire starting the discussion.)

Primary topic was — per weekend list posts — whether we had enough data to proceed with a report, or whether we should collect some more.

Zack was concerned that 35 people isn’t a large enough population to draw statistics out of, or otherwise do math on.

Without disagreeing per se, Deborah countered that our test was never set up as a proper statistical exercise to begin with, so expecting any mathy results now is simply not practical, no matter how much more data we capture (in the form of survey results). Furthermore, 35 survey respondents is far better than typical for any accessibility study regarding an existing product or service.

Everyone present on the call ended up joining me in my lean towards making our report our next goal, and making that report about our method and our (very human, subjective, but experience-driven) observations and recommendations, as well as just sharing all the (anonymized) responses. And not trying to over-math any of it.

I absolutely want to hear from others on this list about this idea, before we commit to anything!

We agreed to defer discussion about the logistics of writing the report, at any rate, and (as Claire’s already posted) agreed that June 30 would be a fine deadline to shoot for, with June 14 — the start of Narrascope — being a good “stretch goal”.

> On Feb 23, 2019, at 10:33 AM, Jason McIntosh <jmac at jmac.org> wrote:
> 
> Hi folks,
> 
> First of all, our next call is set for Monday, Feb 25, at 7 PM eastern; as usual, please come if you can.
> 
> Before then, I’d like to start discussing our next step as a team. The obvious routes from here are:
> 
> A: Decide we have collected enough data to write a meaningful report; write that report.
> 
> B: Decide we don’t have enough data yet; perform another round of tests.
> 
> Being a completion-minded individual, I of course lean towards route A. By my count we have received survey responses from 33 people.
> 
> I would ask the folks here experienced with group-testing: given that number, and given the data that we have in-hand now, is this enough to write a meaningful report, or does it not meet that minimum?
> 
>> On Feb 14, 2019, at 4:35 PM, Jason McIntosh <jmac at jmac.org> wrote:
>> 
>> Present: Austin, Claire, Deborah, jmac, Zack
>> 
>> We spent a lot of the meeting talking about patterns that jumped out at us from the collected responses. A sampling:
>> 
>> (And let me state here that I mean “Opera” to refer to the game, not the browser)
>> 
>> • Many low-vision users wanted more high-contrast / white-on-black text options.
>> 
>> • More people than I would have expected could not see any alt-text for the image in Opera.
>> 
>> • Interesting to learn of the existence and popularity of Nick Stockton’s IF plugin for NVDA. It seems to have known bugs that many veteran players were inured to (but still able to describe, knowing exactly where the blame lay).
>> 
>> • Lots of people had problems with Opera’s plaque (involving a box quote). Others struggled with the slide projector (involving menu selections). I wonder how it correlates with familiarity with parser-game conventions. (Did we ask about that in the survey?)
>> 
>> • I was surprised how few blind testers had much to say about the Twine game’s cycle-links. Taking other feedback into account, I hypothesize that veteran screen-reader users are simply used to getting their focus bounced around by Javascript hacks like this all over the web, and just take it in stride, as annoying as it is?
>> 
>> • More than one screen-reader user suggested making the link between picking up objects and having them appear in your inventory more obvious by placing the inventory list closer to the bottom of the screen. I admit I’m not sure why this suggestion, in particular.
>> 
>> • Every low-vision or blind user was 100% thumbs-down on the text-art map, as expected — except for one dude who said “Easy as a piece of cake”, and I appreciate him proving that *there’s always one*.
>> 
>> Some individual responses that jumped out to me, and which we discussed:
>> 
>> • I loved Felix Grützmacher’s complaint about Opera’s “twist the 6 to turn it into a 9” puzzle, saying “This is a very visual way of thinking, and many blind people do not know what the numerals look like.” I have to wonder, Zarf, was this intentional? Because it’s a sort of second-order accessibility issue I didn’t expect at all before yesterday!
>> 
>> • Melinda Embleton has mobility issues that reduce her ability to type more than a little, and she uses no AT. She was able to play through Opera but it was a struggle. I thought that an interesting case… the naive solution is “well, use AT, then” but that seems like assuming a lot. It might be nice to have a “community official” list of AT that plays nice with IF (and doesn’t break anyone’s budget, if possible)?
>> 
>> • Laura Swetz suggested playing a sound effect whenever anything pops up. This brought to kind the recent AbleGamers guidelines, as Mark has described them to me, to alert players of stuff via multiple sensory channels (e.g. when you level up, print “You leveled up!” *and* play a unique sound effect, consistently). What an interesting idea, to make that a habit in all kinds of IF, which is traditionally stone-silent (no matter how visually noisy it might get)
>> 
>> 
>> 
>> _______________________________________________
>> Accessibility-testing mailing list
>> Accessibility-testing at iftechfoundation.org
>> https://iftechfoundation.org/cgi-bin/mailman/listinfo/accessibility-testing
>