How can web analytics improve accessibility?

Editor's Notes

• #2: How do we know it works for anyone? Not just how many users, but How they behave Use it as expected Pages hard to use? Improvement work? Hard to do with AT - why?
• #6: Difficulties and possibilities
• #7: So what is web analytics? The web log stores a lot of information about the user. Web analytics convert this raw data into statistics, so we can easily know which pages they see, what kind of device they are using, how much time it takes to download the a page and so on. All this is useful information for us. We know what attracts people, how much content we can put on a page, and when we can stop worrying about outdated versions of Internet Explorer. But we don’t know if the user is using a screen reader. We know if how the site works for someone who is still using Windows 7, but we don’t know if it works just as good for people who can just hear the content, as for people who can see it.
• #8: They don’t want to be treated differently. This is because they fear that they will be directed to a different page than the other users. If the webpage knows they are using a screen reader, it might redirect them to another so called accessible page which is a lot duller than the cool and fancy one everybody else are using. And that happens. Some websites do this, so that they won’t have to worry about accessibility on the rest of the site. I think that’s a bad idea. That’s twice as many pages to maintain, and my bet is that one of the versions will fall behind when new features are added. I’m a fan of universal design, as I think most of you are as well. They don’t want to share personal information This is a common concern for everyone these these days. We don’t want people we don’t know to know more about us than they really have to. Facebook and Google probably know more about me than I do, and selling this information has become big business. People with disabilities don’t want that information to be sold. For web analytics however, these data are made anonymous. I don’t care what each single user is doing. It’s what each group of users do that I find interesting. You’re not counting people When I try to explain what accessibility is about, it’s very easy to use blind users as an example. Then I always get asked, how many blind users do we have? sounds familiar? Even if we could detect screen readers, we couldn’t know if these users were actually blind, or they were using it for other reasons. But web analytics is not about counting users. It’s about watching groups of users, and how they are using different parts of the website. If we could detect screen readers, we wouldn’t know how many blind users we have, but we could have a nice indication if they have problems on our site that other users don’t. You may agree or disagree with these arguments. But whatever you decide, you should not ignore them. If you somehow are trying to identify AT users, either for web analytics or other purposes, ask yourself if you are violating any of these concerns.
• #9: Then there is the issue that causes panic all over Europe this winter. GDPR and ePrivacy are new regulations for what information about users that we can store and share. I’m not going into details about this, but the important issue for this talk is that we can not log personal information without the user’s consent. What personal information really means in this context is not really clear to me, but in worst case we will lose a big parts of our traffic data. We don’t know how many users who’s going to agree that we can use their data for web analytics, but I fear that most people will refuse by instinct, like most of us do when we can refuse something when we don’t want to dig into it. When the amount of AT users is quite low to begin with, and we can only use data from a fraction of them, I doubt we will have the volume we need to get reliable statistics. But GDPR also provide some other possibilities, and I will get back to that.
• #10: So, why should we do this? What are the benefits? You may have seen this lean startup model before. When you create something new, you build, you measure, you learn. Then you start over and build something better. If you think you have a good idea for a website, you might want to test it in a small scale before investing a lot of money on it to see if people are going to use it. You start by building a «minimum viable product», which means that you make a simple solution as possible. Then you measure it. You deploy your first version and let some people try it, and gather data. To do that, you will need to use web analytics. Then you learn. What worked in this version and what should you improve? Then you figure out how to improve it, and start over with next iteration.
• #11: But if you seriously want to include all users, you shouldn’t just look at the majority. You should separate them into groups. You can easily split them into desktops users and smartphone users, because they might use the site differently. But it could also be interesting to have a group of assistive technology users.
• #12: When you deploy a change and the reaction is like this
• #13: You won’t see these people Put them into a group and make them visible
• #15: Let’s start with measuring accessible alternatives. Sometimes different users need different features, so that we should provide more than one way to do the same thing. The last time I was was buying a car, I was of course using finn, because it’s the best online marketplace in the world. Back then we only had these sliders to adjust the price. Sliders are great when you only need a rough estimate of what you can afford, but when you are actually going to buy something, you need to be more accurate. And when I was searching for a car I could afford, I was sitting on a very shaking bus, so it was impossible to hit the correct maximum price. So I ended up by buying a car that was a lot more expensive. The sliders were very popular, so we couldn’t just remove them. So the solution was to provide an alternative way to set such values. The challenge was that if we displayed both the text boxes and the sliders, the filters would occupy twice as much space, so we hid them and made them visible by clicking these buttons. The first attempts scored very low so was I wrong or the users didn’t understand what those buttons were. Finally, we came up with this design, and the usage escalated. We see here that a little less than 3 percent of the visitors on our car search page have used it. It’s not overwhelming, but it’s a good indication that there are users who think that this feature is a better alternative.
• #16: When you struggle to convince your bosses that a component should be more accessible, you can make an AB test to get hard data that proves that your solution is the better. In an AB test you give 2 equal groups 2 different alternatives. Then you you measure which one that works best. It doesn’t even have to convert better. As long as your alternative is more accessible, all you have to prove is that is not worse. I will show you an example of an AB test later.
• #17: If you have logged in users who has entered their birthdate when they registrate, make sure this is logged for the statistics. I have created a segment for users between 80 and 90, and I’m soon going to show you how this group stood out compared with the rest.
• #18: We may try to recognize them by looking for typical behaviour. This is a quite difficult and not very accurate method, so it won’t give us exact data, but perhaps good enough to spot differences in the user patterns. I have come up with two possible things to look for. One is scaling a web page, making the content larger. The other is to track focus events on skip links. The biggest problem about the first one, is that my script failed completely. It was supposed to only log zoom events, but it logged every page view. So the analytics team became quite mad at me. I still think it’s possible to it right, but I haven’t made it yet. Another problem is that this group simply might be too big. I mean, who hasn’t zoomed in on a web page on a mobile before? The second approach is to find users who are using skip links. Skip links are hidden links that only get visible when they get focus from users with screen readers, or who navigate with the keyboard. They are used to skip headings and navigational stuff, and go straight to the main content. This method is not perfect either. People may still open them by accident, or there may be users who just prefer to use the keyboard, but picks up the mouse when things get difficult.
• #19: Skip links are hidden until they get focus
• #20: Logging this event is quite simple. You add an event listener to the link that reacts when it gets focus, then you call the tracking script to track the event for web analytics. Then you can use that event to create a segment in your web analytics tool. Filtering with this segment in your ordinary reports may give you some interesting information on how this group use the site compared with the total.
• #23: If we look at the time spent per visit, we see the same trend. We also see that older users spend a little more time than the rest, which may be expected.
• #25: We had a workshop recently, where we were teaching some colleagues how to test accessibility. An eager participant found some buttons that wasn’t accessible by keyboard. These are different features for editing your own ads. Instead of fixing this bug right away, I made an A/B-test to measure if the fixed version got more clicks than the buggy one.
• #26: But I couldn’t see any noticeable improvement. Could it be that the users who couldn't use them simply drowned in the masses?
• #27: So I ran the same report with skip link users only, and here we can see a remarkable change. Don’t look at the one the top. That was accessible to begin with. But look at the last one here, the accessible version has almost 4 times as many clicks. And this is the benefit of identifying assistive technologies.
• #28: When we know that there is a lot of skepticism about doing this, we could just simply ask them. This should provide more accurate data, since we can ask specifically what their challenges are. The big question is: will it be enough data? I haven’t had the chance to test this method yet, so I don’t know. So when we ask the users for the consent we need for GDPR, we can add an extra checkbox that users who claim to have a disability may check if they accept that we use their traffic data to improve our site for their group.