Luisa- Usability Test Google Lens

1. I would like to ask a few questions to understand your background and experience with
technology. Can you lease give me a little introduction about yourself, please?

- uh My name is Luisa Fernanda Revelo Escobar. I'm Colombian. I'm 37 years old. Um I have a
bachelor degree in foreign language education. Um Currently I'm living in Norway, specifically
Bergen. Um Yeah, what else can I say? Working as a... Well, actually I have two jobs. One is a BPA
assistant, which is being an assistant for a handicapped person. And I'm also a tourist guide for a
Spanish speaker, a Spanish-speaking tourist.

2. And what kind of mobile devices do you use?

- I use a Samsung Galaxy. It's an Android. Uh I think it's S21. It's the model.

3. how often do you use image recognition tools like Google Lens?

- Not that often. I know the tool, but I don't know why, but I don't need it so often. So I would say
maybe once a month.

4. What do you typically use your mobile device for?

- Uh Chatting, basically. to use apps like WhatsApp to make calls with my family in Colombia, to text
yeah family and friends, to check social media, Facebook, Instagram, uh to take pictures, make
videos, yeah to check my e-mail. uh To make calls, of course, and um use send or receive SMS,
which is very common here. And yeah, that's pretty much it.

5. And have you ever used Google Lens before?

- Yes, I have used it before. And what for? Um When I need to, like I find an item, for example, which
I'm not sure what it is, and I need to know a little bit more about it, but I don't know how to name it,
or especially because I'm living in a foreign country, so sometimes if I'm not sure about what that is,
I just just Google Lens and yeah find some information on the web about it, and that gives me a hint
or an answer of what the item is or what is it

6. we are going to start by opening the Google Lens app. And before doing anything, I would like
you to take a moment and look at the first screen, and tell me what do you see, what do you
think you can do with this app, and what stands out to you. So can you please show me your
screen? Okay, I can see your screen. I can see my screen, so should I go to the first picture? I
would like you just to take a moment with the app, explore a little bit, and then you can tell me
what you see. Uh something that stands out to you at first sight, and maybe tell me, what do
you think you can do with this app?

- Well, I haven't explored it that much, so right now, what I have used so far is the option of taking the
picture with my phone, and 137 yeah, search, yeah, like just let the machine search. But I can see
now that there are some options below the bar. So I guess that there's something related to music
because I see a music symbol, music fine. Um Something related with education, I don't know what
that is. Options of different languages, I guess it's the language setting, maybe. And I don't know. I
suppose that with the first option, the one that is like yellow, I can load the image and maybe it's
going to start searching based on that image that I load, I guess.

7. Is there anything that stands out to you?

- Yeah, the main bar with the search word, the the icon of the microphone, so I guess I can provide
some sound, and the icon of the of the camera, besides the name of Google, of course. Yeah, if I
scroll down, I see that, yeah, these are just like news, suggested news.All right, thank you so much.
All right,

8. So now, before we start, I'm going to introduce to you a scenario. I would like you to think, uh
to put yourself into that context and try to perform all the tasks that we are going to perform
during the test, thinking that you are the person in that scenario. You are a frequent traveler
and today you find yourself in a large and vibrant market. The market is filled with colorful
stalls, people, and plenty of attractive objects and food. Some local made others from around
the world. You want to buy interesting souvenirs for your loved ones, but since you can't speak
the local language, you only rely on your smartphone for assistance. As you explore, you'll find
5 things that intrigue you. Before purchasing them, you have taken picture of them and want
to learn more about their history, origin and significance. To do this, you will use Google
Google Lens to upload the photos and gather relevant information.

We are going to start working on the task number one. So for task number one, I would like
you to use the first picture, and you need to upload this object into Google Lens. I would like
you to identify the object using Google Lens and tell me what did you expect to see in the
results and if the app identified the object correctly. So you can explore with the suggestions
that you are having and see how it goes and how you can interact with the app.

- Yeah, I can see the name of this, Arruaco Mochila Arruaca, which is something that I expected
because that's actually the item in the picture. Um I can see that the first websites with the
information, they're in English. And I think I saw one in Spanish here, Boles artes en el Colombiana.
Um most of the results are like maybe websites where they sell these types of mochilas. Um Yes, so,
yeah, maybe for the scenario that you explained to me before, if, for example, this was one of the
items that I found in that market,So yeah, I would expect to find maybe the price and compare,
because I'm thinking of this as a souvenir, so I would like to know the prices. Then, yeah, I don't
know, should I click in one of the websites or...somewhere. I don't know what happened. Okay, I
have to start again to search. Probably it will happen the same here, I don't know. That can be
expensive.

9. how confident are you in the accuracy of the results that you are exploring right now?

- I'm very confident, I guess, because even if I don't find exactly the same Mochila exactly the same
pattern or you know the same design. Um I can understand that it's the it it belongs to the same type
of bag of Mochila, yeah just different models, different styles or even sizes, but it follows the same
type of Mochila that I'm looking for. SoYeah, I can recognize that. Then, yeah, if I want to buy one, I
guess I would start just going from one website to another, checking the, well, the pattern that I like
or and the price.

10. task number two. So I would like you to upload the second object in Google Lens. And for this
one, uh I would like you to to test the relevance of information. So how relevant is the
information regarding the image that you are using for searching into Google Lens. So um
review the provided information and see if the information is relevant and helpful for you.
Okay.

- Well, first of all, I have to say, I don't know what this is. So now I would just like to know what this
is. It looks like a musical instrument. Magali kalimba. Okay. In stocks, so since I want to know what
it is, probably I would like to go to the video. ... ... ... poly rhythm. poly rhythm. I'm a folk hub. We'll
start with these three notes, and place your right hand thumb on the white key to the left. Cool.
 Looks like a nice present, nice souvenir. Umm So YouTube, YouTube, YouTube. Here I have prize.
Yeah, I know pretty much what it is. I don't know too much about the history, where it's from. So

11. what kind of information would you expect to see?

- Uh In my particular case, since I didn't know what that was, I was expecting to see something that
showed me. Yeah, maybe, well, I I wouldn't expect a definition, but a video was a good, yeah, it was
was a good thing to find because then I could see how it works. And now I'm sure that it's a musical
instrument, but in the scenario, finding a souvenir, probably. I would need to find more websites
where this thing is sold so I can compare prices. But there aren't too many.

12. Is any important data missing?

- Umm Well,It depends on the purpose. So again, like looking for a souvenir, then maybe I would need
more prices. And well, right now I'm searching from Norway, right? So of course all the websites, or
not all, but at least some, they show me the prices in the currency, like in the Norwegian currency,
and they aretaking into account the shipping from X place to here. So umm yeah, I guess that that
would be out of the scenario of being in a market as I travel, unless the place where I'm traveling
around is actually in Norway, but I wouldn't find something like this here, I guess, because this
doesn't look like a Scandinavian. I don't know, maybe Okay, this can be maybe from, I don't know,
China, or I don't know, or maybe African. I have no idea. I would need to read more. All right, then.
So since we have accurately uh finished with task number two,

13. we can move on with task number three. So, can you please upload the following picture into
Google Lens? And for this task, I would like you to assess the search of reliability. I want you to
um search and see if the searches that appear into the suggestions are credible for you, for
instance. uh If we if you have a Wikipedia suggestion on our e-commerce site or a blog, just
check how reliable are those sources for you regarding the information that you want from
that specific picture.

- Okay, so info Florida, please. Okay, Info Florida sounds like a reliable site. Let's see. But it's in
German, I think. Switzerland. So I cannot read it right now. Umm see. It's the same website. Okay,
close here. Name that plant. So Diego Telaris. Telaris, okay. Pictures. umm of the Carolinas and
Georgia. Okay, I don't know too much about plants either, but um this is a forum, probably
somebody like me asking just, uh but it's a thing about the red spot. I don't want to know about that.
Yeah, but another language, three identifications, please. Is it wheat? Yeah. This is a little bit more
difficult for me because I don't find like enough information. It's like I don't find right away maybe I
would need to spend more time, but I don't get enough information like you know like right now,
immediately or just randomly going to a website. Some of them I'm not intending to go like TikTok.
No. So I tried to check the name of the of the address or the website so it can give me like a little bit
of confidence in the content. But yeah, I don't know. So, um so far.

14. which sources do you trust the most and why?

- The ones that have a name included, like a name related to nature or to plants or to botanics or
something like that.

15. So in that case, would you rely on Google Lens to make purchase decisions based on this
information? For instance, if you want to buyuh I don't know, maybe this plant, would you use
this information to assess?

- I'm not sure. I'm aware of some apps which actually do that. You can take a picture and the app
identifies from a bank, I guess, of data, what type of plant it is. And so if I would be that
 interested,um Probably I would try one of those apps instead of Google Lens. But if I don't know too
much about plants and trees, maybe I would try Google Lens. But actually, I just remember that once
I tried it with a plant and using Google Lens, and it wasn't that successful. I mean, I didn't get that
much information, not like direct information. It takes more time, I guess, especially because the
picture is, um it looks like a regular, well, for someone who doesn't know about plants like me, it
looks just like any regular green plant. I don't see anything special. probably it makes it more
difficult than taking a flower, which is, it has like a more, you know, characteristic view. But here
it's, yeah, it looks pretty common to me and maybe, yeah I don't know, I would need to take much
more time to actually know. I don't know if I could make a decision nabout buying a plant just based
on a couple of minutes after using Google Lens with the plant.

16. task number four. And the task for this um picture will be comparison. I would like you to
compare with similar objects found in the results and see if the alternative suggestions match
your expectations. You can share now.

- Well, the reason to yeah go into a search of a coin would be because I like to collect coins, for
example. This is an old coin because people are not using this coin anymore, I mean this value
anymore. So probably if I'm a person who is collecting coins then this is what I would need like to
check the price. Uh Now I can see that there's a similar one in Argentina and in Colombia. They're
similar, but they're not the same. So yeah, if I would have, I don't know, found it without looking at
the reverse, and I would like to know where the coin is from, from which country. So probably I just
got the answer because I can read Colombia here and it says here 27,000 coins of Colombia. So these
suggestions were helpful for you. Yeah, Yeah I find them accurate.

17. So in your experience so far, how could this comparison feature be improved?

- Umm Maybe I don't know if actually what I'm going to say it already, it does exist or not, but um I
would like to like try to change the settings or customize them into a specific region of the world and
not like, I don't know if the reason why it's, again, because of the place where I am now. But I would
like to, for example, if I want to find this coin and how much does it cost and everything, I wouldn't
like to go to eBay, but I would like to find it in their own country, in Colombia. How much is it cost
in Colombia?So I would like to maybe be able to, yeah, customize it like only search in the
Colombian region, for example, or yeah, the same for everything. And maybe it would be easier if,
well, if I don't have any problem with the language, of course. And yeah, that's something that maybe
could improve all the searches in general. I wouldn't know what to say with the one of the plant
because I I think it was the less successful search. I I didn't get too much information about the plant.
I'm not sure about what the name of the plant is. Umm I think it's maybe more difficult with things
like that we can find in nature, like plants or, I don't know, or rocks, for example, or yeah, that
wouldn't that would take more. I don't know more time, more more time. Yeah, to find out like an
accurate, an accurate answer for you.

18. task number five, and that will be the final object exploration. For this picture, um I would like
you to explore the the app, and try to find information, or what can you do with the
information that you have in the picture. and see how Google Lens identified the object, if it
allows you to explore its history. I would like you to please describe what you learned.

- Okay. Uh I just noticed these options, so I'm just going through them to see how they work. Umm
OK, I guess it's a song. Have a look. Now let me check the picture again. Maybe I can do this. It's
translated to Spanish. Let's write in English.... your coin over your end little boy. What's the detective
language? I don't know, but I read Tagalog, so I guess that's the language. Ohh To just stop
translating. One will die. Such a dream. Judge, you're going. Umm What language is this? Now
some patience. These words. Umm I don't know. I think I read Tagalog somewhere, but no, I didn't
 find it. It can be anything, but maybe, okay, my interest would be to find out what language is that.
So again, I will try to translate, but then I have to go back maybe.

19. So if you translate it, do you have uh something useful? Do you think it's it's a good translation
to you or are you confident about this information?

- Probably it's the best that I can get from Google Translate. Of course it's not perfect. There are some
words which are not translated. I don't know if they are proper nouns. Um But I have a very basic
idea of what it it is said. Now I would like to find what language is that. That's the thing that I don't
know how to find. It was good to notice that I can translate directly from Google Lens without going
to to another app. Um I know that I can use, well, it works actually Yeah, just the same, not the
opposite, because I knew that from Google Translate it was possible to connect with Google Lens.
So I guess they, yeah, they work together anyway. Um My guess is that this can be a poem or a song,
uh yeah, or something similar. But I don't know how to identify the language because I asked, once
again, umm We actually... One will die. Such a dream. Okay, struggle here a little bit. I want to
translate just a picture. Search. If I ask here, uh which is actually one option. So it's good to know
that they give me options that are what I'm what i'm thinking. Yeah Like, yeah, I need to find out
what language is this. So the option is here. That's good. Umm But, well, I don't think TikTok is
going to give me the answer. Neither Facebook. I would have to go back to the picture and compare
if what I found, like what is written there, it's the same as I see here in the pictures, but I don't think
it is. They look just like different things written in different languages. So yeah, at the very first view,
I don't trust in any of these sources. It's like Google Lens is trying to find a similar match, like
pictures that look similar, but yeah, it's not it's not going to give me the answer that I'm looking for.
Probably in this case, with the knowledge that I have about this, I would go to...to Google Translate.
I don't know if they can tell me the language, like where is the detected language, because it only
says detected language, but it doesn't tell me which language it is. So in that case, you wouldn't feel
confident purchasing something related to something like this, based on the provided information
that Google Lens has given to you. Yeah, no, I don't feel that confident because I can see that they
are just similar matches because it's a piece of paper with words on it, like a list. But yeah, but
obviously it's not, not all of them are the same language, are using the same language and yeah, no,
it's expecting something else.

20. task number six. And we are going to explore the real-time scanning features. And I would like
you to scan a to point your camera at a real object near to you. like a book, a label, a food item,
or plant that you have around you. And without taking the picture, I would like you to use the
real-time scanning feature to gather information. And while doing this, you can try adjusting
the lighting, perhaps like moving to a brighter spot to change the camera angle, Or maybe try
to zoom in or out, if you want to take like a specific part of the object that you would like to
scan, or trying changing the angle of the object, or trying to tap in on the screen to refine your
shift or shift focus. Tell me if the results are useful for you.

- OK, I'm going to try with the Closest thing that I have here. Um Let's see. I the box Wi-Fi of adult.
Okay. Maybe it's too obvious. Let's try to put it. From the website. How to reset? Interesting
information. OK, maybe it's not that. Yeah, it's like. And see it's. Good information about this item
maybe. It gets more difficult with...

Can I try with something else? Maybe if I try with the plant, seems to be more difficult. Okay.
Maybe you can zoom in to the to the flowers. As you can see, you can maybe have more information
or maybe, uh yeah. This is the same one. Coralberry, Rowing and Curve. Yeah, I would like to know
about it. This is just the general website. Gardening Express, this prices. Project plans for pots
outdoors. I guess my plan is somewhere in this list. Yep. Okay. Nice. I'm going to try with something
else. I'm glad with the information that I found. And more plants. What type of plant is this? Yep, it
looks like a plant. Which actually, I don't know the name. My friend gave me this as a moving gift.
 Okay, same here. And it's thriving, but I don't know what content this is. Please help. Spider plant,
spider plant. Whatever spider plant. Piece of love. Okay. When my spider plant needs a bigger pot.
Ah, that's a good question. That's something that I would like to know. But again, it just brings me to
the website, but not to the specific article or information that I have. I'm going to find one last item.
Maybe if you yeah put in a different angle, maybe from the top or maybe from the bottom, you will
have like a more interesting view. I don't think... from this angle. Exactly. So maybe you can change
the angle to find the answer that you are looking for. Yeah. Better. Okay. Pretty silicon body mold.
Audicondo, Australia. Products. No matches. Okay. Results. Etsy, Etsy, Etsy, Amazon. I need to
close it. Well, I would like to maybe to find where to buy one of these candles, but yeah all the
results are based on the UK or Australia. I don't know where, I don't know if Etsy Okay, I have no
prices in Kronor, so probably I can find a place and then ship it here. No? Okay.

21. how did changing the lighting or angle effects affect the results for you?

- Umm Well, about the lighting, I don't know, because I use the same spot for all the items. Well, I just
suppose that I will have a better picture if there is a good light over the item. About the angle well
when I try with the last one with the candle obviously, I need to find a good angle of the item
otherwise like you saw that when I took the picture from the top which is yeah you know like the
candle itself it gave me other results like just because I guess it's too difficult to to guess what it was
from that angle. So, yeah, it's my common sense says that I should take a good angle of the picture,
yeah, or a good picture with a good angle.

22. did you notice any improvements or limitations in the performance of the app while scanning
live?

- um Compared to the loaded pictures, you mean? Yeah, exactly. Well, I wouldn't know, no. They were
just fine, I guess. If there are any improvements or maybe if In the case of the plants, since you took
the pictures from different type of plants,

23. how the performance was for you in that matter while sending live a plant and having a
picture from a plant that you have to look to the app?

- Well, with the plants it was way much better with my pictures. I mean the ones that I took directly
from my phone and being in front of the plant. Because yeah, I just got the information that was
good enough for me. I get to know about the plant, what was the name of the plant, and maybe some
prices also. It was like, yeah, good enough for me. Better than with the picture that I had to load and
yeah, it's fine. But yeah, I don't know if it is related to with the angle of the picture, or... Yeah, I
wouldn't know why but it worked way much better when I took the picture. Amazing. Okay,

24. I would like you to ask some final questions and some feedback for you, and tell me regarding
to overall usability, so what aspects of Google Lens did you find most useful?

- Umm Well, I think the fact of being able to have a search only with an image of something, well,
that's already amazing. That's something that we didn't have before some years ago. Um It's good to
know that we that I can either take the picture or have the picture in my phone and I can do the
search with any of these Um It's also, it was very nice with the list that, well, when I did the exercise
with the list, I noticed that then I also had a translation option and yeah, I hadn't realized about this
this options here that I'm checking right now. Umm Well, maybe it doesn't work too much with the
candle, but but yeah, I guess it it helps to specify some fields of the search. Um Yeah, generally it's a
very good tool. I mean, I I wouldn't expect it to be perfect because it's many people who are using
this and maybe all of us, we're not expecting the same information. Maybe we're looking for
 something specific. So this is just a tool that is providing a lot of information and Yeah, it gives a lot
of matches with tons of possibilities to just be useful for the user. So I think it's good enough.

25. And what, if anything, did you find confusing or frustrating when using the app?

- Um maybe again with the list of words, like I couldn't get to know what language was that. So it was
frustrating not to find it so easily. But yeah, but I mean, maybe if I have more time and I can be with
mylaptop or something like that and I really want to know, yeah probably I will figure it out at some
point, but it's going to take more time. So the only frustrating thing was like not to find the answers
so quickly. Yeah, but but it's okay. I mean... So time consuming during the searching is important for
you. Like you want to beuh using an app that provides you information straight away and not
wasting much time, as I can understand you. Yeah, but again, it depends on my purpose. I mean, if
I'm, if that's super important for research that I'm conducting, so probably I need to know that I need
more time to find information and like relevant and reliable information. But if I just want to know
what that is, just yeah for the sake of knowing it or just to give a quick answer, yeah the amount of
time that it it takes to find like the right answer, it yeah can be an issue

26. Which task was the easiest and why?

- Umm Maybe the first one with Mochila. Um Because yeah with this scenario of looking for a
souvenir, so yeah, the results that popped up. were just yeah websites where I could buy similar
mochila. So that was easy and that was fast. And was it difficult? The plant.

And why? Umm

- Because I think I didn't find like an answer that I can remember, like a specific answer or what plan
it was because it looks so standard that the results were just similar pictures, but yeah. Well, I'm
between that one and the one of the list, the list with the words. because of what I already said about
the list. Yeah, I never get to know my choice.

So you think that you successfully accomplished those tasks?

- Those two? No.

Did you trust the information provided by Google Lens?

- Yes. More or less, because um For example, here with this this these results that I have here, I can
see that most of the websites where I can find it's it's just the same, Etsy. Etsy. So unless Etsy is like
the official, I think it is like the official, I don't know, company that makes this or whatever, I think
it's too limited. I would like to find more like more options. At least um the ones that like the ones
that comes first. Yeah, because if I go down, I notice that I find yeah maybe another website here.
Yeah, again, I don't know. I would like just to have the option to. I don't know if I can do that. If I
can, for example, give a location and it's going to start searching. In the location, yeah, probably it
doesn't. So that's. That's one of the things that I think it's limited. I guess that. No, I don't know. It
works just like Google. When you go to Google something, yeah there's a reason why you you get
some results, like the very first results. It doesn't mean that they are all of the existent ones. It means
that maybe they pay or whatever. So in that case.

27. would you use this tool again for object identification?

- Yes, of course. I have done it before. I will continue doing it. All right.

28. And how did using Google Lens in real time feel compared to uploading pictures?
 - Umm I feel more confident with the pictures that I'm taking in real time. because I could change the
angle or maybe see if I can find different results. But with the picture, yeah, it's it's all that I have. I
cannot just try a different angle, of course. So I prefer to use the camera.

29. If you could change one thing about Google Lens, what would it be?

- Umm In a few places. OK, maybe this is what I would umm I would need to check more but I would
like to learn maybe it's maybe it does exist and I don't know it I don't know but I would like to learn
how to change the settings for the location of the search like I said it before like I would like to find
something in a specific location or not so specific, but at least in one country or in one continent and
not yeah maybe all over the world, but yeah This is the search here. What I understand is that it's
being made based on my my location. Yeah So I don't know if I can change that, but yeah, I would
need more time to check it out.

Luisa Revelo, Task-completion time:

Task Name Time taking Why It Took That Long

Task 1- Mochila (05:12) Task was accomplished briefly without any issues. The search was smooth
Bag: 5’12’’ and it was the shortest.

Task 2 Kalimba (07:11) User took more time because of the how interesting the results were. Time
instrument: 7’11’’ was invested in learning and testing information (tutorials, history and
materials)
Task 3: Plant (07:33) Customer invested more time trying to find relevant information about the
7’33’’ object unsuccessfully. The task was not accomplished.
Task 4: Coin (06:03) 6’3’’ Task with the second shorter out of the 5 first ones. Smoothed experience
without issues and easy to accomplish.
Task 5: List of (08:00) 8’ Took a lot of time without reliable results, they were no accurate, the task
words failed.
Task 6: Scaning : (18:30) Longest task because of changing angles and searching for propper light
in real time (3 18’30’’ adjustments. The first attempt was successful and the user wanted to try
objects) different objects.

Usability issues and emotions:

Task 1: This was the one where she felt confident and accomplished with the results she got, it was the
task with least time invested, followed by task 4, regarding to the coin, which also provided reliable
information to compare and meet the expectations of the scenario.

Task 2: Luisa enjoyed learning about a new musical instrument and found useful the information
provided; she took time to explore google lens insights as well as check on the tutorials for learning how
to play it.

Task 3: Felt frustrated and confused about looking for answers for the plant. She didn’t find anything
clear and wasted time trying to find something she was not uncertain she could find. The sources were
reliable yet did not show any direct relation to the object to find out. The lacked-on options to customise
her searching.
Task 4: Second least lasting task out of the 5 first tasks. The results allowed the user to compare the
object gathering enough information to validate the resemblance and similarities confirming she has
found the exact same object. The task was accomplished and generated positive feedback and emptions
such as satisfaction, confidence and trust.

Task 5: Task 5 took most time from the 5 tasks the included uploading a picture, since the list of word
picture was not possible to find reliable information and accurate translation. She tried to find options to
customise her search and the few tools that google lens provided, but not they could not translate all the
words and it did not show logic or cohesion in its output. This task left frustration, unsatisfaction and
uncertainty the user. She was impressed on the interaction regarding translating from an image but not
on the results.

Task 6: The optional task 6, took more time because it needed to scan an object in real time. The user
tried different objects and got reliable sources and the exact same object. However, the app lack on
personalised research since for some specific context such as her residency country, when she wanted to
find one of the objects locally, she couldn’t even when the app used her current location, the results did
not meet contextual expectations such as local stores for purchasing.