Radio Inspire

How To Learn Sign Language

IJQwerty: What Difference Does One Key Change Make? Gesture Typing Keyboard Optimization Bounded …


– I’m not Xiaojun, but
I wanted just to say a few words about this. So, this paper is about gesture typing, which is a paradigm that
is a truly innovative new paradigm that’s already on perhaps a billion devices. But if you look at the very first paper Per Ola and I published in CHI 2013. 2003. Three. (laughs) – [Man] I know. – 13 years ago. The big part of it was about defining the keyboard layout so the gestures. – [Man] Use the mic. – So the gestures on a given layout will be all well-defined
and well separated, and so therefore people can learn and remember the shapes better, therefore, you can get into
a truly shorthand writing, a new way of writing stage. But, I would say that’s still
far from being a reality, so we keep on returning to this topic that is how we can actually find a layout that is both much more clear, much more efficient and also much
more easier to learn, and unfortunately, the three
all together don’t seem to come together easily
and here is another attempt at making it at least easier to learn. Because the Qwerty is just
about the most difficult, and sometimes worst
keyboard for gesture typing in terms of defining shapes. – As you can see here,
one reasons that Qwerty is pretty bad for gesture typing, if you look at the top
right corner of the layout, the three vowels, U, I, O,
they are placed together, which causes problem for
gesture typing because in gesture typing, you are inevitably cross on the letters and cause problem. Here I show you some examples. See, tip and top is two very common words, and they have identical
traces on gesture keyboard. And here, for in and on,
two very common words, because I and O are placed together, they have very similar
shape, so the decoder often get confused by which
word you intended to input. Here is another example, nit and not, they have very similar shapes. Because Qwerty is inefficient and inaccurate for typing, there has been a lot of effort
involved by the keyboard. Where previous, we would
mostly focus on how to improve the performance, how we
improve speed and accuracy. But later we found out that
one of the biggest challenges of that layout is about learning. We have been using Qwerty for 100 years, and we use it on daily basis. When have a new layout,
user have spend more time to switch to Qwerty,
even though we know that they will eventually
have good performance, they are unlikely to switch
from Qwerty to other layouts. And, this will focus on
how to improve the learning of the optimized layout. More specifically, we try
to, say if we introduce some Qwerty constraints,
specifically, we are trying to keep optimized layout
different from Qwerty, but obviously there is goal
to improve the learning and also under its constraints, where we can still get optimized layout. So, we look at three kinds
of Qwerty constraints here. The first one, we call it
Qwerty-like constraint. It is also known as
Quasi-Qwerty in previous work. Basically for given letter on layout, it’s new positions coming at most one key away from its Qwerty position, we call this position a Qwerty neighbor. So for letter G on layout. The possible new location for G, couple location on this eight slots. And secondly our second
constraint’s even stronger we call it QwertyH1 constraint. Basically a case for
only move horizontally on the same row and of
course it has at most one key away from it Qwerty position. So for Latin G here, it has only three
possibilities in the layout. And the third one is the strongest. So we’ll only do one swap, we’ll only swap a pair of keys on a layout of a Qwerty. And these two keys have
to be Qwerty neighbor for example, G and H, one possibility. So if we want to investigate
only three constraints can we still layout that
better Qwerty and hope that its constraints will help
you to learn the new layout. The optional layout for just typing. So the procedure was very similar to what I did in last year’s
Chi research project. We hope to adjust typing layout for both accuracy and speed. For accuracy we defined,
we introduced a gesture character metric, it’s
basically quantified to one degree at given gesture, a gesture on given layout,
a separate default gestures. So the higher our suggested clarity, the better the keyboard for
just typing and writing. For details, please refer to this paper. And the second objective
is their typing speed. We have a model to predict the time you would need to enter a word given its gesture traits. In the next optimization
program we tried to optimize our two objectives. Speed and accuracy. Mostly speaking we want to be escape if we add these three
constraints separately, how well they affect the performance. And whether it will improve the learning. We learned that it is very typical parental optimization program. We have two objectives and final outcome of the optimization procedure, is a set of parental optimal layout. And we picked the most balanced one, the one has the balance between
just the accuracy and speed. And here are the optimized results. Under one sub-constraint
we happen to state that swapping I and G, you
will get a layout that has a most balance ratio
between speed and accuracy. Interestingly, swapping I and
G has some of strong pairing that we did not expect. For example I and J, they
are next to each other, diagonally on Qwerty and
also they happen to be consecutive letters in Roman alphabet. And also if you look a these two letters, they have very similar appearance. So there’s a strong
pairing that we hope that it might help you to remember after you swap the two keys and
remember this position. And this is the optimized
layout under QwertH1 constraint. Can see that we have 16 piece move to new locations. And this is the layout
we got after the QwertyH1 so Qwerty1 constraint, 22 piece changed their positions. And this figure actually
shows the performance of the different layout. The metrics optimized between V and one. This orange dot indicates a layout that it optimized for clarity only, and we didn’t add any constraint. Basically the maximum
currently you have again. In case you have before
and after clarity exist. Here we optimized for
speed only and we didn’t add any constraint,
particularly any case with any location. Can see here, Qwerty is at
the very low end in both axis. It confirms our observation
that it’s greatly inefficient and ambiguous for just typing. And this is a layout
optimized for both metrics, and you can add no hand strength. What you can get, it’s
kinda in the middle between speed and clarity. Here the performance of
the three optimized layout under three constraints. Interestingly you can
see that the Qwerty trial and the IJQwerty have
very similar performance. And both are better
than Qwerty, especially in gesture clarity. It shows that by swapping
only two keys, I and J, you can get these character
metrics improved by a lot, and you need to look as
some complex examples for you can get a feeling
for why it happens. You see here, here’s some work I showed at the beginning of presentation. Can see tip and top. Simply three letter I for second row. These two traces are clearly
different from each other. On iQwerty they’re identical. And for word in and on, can see here, you have more separate traces
on IJQwerty than on Qwerty. And this is the gesture
traces for not and nit, these two words. Clearly they are more
separated on IJQwerty, because there were more
people get it correct. So far our solid analysis
shows that IJQwerty and all the other layouts we have higher gesture clarity and
also higher entry speed than Qwerty but they
are all append based on solid analysis. Next step, we can talk
it in critical studies. The purpose are first we
want to know the reserves, Of the layout after three
constraints can improve them if anything. Second, we want to see if I
can observe any performance gain we just carry out. Conducted within-subject user study. We test three optimized
layouts together with Qwerty and also we picked two tasks here. First it’s a phrase input task. Basically one test will
first pick up the keyboard and enter for instance
whether you have performance improvement under optimized layout. A second task is a word repetition task. We picked 20 individual words and for each word you would enter seven times in a row. The purpose of that, when you repeat a same
word again and again, the last few repetitions
you reach expert mode. We’ll see when we enter a performance gain at this stage. And we would have to speed
measured by word per minute and word entry rate. Here’s the result for phrase input task. Interestingly you can see
that for IJQwerty is has same rate for speed with Qwerty. Static analysis did not show difference between these two layout. And both of them are similarly faster than QwertyH1 and Qwerty1. It showed that the user
first pick up the keyboard, first pick up the IJQwerty,
it did not have any performance degradation. But for other clues we observed that they actually resolved with at QwertyH1 and Qwerty1 constraint they still had their performance degradation
with these two constraints. And for error rate we didn’t notice any difference character rate
because it has you know three they allow to correct mistakes, and use backspace. In the second study for
the word repetition tasks, some findings confirm what we observed in the previous study. So actually for the first
repetition of the word, the first time user
encountered the word and enter on the newly up we can see that IJQwerty and Qwerty people
have shorter intro speed than other two layouts. It indicates that IJQwerty didn’t slow down when they first encounter this IJQwerty new layout. Starting from the repetition
three you can see that Qwerty, IJQwerty, all
the three optimal layouts have shorter input time that Qwerty. Faster, the difference
between Qwerty and others, they are significant here. Here are error rates of
three of the four layouts. Can see here for IJQwerty, it actually has a lower
error rate than Qwerty. It’s consistent with what we showed in just the clarity debt,
we can see that because, in real day study they
separate the three vowels, actually get separate just
the three or four models. The moral of our study. So in this research we
investigated how to improve the ability to learn a new layout. And in the meantime to rescue a new layout that is better than Qwerty. And we showed that our, our research rests with three constraints. So in Qwerty-like constraints, to see whether they would
help with just typing, we find out that IJQwerty is
a pretty easy learn layout and introduced no performance
degradation over Qwerty when you initially use it. And also this new layout
is faster and more accurate than Qwerty when the
user reach expert mode. That’s the presentation, thank you. (audience applauding) – [Man] Please line up in the middle, the mic is in the middle. Any questions? – [Man] I’m from University
of Hanover, hello. I think it’s not. – [Man] It’s on, it’s on. – Okay. So my question is you
basically allow single swap. With the I and the J and
you get this nice property. But if you allow a single
swap but between using any keys, so did you
look at reconstructed. Do you expect could be even better or? – Yeah so. Yeah, yeah, yeah, yeah, more freedom, yeah, had more freedom in, if we had not constrained
the Qwerty neighbor, first thing I think it
wouldn’t be worse than this situation because
it’s what is happening. The swap set that new approach. You can swap any two keys on the layout. Actually with this
experiment it shows that if you swap other two keys, it says on the statistical
analysis it didn’t improve much over the IJQwerty. And we can see even in
QwertyH1 constraint. So QwerryH1 you have more freedom in getting the optimal layout, still have a similar
gesture clarity and speed over IJQwerty, IJQwerty pretty sweet spot. Second point of that,
if we allowed to freely swap any two keys I think
it would hurt learnability. Actually with this project
our last year’s research we introduced the Qwerty
similarity so you can move the keys away. We didn’t really confine
them to Qwerty neighbors. So the problem that we
found out is isn’t really improve learning because
like sometime when you encounter the key you
don’t know the location. The idea of Qwerty constraint that, when you try to look for a key, if you don’t find a Qwerty neighbor it’s not in the Qwerty position you are, first look at its neighbor. And then you might increase
your search gradient to the second keys. But if you randomly
swap two keys that just happen to be very far away it, it would enter a lot of time
searching for these two keys and overall it will
add more learning time. More frustrations. – [Man] Thank you. – [Ted] Hi, Ted Pack, Microsoft Research. To some extent, when you’re
dealing with ambiguity, some ambiguity is worse
than others depending on the language model content. So for example, hi and
go is much worse than tip and top cause then
you have the context. So why didn’t you guys consider using that as a constraint,
the larger, you know, constraint of the language model context. And then the other thing
is when you’re actually evaluating the word error rate,
you’re testing on phrases, but are you evaluating on
single word recognition? So is it top best for single world, without the context? – So for the phrase it’s reading content. It has a fully functional
keyboard with language model error we just basically without change. – [Ted] You can see that there’s sort of like a conflation there
because of the context. – So I think we must, after the first. – I was too excited, I
forgot why I was giving a few more words in the beginning. I want to say that
Xiaojun and I have worked together for many years but this fight, free food and other more significant perks at Google, Xiaojun is
going to join Stonybrook next year so that was the reason. To that question Tim, it’s
a very complex question. Actually last year, in
our last year’s work in defining keyboard
gesture typing clarity, we spent a lot of time, a
lot of back and forth on that very topic, that is, how
do you define clarity if you do consider language model context. We decided to cut it at unigram level. The reason is that if you consider bigram or ingrams and there’s the next level, like what if we use LSTM, what if we use, no cementing models and
then the context will be changing even further,
the spaces will just become so complex that it
would be very hard to optimize. Maybe you have better ideas. Tell us (laughs) if you choose to. – [Man] Last question. – [Man] My question is
there for your experiment, where you mentioned
that you had experiment by multiple levels of
typing but same user. Expert behavior and most of. What is the limit, when do you see that both behavior has most of, because this is a
problem which I have been facing with my experiment and every time I make users repeat the same phrases, the next time they do it faster so, what kind of limitations to you see there? – So actually for to similar expert mode in the second study
which is word repetition, it’s not phrase repetition. It’s basically you enter the same word multiple times in a row. The reason we believe
it’s similar expert speed because if you look at the time you need to enter the word, the word repetition can see starting from the third repetitions,
you can now reach ceiling. It means that in this stage
mostly your performance is determined by your multivariability. Not by whether if anyone would be out, although you need to search for that. But I think that it’s kind of a way to, it’s approach to similar to expert mode in a short period of time. (mumbling) – [Man] Would you like to say that? By repeating the same word
approximately seven times, the motor speed has been achieved. Over time. – Yeah sometimes I think just in typing basically your influenced
time will be constrained by first your mobility and
something other like the radial touch time and whether you remember the word. The idea like you repeat a
single word multiple times. We mean at the first,
second trials you might be still need to do real search. Once you repeat three times in a row, less fewer trials we expect that you will eliminate the view search
time and commit it below, remember the shape and you adjust the line of mobility to enter the word. And I think that’s what expert mode doing than just typing. And to test our hypothesis
you can look at the curve of the intro speed you can see that the speed definitely
increased in last few trials and then actually reached
a pretty stable level which indicates that how
to get some improvement off of that stage, I think that that’s confirmed that party reached actually low. At this stage. – Perhaps what he meant is. Whether that’s a mistake but it’s actually done on purpose. It’s actually precisely to a point. Because people will do better next trial if you do it repeatedly. We do that to measure
what eventual performance people would be able to
reach, it doesn’t mean in actual use they will reach that faster. But eventually they will reach that. But it cannot do that long experiment in order to prove a point. So that’s precisely and to a point that is we wanted to take advantage of the fact that people learn
faster if you repeat the same motor control pattern. But the clarity would
still show the same way. So the idea is we tried to test clarity, after a larger amount of learning. So it’s essentially acceleration. That’s the idea. – [Man] Thank you, alright. Thank you. (audience applauding)

One Reply to “IJQwerty: What Difference Does One Key Change Make? Gesture Typing Keyboard Optimization Bounded …”

Leave a Reply

Your email address will not be published. Required fields are marked *