US011360509B2

( 12 ) King
United
et al.
States Patent ( 10 ) Patent No.: US 11,360,509 B2
(45 ) Date of Patent : * Jun . 14 , 2022

( 54 ) ELECTRONIC DEVICE HAVING DISPLAY ( 58 ) Field of Classification Search

AND SURROUNDING TOUCH SENSITIVE ??? G06F 3/0412 ; G06F 1/1643 ; G06F 3/044 ;
SURFACES FOR USER INTERFACE AND GO6F 3/03547 ; G06F 1/1626 ;
CONTROL (Continued )
( 71 ) Applicant: Apple Inc. , Cupertino, CA (US )
9 ( 56 ) References Cited
( 72 ) Inventors : Nicholas King , San Jose , CA (US ) ; U.S. PATENT DOCUMENTS
Duncan Robert Kerr , San Francisco , 3,333,160 A 7/1967 Gorski
CA (US ); Paul Herbst , Austin , TX 3,541,541 A 11/1970 Engelbart
(US ) ; Steven P. Hotelling , Los Gatos, (Continued )
CA (US)
FOREIGN PATENT DOCUMENTS
( 73 ) Assignee : Apple Inc. , Cupertino , CA (US)
CA 1243096 A 10/1988
( * ) Notice: Subject to any disclaimer, the term of this CN 1173672 A 2/1998
patent is extended or adjusted under 35 (Continued )
U.S.C. 154 ( b ) by 0 days.
This patent is subject to a terminal dis OTHER PUBLICATIONS
claimer .
4 -Wire Resistive Touchscreens, Available online at: <http ://www .
( 21) Appl . No .: 17 / 175,387 touchscreens.com/intro-touchtypes-4resistive.html> , Accessed on Aug.
5 , 2005 .
(22) Filed : Feb. 12 , 2021 (Continued )
(65 ) Prior Publication Data Primary Examiner - Michael A Faragalla
US 2021/0165449 A1 Jun . 3 , 2021 (74 ) Attorney, Agent, or Firm — Kubota & Basol LLP
( 57 ) ABSTRACT
Related U.S. Application Data An electronic device has a display and has a touch sensitive
( 63 ) Continuation of application No. 16/ 529,690 , filed on bezel surrounding the display. Areas on the bezel are des
Aug. 1 , 2019 , now Pat. No. 10,921,941 , which is a ignated for controls used to operate the electronic device .
Visual guides corresponding to the controls are displayed on
(Continued ) the display adjacent the areas of the bezel designated for the
(51 ) Int. Ci.
controls. Touch data is generated by the bezel when user
touches an area of the bezel . The device determines which
G06F 3/044 ( 2006.01 ) of the controls has been selected based on which designated
GO6F 1/16 ( 2006.01) area is associated with the touch data from the bezel . The
( Continued ) device then initiates the determined control. The device can
( 52 ) U.S. Cl. have a sensor for determining the orientation of the device .
CPC GO6F 1/1626 ( 2013.01 ) ; G06F 1/169 Based on the orientation, the device can alter the areas
( 2013.01 ) ; G06F 1/1637 ( 2013.01 ) ; designated on the bezel for the controls and can alter the
( Continued ) ( Continued )
120
1029
LINKPOWER The

SELECT
 US 11,360,509 B2
Page 2

location of the visual guides for the display so that they 4,771,276 A 9/1988
11/1988
Parks
Bruere -Dawson et al.
match the altered areas on the bezel . 4,788,384 A
4,806,846 A 2/1989 Kerber
4,857,916 A 8/1989 Bellin
20 Claims , 13 Drawing Sheets 4,866,602 A 9/1989 Hall
4,891,508 A 1/1990 Campbell
4,896,370 A 1/1990 Kasparian et al .
4,898,555 A 2/1990 Sampson
4,917,516 A 4/1990 Retter
Related U.S. Application Data 4,922,236 A 5/1990 Heady
continuation of application No. 15 / 970,571 , filed on 4,933,660 A 6/1990 Wynne, Jr.
May 3 , 2018 , now Pat. No. 10,386,980 , which is a 4,968,877 A 11/1990 Mcavinney et al .
4,970,819 A 11/1990 Mayhak
continuation of application No. 14 /724,753 , filed on 5,003,519 A 3/1991 Noirjean
May 28 , 2015 , now Pat . No. 9,983,742 , which is a 5,017,030
5,027,690
A
A
5/1991 Crews
7/1991 Wachi et al .
continuation of application No. 12 /486,710 , filed on 5,125,077 A 6/1992 Hall
Jun . 17 , 2009 , now Pat . No. 9,047,009 , which is a 5,178,477 A 1/1993 Gambaro
continuation of application No. 11 / 426,078 , filed on 5,179,648 A 1/1993 Hauck
Jun . 23 , 2006 , now Pat . No. 7,656,393 , which is a 5,186,646 A 2/1993 Pederson
continuation - in -part of application No. 11 / 367,749, 5,189,403
5,194,862
A
A
2/1993
3/1993
Franz et al .
Edwards
filed on Mar. 3 , 2006 . 5,224,151 A 6/1993 Bowen et al .
( 60 ) Provisional application No. 60/ 663,345 , filed on Mar. 5,224,861 A 7/1993 Glass et al .
16 , 2005 , provisional application No. 60/ 658,777 , 5,225,959 A 7/1993 Stearns
5,237,311 A 8/1993 Mailey et al .
filed on Mar. 4 , 2005 . 5,241,308 A 8/1993 Young
5,252,951 A 10/1993 Tannenbaum et al .
(51 ) Int. Ci. 5,281,966 A 1/1994 Walsh
G06F 3/0488 ( 2022.01 ) 5,305,017 A 4/1994 Gerpheide
G06F 3/04883 (2022.01 ) D349,280 S 8/1994 Kaneko
5,339,213 A 8/1994 O'callaghan
G06F 3/04886 ( 2022.01 ) 5,345,543 A 9/1994 Capps et al .
G06F 3/0354 ( 2013.01 ) 5,355,148 A 10/1994 Anderson
GO6F 3/041 ( 2006.01) 5,376,948 A 12/1994 Roberts
G06F 3/04847 ( 2022.01 ) 5,379,057 A 1/1995 Clough et al .
5,398,310 A 3/1995 Tchao et al .
(52) U.S. CI. 5,404,152 A 4/1995 Nagai
CPC G06F 1/1643 (2013.01 ) ; G06F 3/03547 5,414,445 A 5/1995 Kaneko et al .
( 2013.01 ) ; G06F 3/044 ( 2013.01 ) ; G06F 5,438,331 A 8/1995 Gilligan et al .
3/0412 (2013.01 ) ; G06F 3/04847 ( 2013.01 ) ; 5,442,742 A 8/1995 Greyson et al .
G06F 3/04883 (2013.01 ) ; G06F 3/04886 D362,431 S 9/1995 Kaneko et al.
5,463,388 A 10/1995 Boie et al .
(2013.01 ) ; G06F 3/0445 (2019.05 ) ; GOOF 5,463,696 A 10/1995 Beernink et al .
3/0446 (2019.05 ) ; GOOF 2200/1612 ( 2013.01 ) ; 5,473,343 A 12/1995 Kimmich et al .
GO6F 2203/0339 ( 2013.01 ) ; G06F 2203/04101 5,473,344 A 12/1995 Bacon et al .
( 2013.01 ) ; G06F 2203/04808 (2013.01 ) 5,483,261 A 1/1996 Yasutake
5,488,204 A 1/1996 Mead et al .
( 58 ) Field of Classification Search 5,491,706 A 2/1996 Tagawa et al .
CPC G06F 2203/04105 ; G06F 3/0446 ; G06F 5,495,077 A 2/1996 Miller et al .
2203/0339 ; G06F 1/169 ; G06F 5,495,566 A 2/1996 Kwatinetz
2203/04101 ; G06F 2200/1612 ; GO6F 5,510,813 A 4/1996 Makinwa et al.
5,513,309 A 4/1996 Meier et al .
3/04847 ; G06F 3/0445 ; G06F 3/04886 ; 5,523,775 A 6/1996 Capps
G06F 3/04883 ; G06F 1/1637 ; G06F 5,528,265 A 6/1996 Harrison
2203/04808 5,530,455 A 6/1996 Gillick et al .
See application file for complete search history. 5,541,372 A 7/1996 Baller et al .
8/1996 Gillespie et al .
5,543,590 A
( 56 ) References Cited 5,543,591 A 8/1996 Gillespie et al .
5,559,943 A 9/1996 Cyr et al .
U.S. PATENT DOCUMENTS 5,561,445 A 10/1996 Miwa et al.
5,563,632 A 10/1996 Roberts
5,563,996 A 10/1996 Tchao
3,662,105 A 5/1972 Hurst et al . 5,565,658 A 10/1996 Gerpheide et al.
3,798,370 A 3/1974 Hurst 5,565,887 A 10/1996 Mccambridge et al .
4,110,749 A 8/1978 Janko et al . 5,578,817 A 11/1996 Bidiville et al.
4,246,452 A 1/1981 Chandler 5,579,036 A 11/1996 Yates, IV
4,264,903 A 4/1981 Bigelow 5,581,681 A 12/1996 Tchao et al .
4,550,221 A 10/1985 Mabusth 5,583,946 A 12/1996 Gourdol
4,566,001 A 1/1986 Moore et al . 5,585,823 A 12/1996 Duchon et al .
4,672,364 A 6/1987 Lucas 5,589,856 A 12/1996 Stein et al .
4,672,558 A 6/1987 Beckes et al .
5,590,219 A 12/1996 Gourdol
4,689,761 A 8/1987 Yurchenco 5,592,198 A 1/1997 Fagard et al .
4,692,809 A 9/1987 Beining et al. 5,592,566 A 1/1997 Pagallo et al .
4,695,827 A 9/1987 Beining et al. 1/1997 Deeran et al .
4,719,524 A 1/1988 Morishima et al . 5,594,471 A
4,733,222 A 3/1988 Evans 5,594,810 A 1/1997 Gourdol
4,734,685 A 3/1988 Watanabe 5,596,347 A 1/1997 Robertson et al .
4,736,191 A 4/1988 Matzke et al . 5,596,694 A 1/1997 Capps
4,746,770 A 5/1988 Mcavinney 5,598,183 A 1/1997 Robertson et al .
 US 11,360,509 B2
Page 3

( 56 ) References Cited 5,900,848

5,903,229
A
A
5/1999
5/1999
Haneda et al .
Kishi
U.S. PATENT DOCUMENTS 5,907,152 A 5/1999 Dandliker et al .
5,907,318 A 5/1999 Medina
5,598,527 A 1/1997 Debrus et al . 5,909,211 A 6/1999 Combs et al .
5,611,040 A 3/1997 Brewer et al . 5,910,799 A 6/1999 Carpenter et al .
5,611,060 A 3/1997 Belfiore et al . 5,914,705 A 6/1999 Johnson et al .
5,612,719 A 3/1997 Beernink et al . 5,914,706 A 6/1999 Kono
5,631,805 A 5/1997 Bonsall 5,920,309 A 7/1999 Bisset et al .
5,633,955 A 5/1997 Bozinovic et al . 5,923,319 A 7/1999 Bishop et al .
5,634,102 A 5/1997 Capps 5,931,906 A 8/1999 Fidelibus et al .
5,636,101 A 6/1997 Bonsall et al . 5,933,134 A 8/1999 Shieh
5,642,108 A 6/1997 Gopher et al . 5,943,044 A 8/1999 Martinelli et al .
5,644,657 A 7/1997 Capps et al . 5,956,019 A 9/1999 Bang et al .
D382,550 S 8/1997 Kaneko et al. 5,959,611 A 9/1999 Smailagic et al .
5,661,505 A 8/1997 Livits 5,977,867 A 11/1999 Blouin
5,663,070 A 9/1997 Barr et al . 5,977,869 A 11/1999 Andreas
5,666,113 A 9/1997 Logan 5,977,952 A 11/1999 Francis
5,666,502 A 9/1997 Capps 5,982,302 A 11/1999 Ure
5,666,552 A 9/1997 Greyson et al . 5,991,431 A 11/1999 Borza et al .
D385,542 S 10/1997 Kaneko et al . 5,996,080 A 11/1999 Silva et al .
5,675,361 A 10/1997 Santilli 5,999,166 A 12/1999 Rangan
5,675,362 A 10/1997 Clough et al . 6,002,389 A 12/1999 Kasser
5,677,710 A 10/1997 Thompson - rohrlich 6,002,808 A 12/1999 Freeman
5,686,720 A 11/1997 Tullis 6,005,299 A 12/1999 Hengst
5,689,253 A 11/1997 Hargreaves et al . 6,020,881 A 2/2000 Naughton et al.
5,710,844 A 1/1998 Capps et al . 6,029,214 A 2/2000 Dorfman et al .
5,726,687 A 3/1998 Belfiore et al . 6,031,518 A 2/2000 Adams et al .
5,729,219 A 3/1998 Armstrong et al . 6,031,524 A 2/2000 Kunert
5,729,250 A 3/1998 Bishop et al. 6,037,882 A 3/2000 Levy
5,730,165 A 3/1998 Philipp 6,041,134 A 3/2000 Merjanian
5,736,976 A 4/1998 Cheung 6,049,328 A 4/2000 Vanderheiden
5,741,990 A 4/1998 Davies 6,050,825 A 4/2000 Nichol et al.
5,745,116 A 4/1998 Pisutha - arnond 6,052,339 A 4/2000 Frenkel et al .
5,745,591 A 4/1998 Feldman 6,064,370 A 5/2000 Wang et al .
5,745,716 A 4/1998 Tchao et al . 6,069,648 A 5/2000 Suso et al .
5,746,818 A 5/1998 Yatake 6,072,471 A 6/2000 Lo
5,748,185 A 5/1998 Stephan et al . 6,072,475 A 6/2000 Van
5,748,269 A 5/1998 Harris et al . 6,072,494 A 6/2000 Nguyen
5,751,274 A 5/1998 Davis 6,075,533 A 6/2000 Chang
5,754,890 A 5/1998 Holmdahl et al . 6,084,574 A 7/2000 Bidiville
5,764,222 A 6/1998 Shieh 6,084,576 A 7/2000 Leu et al .
5,764,818 A 6/1998 Capps et al . 6,097,372 A 8/2000 Suzuki
5,766,493 A 6/1998 Shin 6,107,997 A 8/2000 Ure
5,767,457 A 6/1998 Gerpheide et al. 6,111,563 A 8/2000 Hines
5,767,842 A 6/1998 Korth 6,115,028 A 9/2000 Balakrishnan et al .
5,786,804 A 7/1998 Gordon 6,118,435 A 9/2000 Fujita et al .
5,786,818 A 7/1998 Brewer et al . 6,124,587 A 9/2000 Bidiville et al.
5,790,104 A 8/1998 Shieh 6,125,285 A 9/2000 Chavez et al .
5,790,107 A 8/1998 Kasser et al . 6,128,003 A 10/2000 Smith et al.
5,793,881 A 8/1998 Stiver et al. 6,128,006 A 10/2000 Rosenberg et al .
5,802,516 A 9/1998 Shwarts et al . 6,130,664 A 10/2000 Suzuki
5,808,567 A 9/1998 Mccloud 6,131,047 A 10/2000 Hayes et al .
5,808,602 A 9/1998 Sellers 6,131,299 A 10/2000 Raab et al .
5,809,267 A 9/1998 Moran et al . 6,135,958 A 10/2000 Mikula -Curtis et al.
5,812,114 A 9/1998 Loop 6,144,380 A 11/2000 Shwarts et al .
5,816,225 A 10/1998 Koch et al . 6,163,312 A 12/2000 Furuya
5,821,690 A 10/1998 Martens et al . 6,163,616 A 12/2000 Feldman
5,821,930 A 10/1998 Hansen 6,166,721 A 12/2000 Kuroiwa et al .
5,823,782 A 10/1998 Marcus et al . 6,179,496 B1 1/2001 Chou
5,825,351 A 10/1998 Tam 6,181,322 B1 1/2001 Nanavati
5,825,352 A 10/1998 Bisset et al. 6,184,868 B1 2/2001 Shahoian et al .
5,825,353 A 10/1998 Will 6,188,389 B1 2/2001 Yen
5,828,364 A 10/1998 Siddiqui 6,188,391 B1 2/2001 Seely et al .
5,835,079 A 11/1998 Shieh 6,188,393 B1 2/2001 Shu
5,838,304 A 11/1998 Hall 6,191,774 B1 2/2001 Schena et al .
5,841,078 A 11/1998 Miller et al . 6,198,473 B1 3/2001 Armstrong
5,841,425 A 11/1998 Zenz , Sr. 6,198,515 B1 3/2001 Cole
5,841,426 A 11/1998 Dodson et al. 6,208,329 B1 3/2001 Ballare
D402,281 S 12/1998 Ledbetter et al. 6,211,860 B1 4/2001 Bunsen
5,850,213 A 12/1998 Imai et al . 6,211,861 B1 4/2001 Rosenberg et al .
5,854,625 A 12/1998 Frisch et al . 6,219,038 B1 4/2001 Cho
5,856,822 A 1/1999 Du et al . 6,222,465 B1 4/2001 Kumar et al .
5,880,411 A 3/1999 Gillespie et al . 6,222,528 B1 4/2001 Gerpheide et al.
5,880,715 A 3/1999 Garrett D442,592 S 5/2001 Ledbetter et al .
5,883,619 A 3/1999 Ho et al . 6,225,976 B1 5/2001 Yates et al.
5,886,687 A 3/1999 Gibson 6,225,980 B1 5/2001 Weiss et al.
5,889,236 A 3/1999 Gillespie et al . 6,226,534 B1 5/2001 Aizawa
5,898,434 A 4/1999 Small et al . 6,232,957 B1 5/2001 Hinckley
 US 11,360,509 B2
Page 4

( 56 ) References Cited 6,610,936 B2 8/2003 Gillespie et al .

6,614,422 B1 9/2003 Rafii et al .
U.S. PATENT DOCUMENTS 6,624,803 B1 9/2003 Vanderheiden et al .
6,624,833 B1 9/2003 Kumar et al .
6,239,790 B1 5/2001 Martinelli et al . 6,636,197 B1 10/2003 Goldenberg et al.
D443,616 S 6/2001 Fisher et al . 6,639,577 B2 10/2003 Eberhard
6,243,071 B1 6/2001 Shwarts et al . 6,639,584 B1 10/2003 Li
6,243,078 B1 6/2001 Rosenberg 6,650,319 B1 11/2003 Hurst et al .
6,246,862 B1 6/2001 Grivas et al . 6,650,975 B2 11/2003 Ruffner
6,249,606 B1 6/2001 Kiraly et al . 6,658,994 B1 12/2003 Mcmillan
6,256,011 B1 7/2001 Culver 6,661,410 B2 12/2003 Casebolt et al .
6,262,716 B1 7/2001 Raasch 6,670,894 B2 12/2003 Mehring
6,262,717 B1 7/2001 Donohue et al . 6,677,927 B1 1/2004 Bruck et al .
6,266,050 B1 7/2001 Oh et al . 6,677,932 B1 1/2004 Westerman
6,288,707 B1 9/2001 Philipp 6,677,934 B1 1/2004 Blanchard
6,289,326 B1 9/2001 Lafleur 6,686,904 B1 2/2004 Sherman et al .
6,292,178 B1 9/2001 Bernstein et al . 6,690,387 B2 2/2004 Zimmerman et al.
6,297,795 B1 10/2001 Kato et al . 6,700,564 B2 3/2004 Mcloone et al .
6,297,811 B1 10/2001 Kent et al . 6,703,550 B2 3/2004 Chu
6,310,610 B1 10/2001 Beaton et al . 6,703,599 B1 3/2004 Casebolt et al .
6,313,828 B1 11/2001 Chombo 6,707,027 B2 3/2004 Liess et al .
6,323,843 B2 11/2001 Giles et al . 6,717,569 B1 4/2004 Gruhl et al.
6,323,845 B1 11/2001 Robbins 6,717,573 B1 4/2004 Shahoian et al .
6,323,846 B1 11/2001 Westerman et al . 6,724,366 B2 4/2004 Crawford
6,323,849 B1 11/2001 He et al . 6,724,817 B1 4/2004 Simpson et al.
6,327,011 B2 12/2001 Kim 6,725,064 B1 4/2004 Wakamatsu et al .
6,333,734 B1 12/2001 Rein 6,727,889 B2 4/2004 Shaw
6,337,678 B1 1/2002 Fish 6,738,045 B2 5/2004 Hinckley et al .
6,337,919 B1 1/2002 Dunton 6,740,860 B2 5/2004 Kobayashi
6,340,800 B1 1/2002 Zhai et al . 6,757,002 B1 6/2004 Oross et al.
6,347,290 B1 2/2002 Bartlett D493,157 S 7/2004 Yang
D454,568 S 3/2002 Andre et al . D493,158 S 7/2004 Yang
6,356,287 B1 3/2002 Ruberry et al . 6,762,751 B2 7/2004 Kuan
6,356,524 B2 3/2002 Aratani 6,788,288 B2 9/2004 Ano
6,362,811 B1 3/2002 Edwards et al. 6,791,533 B2 9/2004 Su
6,369,797 B1 4/2002 Maynard, Jr. 6,792,398 B1 9/2004 Handley et al .
6,373,470 B1 4/2002 Andre et al . 6,795,056 B2 9/2004 Norskog et al.
6,377,009 B1 4/2002 Philipp 6,795,057 B2 9/2004 Gordon
6,377,530 B1 4/2002 Burrows D497,606 S 10/2004 Yang
6,380,931 B1 4/2002 Gillespie et al . 6,803,906 B1 10/2004 Morrison et al .
6,384,743 B1 5/2002 Vanderheiden 6,816,149 B1 11/2004 Alsleben
6,392,632 B1 5/2002 Lee 6,816,150 B2 11/2004 Casebolt et al.
6,392,634 B1 5/2002 Bowers et al . 6,819,312 B2 11/2004 Fish
6,392,636 B1 5/2002 Ferrari et al . 6,822,635 B2 11/2004 Shahoian et al .
6,396,477 B1 5/2002 Hinckley et al . D500,298 S 12/2004 Yang
6,411,287 B1 6/2002 Scharff et al . 6,828,958 B2 12/2004 Davenport
6,413,233 B1 7/2002 Sites et al . 6,833,825 B1 12/2004 Farag et al .
6,414,671 B1 7/2002 Gillespie et al . 6,834,195 B2 12/2004 Brandenberg et al.
6,421,234 B1 7/2002 Ricks et al . 6,834,373 B2 12/2004 Dieberger
6,429,846 B2 8/2002 Rosenberg et al . 6,842,672 B1 1/2005 Straub et al .
6,429,852 B1 8/2002 Adams et al . 6,844,871 B1 1/2005 Hinckley et al.
6,433,780 B1 8/2002 Gordon et al . 6,844,872 B1 1/2005 Farag et al .
6,452,514 B1 9/2002 Philipp 6,847,352 B2 1/2005 Lantigua
6,456,275 B1 9/2002 Hinckley et al. 6,848,014 B2 1/2005 Landron et al .
6,457,355 B1 10/2002 Philipp 6,853,850 B2 2/2005 Shim et al .
6,466,036 B1 10/2002 Philipp 6,856,259 B1 2/2005 Sharp
6,466,203 B2 10/2002 Van 6,865,718 B2 3/2005 Levi Montalcini
6,469,693 B1 10/2002 Chiang et al . 6,873,715 B2 3/2005 Kuo et al .
6,489,947 B2 12/2002 Hesley et al. 6,876,891 B1 4/2005 Schuler et al .
6,492,979 B1 12/2002 Kent et al . 6,888,532 B2 5/2005 Wong et al.
6,498,601 B1 12/2002 Gujar et al . 6,888,536 B2 5/2005 Westerman et al .
6,504,530 B1 1/2003 Wilson et al . 6,900,795 B1 5/2005 Knight, III et al .
6,505,088 B1 1/2003 Simkin et al . 6,909,424 B2 6/2005 Liebenow et al .
6,513,717 B2 2/2003 Hannigan 6,924,789 B2 8/2005 Bick
6,515,669 B1 2/2003 Mohri 6,927,761 B2 8/2005 Badaye et al .
6,525,749 B1 2/2003 Moran et al . D509,819 S 9/2005 Yang
6,535,200 B2 3/2003 Philipp D509,833 S 9/2005 Yang
6,543,684 B1 4/2003 White et al. D510,081 S 9/2005 Yang
6,543,947 B2 4/2003 Lee 6,942,571 B1 9/2005 Mcallister et al .
6,545,665 B2 4/2003 Rodgers 6,943,779 B2 9/2005 Satoh
6,559,830 B1 5/2003 Hinckley et al . 6,950,094 B2 9/2005 Gordon et al.
6,559,831 B1 5/2003 Armstrong D511,512 S 11/2005 Yang
6,567,073 B1 5/2003 Levin D511,528 S 11/2005 Yang
6,570,557 B1 5/2003 Westerman et al . 6,965,375 B1 11/2005 Gettemy et al.
6,587,091 B2 7/2003 Serpa D512,403 S 12/2005 Yang
6,587,093 B1 7/2003 Shaw et al . D512,435 S 12/2005 Yang
6,593,916 B1 7/2003 Aroyan 6,972,401 B2 12/2005 Akitt et al .
6,597,347 B1 7/2003 Yasutake 6,977,666 B1 12/2005 Hedrick
6,597,384 B1 7/2003 Harrison 6,985,801 B1 1/2006 Straub et al .
 US 11,360,509 B2
Page 5

( 56 ) References Cited 2002/0093487 Al 7/2002 Rosenberg

2002/0093492 A1 7/2002 Baron
U.S. PATENT DOCUMENTS 2002/0118169 Al 8/2002 Hinckley et al .
2002/0118174 A1 8/2002 Rodgers
6,992,656 B2 1/2006 Hughes 2002/0118848 Al 8/2002 Karpenstein
6,992,659 B2 1/2006 Gettemy 2002/0130839 A1 9/2002 Wallace et al .
6,995,744 B1 2/2006 Moore et al . 2002/0130841 Al 9/2002 Scott
7,013,228 B2 3/2006 Ritt 2002/0140676 A1 10/2002 Kao
7,015,894 B2 3/2006 Morohoshi 2002/0154090 A1 10/2002 Lin
7,031,228 B2 4/2006 Born et al . 2002/0158838 A1 10/2002 Smith et al .
7,034,802 B1 4/2006 Gettemy et al . 2002/0164156 A1 11/2002 Bilbrey
D520,516 S 5/2006 Yang 2002/0180763 Al 12/2002 Kung
7,046,230 B2 5/2006 Zadesky et al . 2002/0190975 A1 12/2002 Kerr
7,051,291 B2 5/2006 Sciammarella et al . 2003/0002246 Al 1/2003 Kerr
7,068,256 B1 6/2006 Gettemy et al. 2003/0006974 A1 1/2003 Clough et al .
7,109,978 B2 9/2006 Gillespie et al . 2003/0011574 A1 * 1/2003 Goodman G06F 3/0237
7,113,196 B2 9/2006 Kerr 345/172
7,119,792 B1 10/2006 Andre et al . 2003/0025735 A1 2/2003 Polgar et al .
7,129,416 B1 10/2006 Steinfeld et al . 2003/0038776 Al 2/2003 Rosenberg et al .
7,142,193 B2 11/2006 Hayama et al . 2003/0043121 A1 3/2003 Chen
7,145,552 B2 12/2006 Hollingsworth 2003/0043174 Al 3/2003 Hinckley et al .
7,164,412 B2 1/2007 Kao 2003/0048260 A1 3/2003 Matusis
7,168,047 B1 1/2007 Huppi 2003/0050092 Al 3/2003 Yun
7,170,488 B2 1/2007 Kehlstadt et al . 2003/0063072 A1 4/2003 Brandenberg et al .
7,170,496 B2 1/2007 Middleton 2003/0074977 Al 4/2003 Doemens et al .
7,183,948 B2 2/2007 Roberts 2003/0076301 Al 4/2003 Tsuk et al .
7,184,064 B2 2/2007 Zimmerman et al. 2003/0076303 A1 4/2003 Huppi
7,191,024 B2 3/2007 Kano 2003/0076306 A1 4/2003 Zadesky et al .
7,233,318 B1 6/2007 Farag et al . 2003/0085870 Al 5/2003 Hinckley
7,236,159 B1 6/2007 Siversson 2003/0085882 A1 5/2003 Lu
7,239,800 B2 7/2007 Bilbrey 2003/0095095 A1 5/2003 Pihlaja
7,240,289 B2 7/2007 Naughton et al . 2003/0095096 A1 5/2003 Robbin et al .
7,312,981 B2 12/2007 Carroll 2003/0098851 Al 5/2003 Brink
7,336,260 B2 2/2008 Martin et al . 2003/0098858 Al 5/2003 Perski et al .
RE40,153 E 3/2008 Westerman et al . 2003/0107551 Al 6/2003 Dunker
7,366,995 B2 4/2008 Montague 2003/0107552 A1 6/2003 Lu
7,388,578 B2 6/2008 Tao 2003/0107607 A1 6/2003 Nguyen
7,423,636 B2 9/2008 Sano et al . 2003/0117377 A1 6/2003 Horie et al .
7,452,098 B2 11/2008 Kerr 2003/0122779 Al 7/2003 Martin et al .
7,453,439 B1 11/2008 Kushler et al. 2003/0142855 Al 7/2003 Kuo et al .
7,495,659 B2 2/2009 Marriott et al . 2003/0156098 A1 8/2003 Shaw et al .
7,499,039 B2 3/2009 Roberts 2003/0179223 A1 9/2003 Ying et al .
7,499,040 B2 3/2009 Zadesky et al . 2003/0184517 Al 10/2003 Senzui et al .
7,505,785 B2 3/2009 Callaghan et al . 2003/0184520 A1 10/2003 Wei
D592,665 S 5/2009 Andre et al . 2003/0206162 Al 11/2003 Roberts
RE40,993 E 11/2009 Westerman 2003/0206202 A1 11/2003 Moriya
7,629,961 B2 12/2009 Casebolt et al . 2003/0210286 Al 11/2003 Gerpheide et al.
7,652,589 B2 1/2010 Autor 2003/0222848 Al 12/2003 Solomon et al .
7,663,607 B2 2/2010 Hotelling et al . 2003/0222858 A1 12/2003 Kobayashi
7,683,889 B2 3/2010 Rimas et al . 2003/0234768 A1 12/2003 Rekimoto et al .
7,800,592 B2 9/2010 Kerr et al . 2004/0003947 Al 1/2004 Kesselman et al .
7,808,479 B1 10/2010 Hotelling et al . 2004/0012572 Al 1/2004 Sowden et al .
7,932,893 B1 4/2011 Berthaud 2004/0021643 Al 2/2004 Hoshino et al .
8,479,122 B2 7/2013 Hotelling et al . 2004/0021681 Al 2/2004 Liao
8,537,115 B2 9/2013 Hotelling et al. 2004/0046739 A1 3/2004 Gettemy
8,665,209 B2 3/2014 Rimas -Ribikauskas et al . 2004/0046741 A1 3/2004 Low et al .
8,704,769 B2 4/2014 Hotelling et al . 2004/0075676 A1 4/2004 Rosenberg et al .
8,704,770 B2 4/2014 Hotelling et al . 2004/0099131 A1 5/2004 Ludwig
9,047,009 B2 6/2015 King et al . 2004/0109013 Al 6/2004 Goertz
9,335,868 B2 5/2016 Hotelling et al . 2004/0125086 Al 7/2004 Hagermoser et al.
9,785,258 B2 10/2017 Hotelling et al . 2004/0130526 A1 7/2004 Rosenberg
9,794,397 B2 10/2017 Min et al . 2004/0138569 Al 7/2004 Grunwald et al .
9,983,742 B2 5/2018 King et al . 2004/0139348 A1 7/2004 Norris
10,156,914 B2 12/2018 Hotelling et al . 2004/0156192 Al 8/2004 Kerr et al .
2001/0011991 A1 8/2001 Wang et al . 2004/0212586 Al 10/2004 Denny
2001/0011995 A1 8/2001 Hinckley et al . 2004/0222979 A1 11/2004 Knighton
2001/0035854 A1 11/2001 Rosenberg et al. 2004/0227736 A1 11/2004 Kamrath et al .
2001/0043189 Al 11/2001 Brisebois et al . 2004/0239622 A1 12/2004 Proctor et al .
2001/0043545 A1 11/2001 Aratani 2004/0242269 Al 12/2004 Fadell
2001/0050673 Al 12/2001 Davenport 2004/0242295 Al 12/2004 Ghaly
2001/0051046 A1 12/2001 Watanabe et al. 2004/0246231 A1 12/2004 Large
2002/0008691 Al 1/2002 Hanajima et al. 2004/0252109 Al 12/2004 Trent et al .
2002/0015024 A1 2/2002 Westerman et al . 2004/0263483 A1 12/2004 Aufderheide
2002/0033795 A1 3/2002 Shahoian et al . 2004/0263484 Al 12/2004 Mantysalo et al.
2002/0033848 A1 3/2002 Sciammarella et al . 2005/0012723 Al 1/2005 Pallakoff
2002/0063688 A1 5/2002 Shaw et al . 2005/0030278 A1 2/2005 Fu
2002/0067334 Al 6/2002 Hinckley et al . 2005/0035951 A1 2/2005 Bjorkengren
2002/0084986 Al 7/2002 Armstrong 2005/0035955 Al 2/2005 Carter et al .
2002/0089545 Al 7/2002 Levi Montalcini 2005/0043060 A1 2/2005 Brandenberg et al .
 US 11,360,509 B2
Page 6

( 56 ) References Cited 2008/0266257 Al 10/2008 Chiang

2008/0278450 Al 11/2008 Lashina
U.S. PATENT DOCUMENTS 2008/0297476 Al 12/2008 Hotelling et al .
2008/0297477 Al 12/2008 Hotelling et al .
2005/0048955 A1 3/2005 Ring 2008/0297478 Al 12/2008 Hotelling et al .
2005/0052425 A1 3/2005 Zadesky et al . 2009/0059730 Al 3/2009 Lyons et al .
2005/0057524 A1 3/2005 Hill et al . 2009/0066670 A1 3/2009 Hotelling et al .
2005/0068322 Al 3/2005 Falcioni 2009/0085894 Al 4/2009 Gandhi et al .
2005/0084138 A1 4/2005 Inkster et al . 2009/0096757 Al 4/2009 Hotelling et al .
2005/0104867 A1 5/2005 Westerman et al . 2009/0096758 Al 4/2009 Hotelling et al .
2005/0110767 A1 5/2005 Gomes et al . 2009/0236648 Al 9/2009 Maeda et al.
2005/0110768 A1 5/2005 Marriott et al . 2009/0295738 A1 12/2009 Chiang
2005/0115816 A1 6/2005 Gelfond et al . 2009/0295753 A1 12/2009 King et al.
2005/0135053 A1 6/2005 Carroll 2010/0026656 A1 2/2010 Hotelling et al .
2005/0146509 Al 7/2005 Geaghan et al . 2010/0112964 Al 5/2010 Yi et al.
2005/0146513 A1 7/2005 Hill et al . 2010/0164878 A1 7/2010 Bestle et al .
2005/0154798 A1 7/2005 Nurmi 2011/0012835 A1 1/2011 Hotelling et al .
2005/0168488 A1 8/2005 Montague 2012/0075238 A1 3/2012 Minami et al .
2005/0183035 A1 8/2005 Ringel et al . 2012/0099406 A1 4/2012 Lau et al .
2005/0190158 A1 9/2005 Casebolt et al . 2012/0299859 Al 11/2012 Kinoshita
2005/0212760 A1 9/2005 Marvit et al . 2014/0171156 Al 6/2014 Pattikonda et al .
2005/0219228 A1 10/2005 Alameh et al. 2014/0375577 Al 12/2014 Yeh et al .
2005/0228320 A1 10/2005 Klinghult 2015/0002440 A1 1/2015 Huang et al .
2005/0253818 A1 11/2005 Nettamo 2015/0062050 Al 3/2015 Zadesky et al .
2005/0259077 Al 11/2005 Adams et al . 2015/0091790 A1 4/2015 Forutanpour et al.
2005/0264540 A1 12/2005 Niwa 2015/0111558 A1 4/2015 Yang
2005/0275637 Al 12/2005 Hinckley et al . 2015/0153895 Al 6/2015 Hotelling
2006/0010400 A1 1/2006 Dehlin et al . 2015/0186092 A1 7/2015 Francis et al .
2006/0022955 Al 2/2006 Kennedy 2015/0261310 Al 9/2015 Walmsley et al .
2006/0022956 A1 2/2006 Lengeling et al . 2015/0261362 Al 9/2015 King et al.
2006/0026521 A1 2/2006 Hotelling et al . 2015/0346026 A1 12/2015 Maass et al .
2006/0026535 A1 2/2006 Hotelling et al . 2016/0070399 Al 3/2016 Hotelling
2006/0026536 Al 2/2006 Hotelling et al . 2016/0197753 A1 7/2016 Hwang et al .
2006/0032680 A1 2/2006 Elias et al . 2017/0068352 Al 3/2017 Blondin et al .
2006/0033724 Al 2/2006 Chaudhri et al . 2017/0336964 A1 11/2017 Kim et al .
2006/0044259 Al 3/2006 Hotelling et al. 2018/0032158 Al 2/2018 Hotelling et al .
2006/0050011 A1 3/2006 Kamio 2018/0059809 Al 3/2018 Mcclendon et al .
2006/0053387 A1 3/2006 Ording 2018/0067639 Al 3/2018 Balaram
2006/0066582 Al 3/2006 Lyon et al. 2018/0217709 Al 8/2018 Hotelling
2006/0073272 Al 4/2006 Carel 2018/0253172 A1 9/2018 King et al.
2006/0079969 A1 4/2006 Seguin 2019/0041991 Al 2/2019 Hotelling et al .
2006/0085757 A1 4/2006 Andre et al . 2019/0113988 A1 4/2019 Hotelling et al .
2006/0097991 Al 5/2006 Hotelling et al . 2019/0258395 Al 8/2019 Balaram
2006/0109252 A1 5/2006 Kolmykov - zotov et al. 2019/0354215 Al 11/2019 King et al.
2006/0109256 Al 5/2006 Grant et al .
2006/0146036 Al 7/2006 Prados et al . FOREIGN PATENT DOCUMENTS
2006/0181517 Al 8/2006 Zadesky et al .
2006/0181518 A1 8/2006 Shen et al . 1343330 A 4/2002

22
2006/0183505 Al 8/2006 Willrich 1397870 A 2/2003
2006/0197750 A1 9/2006 Kerr et al . 1529869 A 9/2004
2006/0197753 A1 9/2006 Hotelling 1588432 A 3/2005
2006/0232567 Al 10/2006 Westerman et al . 1720499 A 1/2006
2006/0238517 A1 10/2006 King et al . 1737827 A 2/2006
2006/0238518 A1 10/2006 Westerman et al . CN 1942853 A 4/2007
2006/0238519 Al 10/2006 Westerman et al . DE 4125049 A1 1/1992
2006/0238520 A1 10/2006 Westerman et al . DE 19722636 A1 12/1998
2006/0238521 Al 10/2006 Westerman et al . DE 10022537 A1 11/2000
2006/0238522 Al 10/2006 Westerman et al . DE 10201193 A1 7/2003
2006/0244733 A1 11/2006 Geaghan DE 10251296 Al 5/2004
2006/0250357 A1 11/2006 Safai EP 0288692 A2 11/1988
2006/0267934 A1 11/2006 Harley et al . EP 0464908 A2 1/1992
2006/0267953 A1 11/2006 Peterson et al . EP 0498540 A2 8/1992
2006/0279548 Al 12/2006 Geaghan EP 0288692 B1 7/1993
2006/0284855 Al 12/2006 Shintome EP 0653725 A2 5/1995
2007/0043725 A1 2/2007 Hotelling et al. EP 0664504 A2 7/1995
2007/0046646 A1 3/2007 Kwon et al. EP 0464908 B1 9/1996
2007/0063923 A1 3/2007 Koenig EP 0768619 Al 4/1997
2007/0075968 A1 4/2007 Hall et al . EP 0795837 A1 9/1997
2007/0136064 Al 6/2007 Carroll EP 0880091 A2 11/1998
2007/0229464 A1 10/2007 Hotelling et al . EP 1026713 A1 8/2000
2007/0236466 Al 10/2007 Hotelling EP 1014295 A3 1/2002
2007/0247429 A1 10/2007 Westerman EP 1241557 A2 9/2002
2007/0257890 A1 11/2007 Hotelling et al . EP 1241558 A2 9/2002
2008/0012835 Al 1/2008 Rimon et al . EP 1505484 A1 2/2005
2008/0012838 Al 1/2008 Rimon EP 1571537 A1 9/2005
2008/0088602 Al 4/2008 Hotelling EP 0899650 B1 6/2011
2008/0158172 Al 7/2008 Hotelling et al . GB 2380583 A 4/2003
2008/0211772 Al 9/2008 Loucks GB 2393688 A 4/2004
2008/0238879 A1 10/2008 Jaeger et al . GB 2393688 B 1/2006
 US 11,360,509 B2
Page 7

( 56 ) References Cited WO 2006094308 A2 9/2006

WO 2006096501 A1 9/2006
FOREIGN PATENT DOCUMENTS WO 2006094308 A3 12/2006
WO 2006132817 A2 12/2006
JP 63106826 A 5/1988 WO 2006132817 A3 8/2007
JP S63257824 A 10/1988 WO 2007103631 A2 9/2007
JP S63292774 A 11/1988 WO 2007103631 A3 11/2008
JP H03237520 A 10/1991 WO 2010014560 A1 2/2010
JP H0764725 A 3/1995
JP H07160396 A 6/1995 OTHER PUBLICATIONS
JP H07182101 A 7/1995
JP H07319001 A 12/1995
JP H08161138 A 6/1996 5 - Wire Resistive Touchscreens, Available online at: <http ://www .
JP H08166866 A 6/1996 touchscreens.com/intro-touchtypes-resistive.html> , Accessed on Aug.
JP H08211992 A 8/1996 5 , 2005 .
JP H096525 A 1/1997 A Brief Overview of Gesture Recognition, Available online at:
JP H09244810 A 9/1997 < http://www.dai.ed.ac.uk/Cvonline/LOCA_COPIES/COHEN/gesture_
JP H09305262 A 11/1997
JP H10228350 A 8/1998 overview.html > , Accessed on Apr. 20 , 2004 .
JP H10326149 A 12/1998 About Quicktip , Available online at: < www.logicad3d.com/docs/qt.
JP H11143606 A 5/1999 html > , Accessed on Apr. 8 , 2002 , 2 pages .
JP H11194863 A 7/1999 Advisory Action received for U.S. Appl. No. 10/ 654,108 , dated Apr.
JP H11194872 A 7/1999 17 , 2007 , 3 pages .
JP H11194883 A 7/1999 Advisory Action received for U.S. Appl. No. 10/ 654,108 , dated Apr.
JP H11215217 A 8/1999
JP 2000163031 A 6/2000 20, 2009 , 3 pages.
JP 2000215549 A 8/2000 Advisory Action received for U.S. Appl. No. 10 /654,108 , dated Feb.
JP 2000242424 A 9/2000 12, 2008 , 3 pages.
JP 2000242428 A 9/2000 Advisory Action received for U.S. Appl. No. 10 /654,108 , dated Mar.
JP 2000293289 A 10/2000 25 , 2010, 3 pages.
JP 2000330946 A 11/2000 Advisory Action received for U.S. Appl. No. 11 /204,873 , dated Jun .
JP 2001051790 A 2/2001
JP 2001356878 A 12/2001 17 , 2009 , 3 pages.
JP 2002501271 A 1/2002 Advisory Action received for U.S. Appl. No. 11 /204,873 , dated Mar.
JP 2002062972 A 2/2002 3 , 2017 , 7 pages.
JP 2002185630 A 6/2002 Advisory Action received for U.S. Appl. No. 11 /204,873 , dated Sep.
JP 2002229719 A 8/2002 17 , 2018 , 4 pages.
JP 2002342033 A 11/2002 Advisory Action received for U.S. Appl. No. 11 /367,749 , dated
JP 2002342034 A 11/2002 Aug. 25 , 2010 , 33 pages .
JP 2003058316 A 2/2003 Advisory Action received for U.S. Appl. No. 11 /966,948 , dated Sep.
JP 2003122506 A 4/2003 23 , 2016 , 3 pages .
JP 2003241872 A 8/2003 Advisory Action received for U.S. Appl. No. 12 / 184,190 , dated Feb.
JP 2003271309 A 9/2003 25 , 2013 , 5 pages.
JP 2003280807 A 10/2003 Advisory Action received for U.S. Appl. No. 12 / 189,030 , dated
JP 2003330611 A 11/2003
JP 2004021933 A 1/2004 Nov. 8, 2013 , 3 pages.
JP 2004070920 A 3/2004 Advisory Action received for U.S. Appl. No. 12 / 189,030 , dated
JP 2004177993 A 6/2004 Nov. 12 , 2013 , 3 pages .
JP 2004226715 A 8/2004 Apple Unveils Optical Mouse and New Pro Keyboard , Available
JP 2004527847 9/2004 online at: http://www.apple.com/pr/library/2000/jul/19mousekeyboard.
JP 2004340991 A 12/2004 html, Jul. 19 , 2000 , 2 pages .
JP 2005006259 A 1/2005 Capacitive Position Sensing , Available online at: < http : //www .
JP
KR
2005346244
20010047975
A
A
12/2005
6/2001
synaptics.com/technology/cps.cfm >, Accessed on Aug. 5 , 2005 .
KR 20020016080 A 3/2002 Capacitive Touchscreens, Available online at: <http ://www.touchscreens.
TW 431607 U 4/2001 com /intro -touchtypes - capacitive.html> , Accessed on Aug. 5 , 2005 .
WO 9005972 A1 5/1990 Comparing Touch Technologies, Available online at: < http : //www.
WO 9210823 A1 6/1992 touchscreens.com/intro-touchtypes.html> , Accessed on Oct. 10 , 2004 .
WO 9417494 A2 8/1994 EPO Form 1507 received for European Patent Application No.
WO 9718547 A1 5/1997 02761784.4 , dated Nov. 19 , 2004 , 6 pages .
WO 9723738 A1 7/1997 Examination Report received for European Patent Application No.
WO 9814863 A2 4/1998 06737515.4 , dated Apr. 21 , 2008 , 5 pages .
WO 9926330 A2 5/1999 Examiner's Answer to Appeal Brief received for U.S. Appl. No.
WO 9938149 A1 7/1999 11 / 204,873 , dated Mar. 29 , 2013 , 22 pages .
WO 9949443 A2 9/1999 Examiner's Answer to Appeal Brief received for U.S. Appl. No.
WO 0039907 A1 7/2000 11 /367,749 , dated Mar. 15 , 2019 , 5 pages.
WO 0055716 A1 9/2000 Examiner's Answer to Appeal Brief received for U.S. Appl. No.
WO 0235461 A1 5/2002
WO 02052494 A2 7/2002 12/184 , 190 , dated Aug. 16 , 2013 , 6 pages .
WO 03007227 A2 1/2003 Examiner's Answer to Appeal Brief received for U.S. Appl. No.
WO 03001576 A3 4/2003 12/ 189,030 , dated May 30 , 2014 , 10 pages .
WO 03065192 A1 8/2003 Extended Search Report received for European Patent Application
WO 03077110 A2 9/2003 No. 09170572.3 , dated Nov. 7 , 2013 , 8 pages .
WO 03088176 A1 10/2003 Final Office Action received for U.S. Appl. No. 10 /654,108 , dated
WO 2004111816 A2 12/2004 Feb. 2, 2007 , 17 pages.
WO 2005052773 A2 6/2005 Final Office Action received for U.S. Appl. No. 10 /654,108 , dated
WO 2006020305 A2 2/2006 Feb. 18, 2009, 13 pages.
WO 2006023569 A1 3/2006 Final Office Action received for U.S. Appl. No. 10 /654,108 , dated
WO 2004111816 A3 4/2006 Jan. 19 , 2010 , 16 pages .
 US 11,360,509 B2
Page 8

( 56 ) References Cited Final Office Action received for U.S. Appl. No. 14 /940,010 , dated
Aug. 23 , 2016 , 12 pages .
OTHER PUBLICATIONS Final Office Action received for U.S. Appl. No. 14 /940,010 , dated
Feb. 2 , 2018 , 14 pages.
Final Office Action received for U.S. Appl. No. 10 /654,108 , dated Final Office Action received for U.S. Appl. No. 14 /940,010 , dated
Nov. 16, 2007 , 13 pages. Feb. 26, 2019, 12 pages.
Final Office Action received for U.S. Appl. No. 11 / 115,539 , dated Final Office Action received for U.S. Appl. No. 15 /933,196 , dated
Nov. 21 , 2008, 24 pages. Oct. 31 , 2019 , 10 pages.
Final Office Action received for U.S. Appl. No. 11 / 115,539 , dated Final Office Action received for U.S. Appl. No. 16 / 155,508 , dated
Oct. 14 , 2009 , 11 pages. Oct. 11 , 2019 , 17 pages.
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated FingerWorks Gesture Guide Application Switching , Available
Apr. 11 , 2018 , 29 pages. online at : < http://www.fingerworks.com/gesture_guide_apps.
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated html> , Accessed on Aug. 27 , 2004 , 1 page .
Apr. 28 , 2009 , 19 pages . FingerWorks Gesture Guide Editing, Available online at : < http : //
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated www.fingerworks.com/gesure guide_editing.html> Feb. 13 , 2004 ,
Aug. 13 , 2012 , 22 pages . Accessed on Aug. 27 , 2004 , 1 page .
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated FingerWorks Gesture Guide File Operations, Available online
Jan. 15 , 2010 , 24 pages. at: < http://www.fingerworks.com/gesture_guide_files.html > Jun . 18 ,
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated 2004 , Accessed on Aug. 27 , 2004 , 1 page .
Jun. 20 , 2011 , 17 pages.
FingerWorks Gesture Guide — Text Manipulation, Available online
at: < http://www.fingerworks.com/gesture_guide_text_manip.html >
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated Jun. 6 , 2004 , Accessed on Aug. 27 , 2004 , 2 pages .
Sep. 7 , 2016 , 25 pages . FingerWorks Gesture Guide Tips and Tricks , Available online
Final Office Action received for U.S. Appl. No. 11 /204,873 , dated at: < http://www.fingerworks.com/gesture_guide_tips.html> , Accessed
Sep. 20 , 2010 , 15 pages . on Aug. 27 , 2004 , 1 page .
Final Office Action received for U.S. Appl. No. 11 /367,749 , dated FingerWorks Gesture Guide — Web , Available online at: < http : //
Aug. 25 , 2009 , 13 pages . www.fingerworks.com/gesture guide_web.html> Jun . 5 , 2004 , Accessed
Final Office Action received for U.S. Appl. No. 11 /367,749 , dated on Aug. 27 , 2004 , 1 page .
Jan. 8, 2016, 11 pages. FingerWorks Guide to Hand Gestures for USB Touchpads, Avail
Final Office Action received for U.S. Appl. No. 11 /367,749 , dated able online at: < http://www.fingerworks.com/igesture_userguide.
Jan. 17 , 2014, 13 pages . html > , Accessed on Aug. 27 , 2004 , 1 page .
Final Office Action received for U.S. Appl. No. 11 /367,749 , dated FingerWorks — iGesture — Technical Details , Available online at:
Jul. 14, 2017, 11 pages. < http://www.fingerworks.com/igesture_tech.html> , Accessed on Aug.
Final Office Action received for U.S. Appl. No. 11 /367,749 , dated 27 , 2004, 1 page.
Jun. 30, 2010, 10 pages . Finger Works — The Only Touchpads with Ergonomic Full -Hand
Final Office Action received for U.S. Appl. No. 11 /367,749 , dated Resting and Relaxation !, Available online at: < http ://www.fingerworks.
Mar. 16, 2018, 13 pages. com /resting.html> , 2001 , 1 page .
Final Office Action received for U.S. Appl. No. 11 /966,948 , dated FingerWorks — Tips for Typing on the Mini , Available online at:
Jul . 13 , 2011 , 10 pages . < http://www.fingerworks.com/mini_typing.html> Jun . 5 , 2004 , Accessed
Final Office Action received for U.S. Appl. No. 11 /966,948 , dated on Aug. 27 , 2004 , 2 pages .
Mar. 10, 2016, 15 pages. Gesture Recognition, Available online at: < http : //www.fingerworks.
Final Office Action received for U.S. Appl. No. 11 /966,948 , dated com / gesture_recognition.html>, downloaded on Aug. 30 , 2005 , 2
Oct. 9, 2014, 11 pages . pages.
Final Office Action received for U.S. Appl. No. 12 / 184,190 , dated Glidepoint, Available online at: < http://www.cirque.com/technology/
Nov. 8, 2012, 21 pages. technology_gp.html>, Accessed on Aug. 5 , 2005 .
Final Office Action received for U.S. Appl. No. 12 / 188,948 , dated How Do Touchscreen Monitors Know Where You’re Touching ?,
Dec. 7 , 2011 , 13 pages. Available online at: < http://electronics.howstuffworks.com/question716 .
Final Office Action received for U.S. Appl. No. 12 / 188,988 , dated html> , Jul. 7 , 2008 , 2 pages.
Dec. 8 , 2011 , 15 pages. How Does aa Touchscreen Work ?, Available online at: < http ://www .
Final Office Action received for U.S. Appl. No. 12 / 189,030 , dated touchscreens.com/intro-anatomy.html> , Accessed on Aug. 5 , 2005 .
Jun. 20, 2013 , 17 pages. IGesture Pad - the MultiFinger USB TouchPad with Whole -Hand
Final Office Action received for U.S. Appl. No. 12 / 189,030 , dated Gestures, Available online at: < http://www.fingerworks.com/igesture.
Oct. 14, 2011 , 15 pages. html > , Accessed on Aug. 27 , 2004 , 22 pages .
Final Office Action received for U.S. Appl. No. 12 /486,710 , dated IGesture Products for Everyone ( learn in minutes) Product Over
Jan. 2 , 2014, 30 pages. view , Available online at: < FingerWorks.com >, Accessed on Aug.
Final Office Action received for U.S. Appl. No. 12 /486,710, dated 30, 2005 .
Jul. 1 , 2011 , 20 pages. Infrared Touchscreens, Available online at: < http ://www.touchscreens.
Final Office Action received for U.S. Appl. No. 12 / 486,710 , dated com /intro - touchtypes -infrared.html> , Accessed on Aug. 5 , 2005 .
Sep. 4 , 2012 , 25 pages . Intention to Grant received for European Patent Application No.
Final Office Action received for U.S. Appl. No. 12 / 890,437 , dated 16186726.2 , dated Jan. 24 , 2020 , 7 pages .
Apr. 19 , 2011 , 20 pages . International Search Report and Written Opinion received for PCT
Final Office Action received for U.S. Appl. No. 14 / 535,101 , dated Patent Application No. PCT/US2006 /007585 , dated Jul. 25 , 2006,9
Apr. 28 , 2017 , 9 pages . pages.
Final Office Action received for U.S. Appl. No. 14 / 535,101 , dated International Search Report received for PCT Patent Application
Feb. 4 , 2016 , 7 pages. No. PCT/US2005 /003325, dated Mar. 3 , 2006 .
Final Office Action received for U.S. Appl. No. 14 / 535,101 , dated International Search Report received for PCT Patent Application
May 18 , 2018 , 10 pages . No. PCT/US2006 /008349, dated Oct. 6 , 2006 , 3 pages .
Final Office Action received for U.S. Appl. No. 14 / 595,032 , dated International Search Report received for PCT Patent Application
Aug. 3 , 2015 , 10 pages. No. PCT/US2006 /020341, dated Jun . 12 , 2007 , 7 pages .
Final Office Action received for U.S. Appl. No. 14 / 595,032 , dated International Search Report received for PCT Patent Application
Feb. 15 , 2017 , 10 pages. No. PCT/US2007/ 062474 , dated Jan. 3 , 2008 , 3 pages .
Final Office Action received for U.S. Appl. No. 14 /724,753 , dated International Search Report received for PCT Patent Application
Nov. 25 , 2016, 16 pages. No. PCT/US2009 /051874, dated Nov. 27 , 2009 , 3 pages .
 US 11,360,509 B2
Page 9

( 56 ) References Cited Non - Final Office Action received for U.S. Appl. No. 12/ 184,190 ,
dated Jul. 5 , 2012 , 20 pages.
OTHER PUBLICATIONS Non -Final Office Action received for U.S. Appl. No. 12/ 188,948 ,
dated Aug. 21 , 2013 , 14 pages .
IRiver Clix Quick Start Guide, 2 pages . Non -Final Office Action received for U.S. Appl. No. 12/ 188,948 ,
IRiver Clix User Guide, 1999-2006 , 38 pages . dated May 6 , 2011 , 11 pages .
Letter Restarting Period for Response received for U.S. Appl. No. Non - Final Office Action received for U.S. Appl. No. 12/ 188,988 ,
12/ 189,030 , dated May 26 , 2011 , 16 pages . dated Aug. 14 , 2013 , 18 pages .
Mouse Emulation , FingerWorks, Available online at: < http ://www . Non - Final Office Action received for U.S. Appl. No. 12/ 188,988 ,
fingerworks.com/gesture guide_mouse.html> Dec. 10 , 2002 , Accessed dated May 16 , 2011 , 15 pages .
on Aug. 30 , 2005 . Non - Final Office Action received for U.S. Appl. No. 12/ 189,030 ,
Mouse Gestures , Optim oz , May 21 , 2004 . dated Aug. 24 , 2016 , 19 pages .
Mouse Gestures in Opera , Available online at : < http ://www.opera. Non - Final Office Action received for U.S. Appl. No. 12/ 189,030 ,
com / products /desktop /xmouse / index.dml >, Accessed on Aug. 30 , dated Feb. 22 , 2017 , 23 pages.
2005 .
MultiTouch Overview , FingerWorks, Available online at: < http : // Non - Final Office Action received for U.S. Appl. No. 12/ 189,030 ,
www.fingerworks.com/multoverview.html> , Accessed on Aug. 30 , dated May 6 , 2011 , 8 pages.
2005 . Non -Final Office Action received for U.S. Appl. No. 12/ 189,030 ,
Near Field Imaging Touchscreens, Available online at: < http : //www . dated Nov. 28 , 2012 , 16 pages.
touchscreens.com/intro-touchtypes-nfi.html> , Accessed on Aug. 5 , Non - Final Office Action received for U.S. Appl. No. 12/486,710 ,
2005 . dated Aug. 25 , 2014 , 29 pages .
Neuros MP3 Digital Audio Computer, Available online at: < www . Non - Final Office Action received for U.S. Appl. No. 12/486,710 ,
neurosaudio.com > , Accessed on Apr. 9 , 2003. , 1 page . dated Jun . 6, 2013 , 29 pages.
Non - Final Office Action received for U.S. Appl. No. 10 /654,108 , Non -Final Office Action received for U.S. Appl. No. 12/486,710 ,
dated Jul. 6 , 2007 , 14 pages. dated Mar. 9 , 2011 , 14 pages.
Non - Final Office Action received for U.S. Appl. No. 10 /654,108 , Non - Final Office Action received for U.S. Appl. No. 12/486,710 ,
dated Jul. 14, 2009 , 17 pages. dated Mar. 28, 2012 , 25 pages.
Non - Final Office Action received for U.S. Appl. No. 10 /654,108 , Non - Final Office Action received for U.S. Appl. No. 12/ 890,437 ,
dated Jun . 16 , 2006 , 8 pages. dated Dec. 28, 2010 , 17 pages.
Non - Final Office Action received for U.S. Appl. No. 10 /654,108 , Non - Final Office Action received for U.S. Appl. No. 14/ 535,101 ,
dated Oct. 28 , 2008 , 11 pages. dated Aug. 11 , 2016 , 9 pages .
Non - Final Office Action received for U.S. Appl. No. 11 / 115,539 , Non - Final Office Action received for U.S. Appl. No. 14/ 535,101 ,
dated Apr. 6, 2009 , 32 pages. dated May 12, 2015 , 8 pages.
Non - Final Office Action received for U.S. Appl. No. 11 / 115,539 , Non - Final Office Action received for U.S. Appl. No. 14/ 535,101 ,
dated Jan. 15 , 2010 , 11 pages . dated Nov. 15, 2017, 9 pages.
Non - Final Office Action received for U.S. Appl. No. 11 / 115,539 , Non -Final Office Action received for U.S. Appl. No. 14/ 595,032 ,
dated Jul. 2 , 2008 , 19 pages. dated Apr. 10 , 2015 , 12 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non - Final Office Action received for U.S. Appl. No. 14/ 595,032 ,
dated Apr. 6 , 2016 , 33 pages . dated Jun . 27 , 2016 , 9 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non -Final Office Action received for U.S. Appl. No. 14/ 724,753 ,
dated Aug. 9 , 2017 , 27 pages. dated Apr. 20 , 2016 , 16 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non - Final Office Action received for U.S. Appl. No. 14/ 724,753 ,
dated Feb. 28 , 2011 , 16 pages . dated Jun . 22, 2017 , 9 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non - Final Office Action received for U.S. Appl. No. 14/ 940,010 ,
dated Mar. 8 , 2012 , 21 pages . dated Feb. 2, 2016 , 9 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non -Final Office Action received for U.S. Appl. No. 14/ 940,010 ,
dated May 20 , 2010 , 14 pages. dated May 17 , 2017 , 11 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non - Final Office Action received for U.S. Appl. No. 14/ 940,010 ,
dated Nov. 13 , 2008, 11 pages. dated Oct. 12 , 2018 , 12 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /204,873 , Non -Final Office Action received for U.S. Appl. No. 15 / 257,713 ,
dated Sep. 30 , 2009 , 21 pages . dated Oct. 17 , 2018 , 9 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /367,749 , Non - Final Office Action received for U.S. Appl. No. 15 / 727,455 ,
dated Feb. 26 , 2009 , 11 pages . dated Nov. 2 , 2017 , 17 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /367,749 , Non - Final Office Action received for U.S. Appl. No. 15 / 933,196 ,
dated Jan. 12 , 2010 , 9 pages. dated Mar. 1 , 2019 , 9 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /367,749 , Non - Final Office Action received for U.S. Appl. No. 16/ 155,508 ,
dated Jul. 5 , 2013 , 9 pages. dated Mar. 14 , 2019 , 15 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /367,749 , Non - Final Office Action received for U.S. Appl. No. 16/ 209,137 ,
dated May 14 , 2015 , 14 pages . dated Jan. 25 , 2019 , 18 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /367,749 , Non -Final Office Action received for U.S. Appl. No. 15 / 933,196 ,
dated Nov. 10 , 2016 , 10 pages. dated Apr. 6 , 2020 , 10 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /426,078 , Non -Final Office Action received for U.S. Appl. No. 16/403,370 ,
dated Apr. 2 , 2009 , 22 pages. dated Jul. 8, 2020, 10 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /966,948 , Non - Final Office Action received for U.S. Appl. No. 16/ 529,690 ,
dated Apr. 6 , 2017 , 14 pages. dated Apr. 6 , 2020 , 17 pages .
Non - Final Office Action received for U.S. Appl. No. 11 /966,948 , Notice of Allowance received for Canadian Patent Application No.
dated Aug. 26 , 2015 , 10 pages. 2,820,737 , dated Feb. 1 , 2019 , 1 page .
Non - Final Office Action received for U.S. Appl. No. 11 /966,948 , Notice of Allowance received for U.S. Appl. No. 10 /654,108 , dated
dated Feb. 27 , 2014 , 11 pages. Jun . 1 , 2010, 6 pages.
Non - Final Office Action received for U.S. Appl. No. 11 /966,948 , Notice of Allowance received for U.S. Appl. No. 11 / 115,539 , dated
dated Jan. 28 , 2011 , 12 pages . Jun. 2, 2010, 99 pages.
Non - Final Office Action received for U.S. Appl. No. 12 / 184,190 , Notice of Allowance received for U.S. Appl. No. 11 /426,078, dated
dated Dec. 2 , 2011 , 22 pages. Sep. 25 , 2009 , 14 pages.
 US 11,360,509 B2
Page 10

( 56 ) References Cited Restriction Requirement received for U.S. Appl. No. 12/ 184,190 ,
dated Sep. 15 , 2011 , 6 pages.
OTHER PUBLICATIONS Search Report received for Chinese Patent Application No.
201310264394.7 , dated Jan. 14 , 2016 , 4 pages ( 2 pages of English
Notice ofAllowance received for U.S. Appl. No. 12 / 184,190 , dated Translation and 2 pages of Official copy) .
Jan. 11 , 2016, 7 pages. Search Report received for Chinese Patent Application No.
Notice of Allowance received for U.S. Appl. No. 12 / 188,948 , dated 201410259240.3 , dated Sep. 27 , 2016 , 4 pages .
Jan. 2, 2014, 7 pages. Search Report received for European Patent Application No. 10010075 .
Notice ofAllowance received for U.S. Appl. No. 12 / 188,988 , dated 9 , dated Nov. 9 , 2010 , 6 pages.
Dec. 30 , 2013 , 10 pages . Search Report received for European Patent Application No. 14169441 .
Notice ofAllowance received for U.S. Appl. No. 12 / 189,030 , dated 4 , dated Jul. 7 , 2014 , 3 pages.
Jun. 5, 2017, 5 pages. Search Report received for European Patent Application No. 1621989 ,
Notice of Allowance received for U.S. Appl. No. 12 /486,710, dated dated Mar. 27 , 2006 .
Jan. 29, 2015 , 5 pages. Search Report received for European Patent Application No. 17203125 .
Notice of Allowance received for U.S. Appl. No. 12 / 890,437 , dated 4 , dated Mar. 5 , 2018 , 4 pages .
Jun. 18 , 2013 , 11 pages.
Notice of Allowance received for U.S. Appl. No. 14 /724,753 , dated Surface Acoustic Wave Touchscreens, Available online at: < http : //
Jan. 24 , 2018, 9 pages. www.touchscreens.com/intro-touchtypes-saw.html> , Accessed on Aug.
Notice of Allowance received for U.S. Appl. No. 15 /257,713 , dated 5 , 2005 .
Feb. 21 , 2019 , 7 pages. Symbol Commander, Available online at: < http://www.sensiva.com/
Notice of Allowance received for U.S. Appl. No. 15 /727,455 , dated symbolcomander/ > , Accessed on Aug. 30 , 2005 .
Apr. 12 , 2018 , 6 pages . Synaptics Touch Pad Interfacing Guide, Synaptics Inc. , Second
Notice of Allowance received for U.S. Appl. No. 15 /727,455 , dated Edition , San Jose , Mar. 25 , 1998 , pp . 1-90 .
Aug. 15 , 2018 , 5 pages. Tips for Typing, FingerWorks, Available online at: < http : //www .
Notice of Allowance received for U.S. Appl. No. 15 /970,571 , dated fingerworks.com/mini_typing.html>, Accessed on Aug. 30 , 2005 .
Apr. 5 , 2019 , 10 pages . Touch Technologies Overview , 3M Touch Systems , Massachusetts,
Notice of Allowance received for U.S. Appl. No. 16 /209,137 , dated 2001 .
Jun. 14 , 2019 , 5 pages. Wacom Components — Technology, Available online at: < http ://
Notice of Allowance received for U.S. Appl. No. 16 / 529,690 , dated www.wacom-components.com/english/tech.asp >, Accessed on Oct.
Oct. 20 , 2020, 8 pages. 10 , 2004 .
Notice ofAllowance received for U.S. Appl. No. 16 /209,137 , dated Watershed Algorithm , Available online at: < http://rsb.info.nih.gov/
Sep. 27 , 2019 , 5 pages . ij /plugins/watershed.html> , Accessed on Aug. 5 , 2005 .
Office Action received for Australian Patent Application No. Abraham et al . , U.S. Appl. No. 60 /364,400 , filed Mar. 13 , 2002 ,
2017258903 , dated Nov. 6 , 2019 , 6 Pages . titled “ Multi- button mouse " .
Office Action received for Canadian Patent Application No. 2600326 ,
dated Oct. 21 , 2019 , 5 Pages . Ahmad Subatai, " A Usable Real - Time 3D Hand Tracker ” , Proceed
Office Action received for Chinese Patent Application No. ings of the 28th Asilomar Conference on Signals , Systems and
201510846284.0 , dated May 5 , 2019 , 14 pages ( 5 pages of English Computers Part 2 ( of 2 ) , vol . 2 , Oct. 1994 , 5 pages .
Translation and 9 pages of Official copy ). Apple Computer, Inc. , “ Apple Pro Mouse ” , Apple Pro Mouse
Office Action received for European Patent Application No.09170572 . Design Innovations Product Specification, Jul. 2000 , pp . 1-11 .
3 , dated Mar. 12, 2020, 55 pages . Bang & Olufsen , “ BeoCom 6000 User Guide and Sales Training
Office Action received for Indian Patent Application No. 2266 / Brochure Cover Sheets ” , 2000 , 53 pages .
KOLNP/2013 , dated Apr. 30 , 2019 , 6 pages. Bier et al . , “ Toolglass and Magic Lenses : The See - Through Inter
Office Action received for Japanese Patent Application No. 2017 face ” , In James Kijiya, editor, Computer Graphics (SIGGRAPH '93
1

224353 , dated May 13 , 2019 , 7 pages (4 pages of English Trans Proceedings ), vol . 27 , Aug. 1993 , pp . 73-80 .
lation and 3 pages of Official Copy) . Chapweske Adam , “ PS / 2 Mouse /Keyboard Protocol ” , Available
Patent Board Decision received for U.S. Appl. No. 11 /204,873 , online at: < http://panda.cs.ndsu.nodak.edu/ -achapwes/PICmicro /
dated Dec. 14, 2015 , 10 pages. PS2 /ps2 . htm > , 1999 , 7 pages .
Patent Board Decision received for U.S. Appl. No. 12 / 184,190 , Cirque Corporation, “ OEM Touchpad Modules ” , Available online
dated Dec. 23 , 2015 , 5 pages. at: < www.glidepoint.com/sales/modules.index.shtml >, Accessed on
Patent Board Decision received for U.S. Appl. No. 12 / 189,030 , Feb. 13 , 2002, 5 pages.
dated Jun. 2, 2016, 8 pages. David Nagel , “ More Details on the New Pro Keyboard and But
Pen Touch Capacitive Touchscreens, Available online at < http : // tonLess Mouse ”, Available online at: < http://www.creativemac.com/
www.touchscreens.com/intro-touchtypes-pentouch.html>, Accessed HTM /News/07_00 /detailskeyboardmouse ,htm >, Jul. 2000 , 2 pages.
on Aug. 5 , 2005 . De Meyer Kevin , “ Buttonless and Finger Mouse : Crystal Optical
Photographs of Innovations 2000 Best of Show Award Presented at Mouse ” , Available online at: http : //www . heatseekerz . net / index .
the 2000 International CES Innovations 2000 Design & Engineering php ? page = articles & id = 19 & pagenum = 2, Feb. 14 , 2002 , 2 pages.
Showcase , Jan. 6 , 2000 , 1 page . Douglas et al . , “ The Ergonomics of Computer Pointing Devices ” ,
Pre - interview First Office Action received for U.S. Appl. No. 1997 .
15 / 257,713 , dated May 9 , 2018 , 5 pages . Dreier Troy, “ The Comfort Zone ”, Available online at http ://www .
Pre - interview First Office Action received for U.S. Appl. No. pcmag.com/print_article/0,3048,a=22807,00asp, Mar. 12 , 2002 , 6
16/403,370 , dated Feb. 20 , 2020 , 3 pages . pages.
Product Overview - ErgoCommander, Available online at: < www . Edwards Andru , “ Gear Live Review : iRiver Clix Review ” , Avail
logicad3d.com/products/ErgoCommander.htm >, Accessed on Apr. able online at: < http://www.gearlive.com/index.php/news/article/
8 , 2002 , 2 pages . gear -live - review -iriver - clix - review -713400 / > , Jul. 13 , 2006 , 5 pages .
1

Product Overview SpaceMouse Classic , Available online at: <WWW . EVB Elektronik , “ TSOP6238 IR Receiver Modules for Infrared
logicad3d.com/products/Classic.htm >, Apr. 8 , 2002 , 2 pages . Remote Control Systems”, Jan. 2004 , 1 page .
Restriction Requirement received for U.S. Appl. No. 10 /654,108 , Fiore Andrew , “ Zen Touchpad ”, Available online at: http : //instruct1.
dated Mar. 20, 2006 , 6 pages. cit.cornell.edu/courses/e476/FinalProjects/s2000/fiore/touchpad . html,
Restriction Requirement received for U.S. Appl. No. 10 /654,108 , May 2000, 6 pages.
dated May 23 , 2008 , 7 pages . Fisher et al . , “ Repetitive Motion Disorders: The Design of Optimal
Restriction Requirement received for U.S. Appl. No. 11 /966,948 , Rate - Rest Profiles ” , Human Factors, vol . 35 , No. 2 , Jun . 1993 , pp .
dated Aug. 17 , 2010 , 5 pages. 283-304 .
 US 11,360,509 B2
Page 11

( 56 ) References Cited Quek ,“ Unencumbered Gestural Interaction ” , IEEE Multimedia , vol .

3 , 1996 , pp . 36-47 .
OTHER PUBLICATIONS Radwin , " Activation Force and Travel Effects on Overexertion in
Repetitive Key Tapping”, Human Factors, vol . 39 , No. 1 , Mar. 1997 ,
Flaminio Michael , “ IntelliMouse Explorer” , Available online at: pp . 130-140.
http://www.insanely-great.com/reviews/intellimouse.html., Oct. 4 , Rekimoto et al . , “ ToolStone : Effective Use of the Physical Manipu
1999 , 4 pages. lation Vocabularies of Input Devices ” , In Proc . of UIST 2000 , 2000 .
Fukumoto et al . , " ActiveClick : Tactile Feedback for Touch Panels ” , Rekimoto J. , “ SmartSkin : An Infrastructure for Freehand Manipu
In CHI 2001 Summary, 2001 , pp . 121-122 . lation on Interactive Surfaces ” , CHI 2002 Conference Proceedings,
Fukumoto et al . , “ Body Coupled Fingering: Wireless Wearable Conference on Human Factors in Computing Systems , Minneapo
Keyboard” , CHI 97 , Mar. 1997 , pp . 147-154 . lis , vol . 4 , No. 1 , Apr. 20-25 , 2002 , pp . 113-120 .
Gadgetboy ," Point and Click with The Latest Mice ” , CNETAsia Rubine et al . , “ Programmable Finger- Tracking Instrument Control
Product Review , Available online at: < www.asia.cnet.com/reviews... lers ” , Computer Music Journal, vol . 14 , No. 1 , 1990 , pp . 26-41 .
are / gadgetboy /0,39001770,38023590,00.htm >, Accessed on Dec. 5 , Rubine Dean, “ Combining Gestures and Direct Manipulation ”,
2001 , 1 page. CHI’92 , May 3-7 , 1992 , pp . 659-660 .
Gizmodo , " Logitech's MX Air Is No Longer Vapor " , Available Rubine Dean H. , “ The Automatic Recognition of Gestures ” ,
online at: < http://www.gizmodo.com.au/2007/07/logitechs_mx_air_ CMU - CS - 91-202 , Submitted in Partial Fulfillment of the Require
is_no_longer.html> , 2007 , 2 pages. ments for the Degree of Doctor of Philosophy in Computer Science
Grevstad Eric , “ Microsoft Wireless IntelliMouse Explorer Review at Carnegie Mellon University, Dec. 1991 , 285 pages.
the Ultimate Pointing Machine” , HardwareCentral Review , Jun. 24 , Rutledge et al . , “ Force - to -Motion Functions for Pointing ” , Human
2003 . Computer Interaction — INTERACT, 1990 .
Grevstad Eric , “ Microsoft Wireless IntelliMouse Explorer Review Safran David, “ Letter re : Bang & Olufsen A / S ” , Nixon Peabody,
the Ultimate Pointing Machine”, Available online at: <http ://www . LLP with BeoCom 6000 Sales Training Brochure , May 21 , 2004 , 7
hardwarecentral.com/hardwarecentral/print/3826/1 >, Oct. 9 , 2001 , pages.
3 pages. Siracusa John , “ The Puck is Dead ( and the Little Keyboard Too ) ” ,
Hardy Ian , “ Fingerworks ” , BBC World on Line , Mar. 7 , 2002 . Available online at: < http://www.arstechnic.com/wanderdesk/3000/
Hillier et al . , “ Introduction to Operations Research ” , 1986 . macworld2k /mwny - 2.html >, Jul. 2000 , 6 pages .
Integration Associates Inc.," Proximity Sensor Demo Kit ”, User Sylvania, “ Intellivision Intelligent Television Master Component
Guide, Version 0.62 - Preliminary, Apr. 13 , 2004 , 14 pages . Service Manual ” , 1979 , pp . 1 , 2 & 8 .
Jacob et al ., “ Integrality and Separability of Input Devices ” , ACM Tessler et al . , “ Touchpads Three new input devices ” , Available
Transactions on Computer - Human Interaction , vol . 1 , Mar. 1994 , online at: < www.macworld.com/1996/02/review/1806.html> , 1996 ,
pp . 3-26 . 2 pages.
Kinkley et al . , “ Touch - Sensing Input Devices ” , CHI ’99 Proceed Texas Instruments," TSC2003 / 12C Touch Screen Controller " , Data
ings , May 1999 , pp . 223-230 . Sheet SBAS 162 , Oct. 2001 , 20 pages.
KIONX ,“ KXP84 Series Summary Data Sheet ” , Oct. 21 , 2005 , 4 Wellner Pierre, “ The Digital Desk Calculators: Tangible Manipu
pages . lation on a Desktop Display ” , In ACM UIST’91 Proceedings, Nov.
Lee et al . , “ A Multi - Touch Three Dimensional Touch -Sensitive 11-13 , 1991 , pp . 27-34 .
Tablet ” , CHI ’85 Proceedings, Apr. 1985 , pp . 121-128 . Westerman Wayne, “ Hand Tracking, Finger Identification, and
Lee et al . , “ A Multi - Touch Three Dimensional Touch - Sensitive Chordic Manipulation on a Multi- Touch Surface ” , A Dissertation
Tablet ” , CHI’85 Proceedings, Apr. 1985 , pp . 21-25 . Submitted to the Faculty of the University of Delaware in Partial
Lee S. , “ A Fast Multiple - Touch -Sensitive Input Device " , A Thesis Fulfillment of the Requirements for the Degree of Doctor of
Submitted in Conformity with the Requirements for the Degree of Philosophy in Electrical Engineering , 1999 , 363 pages.
Master of Applied Science in the Department of Electrical Engi Williams Jim , “ Applications for a Switched -Capacitor Instrumen
neering , University of Toronto , Oct. 1984 , 115 pages . tation Building Block ” , Linear Technology Application Note 3 , Jul.
Matsushita et al . , “ Holo Wall: Designing a Finger, Hand , Body and 1985 , pp . 1-16 .
Object Sensitive Wall ” , In Proceedings of UIST '97 , Oct. 1997 . Yamada et al . , “ A Switched - Capacitor Interface for Capacitive
Mattel , " System Service and Troubleshooting Manual ” , Available Pressure Sensors” , IEEE Transactions on Instrumentation and Mea
online at: < www.dsplib.com/intv/Master >, Accessed on Dec. 11 , surement, vol . 41 , No. 1 , Feb. 1992 , pp . 81-86 .
2002 , 1 page. Yeh et al ., “ Switched Capacitor Interface Circuit for Capacitive
Microsoft Inc ., " Scroll and Zoom on a Microsoft Excel Sheet by Transducers” , IEEE , 1985 .
Using the Microsoft Intellimouse pointing Device ” , Available online Zhai et al . , “ Dual Stream Input for Pointing and Scrolling ” , Pro
at : < mk : @MSITStore : C : \Program % 20Files \Microsoft % ceedings of CHI '97 Extended Abstracts, 1997 .
200ffice \Office 1033 \xlmain9.chm :: /html/xlhowScrol> , 1999 , 1 page . Zimmerman et al., “ Applying Electric Field Sensing to Human
Peter et al., Unpublished U.S. Appl. No. 10 /789,676 , filed Feb. 27 , Computer Interfaces ” , In CHI ’85 Proceedings , 1995 , pp . 280-287 .
2004 , titled “ Shape Detecting Input Device ” . Zuber, “ Der Klanomeister " , Connect Magazine , Aug. 1998 , pp .
Press Release , “ iRiver Clix Delivers Complete Package for Portable 52-53 .
Entertainment Fans ” , Available online at: < www.iriveramerican . Notice ofAllowance received for U.S. Appl. No. 16 /403,370, dated
com /images.pdf /iriv_clix.pdf >, May 17 , 2006 , 3 pages . Nov. 19 , 2020, 5 pages.
Quantum Research Group,“ QT510 /QWheel Touch Slider IC ” , 2004 9

2005 , 14 pages . * cited by examiner

U.S. Patent Jun . 14 , 2022 Sheet 1 of 13 US 11,360,509 B2

OOO 12
14 0

FIG . 1A
(PRIOR ART)
FIG . 1B
( PRIOR ART) FIG . 1C
(PRIOR ART )
12

do

12 1

FIG . 1D meton
100
( PRIOR ART )
FIG . 1E
( PRIOR ART) FIG . 1F
( PRIOR ART)
U.S. Patent Jun . 14 , 2022 Sheet 2 of 13 US 11,360,509 B2

???
V

?
-36

FIG . 2A FIG . 2B
(PRIOR ART) ( PRIOR ART)

ga
0 O

FIG . 2C
(PRIOR ART)
U.S. Patent Jun . 14 , 2022 Sheet 3 of 13 US 11,360,509 B2

52 52 U is CS
****ext

41-41%
US ***** **** -56

is
2
ti
etten
. 3
$

57
FIG . 3A FIG . 3B

**********
ti
111111* * 72 whunti++++++ 82

WWWWWW

188FIG. 3C
*****

:
Web

FIG . 3D
O
U.S. Patent Jun . 14 , 2022 Sheet 4 of 13 US 11,360,509 B2

ww

????? ????? ? 7 www

An e printen

FIG . 4
over

212
8240
230 DISPLAY
DRIVER 5260
232-1
PROCESSOR
X

*
CONTROLS
DRIVER
FG
SCAN SENSOR (S)
CIRCUIT CIRCUIT 3 1270
234J **********WW.AL
FIG . 5
U.S. Patent Jun . 14 , 2022 Sheet 5 of 13 US 11,360,509 B2

3029 DESIGNATE AREAS ON

CONNECT BEZEL FOR CONTROL
Son
3047 GENERATE VISUAL GUIDES

DISPLAY VISUAL GUIDES

3067 ADJACENT DESIGNATED AREAS
CONNECT ROW ON TOUCH BEZEL
TO STRG CAP
308 MOVDX3NQUOD
DATA FROM BEZEL
SEC ION
1314 4101
DESIGNATIONS ON BEZEL
DIGITAL VALUE
4129
3127
DONE ? IMPLICATED BY DESIGNATION

316 )
4147
INITIATE DETERMINED
PROCESS

318 ) 4167
DISPLAY ?
FIG . 6 FIG . 7
U.S. Patent Jun . 14 , 2022 Sheet 6 of 13 US 11,360,509 B2

432
START OBTAIN ORIENTATION DATA
FROM SENSOR 430
.
NO 4542ORIENTATION
CHANGED ?
4367 YES
ALTER DESIGNATION OF
FIG . 8 AREAS FOR CONTROLS
ON TOUCH BEZEL
438
ALTER LOCATION OF VISUAL
END
GUIDES ON DISPLAY
ROTATION 450
450
L
452 470 -460 C 4527 470 -460
MENU POWER MENU MENU POWER MENU

UP UP
471A
UP
456
a
-480
MENU LEFT SELECT RIGHT MENU
Z 454
454 471A -490 -480
FIG . 9A LEFT SELECT RIGT

490 -456
471B' FIG . 9B
U.S. Patent Jun . 14 , 2022 Sheet 7 of 13 US 11,360,509 B2

Ben y

an

$24
570C
$ 26

512

FIG . 10 FIG . 12
TZ

559 I

552

2
FIG . 11
U.S. Patent Jun . 14 , 2022 Sheet 8 of 13 US 11,360,509 B2

0227 6
600

from

27
no

111111
FIG . 13A
620 100 Resistor )

SNSIB

FIG . 13B
U.S. Patent Jun . 14 , 2022 Sheet 9 of 13 US 11,360,509 B2

*****

FANYWA,WYWA +7 ( ) 193?$11 Wir

THE
*
w

VES *****
130

ww
LOWL *****
720
702700 742 14
.
FIG

626
652
13C
.
FIG
650
610
,
602
U.S. Patent Jun . 14 , 2022 Sheet 10 of 13 US 11,360,509 B2

830 828

VO V V V V

Wory

ZINKPHOTS SUNSET SUNRISE CLOUDS FAMILY FREINDS

914
1912 Preview
Amal

ZZ 16
.
FIG

826
824

** )
2045
wwws // 902
15
.
FIG
MENO906
820
MUSIC PHOT S VIDEOS SET INGS EXTRAS
822
U.S. Patent Jun . 14 , 2022 Sheet 11 of 13 US 11,360,509 B2

Theword

832

hors

FIG . 17A
U.S. Patent Jun . 14 , 2022 Sheet 12 of 13 US 11,360,509 B2

ww
802 832 in

WYDDIONA
***

wer ***une*
*****

ZEZILIZZZZZAAK
F FIG . 17B

wertya NY
842

TZ
FIG . 170
U.S. Patent Jun . 14 , 2022 Sheet 13 of 13 US 11,360,509 B2

Z9Z4Z
,
D
850
820
852
,
802

935
PNOWLAYING
Night
'
the
is
Tender
Song
:
BJA:arcotkwisnoet Album
L
:
Love
ina w y e r s w*WAw*

21
(
Bass

2017$** n
870
FIG
.
19

han string mental **

22 The
,
>

,
850
820
852
802 WA
>
SONGS VIEWPTOLAYLISTS
ADD
932Amanda HotNight BSomaebody's TenderistheNight SideDarkOnthe GComraendmt CLoseMakeMeontrol
Drive
ZID FIG
18
.

932
 US 11,360,509 B2
1 2
ELECTRONIC DEVICE HAVING DISPLAY The user interface typically includes an output device in the
AND SURROUNDING TOUCH SENSITIVE form of a display, such as a Liquid Crystal Display ( LCD ) ,
SURFACES FOR USER INTERFACE AND
CONTROL
and one or more input devices, which can be mechanically
actuated ( e.g. , switches , buttons, keys, dials , joysticks, joy
CROSS - REFERENCE TO RELATED
5 pads ) or electrically activated (e.g. , touch pads or touch
APPLICATIONS
screens ) . The display is typically configured to present
visual information such as text and graphics, and the input
This is a continuation of U.S. patent application Ser. No. devices are typically configured to perform operations such
16 / 529,690 , filed Aug. 1 , 2019 , (now U.S. Publication No. 10 or selectorcommands
as issuing
of the
, making selections , or moving a cursor
electronic device . Each of these well
2019-0354215, published on Nov. 21, 2019 ) , which is a known devices has considerations such as size and shape
continuation of U.S. patent application Ser. No. 15 / 970,571 , limitations, costs , functionality, complexity , etc. that must be
filed May 3 , 2018 , (now U.S. Pat . No. 10,386,980 , issued on taken
Aug. 20 , 2019 ) , which is a continuation of U.S. patent deviceinto account when designing the hand - held electronic
. In most cases , the user interface is positioned on the
application Ser. No. 14/ 724,753 , filed May 28 , 2015 ( now
2

U.S. Pat. No. 9,983,742 , issued on May 29, 2018 ) , which is 15 viewing
front faceof(orthefront surface
display and) ofeasy
the manipulation
hand -held device
of theforinput
easy
a continuation of U.S. patent application Ser. No. 12/486 , devices.
710 , filed Jun . 17 , 2009 (now U.S. Pat. No. 9,047,009 , issued
Jun . 2 , 2015 ) , which is a continuation of U.S. patent appli- FIGS . 1A - 1F are diagrams of various hand -held elec
cation Ser. No. 11 /426,078 , filed Jun . 23 , 2006 , (now U.S. tronic devices including for example a telephone 10A ( FIG .
Pat . No. 7,656,393 , issued on Feb. 2 , 2010 ) , which is a 20 1A) , a PDA 10B (FIG . 1B ) , a media player 10C (FIG . 1C ) ,
continuation -in -part of U.S. patent application Ser. No. a remote control 10D (FIG . 1D ) , a camera 10E (FIG . 1E ) ,
11 / 367,749 , filed Mar. 3 , 2006 (U.S. Publication No. 2006- and a Global Positioning System (GPS ) module 10F (FIG .
0197753 , published on Sep. 7 , 2006 ) , which claims priority 1F ) . In each of these devices 10 , a display 12 , which is
to U.S. Provisional Patent Application No. 60 /658,777, filed secured inside the housing of the device 10 and which can
Mar. 4 , 2005 and U.S. Provisional Patent Application No. 25
60/ 663,345 , filed Mar. 16 , 2005 , the entire disclosures of
be seen through an opening in the housing, is typically
which are incorporated by reference herein in their entirety positioned in a first region of the electronic device 10. Each
for all purposes. of these devices is also include one or more input devices 14 ,
This application is also related to the following applica which are typically positioned in a second region of the
tions , which are all herein incorporated by reference in their electronic device 10 next to the display 12 .
entirety for all purposes: ( 1 ) U.S. patent application Ser. No. 30 To elaborate , the telephone 10A typically includes a
10/ 188,182 , entitled “ Touch Pad for Handheld Device , ” filed display 12 such as a character or graphical display, and input
on Jul . 1 , 2002 , ( now U.S. Pat. No. 7,046,230 , issued on devices 14 such as a number pad and in some cases a
May 16 , 2016 ) ; (2 ) U.S. patent application Ser. No. 10/722, navigation pad . The PDA 10B typically includes a display
948 , entitled “ Touch Pad for Handheld Device , " filed on 12 such as a graphical display, and input devices 14 such as
Nov. 25 , 2003 , ( now U.S. Pat . No. 7,495,659 , issued on Feb. 35 a stylus based resistive touch screen and buttons. The media
24 , 2009 ); ( 3 ) U.S. patent application Ser. No. 10 /643,256 , player 10C typically includes a display 12 such as a char
entitled “ Movable Touch Pad with Added Functionality,” acter or graphic display and input devices 14 such as buttons
filed on Aug. 18 , 2003 , (now U.S. Pat . No. 7,499,040 , issued or wheels . The iPod® media player manufactured by Apple
on Mar. 3 , 2009 ) ; ( 4 ) U.S. patent application Ser. No. Computer, Inc. of Cupertino, Calif. is one example of a
10/ 654,108 , entitled “ Ambidextrous Mouse,” filed on Sep. 40 media player that includes both a display and input devices
2 , 2003 , (now U.S. Pat . No. 7,808,479 , issued on Oct. 5 , disposed next to the display. The remote control 10D typi
2010 ) ; ( 5 ) U.S. patent application Ser. No. 10/ 840,862 , cally includes an input device 14 such as a keypad and may
entitled “Multipoint Touch Screen , ” filed on May 6 , 2004 or may not have a character display 12. The camera 10E
( now U.S. Pat . No. 7,663,607 , issued on Feb. 16 , 2010 ) ; ( 6 ) typically includes a display 12 such as a graphic display and
U.S. patent application Ser. No. 10/903,964, Untitled “Ges-
tures for Touch Sensitive Input Devices , ” filed on Jul. 30 ,
45 input devices 14 such as buttons . The GPS module 10F
2004 , ( now U.S. Pat . No. 8,479,122 , issued on Jul . 2 , 2013 ) ;
typically includes a display 12 such as graphic display and
( 7 ) U.S. patent application Ser. No. 11 / 038,590 , entitled input devices 14 such as buttons, and in some cases a joy
pad.
“ Mode - Based Graphical User Interfaces for Touch Sensitive Such prior art devices 10A - 10F often employ a user
Input Devices , ” filed on Jan. 18 , 2005 , ( now U.S. Pat . No. 50 interface in conjunction with the display 12 and input device
8,239,784 , issued on Aug. 7 , 2012 ) ; and ( 8 ) U.S. patent 14. In one example, FIG . 2A shows an electronic device 20 ,
application Ser. No. 11 / 057,050 , entitled “ Display Actua such as a portable media player. The electronic device 20 has
tor,” filed on Feb. 11 , 2005 , (now abandoned ); ( 9 ) U.S.
patent application Ser. No. 11 / 115,539 , entitled “ Hand -Held a display 24 and an input device 26 according to the prior art.
Electronic Device with Multiple Touch Sensing Devices , ” The display 24 can show various forms of information ( e.g. ,
filed Apr. 26 , 2005 , (now U.S.Pat .No. 7,800,592 , issued on 55 menu items , song titles stored in memory, etc.) of a user
Sep. 21 , 2010 ) . interface . The display 24 and input device 26 used in
conjunction with the user interface allows the user to make
FIELD OF THE DISCLOSURE selections ( e.g. , select a song) , to operate functions of the
device (e.g. , play, stop , or pause a song , etc. ) , and to perform
The subject matter of the present disclosure relates to an 60 other functions . In this device 20 , the input devices 26 is a
electronic device having a display and a surrounding touch “ click wheel , ” such as used on an iPod® media player
>

sensitive bezel for user interface and control. manufactured by Apple Computer, Inc. of Cupertino, Calif.
The electronic device 20 has a housing 22 that contains
BACKGROUND OF THE DISCLOSURE the display 24 and the input device 26. The input device 26
65 typically requires a number of components , such as pressure
There exist today many types of hand - held electronic pads , printed circuit board , integrated circuits, etc. Accord
devices, each of which utilizes some sort of user interface . ingly, the housing 22 for the electronic device 20 must
 US 11,360,509 B2
3 4
typically be extended or enlarged beyond the size of the Thus, it is a challenge to design a device with limited
display 24 so that the electronic device 20 can accommodate input mechanisms without adversely affecting the numerous
the components of the input device 26. Consequently , due to possible functions that the device can perform . As will be
the required components for the input device 26 , the size of appreciated , it is preferable not to overload the electronic
the housing 22 may in some cases be larger than is actually 5 devices with a large number of input mechanisms as this
required to house just the display 24 and any other necessary tends to confuse the user and to take up valuable space , i.e. ,
components ( i.e. , processor, memory, power supply, etc. ) for “ real estate . ” In the case of hand -held devices, space is at a
>

the device 20. In addition, placement of the display 24 and premium because of their small size . At some point, there is
the input device 26 typically accommodate only one orien- not enough space on the device to house all the necessary
tation of the device 20 when held by a user. 10 buttons and switches , etc. This is especially true when
In another example, FIG . 2B shows another electronic considering that all these devices need a display that typi
cally takes up a large amount of space on its own . To
device 30 having a display 34 and an input device 36 increase
according to the prior art. The electronic device 30 can be a designersthewould number of input devices beyond some level ,
laptop computer or the like, and the input device 36 can be 15 However ,this willhave oftentoleave
decrease the size of the display.
a negative impression on the
a touch pad used to control functions of the device 30 , such user because the user typically desires the largest display
as moving a cursor, making selections , etc. The touch pad 36 possible . Alternatively, to accommodate more input devices
is positioned on a housing 32 of the device 30 in conjunction designers may opt to increase the size of the device. This ,
with conventional components of a keyboard 38 and other too , will often leave a negative impression on a user because
physical inputs. The touch pad 36 can be categorized as 20 it would make one-handed operations difficult, and at some
either “ resistive ” or “ capacitive . ” In the resistive category, point, the size of the device becomes so large that it is no
the touch pad 36 is coated with a thin metallic electrically
a
longer considered a hand -held device .
conductive layer and aa resistive layer. When the touch pad Therefore, what is needed in the art is an improved user
36 is touched, the conductive layers come into contact interface that works for multi- functional hand -held devices .
through the resistive layer causing a change in resistance 25
( typically measured as a change in current) that is used to SUMMARY OF THE DISCLOSURE
identify where on the touch pad 36 the touch event occurred .
In the capacitive category, a first set of conductive traces run An electronic device has a display and has a touch
in a first direction on the touch pad 36 and are insulated by sensitive bezel surrounding the display. Areas on the bezel
aa dielectric insulator from a second set of conductive traces 30 are designated for controls used to operate the electronic
running in a second direction ( generally orthogonal to the device . Visual guides corresponding to the controls are
first direction) on the touch pad 36. The grid formed by the displayed on the display adjacent the areas of the bezel
overlapping conductive trace creates an array of capacitors designated for the controls. Touch data is generated by the
that can store electrical charge. When an object (e.g. , a user's bezel when a user touches an area of the bezel . The device
finger) is brought into proximity or contact with the touch 35 determines which of the controls has been selected based on
pad 36 , the capacitance of the capacitors at that location which designated area is associated with the touch data from
changes. This change can then be used to identify the the bezel . The device then initiates the determined control.
location of the touch event. The device can also have a sensor for determining the
In yet another example, FIG . 2C illustrates an electronic orientation of the device. Based on the orientation, the
device 40 having a touch screen display 44 according to the 40 device can alter the areas designated on the bezel for the
prior art as an input device . The electronic device 40 is a controls and can alter the location of the visual guides for the
Personal Digital Assistant or the like. The touch screen display so that they match the altered areas on the bezel if
display 44 is positioned on a housing 42 , and the electronic the orientation of the device has changed .
device 40 typically has some physical controls 46 on the The foregoing summary is not intended to summarize
housing 42. A stylus 48 is used to touch locations of the 45 each potential embodiment or every aspect of the present
touch screen display 44 to perform various functions. The disclosure.
stylus 48 is typically used like a mouse and arrow , and the
display 44 can show various menu items and other user BRIEF DESCRIPTION OF THE DRAWINGS
interface features. Touching a menu item on the display 44
with the stylus 48 can generate a pop -up or window 45 in 50 The foregoing summary, preferred embodiments, and
which the user can then make a selection with the stylus 48 . other aspects of subject matter of the present disclosure will
The pop -ups or windows 45 overlay the content being be best understood with reference to a detailed description of
displayed and tend to obscure it . specific embodiments, which follows, when read in con
Recently, traditionally separate hand - held electronic junction with the accompanying drawings, in which :
devices have begun to be combined in limited ways. For 55 FIGS . 1A - 1F are diagrams of various electronic devices
example , the functionalities of a telephone have been com- according to the prior art.
bined with the functionalities of a PDA . One problem that FIG . 2A illustrates an electronic device having a display
has been encountered is in the way inputs are made into the and user controls according to the prior art .
device. Each of these devices has a particular set of input FIG . 2B illustrates an electronic device having a display
mechanisms or devices for providing inputs into the device . 60 and a touch pad according to the prior art .
Some of these input mechanisms are generic to all the FIG . 2C illustrates an electronic device having a touch
devices (e.g. , power button ) while others are not. The ones screen display according to the prior art.
that are not generic are typically dedicated to a particular FIG . 3A is a perspective view of a substantially full screen
functionality of the device . By way of example, PDAs hand -held device with a limited number of buttons accord
typically include a touch screen and a few dedicated buttons 65 ing to certain teachings of the present disclosure .
while cell phones typically include a numeric keypad and at FIG . 3B is a front view of the hand - held device of FIG .
least two dedicated buttons. 3A with at least one button .
 US 11,360,509 B2
5 6
FIG . 3C is a diagram of a graphical user interface sepa- allows the electronic device to house a substantially full
rated into a standard region and a control region for the screen display. As used herein , a full screen display is a
electronic device of FIG . 3A . display that consumes , or at least dominates, a surface ( e.g. ,
FIG . 3D illustrates an exemplary media player interface a front surface ) of the electronic device .
for the electronic device of FIG . 3A . 5 FIG . 3A is a perspective view of a multi- functional
FIG . 4 illustrates an embodiment of an electronic device hand - held device 50 having a housing 52 and a substantially
having a display and a touch sensitive bezel according to full screen display 60. To accommodate the full screen
certain teachings of the present disclosure . display 60 , the multi - functional hand -held device 50 is
FIG . 5 schematically illustrates components of the elec- preferably configured with a limited number of physical
tronic device of FIG . 4 . 10 buttons. Because a limited number of physical buttons are
FIG . 6 illustrates a process of operating the touch sensi- provided , the display 60 of the hand - held device 50 prefer
tive bezel in flow chart form . ably uses a touch screen as the primary input mechanism for
FIG . 7 illustrates a process of operating the electronic the electronic device 50. The touch screen of the display 60
device in flow chart form . is a transparent touch sensing mechanism that is positioned
FIG . 8 illustrates a process of operating the electronic 15 over or incorporated into the display 60. Typically, the touch
device having an orientation sensor in flow chart form . screen display 60 works in conjunction with a graphical user
FIGS . 9A - 9B illustrate an electronic device with an interface (GUI) presented on the display 60. For example,
orientation sensor in two different orientations. the GUI may present an on- screen button or user control on
FIG . 10 illustrates an embodiment of an electronic device the display 60 , and the touch screen display 60 may detect
9

capable of disregarding certain types of touch data . 20 when a user presses the on - screen button (e.g. , places their
FIG . 11 illustrates an embodiment of an electronic device finger or stylus over the on - screen button ). Aspects of the
having a touch sensitive bezel around the display and having touch screen display 60 and GUI for the electronic device 50
addition touch sensitive pads incorporated throughout vari- are described in greater detail below.
ous sides of the housing for the device . The hand - held device 50 may be constructed with only
FIG . 12 illustrates some other possible bezel arrange-
ments for an electronic device according to the present
25 cross -functional physical buttons, i.e., there are no buttons
dedicated to individual device functionalities. These types of
disclosure. buttons may include power buttons and hold switches. In
FIG . 13A illustrates an embodiment of aa touch sensitive another embodiment, the hand - held device 50 may not
bezel having a plurality of conductive pads , a control include any physical buttons at all . In some embodiments,
module, and sensors according to certain teachings of the 30 the physical buttons are limited to only the sides 56 and back
present disclosure, and FIG . 13C is an alternative embodi- surface 58 of the hand - held device 50. In other embodi
ment of FIG . 13A . ments, the physical buttons of the handheld device 50 are
FIG . 13B illustrates a circuit diagram of portion of the limited to the upper and lower portion of the sides 56 so that
control module for the touch sensitive bezel of FIG . 13A . there are no buttons in the regions of the sides 56 where a
FIG . 14 illustrates an embodiment of a touch sensitive 35 user would physically support the device 50 (i.e. , holding
bezel having force detection and location layers according to region ). In still other embodiments, the physical buttons may
certain teachings of the present disclosure . be located on the front surface 54 , but only in the bezel 55
2

FIGS . 15 through 19 illustrate an embodiment of a surrounding the display 60. In some embodiments , the
multimedia device having a touch sensitive bezel and a user buttons may be located on only the top and bottom surfaces
interface according to certain teachings of the present dis- 40 57 of the device 50 .
closure . As shown in the embodiment of FIG . 3A , there are no
While the subject matter of the present disclosure is physical buttons on the front surface 54 of the housing 52 so
susceptible to various modifications and alternative forms, that the front surface 54 can be used almost entirely for the
specific embodiments thereof have been shown by way of display 60. Further, because the side surfaces 56 are used for
example in the drawings and are herein described in detail. 45 grasping the device 50 , it may be preferred to leave the sides
The figures and written description are not intended to limit surfaces 56 free from buttons to prevent accidental actions
the scope of the inventive concepts in any manner . Rather, in the event a user inadvertently presses a button while
the figures and written description are provided to illustrate supporting the device 50. Although the top and bottom
the inventive concepts to a person skilled in the art by surfaces 57 would not typically be used to hold the device
reference to particular embodiments, as required by 35 50 50 , these surfaces 57 are not ideal locations for buttons that
2

U.S.C. 112 . are often actuated because it would be awkward to reach

these buttons when operating the device 50 with one hand .
DETAILED DESCRIPTION Instead , the top surface 57 may be reserved for buttons that
have limited action and generic functions that are cross
Co -pending U.S. patent application Ser. No. 11 /426,078 , 55 functional, for example, power and hold switches . The top
which has been incorporated herein by reference in its and bottom surfaces 57 are also well suited for placement of
entirety, discloses electronic devices capable of configuring I/ O and communication ports. The top surface 57 may, for
user inputs based on how the devices are to be used . The example , include a headset /microphone jack and an antenna ,
electronic devices may be multi - functional hand - held and the bottom surface 57 may include power and data ports.
devices. The electronic devices have a user interface that 60 In some cases , it may be desirable to place buttons in the
requires no (or at most only a few ) physical buttons, keys , upper or lower regions of the side surfaces 56 out of the way
or switches so that the display size of the electronic of the grasping hand of the user. This may be particularly
devices can be substantially increased. Preferably, the well suited when the housing 52 of the device 50 is elon
electronic devices eliminate such physical buttons, keys, or gated more than the standard width of a user's grasping
switches from a front surface of the electronic device so that 65 hand . As shown in FIG . 3B , the hand -held device 50
additional surface area becomes available for a larger dis- includes a button 53 in the upper region on the right side
play on the electronic device. Ultimately, this strategy surface 54 of the housing 52. Because the button 53 is in the
 US 11,360,509 B2
7 8
upper region, it tends to be out of the way of the grasping touch sensitive surfaces may be located on the sides 56 or
hand and therefore accidental activation is substantially back surface 58 of the housing 52 and may even be located
eliminated . In one embodiment, the upper button 53 may be at the bezel (55 ; FIGS . 3A - 3B ) located at the front surface
configured to switch the functionality of the multi - functional 54 of the housing 52. In all of these cases , a large portion of
device 50. For example, by pressing the button 53 , a new 5 the front surface 54 of the housing 52 is saved for the display
device functionality is activated , and the current device 60 so that the viewing area of the hand - held electronic
functionality is deactivated . Although the term “ button ” is device 50 can be maximized .
used , it should be appreciated that the button 53 may The touch sensitive surfaces of the housing 52 may take
correspond to a dial , wheel , switch , or the like. the form of one or more touch panels that are positioned
As discussed above , the touch screen display 60 typically 10 within the housing 52. The touch sensitive surfaces may be
works in conjunction with a GUI presented on the display alternatively or additionally be provided directly by the
60. The GUI shows user controls on the touch screen display housing 52. That is , the touch sensing components of the
60 , which in turn responds to user touches made in regions touch sensitive surfaces may be integrated into , incorporated
of the touch screen display 60 corresponding to the dis
played user controls. The entire touch screen display 60 or 15 into , or disposed underneath the housing 52 such that the
only a portion may be used to show the user controls. housing 52 itself is touch sensitive and forms part of the
Referring to FIG . 3C , for example, a GUI 70 for the touch sensitive surfaces ( rather than using a separate touch
electronic device 50 of FIG . 3A is separated into a standard panel ). Similar to a touch screen , such touch sensitive
region 72 and a control region 74 on the touch screen display surfaces recognize touches and the positions of the touches
60. The standard region 72. represents what would normally 20 on the surfaces. The electronic device 50 has circuitry ( not
be displayed on the display 60 when using the electronic shown ), which can include a controller or the like, and the
device 50. That is , any standard GUI screens associated with circuitry interprets the touches and thereafter performs
the device 50 are displayed in the standard region 72. For actions based on the touch events . Touch sensitive surfaces
example , when the device 50 is operated with a PDA can be constructed in the same manner as a touch screen ,
functionality, a main menu (window with a set of icons ) , 25 except the surfaces need not be substantially transparent. By
calendar, address book or date book may be displayed in the way of example , the touch sensitive surfaces for the elec
standard region 72 . tronic device 50 may generally correspond to the touch
On the other hand , the control region 74 virtually repre- sensitive housing described in detail in U.S. patent applica
sents those physical controls 76 that would normally be tion Ser. No. 11 / 115,539 , entitled " Hand - Held Electronic
physically placed on a particular type of electronic device . 30 Device with Multiple Touch Sensing Devices , ” filed Apr. 26 ,
That is , the virtual controls 76 displayed in the control 2005 , which has been incorporated herein by reference in its
region 74 essentially mimic physical controls for a particular entirety.
type of device . For example, when the device 50 is operated Having a display 60 that encompasses almost the entire
with a PDA functionality, the control region 74 may include front surface 54 of the housing 52 of the electronic device 50
virtual representations of aa hand writing recognition area, a 35 has several advantages discussed herein . In addition , having
navigation pad, and the standard function buttons. The one or more touch sensitive surfaces on various portions of
standard and control regions 72 and 74 can be positioned at the housing 52 that allows a user to control the electronic
any position on the display 60 ( top, bottom , sides, center, device 50 can also provide several advantages discussed
etc. ) . For example, as shown in FIG . 3C , the control regions herein . As alluded to above, one or more touch sensitive
72 and 74 may be positioned vertically relative to one 40 surfaces can be located on the bezel 55 (i.e. , the portion of
another (one on top of the other ) on the display 60 . the front surface 54 of the housing 52 that surrounds the
In another example, FIG . 3D is a diagram of a GUI 80 that display 60 ) . Turning then to FIG . 4 , an embodiment of an
can be used with the electronic device 50 when operated in electronic device 100 having a touch sensitive bezel 120
a music player functionality. Again, the GUI 80 is divided according to certain teachings of the present disclosure is
into a standard region 82 and a control region 84. Located 45 illustrated . As will be discussed in more detail below, the
inside the control region 84 are a virtual scroll wheel 86 and touch sensitive bezel 120 is used for user interface and
five virtual buttons 88. Additional details on a virtual scroll control of the electronic device 100 and is used in conjunc
wheel 86 are provided in U.S. patent application Ser. No. tion with a graphical user interface operating on the device
11 / 038,590 , entitled “ Mode - Based Graphical User Inter- 100 .
faces for Touch Sensitive Input Devices,” filed on Jan. 18 , 50 The electronic device 100 includes a housing 102 and a
2005 , which has been incorporated herein by reference . display 110. The housing 102 holds the display 110 , which
In the embodiments of FIGS . 3A - 3D , the electronic can be any conventional display known and used in the art
device 50 includes the touch screen display 60. In additional for electronic devices. Some common examples for the
embodiments, the electronic device 50 according to certain display 110 include a Liquid Crystal display ( LCD ) , an
teachings of the present disclosure may incorporate one or 55 electroluminescent display, and aa touch screen display. The
more touch sensitive surfaces (not shown) on the housing 52 housing 102 also holds the touch sensitive bezel 120 , which
of the electronic device 50 itself. These touch sensitive is positioned substantially around the perimeter of the dis
surfaces (not shown) can provide a large surface for tracking play 110. (In the present embodiment, the bezel 120 is
touch inputs or can provide small -dedicated areas, such as positioned entirely around the perimeter of the display 110
touch buttons, for performing dedicated functions . Further- 60 so that the bezel 120 essentially frames the display 110. ) The
more , the one or more touch sensitive surfaces can be used housing 102 of the electronic device 100 also contains
in addition to or in the alternative to the touch screen display electronic components that provide a number of operations
60 discussed in the embodiment of FIGS . 3A - 3D . and features, such as memory access , communications,
The touch sensitive surfaces may be located on any sound, power, etc. In addition, the electronic device 100
surface of the housing 52 , any side of the housing 52 , any 65 houses electronic components (discussed in more detail
portion of any side of the housing 52 , or at dedicated below ) that are used to control operation of the display 110
locations on the surface of the housing 52. For example , the and the bezel 120 .
 US 11,360,509 B2
9 10
In one example, the electronic device 100 can be a picture icons , menu items, pictures, or words ( referred to herein as
frame having memory for storing digital pictures and for " visual guides ” ) 180 on the display 110. The visual guides
viewing on the display 110. In another example, the elec- 180 correspond to the user controls designated for areas or
tronic device 100 can be a digital media device having the locations of the bezel 120 and are shown on the display 110
display 110 , the touch sensitive bezel 120 , and lacking most 5 adjacent designated areas on the bezel 120. By way of
or all buttons or similar physical controls on the housing 52 . example, the visual guides 180 in FIG . 4 include “ Menu , " >

In other examples, the electronic device 100 can be an “ Power , ” “ Left,” “ Select,” “ Right , ” etc. It will be appreci
electronic game , a personal digital assistant, a multimedia ated that the visual guides 180 and their arrangement
device , a cellular telephone, a portable video player, a depicted in FIG . 4 are only exemplary. More or fewer visual
portable navigation device, or the like . 10 guides 180 may be provided, and the size of the visual
The bezel 120 is touch sensitive and is used to obtain guides 180 can be larger or smaller than the scale depicted
touch data from the user in response to touch events made in FIG . 4 .
by the user on the bezel 120. The electronic device 100 uses As shown in FIG . 4 , the visual guides 180 are preferably
the touch data obtained with the bezel 120 to perform located near the perimeter of the display 110 so that the
various operations and functions related to user interface and 15 majority of the display 110 can be dedicated to showing
user control of the device 100. For example, the touch data content or the like . Preferably and as shown in FIG . 4 , the
obtained with the bezel 120 can control what is displayed visual guides 180 are superimposed over visual data ( e.g. ,
with the device 100 , what files are played , what the volume content, text, a picture , video , etc. ) shown on the display
level is , what the settings for the display 110 are, etc. 110. The visual guides 180 can be displayed consistently on
A number of techniques can be used to obtain touch data 20 the display 110 while the electronic device 100 is operating.
with the touch sensitive bezel 120. In one embodiment, at Alternatively, the visual guides 180 may not be shown in
least a portion of the bezel 120 includes a multi - touch input most circumstances during operation and can be displayed
surface capable of generating touch data for a plurality of only after the user touches a predetermined portion of the
touch events made by the user simultaneously at different bezel 120 or makes some user configured preference, or
locations of the bezel 120. For example, the bezel 120 can 25 moves the device 100 .
include a capacitive sensor array and data acquisition cir- During operation, the user can touch designated areas
cuitry for detecting when a user touches areas or location s (e.g. , outlined area 121) on the bezel 120 to initiate user
on the bezel 120. The capacitive sensor array and data controls for the electronic device 100. Some examples of
acquisition circuitry can be similar to those disclosed in U.S. possible user controls include menu operations, cursor
patent application Ser. No. 10 / 949,060, filed Sep. 24 , 2004 30 operations, and data entry operations. The user interface
and entitled “ Raw Data Track Pad Device and System , ” software operating on the display 110 shows the visual
which is incorporated herein by reference in its entirety. An guides 180 in positions adjacent the areas 121 on the bezel
example of such an embodiment for the bezel 120 is 120 designated to perform the user controls so the user may
discussed below with reference to FIGS . 5 and 6. During know the general area of the bezel 120 designated for the
operation, the multi- touch input surface of the bezel 120 can 35 corresponding user control indicted by the adjacent visual
be used to generate touch data that can be recognized as guide 180. The designated areas 121 can be arbitrarily
touch gestures discussed in more detail later. positioned and sized around the bezel 120 depending on the
In another embodiment, at least a portion of the bezel 120 context or content of what is being displayed. The number
includes a plurality of resistive or capacitive sensors and an of distinct areas 121 that can be designated depends on the
integrated circuit for analyzing resistive or capacitive values 40 size of the display 110 and the bezel 120 and depends on
caused by a user touching the bezel 120. An example of an what type of touch sensitive sensors are used for the touch
embodiment for such a bezel 120 is discussed below with sensitive bezel 120. In one example, one edge of the bezel
reference to FIGS . 13A and 13B . In yet another embodi- 120 that is about 4 to 5 -inches in length may accommodate
ment, at least a portion of the bezel 120 includes force about one - hundred distinct areas that can be designated for
detection layers as disclosed in U.S. patent application Ser. 45 user controls.
No. 11 /278,080 , filed Mar. 30 , 2006 and entitled “ Force In aa further embodiment, the electronic device 100 may be
Imaging Input Device and System ,” which is incorporated capable of rotation and may have an orientation sensor
herein by reference in its entirety. An example of such an (discussed in more detail below ) for determining the orien
embodiment for the bezel 120 is discussed below with tation of the device 100. Based on the sensed orientation , the
reference to FIG . 14 . 50 areas 121 on the bezel 120 designated for the user controls
During operation of the device 100, areas or location s of can be altered or relocated to match the current orientation
the bezel 120 are designated. for various user controls of the of the device 100. Likewise , the user interface software
device 100. In one embodiment, particular user controls operating on the device 100 can alter the location of the
designated for areas of the bezel 120 may be indicated visual guides 180 to match the current position of the areas
directly on the bezel 120 itself using graphics, words , or the 55 121 on the bezel 120 designated for the user controls.
like . In such an embodiment, the user controls /having indi- Now that details related the electronic device 100 , display
cations directly on the bezel 120 may be fixed and may be 110 , and bezel 120 for user interface and control have been
those user controls that a user would typically use when discussed above in FIG . 4 , we now turn to a more detailed
operating the device 100 in any of the possible modes or discussion of components for an electronic device of the
functionalities of the device 100. In another embodiment, 60 present disclosure .
particular user controls designated for areas of the bezel 120 Referring to FIG . 5 , components of an embodiment of an
may not have any visual indications appearing directly on electronic device 200 according to certain teachings of the
the bezel 120 itself. Instead , the designated user controls present disclosure are schematically illustrated . Some of the
may be in a logical or predetermined location on the bezel components of the electronic device 200 are not shown in
120 that the user may know or expect . 65 FIG . 5 but will be apparent to one skilled in the art . For
In yet another embodiment, the electronic device 100 has example, the electronic device 200 may include aa housing
user interface software or an application for displaying and conventional components, such as power circuitry, a
 US 11,360,509 B2
11 12
central processing unit, memory, and the like . In one sequential. taps made at substantially the same location of
example of a conventional component, for instance, the the array 220 within predefined intervals of one another,
memory can store data , software , etc. and can include touch events occurring substantially simultaneously at two
random access memory, read -only memory, or both . or more location sS of the array 220 , sliding touches of one
The electronic device 200 includes a housing (not shown ), 5 or more fingers by the user over the surface of the array 220 ,
a display 210 , display circuitry 212 , a capacitive sensor sustained touch at one location of the array 220 in conjunc
array 220 , data acquisition circuitry 230 , and processing tion with sliding or tapping touches at other location Ss of the
circuitry 240. The display 210 is positioned on the housing array 220 , and other combinations of the above.
( not shown ) and has a perimeter. The capacitive sensor array To recognize such “ touch gestures,” one or more areas
220 is alsosubstantially
positioned positioned onaround
the housing (not shown
the perimeter of the) display
and is 10 221 of the array 220 are associated with a control 262, and
210 so that the capacitive sensor array 220 forms part of a touch gestures involving one or more touches on those areas
bezel for the display 210. The data acquisition circuitry 230 can be aassociated
221 are with the control 262. The touch gesture
is coupled to the capacitive sensor array 220 and is used to more sequential taps , a sweeptap of
single momentary , a sustained touch , two or
a finger,” and any other
acquire touch data from the array 220. The processing 15
circuitry 240 is coupled to the data acquisition circuitry 230 possible touch arrangement. To then determine if the control
and to the display 210 . 262 has been invoked , the processing circuitry 240 deter
As will be explained in more detail below, the processing mines if the touch data includes those areas 221 associated
circuitry 240 is configured to obtain touch data from the data with the control 262 and determines from the touch data if
acquisition circuitry 230 , determine if at least one user 20 the touch gesture associated with the control 262 has
control is invoked by the obtained touch data, and initiate at occurred on those areas 221 .
least one operation for the electronic device 200 based on the Turning from discussion of the capacitive sensor array
determined user control. The processing circuitry 240 220 , the processing circuitry 240 is also operatively con
includes a processor 250 and includes one or more software / nected to the display 210 by display circuitry 212. The
firmware components that operate on the processor 250. 25 display driver 251 is used to configure visual data ( e.g. ,
These components include a display driver 251 , a sensor content, screens, user interface elements , visual guides, etc.)
driver 253 , and system and /or user applications 260 . and to send or present the visual data to the display circuitry
The applications 260 have one or more user controls 262 212. The electronic device 200 preferably presents one or
that a user can invoke by touching one or more areas of the more visual guides 280 along the perimeter of the display
capacitive sensor array 220 in order to change or control 30 210. In addition , the one or more visual guides 280 are
operation of the electronic device 200. To determine which preferably displayed at location s on the display 210 adja
user control 262 is invoked , the processing circuitry 240 cent to corresponding areas 221 of the capacitive sensor
designates one or more areas 221 of the capacitive sensor array 220 designated for the one or more user controls 262
array 220 for the one or more user controls 262 of the associated with the one or more visual guides 280 .
applications 260. Then , when a user touches one or more 35 Given the overview of the electronic device 200 discussed
areas 221 of the capacitive sensor array 220 , the data above , we now turn to a more detailed discussion of the
acquisition circuitry 230 provides touch data to the process- components of the electronic device 200 of the present
ing circuitry 240. The capacitive sensor array 220 and data embodiment. The capacitive sensor array 220 includes a
acquisition circuitry 230 is preferably capable of generating plurality of rows and columns of capacitive sensors , and the
touch data that describes more than one simultaneously 40 array 220 may or may not be symmetrical. The rows and
touched areas 221 on the bezel 120 so that the touch data can columns of the array 220 are positioned around the perim
cover instances when the user touches one area only, touches eter of the display 210 .
more than one area simultaneously, or makes a pattern or The data acquisition circuit 230 includes multiplexer
gesture of touches on the array 220 . (“ MUX ” ) circuitry coupled to the sensor array 220. In
In turn , the processing circuitry 240 compares the 45 particular, two multiplexer circuits 232-1 and 232-2 (re
obtained touch data to the one or more designated areas and ferred to as the MUX - 1 and MUX - 2 circuits ) are coupled to
determines which of the user controls 262 has been invoked the rows of the sensor array 220. Each row in the sensor
by the user. The comparison of obtained touch data to the array 220 can be electrically coupled to aa reference voltage
designated areas 221 may involve different levels of pro- Vcc through the MUX - 1 circuit 232-1 and can be electri
cessing . In one level of processing , the processing circuitry 50 cally coupled to a storage capacitor 236 through the MUX - 2
240 compares the location ( e.g. , rows and columns ) that the circuit 232-2 . While not shown in detail , each column of
obtained touch data occurred on the array 220 to the desig- sensor array 220 can be similarly coupled to a reference
nated areas 221 for the user controls. If the obtained touch voltage Vcc and to a storage capacitor using column MUX
data occurs in the designated area 221 for aa satisfactory time circuits 234. Thus, a user touching a location or area 221 of
period or over an average extent of the area 221 , for 55 the sensor array 220 can alter the capacitance measured at
example , then the processing circuitry 240 determines that affected rows and columns of the array 220 .
the corresponding user control has been invoked . During operation, the MUX circuits 232 and 234 are
In other levels of processing, the obtained touch data can responsible for coupling and stimulating successive ele
include one or more location s (e.g. , rows and columns) ments of the sensor array 220 (e.g. , rows , columns, or
being touched on the array 220 , can include touch data 60 individual pixels — that is , an element at the intersection of
obtained over an interval of time , can include changes in a row and column ) to the storage capacitor 236 in a
touch data over time , and can include other “ aggregate ” controlled /sequenced manner and indicating that a measure
forms of touch data . In this level of processing , the process- ment cycle has begun to the scan circuit 238. When the
ing circuitry 240 recognizes “ touch gestures ” from the touch charge on storage capacitor 236 reaches a specified value or
data and determines which control is invoked by the “ touch 65 threshold, the scan circuit 238 records the time required to
gesture.” Some examples of “ touch gestures” include a charge the storage capacitor 236 to the specified threshold .
single “ tap ” at a location of the array 220 , two or more Consequently, the scan circuit 238 provides a digital value
 US 11,360,509 B2
13 14
that is a direct indication of the capacitance of the selected of time , or until the voltage on storage capacitor 236 reaches
a
element of the sensor array 220 . a specified threshold ( Block 306 ) .
The sensor driver 240 obtains measured capacitance data While the MUX - 2 circuitry 232-2 couples the selected
from the acquisition circuitry 230. In turn , the sensor driver sensor row to the storage capacitor 236 , a determination is
240 processes the measured capacitance data and configures 5 made whether the storage capacitor's voltage reaches a
a corresponding control, command, operation, or other func specified threshold ( Block 308 ) . If so (i.e. , the “ Yes ” prong
tion designated by the row and column location of the of Block 308 ) , the digital value corresponding to the time it
capacitance data . Depending on what application , content, took to charge the storage capacitor 236 to the specified
or the like is currently operating, the system application 260 threshold is recorded by the scan circuit 238 (Block 310 ) . If
and /orcontrol
user user application 262 implements
262. Implementation the corresponding
may affect what is cur 10 threshold
the storage capacitor's voltage does not reach the specified
rently being displayed on the display 210. Consequently, the couples theduring the time that the MUX - 2 circuitry 232-2
display driver 214 may operate the display circuitry 212 (i.e. , the “ No” prongsensor
selected
of
row to the storage capacitor 236
Block 308 ), then the acts of block
coupled to the display 210 in response to an implemented 302-308 are repeated .
control 262. For example , a new menu may be presented on 15 Once a digital value corresponding to the capacitance of
the display 210 in response to an implemented user control
262 . the selected row has been obtained ( Block 310 ) , a check is
As shown in FIG . 5 , the electronic device 200 can also made to see if there are additional rows in the sensor array
include one or more sensors 270 coupled to the processing 220 that need to be sampled. If more rows need to be
circuitry 240. The one or more sensors 270 can include a 20 sampled ,the process 300 goes to the next row at Block 314
Mercury switch , an acceleration sensor, inclinometer sensor, and repeats the acts of Blocks 302-308 . If it is determined at
an electronic compass , a light sensor , a motion sensor, or an Block 312 that all the rows in the sensor array 220 have been
orientation sensor. In one embodiment, the orientation sen- sampled in accordance with Blocks 302-308 , a similar
sor 270 is a 3 - G accelerometer similar to what is used in gyro sampling process is used to acquire a capacitance value for
remotes or used in hard drives to detect free fall. The 25 each column in the sensor array 220 (Block 316 ) . Once all
accelerometer 270 detects gravity and generates orientation rows and all columns have been processed, the entire
data that can indicate which edge of the display 210 is ‘ up , ” process 300 is repeated , which can be done at a predeter
" down, ” “ north , ” or other direction . The processing circuitry mined interval (Block 318 ) .
240 coupled to the accelerometer 270 determine the orien- Referring to FIG . 7 , a process 400 of operating the
tation of the electronic device 200 from the orientation data 30 electronic device of FIG . 5 is illustrated in flow chart form .
obtained from the sensor 270. The determined orientation For better understanding, element numerals for components
can then be used to designate or alter the location of the areas of FIG . 5 are concurrently provided in the discussion that
221 on the array 220 to match the orientation of the device follows. During operation , areas 221 on the array 220 of the
200. In addition , the determined orientation can then be used touch sensitive bezel are designated for user controls of the
to designate or alter the location of the visual guides 280 on 35 electronic device (Block 402 ) . In designating an area 221 ,
the display 210 to match the newly designated areas 221 on contiguous rows and columns of a portion of the capacitive
the array 220. Furthermore , the determined orientation can sensor array 220 are associated with a user control for the
then be used to rotate or flip the content shown on the display electronic device 200. The electronic device 200 also gen
210 to match the orientation of the device 200. Details erates visual guides 280 for the user controls (Block 404 )
related to how the electronic device 200 can use orientation 40 and displays the visual guides 280 on the display 210
data are discussed below with reference to FIGS . 8 and adjacent the designated areas 221 on the touch sensitive
9A - 9B . bezel 120 to which the visual guides 280 correspond ( Block
In another embodiment, the one or more sensors 270 can 406 ) . For example, the visual guides 280 are preferably
include one or more ambient light sensors for detecting the displayed along the perimeter of the display 210 adjacent
level of ambient light around the device 200. Preferably, the 45 corresponding rows and columns of the capacitive sensor
device 200 includes at least two such ambient light sensors array 220 associated with the user controls for the electronic
270 for redundancy. Based on the level of ambient light device 200. In this way, the majority of the display 210 can
detected , the electronic device 200 can automatically adjust be used to show content.
the contrast and / or brightness of the display 210 accordingly. The electronic device 200 then obtains touch data with the
In yet another embodiment, the one or more sensors 270 can 50 capacitive sensor array 220 using the techniques disclosed
include a motion sensor, such as a passive pyroelectric herein (Block 408 ) . The touch data in aa basic form includes
sensor. The motion sensor 270 can be used to detect motion information of which rows and columns of the capacitive
of the electronic device 200 from a stationary state so that sensor array 220 have been touched ( i.e. , have exceeded the
the device 200 can “ wake up ” ( e.g. , turn on or come out of capacitance threshold ). The electronic device 200 performs
9

a standby mode ) or can show previously hidden visual 55 various forms of processing of the obtained touch data . For
guides 280 on the display 210 in response to being moved . example, the touch data can be processed to determine how
Referring to FIG . 6 , a process 300 of operating the long the rows and columns have reached aa threshold capaci
electronic device having the capacitive sensor array 220 of tance , to determine how long rows and columns have been
FIG . 5 for the touch sensitive bezel is illustrated in flow chart below the threshold capacitance since an initial period of
form . For better understanding, element numerals for com- 60 being above the threshold, and to determine other forms of
ponents of FIG . 5 are concurrently provided in the discus- information . Furthermore, to facilitate processing , the touch
sion that follows. During operation, the MUX - 1 circuitry data can be aggregated together into predefined intervals of
232-1 couples a first row of the sensor array 220 to the time and portions of the array 220. In addition , the touch
reference voltage Vcc for a specified period of time (Block data obtained at a first instance can be stored and later
302 ) and then isolates or disconnects that row from Vcc 65 compared to touch data obtained at a subsequent instance to
( Block 304 ) . Next , the MUX - 2 circuitry 232-2 couples the determine changes in the data overtime, caused by a user's
same row to the storage capacitor 236 for a specified period touch movement on the array 220 , for example. These and
 US 11,360,509 B2
15 16
other forms of processing of the touch data will be apparent controls on the array 220 (Block 438 ) . Then , the process 430
to one skilled in the art with the benefit of the present can end until called again during the operation of the
disclosure. electronic device 200 .
After the touch data has preferably been processed with By way of example, FIG . 9A shows an electronic device
the techniques described briefly above , the electronic device 5 450 having content on the display 460. The device 450 is
200 then compares the information from the touch data to rectangular and can be oriented in either a “ portrait ” or a
the designations on the array 220 associated with the user " landscape" orientation . Accordingly, the display 460 can
controls for the device 200 ( Block 410 ) . From the compari show a picture or other content in either orientation. In some
son , the electronic device 200 determines which user control circumstances, the orientation of what is displayed may not
is invoked by the designated area 221 of the array 220 that 10 match how orthechanges
device the
450orientation
is currentlyoforientated
the device. The
450 user
the user has touched ( Block 412 ) . Then , the electronic may rotate
example, to the orientation shown in FIG . 9B . The orienta
, for
device 200 initiates the determined user control to affect
tion senor 490 is used to determine the new orientation (i.e. ,
processing of the device 200 (Block 414 ) . Once the user rotation
control is implemented, it may be necessary to update the 15 shown) ofof the 90 -degrees ), and the processing circuitry (not
display 210 and the designated user controls ( Block 416 ) . If designated on thedevice 450 determines that the areas 471
bezel 470 for certain user controls of the
the same visual guides 280 and designated user controls can electronic device 450 should be changed . Accordingly, the
be used, then the process returns to Block 408 , for example, processing circuitry alters the designation of the areas 471 of
to obtain any new touch data with the array 220. If, however, the bezel 470 so that they will better match the newly sensed
new visual guides 280 and designated user controls are 20 orientation of the device 450. In addition, because the
needed due to a change in the content of what is displayed electronic device 450 can display visual guides 480 relative
or the context of the device's operation, then the process to the designated areas 471 of the bezel 470 , the processing
returns to Block 402 to designate new areas on the array 220 circuitry also alters location of the visual guides 480 on the
for user controls and proceeds to subsequent steps . display 460 so that their new location s match the new
One skilled in the art will appreciate that the touch 25 location s of the designated areas 471 of the bezel 470 .
sensitive array 220 around the perimeter of the display 210 For example, the area 471A of where the “ Left ” control
of the device 200 can be used to implement various user device 480 in FIG . 9A will remain on the same side 454 of the
controls in some ways similar to how aa conventional touch 450 as shown in FIG . 9B . Yet, the “ Left” control 480
pad is used . In aa brief example , the information at Block 410 is preferably orientated along the new bottom edge 456 of
may indicate a “ tap ” made by the user on the array 220. This 30 the device 450 so that — it is in a position more amenable to
“ tap” ( i.e. , a touch by a finger on the array 220 for a “ short” the user. Thus, the old area 471A is no longer designated for
duration of time ) may have been performed in designated the “ Left " control. Instead , a new area 421B of the bezel 470
is designated for the “ Left ” control 480 , and the visual guide
area 221 of the array 220. The electronic device 200 deter 480 for the “ Left ” control is displayed in the appropriate
mines that the area 221 invoked by the “ tap ” is designated 35 position of the display 460 along the new bottom edge 456
for performing a “ page up ” control of what is being dis to match the new designated area 471B .
played on the device 200. The time duration of the “ tap ” may In the example of FIGS . 9A - 9B , orientation data from the
indicate the amount or extent of the “ page up” control. In orientation sensor 490 is used to alter the designation of the
response to the user control, the electronic device 200 causes areas 471 for the user controls and the location of visual
what is being shown on the display 210 to page up as 40
requested by the user.
guides 480. In other embodiments, the orientation of the
content to be displayed may dictate how the designation of
As noted briefly above, the electronic device 200 of FIG . the areas 471 for the user controls and the location of visual
5 can include one or more sensors 270 , which can include an guides 480 should be for the device 450. For example , the
accelerometer or other orientation sensor Referring to FIG . display 460 of the electronic device 450 in FIGS . 9A - 9B is
8 , a process 430 of operating the electronic device of FIG . 45 rectangular and can be used to show content in “ portrait " or
5 having the orientation sensor 270 is illustrated in flow " landscape ” orientations . Depending then on the desired or
chart form . For better understanding, element numerals for required orientation for particular content (e.g. , image ,
components of FIG . 5 are concurrently provided in the screen , user interface , or picture ) to be shown on the display
discussion that follows. Initially, the electronic device 200 460 , the electronic device 450 can alter the designation of
obtains orientation data from the sensor 270 ( Block 432 ) . 50 the areas 471 for the user controls and the location of visual
The electronic device 200 can obtain the orientation data at guides 480 according to the “ portrait” or “ landscape” ori
periodic intervals or at certain points during operation . The entations of the content. In other words , when the device
electronic device 200 then determines whether the orienta- 450 is preparing to display particular content, the electronic
tion of the device has changed ( Block 434 ) . If not , the device 450 can determine the particular orientation for that
process 500 can end . 55 content. Then , when the device 450 switches to show that
If the orientation has changed at Block 434 , the electronic new content on the display 460 , the electronic device 450
device 200 determines how the orientation has changed and alters the designation of the areas 471 for the user controls
alters the designation of areas for user controls of the touch and the location of visual guides 480 if the orientation of the
bezel (Block 436 ) . In particular, the processing circuitry 240 newly displayed content is different from that previously
alters how the one or more areas 221 of the capacitive sensor 60 displayed. Thus, the user can naturally rotate the device 450
array 220 are designated for the one or more user controls to better view the newly displayed content in its preferred
262 so that the designation better matches the new orienta- orientation (e.g. , “ portrait " or " landscape ” ), and the visual
tion of the device 200. In addition, if the display 210 is guides 480 and designated areas 471 will be already
showing visual guides 280 for the corresponding areas 221 matched to the content's orientation .
of the bezel 120 , then the electronic device 200 also alters 65 Turning to FIG . 10 , the electronic device 500 of the
location of the visual guides 280 on the display 210 so that present disclosure can also be configured to discriminate or
they match the newly designated area 221 for the user ignore certain forms of touch data made on the bezel 520 .
 US 11,360,509 B2
17 18
For example, the housing 502 of the electronic device 500 adjust the value for the selected attribute by touching the
may be designed to fit mainly around the display 510 and the area 552 on the bezel 550 with a finger of the other hand . The
surrounding bezel 520. As a result , when a user holds the side pad 560 can be either a large surface for tracking touch
electronic device 500 , it may be likely that portion of the inputs or can includes a plurality of small dedicated surfaces,
2

user's hand ( e.g. , one of the user's fingers or thumb) will 5 such as touch buttons, for performing dedicated functions. In
maintain consistent contact on portion 522 of the bezel 520 . yet an another alternative, the additional pads 560 can also
In this instance, it is desirable that the electronic device 500 be force sensitive so that a predetermined amount of force or
ignores such consistent contact made on the bezel 520. The pressure caused by a user touch is required to invoke the user
processing circuitry ( not shown) of the device 500 can store control associated with the touched areas 562 of the pads
information tracking how long touch data has occurred on 10 560 .
portions of the bezel 520 and /or how many adjacent, des- In additional alternatives shown in FIG . 12 , a touch
ignated areas have had repeated touch data. Then , after a sensitive bezel 590 according to the present disclosure can
predefined time limit , the processing circuitry can begin to be arranged in a housing 572 around at least a portion of a
ignore that consistent touch data in the portion 522 of the display 580 of an electronic device 570. In general, the bezel
bezel 520 when determining what user controls the user is 15 590 can include one or more discrete touch sensitive sur
implicating. Furthermore , the processing circuitry can des- faces positioned in the housing 572 adjacent one or more
ignated new location s for areas of the bezel 520 for user sides of the display 580. On device 570A , for example, the
controls that are part of the ignored portion 522 of the bezel bezel 590 has a plurality of discrete touch sensitive surfaces
520. In the present example, the areas 524 and “ 526 for the positioned in the housing 572 adjacent each side of the
" page up ” and page down ” user controls on the left side of 20 display 580. On device 570B , for example, the bezel 590 has
the bezel 520 have been moved to new location s outside the a first and second touch sensitive surface positioned in the
ignored portion 522. Likewise , the visual guides 512 asso- housing 572 adjacent three sides of the display 580 and has
ciated with the “ page up ” and page down ” user controls have a second touch sensitive surface positioned in the housing
been shifted to new locations adjacent to the newly desig- 572 adjacent one side of the display 580. On device 570C ,
nated areas 524 and 526 . 25 for example , the bezel 590 has first and second touch
In previous embodiments of the present disclosure , the sensitive surfaces positioned in the housing 572 adjacent
touch sensitive bezel of the present disclosure is arranged opposing sides of the display 580. These and other alterna
substantially around the entire perimeter of the display. In tive arrangements are possible for touch sensitive bezels
one alternative shown in FIG . 11 , an embodiment of an according to the present disclosure .
electronic device 530 can have a touch sensitive bezel 550 30 In the embodiment of FIG . 5 , the touch sensitive bezel of
around a display 540 just as before . In addition , the elec- electronic device 200 has been described as having capaci
tronic device 530 can have one or more additional touch tive sensor array 220 that is used with data acquisition
sensitive pads or surfaces 560 incorporated throughout vari- circuitry 230. As alluded to above , however, a touch sensi
ous sides of the housing 532 for the device 530. These tive bezel for an electronic device according to the present
additional touch sensitive pads 560 can be used to detect 35 disclosure can include other forms of touch sensitive cir
location caused by a user touching the pads 560 and / or can cuitry. Referring to FIG . 13A , another embodiment of a
be used to detect force caused by a user pressing the pads touch sensitive bezel 620 for an electronic device 600 is
560. The additional touch sensitive pads 560 can be posi- illustrated . Only portion of the touch sensitive bezel 620 is
tioned along edges of the housing 532 and can be positioned illustrated in FIG . 13A , and the housing , display, and other
on the back of the housing 532 . 40 components of the electronic device 600 are not shown for
Any user controls designated for areas 562 on these illustrative purposes. In the present embodiment, the touch
additional touch sensitive pads 560 may be preconfigured sensitive bezel 620 includes aa Printed Circuit Board (PCB )
and may not be change during operation. In this way, the 622 formed into a ring or frame shape and defining an inner
user may know the functionality of the various pads 560 and opening 624 in which components of the display (not
can use the areas 562 to control features of the device 530 45 shown) for the electronic device are positioned . A plurality
without the need of any visual guides 542 on the display 540 . of conductive pads 626 are formed on the PCB 622 , and each
Alternatively, the user may be able to designate any user pad 626 is interconnected by a resistive element (not shown )
controls for these additional touch sensitive pads 560 using according to details discussed below . The PCB 622 in this
setup and configuration operations of the device 530. In yet embodiment can have dimensions of approximately 8 by
another alternative, user controls for areas 562 of these 50 10 - inches and can have about 100 pads 626 formed around
additional pads 560 can be designated and re -designated by its perimeter.
the electronic device 530 during operation in much the same The touch sensitive bezel 620 also includes a control
way disclosed herein for areas 552 on the bezel 550. For module 630 , which is housed in the electronic device and is
example, areas 562 on the pads 560 can be designated for shown here relative to the PCB 622 for illustrative purposes .
user controls similar to the areas 552 that can be designated 55 The control module 630 is connected to the pads 626 of the
on the bezel 550 , and visual guides 542 can be displayed PCB 622 by connections (not shown) . The control module
around the perimeter of the display 540 adjacent to corre- 630 has a plurality of components, including an infrared
sponding areas 562 on the additional pads 560 in the same sensor, communication circuitry, accelerometer / inclinom
way that the visual guides 542 are displayed adjacent eter sensor, and other components. A suitable infrared sensor
designated areas 552 of the bezel 550 . 60 is an RE200B pyroelectric passive infrared sensor. A suit
In FIG . 11 , for example, the area 552 on the bezel 550 can able accelerometer /inclinometer sensor is a KXP84 IC .
be designated to adjust values , and the areas 562 of the The electronic device 600 can also have a plurality of
adjacent side pad 560 can be designated to select various ambient light sensors 604 and a plurality of infrared (IR)
attributes of the display 540. Because the device 530 can be modules 606 , which are also shown here relative to the
hand -held , the user can selected from the various attri- 65 control module 630 for illustrative purposes . A suitable
butes — by touching an area 562 on the side pad 560 with the ambient light sensor is an ISL29001 light- to -digital sensor.
hand used to hold the device 530 , and the user can then The ambient light sensors 604 can be positioned in various
 US 11,360,509 B2
19 20
location s on the housing of the electronic device and behind sensor array of FIG . 5. During operation, for example , data
the display. The ambient light sensors 604 detect the level of acquisition circuitry ( not shown) of the device 700 drives the
ambient light near the display so that the electronic device plurality of drive paths 742 ( one at a time) during a first time
can adjust the contrast, brightness, or backlighting of the period. During this same time period, the sense paths 744 on
display accordingly . 5 the other side of the substrate 740 are interrogated to obtain
In FIG . 13B , a schematic diagram of components 632 data representing the location of one or more touches to the
comprising portion of the control module 630 of FIG . 13A cosmetic layer 730. For example , a user's finger placed in
is illustrated . The components 632 in this portion include a proximity to various rows and columns of the drive paths
QT510 Interacted Circuit 634 available from Quantum 742 and sense paths 744 on the top and bottom of the
Research Group, Ltd. The QT510 IC 634 is connected at 10 substrate 740 alters their capacitive relationship when pro
three approximately equidistant points 636A , B , and on the cessed by the data acquisition circuitry.
pad element 620 , which is shown here schematically. Opera- To sense force, circuitry of the device 700 drives the drive
tion and arrangement of QT510 IC 634 and the pad element paths 762 on the base layer 760 (one at a time) during a
620 is similar to that used for the QWheelTM available from second time period. During this same time , the sense paths
Quantum Research Group, Ltd. However, in at least one 15 744 on the bottom side of the substrate 740 are again
exception , the QWheelTM has 18 pads formed into a con- interrogated to obtain data representing the strength or
centric wheel with resistors of about 15K positioned intensity of force applied to cosmetic layer 730 by a user's
between each pad and a total resistance of about 270 k . In touch . For example, when a force is applied by a user's
contrast, the present embodiment of the pad element 620 has finger on the cosmetic layer 730 , the spring layer 750
about 100 pads formed as a frame for the display of the 20 deforms moving the sense paths 744 on the bottom of the
device . The 100 pads are interconnected by about 100 substrate 740 closer to the drive paths 762 on the top of the
resistors . Each resistor has a resistance of about 2.67 k so base layer 760. A resulting change in mutual capacitance is
that the pad element 620 has a total resistance of about 270 then used to generate data indicative of the strength or
k. intensity of an applied force . Additional details related to the
In an additional embodiment, the operation and arrange- 25 layers and other aspects of this embodiment are disclosed in
ment of IC 634 and the pad element 620 of the present incorporated U.S. patent application Ser. No. 11 / 278,080 .
disclosure can use techniques disclosed in U.S. Patent Using force and location detection , the bezel 720 of the
Application Publication No. 2006/0032680 , entitled “ A present embodiment can provide additional user interface
Method of Increasing Spatial Resolution of Touch Sensitive and controls . For example, a user's finger in FIG . 7 is shown
Devices , ” which is incorporated herein by reference in its 30 touching an area 721 of the bezel 720 designated for a “ page
entirety, to expand the detected sensitivity of the pad ele- down” operation (as indicated by the adjacent visual guide
ment 620 . 780 ) . The electronic device 700 uses the sensed location on
In the embodiment shown in FIG . 13A , the pads 626 are the bezel 720 to determine which control is invoked by the
formed on PCB 622. In an alternative embodiment shown in user's touch . In addition , the force applied by the user's
FIG . 13C , pads 628 can be formed as layers on a surface of 35 touch is obtained using the force detection features of the
a display 610 for the electronic device 600. For example, bezel 700. The sensed force can be used to determine the
techniques associated with Indium oxide doped with tin desired speed or extent with which to perform the “ page
oxide ( referred to herein as ITO techniques) can be used to down " operations, for example .
deposit the pads 626 as transparent conductive thin layers on Given all of the previous discussion of the present dis
the surface of the display 610. In this way, the touch 40 closure, we now turn to an embodiment of an electronic
sensitive bezel of the device 600 is essentially the perimeter device that incorporates one or more of the aspects and
of the display 610 , and the housing 602 is practically features discussed above. In FIGS . 15 through 19 , an
consumed by the display 610. In addition, a touch sensitive embodiment of aa multimedia device 800 having a housing
wheel 650 having a plurality of pads 652 can also be 802 , a display 810 , a touch sensitive bezel 820 , and user
deposited as on the display 610 using ITO techniques to 45 interface 900 according to certain teachings of the present
provide additional user controls of the electronic device 600 . disclosure is illustrated in a number of stages of operation
In FIG . 14 , an embodiment of an electronic device 700 and use . The multimedia device 800 in FIGS . 15 through 19
having a touch sensitive bezel 720 capable of force and is meant to be exemplary . It will be appreciated that the user
location detection is illustrated . Portion of the touch sensi- interface 900 , available features and functions, user controls,
tive bezel 720 is illustrated in a detailed cutaway. In this 50 screens , designations of the bezel 820 , and various other
embodiment, the bezel 720 includes a force detector com- details provided in the discussion that follows may be
bined with a location detector so that the bezel 720 can altered depending on the implementation and desired results .
provide both location and force detection . The bezel 720 In FIG . 15 , a menu screen of the user interface 900 is
includes a cosmetic layer 730 , a substrate 740 , a dielectric displayed and lists various functions or features 902 ( e.g. ,
spring layer 750 , and a base or support layer 760. The 55 Music, Photos, Videos, etc. ) that are available on the device
substrate 740 has a plurality of conductive drive paths 742 800. An area 822 of the bezel 820 adjacent a battery symbol
on a “ top ” side and has a plurality of conductive sense paths 906 can be touched at any point during operation to access
744 on the “ bottom ” side . The dielectric spring layer 750 has power settings for the device 800 without the user having to
alternating, or spatially offset, raised structures 752 on both access the settings function 902 on the menu screen 900. A
sides of the layer 750. The base layer 760 has a plurality of 60 plurality of areas 824 on one side of the bezel 820 are
conductive drive paths 762. The drive paths 742 on the designated for selection of one of the available functions or
substrate 740 are laid down in aa first direction to form rows features 902 , and visual guides 904 are provided on the
as are the drive paths 762 on the base layer 760. The sense perimeter of the bezel 820 adjacent the designated areas 824 .
paths 744 on the bottom side of the substrate 740 are laid A user touching one of these areas 824 of the bezel 820
down in a second direction to form columns . 65 accesses a subsequent screen of the user interface 900 for the
To sense location , the device 700 uses many of the same selected function or feature 902. It should be noted that the
techniques discussed above with reference to the capacitive side of the housing 802 may include a touch sensitive pad
 US 11,360,509 B2
21 22
( similar to pads 560 of FIG . 11 ) on a side of the housing 802 , that they are used to adjust values . By reselecting the toggle
and areas (e.g. , areas 562 of FIG . 11 ) of this side pad can be area 831 in the lower right hand corner, the user can remove
similarly designated . the visual guides 926 and 928 for the user controls from the
In FIG . 16 , the user has accessed the photo feature from screen 920 .
the previous menu so that the display 810 shows a photo 5 In FIG . 18 , an example screen 930 listing songs is shown
screen 910 listing various available photos 912. An area 826 on the display 810 of the device 800. A plurality of areas 864
on the left side of the bezel 820 is designated for scrolling on the right side of the bezel 820 adjacent visual guides 934
up and down the list of photos 912 , and a visual scroll guide for the listed songs can be used to select , highlight, or open
916 is provided at the perimeter of the display 810 adjacent the corresponding song . An area 864 on the left side of the
the area 826. A plurality of areas 828 on the right side of the 10 bezel 820 adjacent a scroll guide 934 can be used to scroll
bezel 820 are designated for selecting to open a selected through the list of songs on the screen 930. If the user selects
photo 912 , and visual guides 914 for each photo 912 are or highlights one of the listed songs , the user can select areas
provided adjacent these areas 828. An area 826 in the upper 850 to play / pause the song or areas 852 , 854 to track forward
corner adjacent a close window icon 914 on the screen 910 or back from the selected song . These areas 850 , 852 , 854
is designated on the bezel 820 for closing the current screen 15 have corresponding visual guides 935. The user can also
910 to return to the menu screen of FIG . 15 . select to add the selected song to a play list or can elect to
In FIG . 17A , the display 810 shows a screen 920 having view various play lists by selecting from additional areas
a selected photo (e.g. , sunrise ). A toggle area 831 of the 860 and 862 having visual guides 932. Depending on the
bezel 820 in the lower right corner is designated to access amount of area of the display 810 available , one or more user
and display additional user controls that are discussed below 20 controls 938 ( e.g. , volume level) can be displayed on the
with reference to FIG . 17C . A visual guide 921 is provided screen 930 , and one or more unused areas 868 of the bezel
on the display 810 adjacent this toggle area 831. A first area 820 can be designated for the one or more user controls 938 .
832 on the bezel 820 is designated for moving to a previous In FIG . 19 , an example " now playing ” screen 940 is
photo of those available , and a second area 834 is designated shown on the display 810 of the device 800. The screen 940
for moving to a subsequent photo . Corresponding visual 25 shows information about the song that is currently being
guides 922 are displayed adjacent these areas 832 and 834 played on the device 800. As before, the area 850 on the
on the bezel 820. Additional areas 836 and 838 on adjacent bezel 820 is designated for play /pause user controls, and
sides of the bezel 820 may be designated for any of a number areas 852 and 854 are designated for previous track and next
of operations, such as zoom contrast, brightness, page track , respectively. A plurality of areas 870 and 872 are
down, scroll , etc. In the present embodiment, visual guides 30 provided on the sides of the bezel 820 adjacent various
are not shown adjacent these areas 836 and 838 so that the visual guides 942 corresponding to various user controls
majority of the display 810 is unobstructed with elements of (e.g. , time bar for song , equalizer selection , and volume
the user interface , and the screen 920 can primarily display level ) . In one possibility, the user can select to change the
the content ( i.e. , the sunrise photo ) . The user controls for settings or values of any one of these user controls by
which these areas 836 and 838 are designated may be 35 tapping or holding on the areas 870 or 872 on either side of
already known or readily apparent to the user. the adjacent visual guide 942 for a control to advance or
As shown in FIG . 17A , the device 800 is rectangular and reduce the setting or value . In another possibility, the user
is shown in aa vertical (i.e. , “ portrait ”) orientation . The user can select or highlight one of the user controls by tapping or
may rotate the device 800 so that it has a horizontal (i.e. , holding on the areas 870 or 872 on either side of the adjacent
" landscape ” ) orientation , such as shown in FIG . 17B . As 40 visual guide 942 for a control, and the user can then advance
discussed previously, the device 800 can have an orientation or reduce the setting or value by sliding over an additional
sensor (not shown) , such as an accelerometer or an incli- area 874 of the bezel 820 next to the visual guide 944 for
nometer, and can determine the orientation of the device adjustability.
800. In FIG . 17B , the device 800 has determined the As shown by the example multimedia device 800 of
horizontal or landscape orientation . Based on this determi- 45 FIGS . 15 through 19 , the touch sensitive bezel 820 and the
nation , the device 800 has adjusted the orientation of the user interface 900 according to certain teachings of the
screen 920 showing the photo on the display 810 in a present disclosure can obtain and process various forms of
landscape orientation and has readjusted the location of all touch data. For example, the touch sensitive bezel 820 can
the areas on the bezel 820 designated for the various user be used to sense touch gestures, such as touch and drag
controls . 50 operations, made by the user . Using the touch gestures, a
If the user selects the toggle area 831 in the lower right user can perform a number of user controls, such as move
corner, the screen 920 shows additional user controls . In portions of an image that are being displayed ( e.g. , page up
FIG . 17C , for example, the toggle area 831 has been or page down ), move a curser to drop and drag visual
previously selected so that a new visual guide 925 is elements displayed (e.g. , move a holder or file in aa directory
provided adjacent the area . In addition, a plurality of areas 55 window ) , scroll up and down through a screen , skip through
840 on the bezel 820 are designated for a number of user images in an album or series of images , adjust the setting or
controls , which have visual guides 928 shown on the display value of a user control, or perform similar operations. In
810 adjacent the bezel 820. In this example, the user controls addition , because the touch sensitive bezel 820 can also be
available for viewing photos include contrast, brightness, configured to detect force , the pressure that a user applies to
zoom , and move . A user can select one of these areas 840 to 60 a designated area of the bezel 820 can be used to determine
access that corresponding control. Then , while the corre- the amount, extent, or level of a setting, adjustment, or value
sponding control is activated ( either by highlighting the of a user control.
corresponding visual guide 928 or while the user continues Furthermore, the touch sensitive bezel 820 according to
touching the corresponding area 840 ) , the user can adjust the present disclosure can be used to obtain touch data
values or settings for the selected control using one or more 65 corresponding to multiple user controls simultaneously . For
areas 842 and 844 on other portions of the bezel 820. These example , the user controls of the bezel 820 can be config
areas 842 and 844 have adjacent visual guides 926 showing ured so that one side of the bezel 820 controls brightness
 US 11,360,509 B2
23 24
with a touch and drag motion by the user, while the other detecting the movement of the object, designating the one or
side of the bezel 820 controls contrast with a touch and drag more areas in the second region of the touch screen as the
motion by the user. Thus, using both of these sides of the input areas.
bezel 820 , the user can simultaneously adjust both the 6. The apparatus of claim 1 , the processor further con
contrast and the brightness of the display 810 using touch 5 figured for detecting a gesture at the one or more input areas
and drag motions on the sides of the bezel 820. These and of the second region , and performing the first operation in
other possibilities and combinations will be apparent to one response to the detected gesture .
skilled in the art having the benefit of the present disclosure . 7. The apparatus of claim 1 , the processor further con
The foregoing description of preferred and other embodi figured for detecting movement of an object at the one or
ments shows several different configurations of electronic 10 more input areas of the second region , and performing the
first operation in response to the detected movement.
devices. Certain features, details, and configurations were 8. The apparatus of claim 1 , the processor further con
disclosed in conjunction with each embodiment. However, figured for detecting a tap at the one or more input areas of
one skilled in the art will understand ( 1 ) that such features , the second region , and performing the first operation in
details, and configurations can be used with the various 15 response to the detected tap.
different embodiments, even if such features, details, and 9. The apparatus of claim 1 , the processor further con
configurations were not specifically mentioned in conjunc figured for re -designating a function of the one or more
tion with a particular embodiment, and (2 ) that this disclo- configurable control areas in the first touch sensitive region
sure contemplates various combinations of the features, during operation of the apparatus .
details, and configurations disclosed herein . More specifi- 20 10. A method for performing operations on a hand -held
cally, the foregoing description of preferred and other device, comprising :
embodiments is not intended to limit or restrict the scope or displaying one or more first images in aa first region of a
applicability of the inventive concepts conceived of by the touch screen located on a front surface of the device ;
Applicants. In exchange for disclosing the inventive con- providing one or more configurable control areas at a first
cepts contained herein , the Applicants desire all patent rights 25 touch sensitive region located on a side surface of the
afforded by the appended claims . Therefore, it is intended device, the side surface adjacent to but non -planar with
that the appended claims include all modifications and respect to the front surface;
alterations to the full extent that they come within the scope detecting a first touch at a first control area of the first
of the following claims or the equivalents thereof. touch sensitive region;
30 upon detecting the first touch, configuring one or more
What is claimed : areas in a second region of the touch screen as input
1. An apparatus, comprising: areas, the one or more input areas correlated to a
a touch screen located on a front surface of the apparatus; location of the first touch ;
a first touch sensitive region located on a side surface of detecting one or more second touches at the one or more
the apparatus, the side surface adjacent to but non- 35 input areas in the second region ; and
planar with respect to the front surface, the first touch performing a first operation in response to the detected
sensitive region providing one or more configurable one or more second touches .
control areas; and 11. The method of claim 10 , further comprising display
a processor communicatively coupled to the touch screen ing the first region and the second region adjacent to each
and the first touch sensitive region, the processor con- 40 other on the touch screen .
figured for 12. The method of claim 10 , further comprising locating
displaying one or more first images in aa first region of the second region at an edge of the touch screen .
the touch screen, 13. The method of claim 10 , further comprising detecting
detecting a first touch at a first control area of the first a gesture at the first control area of the first touch sensitive
touch sensitive region , 45 region, and upon detecting the gesture, designating the one
upon detecting the first touch , configuring one or more or more areas in the second region of the touch screen as the
areas in aa second region of the touch screen as input input areas.
areas, the one or more input areas correlated to a 14. The method of claim 10 , further comprising detecting
location of the first touch , movement of an object over the first control area of the first
detecting one or more second touches at the one or 50 touch sensitive region , and upon detecting the movement of
more input areas in the second region , and the object, designating the one or more areas in the second
performing a first operation in response to the detected region of the touch screen as the input areas .
one or more second touches. 15. The method of claim 10 , further comprising detecting
2. The apparatus of claim 1 , the processor further con- a gesture at the one or more input areas of the second region ,
figured for displaying the first region and the second region 55 and performing the first operation in response to the detected
adjacent to each other on the touch screen . gesture.
3. The apparatus of claim 1 , the processor further con-
2 16. The method of claim 10 , further comprising detecting
figured for locating the second region at an edge of the touch movement of an object at the one or more input areas of the
screen . second region , and performing the first operation in response
4. The apparatus of claim 1 , the processor further con- 60 to the detected movement.
figured for detecting a gesture at the first control area of the 17. The method of claim 10 , further comprising detecting
first touch sensitive region, and upon detecting the gesture , a tap at the one or more input areas of the second region, and
designating the one or more areas in the second region of the performing the first operation in response to the detected tap .
touch screen as the input areas . 18. The method of claim 10 , further comprising re
2

5. The apparatus of claim 1 , the processor further con- 65 designating a function of the one or more configurable
figured for detecting movement of an object over the first control areas in the first touch sensitive region during the
control area of the first touch sensitive region, and upon performance of the operations.
 US 11,360,509 B2
25 26
19. A non - transitory computer - readable storage medium
having stored thereon instructions which, if executed by a
specialized electronic device, causes the electronic device to
perform a method comprising:
displaying one or more first images in a first region of a 5
touch screen located on a front surface of the device ;
providing one or more configurable control areas at a first
touch sensitive region located on a side surface of the
device, the side surface adjacent to but non -planar with
respect to the front surface; 10
detecting a first touch at a first control area of the first
touch sensitive region;
upon detecting the first touch, configuring one or more
areas in a second region of the touch screen as input
areas , the one or more input areas correlated to a 15
location of the first touch ;
detecting one or more second touches at the one or more
input areas in the second region; and
performing a first operation in response to the detected
one or more second touches . 20
20. The non - transitory computer - readable storage
medium of claim 19 , the method further comprising detect
ing a gesture at the first control area of the first touch
sensitive region, and upon detecting the gesture , designating
the one or more areas in the second region of the touch 25
screen as the input areas.
* *