You are on page 1of 78

UNITED STATES PATENT AND TRADEMARK OFFICE

____________

BEFORE THE PATENT TRIAL AND APPEAL BOARD

____________

UNIFIED PATENTS INC.

Petitioner,

v.

AMBUSH INTERACTIVE, INC.

and

SPEAKWARE, INC.

Patent Owners.

____________

Case IPR2019-00495

U.S. Patent No. 6,397,186

____________

PETITION FOR INTER PARTES REVIEW


U.S. Patent No. 6,397,186
Petition for Inter Partes Review

TABLE OF CONTENTS

TABLE OF CONTENTS ........................................................................................ i

LIST OF EXHIBITS.............................................................................................. iii

I. MANDATORY NOTICES PURSUANT TO 37 C.F.R. § 42.8 .................1

A. Real Party-In-Interest ............................................................................1

B. Related Matters......................................................................................1

C. Lead Counsel, Back-Up Counsel, and Service Information .................3

II. PAYMENT OF FEES ...................................................................................4

III. REQUIREMENTS FOR INTER PARTES REVIEW................................4

A. Grounds for Standing ............................................................................4

B. Identification of Challenge ....................................................................4

1. Prior Art ......................................................................................4

2. Grounds .......................................................................................7

IV. U.S. Patent 6,397,186 .....................................................................................7

A. Summary ...............................................................................................7

B. Prosecution History ...............................................................................9

V. LEVEL OF ORDINARY SKILL IN THE ART ......................................10

VI. CLAIM INTERPRETATION ....................................................................11

VII. GROUNDS OF UNPATENTABILITY.....................................................11

A. Ground 1: Claims 1-4, 7, 9-10, 12, 14, 16, 19, 21, 23-26, 28-31, 39,
41, 43 are rendered obvious by Bissonnette in view of Miyazawa
under 35 U.S.C. § 103(a).....................................................................12

i
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

1. Overview ...................................................................................12

2. Motivation to Combine Bissonnette and Miyazawa .................18

3. Claim Charts .............................................................................20

B. Ground 2: Claims 17-18 are rendered obvious by Bissonnette in view


of Miyazawa and Salazar under 35 U.S.C. § 103(a) ...........................65

C. Ground 3: Claims 19-20 are rendered obvious by Bissonnette in view


of Miyazawa and McCall under 35 U.S.C. § 103(a)...........................68

VIII. CONCLUSION ............................................................................................71

ii
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

LIST OF EXHIBITS

1001 U.S. Patent No. 6,397,186 by Bush et al.

1002 Prosecution History File of Application No. 09/469,707, which


matured into U.S. Patent No. 6,397,186

1003 Declaration of Bruce McNair

1004 Unified Patent’s Voluntary Interrogatory Responses

1005 USPTO Assignment Abstract of Title for U.S. Patent No. 6,397,186

1006 WIPO International Application Publication No. WO 1994/003020 by


Bissonnette et al.

1007 U.S. Patent No. 6,591,239 by McCall et al.

1008 U.S. Patent No. 5,983,186 by Miyazawa et al.

1009 U.S. Patent No. 5,774,841 by Salazar et al.

1010 Complaint for Patent Infringement, SpeakWare, Inc. v. Amazon.com,


Inc., 8:18-cv-01303 (CACD July 26, 2018)

1011 Complaint for Patent Infringement, SpeakWare, Inc. v. Microsoft


Corporation, 8:18-cv-01293 (CACD July 26, 2018)

1012 Complaint for Patent Infringement, SpeakWare, Inc. v. Google LLC,


8:18-cv-01299 (CACD July 26, 2018)

1013 Complaint for Patent Infringement, SpeakWare, Inc. v. Samsung


Electronics Co., Ltd. et al., 8:18-cv-01300 (CACD July 26, 2018)

1014 Complaint for Patent Infringement, SpeakWare, Inc. v. Apple Inc.,


8:18-cv-01302 (CACD July 26, 2018)

1015 R.L. Morris & J.R. Miller (ed), Designing with TTL Integrated
Circuits, McGraw-Hill (1971), pp. 131, 175

iii
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

1016 Tobey, Graeme, & Huelsman, Operational Amplifiers – Design and


Applications, Burr-Brown (1971), pp. 358-366

1017 F. Bates & M. Douglas, Programming Language One, Prentice-Hall


(1967), pp. 37-38

1018 B. Kernighan, The C Programming Language, Bell Telephone


Laboratories (1988), p. 41

1019 W.J. Weller, Assembly Level Programming for Small Computers,


D.C. Heath and Company (1975), p. 37, 41

1020 C. Melear, Hardware and Software Techniques for Power


Conservation in Portable Devices, Proceedings of WESCON ’94
(September 1994), pp.453-461

1021 M. Miyakate, S. Tsurufuji, M. Iida, & N. Mori, A Voice Activated


Telephone, IEEE Transactions on Consumer Electronics, Vol. CE-32,
No. 3 (August 1986)

1022 L.R. Rabiner, J.G. Wilpon, & A.E. Rosenberg, A Voice-Controlled


Repetory-Dialer System, The Bell System Technical Journal, Vol. 59,
No. 7 (September 1980)

1023 E.L. Campbell, VOX in a box, QST, American Radio Relay League
(March 1964) pp. 11-13

1024 U.S. Patent No. 5,519,762 by Bartlett

iv
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

I. MANDATORY NOTICES PURSUANT TO 37 C.F.R. § 42.8

A. Real Party-In-Interest

Pursuant to 37 C.F.R. § 42.8(b)(1), Petitioner Unified Patents Inc. (“Unified”)

certifies that it is the real party-in-interest, and further certifies that no other party

exercised control or could have exercised control over Unified’s participation in this

proceeding, the filing of this petition, or the conduct of any ensuing trial. In view of

Worlds Inc. v. Bungie, Inc., 903 F.3d 1237, 1242-44 (Fed. Cir. 2018), Unified has

submitted voluntary discovery in support of its certification. See Ex. 1004

(Petitioner’s Voluntary Interrogatory Responses).

B. Related Matters

U.S. Patent 6,397,186 (“the ’186 patent) is the subject of pending litigation

in the following cases:

1
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Name Case No. Filing Date

SpeakWare, Inc. v. Microsoft


8-18-cv-01293 (CACD) 7/26/2018
Corporation

SpeakWare, Inc. v. Google LLC 8-18-cv-01299 (CACD) 7/26/2018

SpeakWare, Inc. v. Samsung


8-18-cv-01300 (CACD) 7/26/2018
Electronics Co., Ltd. et al.

SpeakWare, Inc. v. Apple Inc. 8-18-cv-01302 (CACD) 7/26/2018

SpeakWare, Inc. v.
8-18-cv-01303 (CACD) 7/26/2018
Amazon.com, Inc.

Google LLC v. SpeakWare, Inc. IPR2019-00340 11/30/2018

Google LLC v. SpeakWare, Inc. IPR2019-00342 11/30/2018

According to the assignment records of the USPTO, the ’186 patent is

assigned to Ambush Interactive, Inc. (Ex. 1005, Assignment Abstract of Title.)

However, an entity named “SpeakWare, Inc.” has stated that it is the owner of the

’186 patent in the complaints filed in each of the above-identified district court cases.

(Ex. 1010, 2 (“SpeakWare is the owner of U.S. Patent 6,397,186, entitled “Hands-

Free, Voice-Operated Remote Control Transmitter,” which issued on May 28,

2002.”); Ex. 1011, 2 (same); Ex. 1012, 2 (same); Ex. 1013, 2 (same); Ex. 1014, 2

(same).) Due to the discrepancy over the ownership of the ’186 patent, Petitioner has

identified both Ambush Interactive, Inc. and SpeakWare, Inc. as the owners of the

patent.

2
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

C. Lead Counsel, Back-Up Counsel, and Service Information

Lead Counsel Backup Counsel


Scott A. McKeown Jonathan R. Bowser
Reg. No. 42,866 Reg. No. 54,574
ROPES & GRAY LLP UNIFIED PATENTS INC.
2099 Pennsylvania Avenue, NW 1875 Connecticut Ave. NW, Floor 10
Washington, D.C. 20006-6807 Washington, D.C. 20009-6046
Phone: +1-202-508-4740 Phone: +1-202-701-1015
Fax: +1-617-235-9492 jbowser@unifiedpatents.com
scott.mckeown@ropesgray.com
Victor Cheung
Reg. No. 66,229
Mailing address for all PTAB ROPES & GRAY LLP
correspondence: 2099 Pennsylvania Avenue, NW
ROPES & GRAY LLP Washington, D.C. 20006-6807
IPRM—Floor 43 Phone: +1-202-508-4641
Prudential Tower Fax: +1-617-235-9492
800 Boylston Street victor.cheung@ropesgray.com
Boston, Massachusetts 02199-3600
Roshan Mansinghani
Reg. No. 62,429
UNIFIED PATENTS INC.
1875 Connecticut Ave. NW, Floor 10
Washington, D.C. 20009-6046
Phone: +1-214-945-0200
roshan@unifiedpatents.com

Please address all correspondence to lead and back-up counsel. Petitioner

consents to electronic service via email.

3
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

II. PAYMENT OF FEES

The undersigned authorizes the Office to charge the fees required by

37 C.F.R. § 42.15(a) for this Petition for inter partes review to Deposit Account No.

50-6990. Any additional fees that might be due are also authorized.

III. REQUIREMENTS FOR INTER PARTES REVIEW

A. Grounds for Standing

Pursuant to 37 C.F.R. § 42.104(a), Petitioner certifies that U.S. Patent

6,397,186 (“’186 patent,” Ex. 1001) is available for inter partes review and that

Petitioner is not barred or estopped from requesting inter partes review challenging

the claims of the ’186 patent on the grounds identified herein.

B. Identification of Challenge

Pursuant to 37 C.F.R. §§ 42.104(b) and (b)(1), Petitioner requests inter partes

review of claims 1-4, 7, 9-10, 12, 14, 16-21, 23-26, 28-31, 39, 41, and 43 of the ’186

patent, and that the Board cancel the same as unpatentable. The ’186 patent issued

from U.S. Patent Application No. 09/469,707 (filed 12/22/1999).

1. Prior Art

Petitioner relies upon the following prior art:

• Ex. 1006 – PCT Publication WO 94/03020 (“Bissonnette”) published on

2/3/1994 and is prior art to the ’186 patent under 35 U.S.C. § 102(b).

4
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• Ex. 1007 – U.S. Patent 6,591,239 (“McCall”) issued on 7/8/2003, is based on

U.S. Patent Application 09/458,175 (filed 12/9/1999), and is prior art to the ’186

patent under 35 U.S.C. § 102(e).

• Ex. 1008 – U.S. Patent 5,983,186 (“Miyazawa”) issued on 11/9/1999 and is

based on U.S. Patent Application 08/700,181 (filed 8/20/1996). Miyazawa is prior

art to the ’186 patent under 35 U.S.C. § 102(a).

• Ex. 1009 – U.S. Patent 5,774,841 (“Salazar”) issued on 6/30/1998 and is prior

art to the ’186 patent under 35 U.S.C. § 102(b).

Neither Bissonnette, McCall, nor Salazar were considered during

prosecution of the ’186 patent, and they are not cumulative of any prior art

considered by the original patent examiner. Miyazawa was identified in Applicant’s

Information Disclosure Statement but was not substantively addressed during

prosecution of the ’186 patent.1 As a result, the prior art asserted in the challenges

1
Grounds based on a reference cited in an IDS but not considered at any length (e.g.,

Miyazawa) should not serve as the basis for the Board to deny institution under 35

U.S.C. § 325(d). See, e.g., Digital Check Corp. d/b/a ST Imaging v. E-Imagedata

Corp., IPR2017-00178, Paper 6 at 12-13 (PTAB April 25, 2017); Fox Factory, Inc.

v. SRAM, LLC, IPR2016-01876, Paper 8 at 7-9 (PTAB April 3, 2017); and Praxair

5
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

below do not constitute the same or substantially the same prior art or arguments

that were previously presented to the Office, as required by 35 U.S.C. § 325(d).

Moreover, the Board should not exercise its discretion under 35 U.S.C. §

314(a) to deny institution of this Petition in view of Google LLC’s challenges in

IPR2019-00340 (challenging claims 1-20) and IPR2019-00342 (challenging claims

21-50). Notably, Unified did not previously challenge the ’186 patent, Unified’s

petition was filed before Patent Owner(s) or the Board substantively addressed

Google’s challenges, and Unified presents different challenges based on different

prior art combinations supported by different expert testimony. In particular, as

shown below, Unified’s challenges are based primarily on the combination of

Bissonnette and Miyazawa. Google’s challenges in IPR2019-00340 and IPR2019-

00342 are based primarily on the combination of Salazar and Miyazawa, or

Salazar, Miyazawa and Bossemeyer (U.S. 6,012,027). Unified’s challenges

relying on Salazar are directed to only dependent claims 17 and 18. Unified’s

challenges are not cumulative of Google’s challenges. Accordingly, none of

discretionary factors 1—7 as set forth in General Plastic weigh in favor of denying

Distribution, Inc. v. INO Therapeutics LLC, IPR2015-00893, Paper 14 at 8 (PTAB

Sept. 22, 2015).

6
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

institution. General Plastic Indus. Co., Ltd. v. Canon Kabushiki Kaisha, IPR2016-

01357, Paper 19 at 15 (PTAB Sept. 6, 2017) (Section II.B.4.i. precedential).

2. Grounds

# Claims Statute Prior Art

1-4, 7, 9-10, 12, 14, 16, 21,


1 § 103(a) Bissonnette in view of Miyazawa
23-26, 28-31, 39, 41, 43

Bissonnette in view of Miyazawa


2 17-18 § 103(a)
and Salazar

Bissonnette in view of Miyazawa


3 19-20 § 103(a)
and McCall

Pursuant to 37 C.F.R. § 42.104(b)(4)-(5), Petitioner provides in Section VII

below an explanation of how the Challenged Claims of the ’186 patent are

unpatentable under the grounds identified above, including an identification of

where each claim element is found in the prior art, to demonstrate that Petitioner has

at least a reasonable likelihood of prevailing on these grounds.

IV. U.S. Patent 6,397,186

A. Summary

The ’186 patent, titled “Hands-Free, Voice-Operated Remote Control

Transmitter,” discloses a device that automatically activates and operates remote

devices (appliances) using voice commands. (Ex. 1001, Abstract.)

7
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Fig. 1 of the ’186 patent, annotated below, shows an example of the electronic

components in a voice-operated system 10:

System 10 includes, among other things, a microphone 20, a speech

recognition system (including an audio switching circuit 30, a sound activation

circuit 40, and a speech recognition circuit 50), and an appliance control system

(including transmitter 70 and infrared light-emitting diode 80). (Id., 7:16-32.) By

determining whether the amplitude of a voice signal received at microphone 20

exceeds a certain threshold, system 10 is switched between a low-power sleep state

(a “sound activation mode”) and an awake state (a “speech recognition mode”) to,

inter alia, minimize power consumption. (Id., 4:27-32, 7:45-8:16). When system 10

is in a low-power sleep mode, an incoming voice signal is routed to the sound

8
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

activation circuit 40 to determine whether the amplitude of the voice signal exceeds

a threshold. (Id., 7:45-8:16.) When the amplitude exceeds the threshold, system 10

enters an awake state and the voice signal is routed to and recognized by the speech

recognition system to determine transmission of a control signal to one or more

remote devices (appliances). (Id.)

The ’186 patent purports to enable a simple procedure for efficient operation

by automatically switching between a low-power sleep mode and an awake mode.

(Id., 4:27-32, 7:45-8:16.) But such techniques were well-known prior to the filing of

the ’186 patent. (Ex. 1003 ¶¶ 26-28, 36-39.)

B. Prosecution History

During prosecution of the ’186 patent, the examiner identified prior art

systems that switched from low-power sleep modes to active modes by manual

switching (e.g., by depressing a button). To overcome prior art rejections, the

applicant: amended the claims to include converting audio signals into electrical

representative signals and argued that the applied prior art were deficient with

respect to automatically switching between operation modes (e.g., sound

9
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

activation/sleep and speech recognition/awake modes 2 (Ex. 1002 at 42-49);

amended the claims to include having the processor enter a low power state when in

the sound activation mode (id. at 71-88); and argued that the prior art taught away

from being combined due to allegedly conflicting discussions about the desirability

of push-to-talk switches to operate with automatic switching (id. at 71-88). The

USPTO issued a Notice of Allowance on 1/2/2002 without providing any reasons

for allowance. (Id. at 89-91.)

But the features that purportedly distinguished the claims of the ’186 patent

from the prior art of record were indeed well-known in the art before the earliest

possible priority date of the ’186 patent, as demonstrated by the prior art presented

herein.

V. LEVEL OF ORDINARY SKILL IN THE ART

The prior art discussed herein, and in the declaration of Bruce McNair (Ex.

1003) demonstrates that a person of ordinary skill in the art (“POSITA”), at the time

the ’186 patent was filed, would have been a person having a bachelor’s degree in

2
Providing an automatic means to replace a manual activity for achieving a same

result is not sufficient to distinguish over the prior art. In re Venner, 262 F.2d 91,

95 (CCPA 1958).

10
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

computer science, computer engineering, electrical engineering, or a related subject,

and one year of work experience with devices or systems for remotely controlling

electronic equipment. Less work experience may be compensated by a higher level

of education, such as a Master’s Degree, and vice versa. (Ex. 1003, ¶¶ 23-24.)

VI. CLAIM INTERPRETATION

At this time, Petitioner does not believe that any terms of the Challenged

Claims require construction beyond their plain and ordinary meaning.

VII. GROUNDS OF UNPATENTABILITY

Although the ’186 patent alleges to have invented a voice-operated remote

controller that automatically switches between different modes of operation, such

techniques were known in the communications device art prior to the earliest

possible priority date of the ’186 patent. As demonstrated below, the prior art

references render the Challenged Claims unpatentable.

This Petition is supported by the Declaration of Bruce McNair, which

describes the scope and content of the prior art at the time of the alleged invention

of the ’186 patent. (Ex. 1003, ¶¶ 10-11, 25-35, 41, 56, 64.)

11
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

A. Ground 1: Claims 1-4, 7, 9-10, 12, 14, 16, 19, 21, 23-26, 28-31, 39,
41, 43 are rendered obvious by Bissonnette in view of Miyazawa
under 35 U.S.C. § 103(a)

1. Overview

Bissonnette, titled “Voice Operated Remote Control Device,” discloses a

device that receives voice commands and performs voice recognition to determine

and transmit appropriate commands for controlling a variety of electronic

equipment. (Ex. 1006, Abstract.)

Bissonnette discloses that the device remotely controls electronic equipment

using customizable voice commands that are associated with control signals such as

IR codes. (Id., 2:34-3:20.) Like the ’186 patent, Bissonnette discloses that the device

is “normally in idle mode 200” (id., 33:27-29) to minimize power consumption (id.,

1:25-27, 11:28-31, 12:11-13).

For example, Bissonnette discloses that a user may press a voice key on the

device so that the device exits the idle mode and enters a state for processing voice

commands so that “the user can voice control: (1) all of the VCR playback functions;

(2) channel numbers; and (3) programming of recording events.” (Id., 33:20-26, Fig.

3.)

12
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Fig. 3 of Bissonnette

shows one possible flow of steps

for the operation of a voice

controlled device. At step 307, a

user can press the Voice VCR key

and record voice events directed

to a VCR using voice commands.

Subsequently, at steps 308-309,

the system performs voice

recognition matching to

determine the appropriate voice

command and transmits an

associated IR code to the VCR.

Similar voice controlled

operations of the Record and

Channel functions are shown in steps 301-304 and 310-313, respectively. The device

then returns to idle mode. (Id., 37:5-39:29).

Therefore, Bissonnette discloses that activating a voice key (e.g., by pressing

it) switches the device from an idle low-power state to an active state of voice

recognition (e.g., to “listen” to the spoken voice command) and initiates voice

13
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

recognition to determine and transmit an associated control signal. (Id., 37:5-39:29.)

Bissonnette discloses switching between idle and active modes without additional

steps after pressing the voice key. (Id., 37:5-39:29, 38:25-37.) (Ex. 1003 ¶¶ 42-46.)

While Bissonnette discloses switching between an idle mode and an active

mode by pressing a voice key, Bissonnette does not explicitly disclose switching

automatically based on the amplitude of the amplitude of the received voice

command exceeding a threshold, and switching based on timing or errors.

Miyazawa does.

Miyazawa is titled “Voice-Activated Interactive Speech Recognition Device

and Method” and discloses a device that automatically switches its speech-

processing CPU between sleep and active states based, e.g., on a volume of a voice

command exceeding a threshold. (Ex. 1008, Abstract.) Miyazawa discloses that

known speech recognition devices would typically perform operations when

switches were manually turned on. Miyazawa sought to improve on those manual

processes with automatic switching.

Miyazawa discloses that it is beneficial for devices to remain in standby/sleep

modes, wherein electronic components are kept in low-power states, to minimize

battery drain. (Id., 2:30-42, 2:46-50.) When idle, Miyazawa’s device “determines

whether input sound signals…are at least at a predetermined threshold volume level

above the background noise [and, if] so, a determination is then made whether or not

14
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

threshold filtered input sound signal corresponds to a recognizable phrase, and, if so,

shifts the device from the sleep mode into the active mode.” (Id., 3:13-28, 4:10-22.)

And, with regard to the alleged point of distinction that was argued during

prosecution (see Section IV(B) above), Miyazawa discloses that the system

performs its volume-based switching automatically (“[a] device…in a standby state

waiting for speech input at all times, and perform[s] recognition operations by

sensing speech input, without the need for the user to turn on the switch every time”).

(Id., 2:13-18.)

Miyazawa discloses that the device includes a sleep mode, in which only

certain portions of the device remain active and consume small amounts of power,

and the device becomes fully activated only when it detects recognizable speech

input above background noise levels. (Id., 4:14-5:7.) As shown in Fig. 1, annotated

below, Miyazawa teaches using an input sound signal power detector 9 to first detect

a threshold level of input before activating the more power-consuming components,

such as CPU 10, which is kept in a power-saving state during the sleep mode. (Id.,

11:50-67.)

15
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Miyazawa’s technique of automatic activation switching minimizes power

consumption and reduces accidental activations. (Id.)

Fig. 4 of Miyazawa, annotated below, shows one possible flow of steps for

the switching operations of a voice controlled device. At step s1, the device’s CPU

is sleeping (see step s7, for example) and the device compares the level of a received

voice command to a threshold level. (Id., 10:8-16.) Once a volume of a received

voice command (e.g., an amplitude) exceeds a threshold level, the CPU wakes and

exits the sleep mode, step s2. (Id., 10:14-18, 3:13-17.) Upon exiting the sleep mode,

16
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

the CPU detects and decodes the voice command to determine whether it is a

keyword (e.g., designated word or phrase such as “good morning” and “time”), at

steps s3 through s6. (Id., 10:17-45, 8:64-9:4.) If it is determined, at s6, that the voice

command is not a keyword, then the CPU re-enters sleep mode in order to conserve

power, step s7. (Id., 10:41-47.)

(Ex. 1003 ¶¶ 47-52.)

17
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

2. Motivation to Combine Bissonnette and Miyazawa

A POSITA would have been motivated to replace Bissonnette’s switching

functionality, which uses a manually operated voice key, with the automatic

switching techniques taught by Miyazawa—switching to a speech recognition mode

based on an input sound signal power detector that detects the amplitude of a voiced

command, and switching to a low-power idle mode when speech recognition is no

longer needed. Miyazawa specifically discloses that while some speech recognition

devices became ready to perform recognition operations in response to turning on a

switch (like in Bissonnette), it would have been more convenient to have a device

in a standby state waiting for speech input at all times, so the user did not need to

turn on the switch every time. (Ex. 1008, 2:8-18.) Because both Bissonnette and

Miyazawa were concerned with maximizing battery life by using idle and sleep

modes to reduce power drain from the voice recognition components, a POSITA

would have been motivated to look to analogous systems like Miyazawa to automate

and improve on the inconvenience of Bissonnette’s manual switching. (Ex. 1006,

11:28-31; Ex. 1008, 2:45-53, 11:33-42.) This would have been, at least, the simple

substitution of known elements (Miyazawa’s automatic switching techniques using

a sound signal power detector, timing criteria, and/or detected keywords) for another

(Bissonnette’s manual voice key) to obtain the predictable result of automatic

switching. (Ex. 1003 ¶¶ 26-32, 53.)

18
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

A POSITA would have had a reasonable expectation of success in making the

above modification. Bissonnette already disclosed components and methods for

switching between modes of a speech recognition device, and Miyazawa’s

teachings added checks of the speech inputs’ amplitude levels against some

threshold level and of speech inputs against time periods. Such checks were simply

comparisons of some values against others—basic functions found in standard off-

the-shelf components, from op-amps to more sophisticated microprocessors. As

shown by Bissonnette’s volume compensation techniques, for example, comparing

the values of signals was a well-known function that was within a POSITA’s ability

to apply. (Ex. 1006, 11:2-35; Ex. 1003 ¶¶ 28, 33-35, 54.)

19
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

3. Claim Charts 3

Claim 1 Bissonnette (Ex. 1006) and Miyazawa (Ex. 1008)


1[P] To the extent the preamble is considered limiting, Bissonnette
An audio discloses or at least renders obvious an audio signal control
signal activated system (including voice controller 1) for controlling appliances
control system (electronic equipment and electronic components).
for controlling • “A voice operated remote control device [that] accepts voice
appliances commands…determines the specific IR remote control code
comprising: or set of IR codes that represent the function corresponding
to the command and transmits the IR remote control codes
to the electronic equipement [sic] to be controlled….” (Ex.
1006, Abstract)
• “[T]here is provided a portable remote control device
operated by a voice command for controlling one or more
electronic components each which in turn is responsive to a
set of respective remote control codes.” (Id., 2:34-3:2)
• See annotated Fig. 1, showing voice controller 1, including
analog voice input 2 (for accepting the user’s voice input)
and IR output 12 (for outputting the IR codes to control
electronic equipment):

3
As discussed in Section VII.A.2, a POSITA would have been motivated to modify

Bissonnette with the teachings of Miyazawa to provide the voice input controls

in a more automatic and convenient way, including using a voice activated system.

(Ex. 1003 ¶¶ 53-54.). The claim charts are based on this obvious modification.

Bold and italicized text in the claim charts represent text from the Challenged

Claims.

20
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

To the extent the preamble is considered limiting, Miyazawa


discloses or at least renders obvious an audio signal activated
control system (a voice activated interactive speech
mechanism) for controlling appliances (electronic appliances
and similar devices):
• “[A] voice activated interactive speech mechanism….
performs various controls on the mechanism based on the
interpreted speech.” (Ex. 1008, 2:56-3:7)
• “All of the above embodiments include examples wherein
the present invention may be applied to a stuffed doll toy,
the present invention in no means limited to stuffed dolls….
[I]n addition to toys, the invention can be applied to
interactive game machines and electronic appliances that are
used in every day life.” (Id., 14:8-14.)
• “This may include … applications and embodiments as
scanning machines, home electronics, computer subsystems,
electronic appliances or similar devices.” (Id., 14:23-26.)

21
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

1[A] Bissonnette discloses or at least renders obvious a microphone


a microphone (e.g., analog voice input 2 that can be or include a microphone)
for receiving for receiving audio signals (e.g., from the user’s voice) and
audio signals converting the audio signals to electrical signals (e.g.,
and converting conditioned signals for microcontroller 20).
said audio • “An Analog Voice Input 2 converts the audio information in
signals to the user's voice to an analog electrical signal and also
electrical conditions this electrical signal for processing by
signals; Microcontroller 20….Analog Voice Input 2 includes a
microphone that is mounted against the front of the casing
with a small opening through the plastic located at the center
of the microphone.” (Ex. 1006, 10:8-19; Fig. 1)
• See annotated Fig. 1, showing analog voice input 2:

• “In operation, the voice signal output of the microphone is


fed into an analog input section of Analog Voice Input 2.
The signal is then conditioned by well known electronic
circuits that amplify and filter the voice input signal from
the microphone prior to going to an analog to digital
converter (ADC) 24 in microcontroller 20…. In a preferred
embodiment, Analog Voice Input 2 consists of three stages
of gain and filtering…. The overall frequency response of

22
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

the analog section is 300 to 4800 Hz. A third and final stage
of Analog Voice Input 2 provides for analog gain control
(AGC) of the voice input signal.” (Id., 10:30-11:22.)
• “[V]oice commands are received, filtered and conditioned
by Analog Voice Input 2.” (Id., 32:15-17.)
• “[A]n analog input circuit for receiving the voice command
and generating an analog voice signal based on the voice
command” (Id., 48:7-9 (claim 1).)
1[B] Bissonnette discloses or at least renders obvious a speech
a speech recognition system that includes a processor (e.g.,
recognition microcontroller 20 that includes microprocessor 21 and voice
system for recognition software stored in ROM 22 of Fig. 1) for receiving
receiving said the electrical signals (output from the Analog Voice Input 2, as
electrical discussed in claim 1[A]).
signals, said • “The input ports of Microcontroller 20 thus include: a voice
speech data port for receiving the analog voice signal from Analog
recognition Voice Input 2….” (Ex. 1006, 14:25-27)
system • “The output of Analog Voice Input 2 feeds into an 8-bit
including a ADC 24 within Microcontroller 20 which samples the data
processor and at 9.6 Khz…. Microcontroller 20 then processes the digital
having a low voice signal by means of microprocessor 21 and a voice
power sound recognition software routine that is part of a control program
activation stored in ROM 22. The digital voice signal is converted into
mode for a voice template that is compared against previously stored
detecting the voice templates of the user's voice. The program then
presence of decodes the voice templates as explained further below.”
said electrical (Id., 11:36-12:10)
signals and a
speech • See annotated Fig. 1, showing microcontroller 20 and
recognition microprocessor 21:
mode for
converting said
electrical
signals to
electrical
representative
signals,

23
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Bissonnette discloses or at least renders obvious the speech


recognition system having a low power … mode (low power
idle mode 200).
• “[I]n Figure 3, the control program of the present invention
is normally in idle mode 200.” (Id., 33:27-29)
• “To maximize on battery life, Analog Voice Input 2 is also
powered up only when the user activates one of three voice
keys described below to speak into the unit.” (Id., 11:28-31)
Bissonnette discloses or at least renders obvious the speech
recognition system having a speech recognition mode for
converting said electrical signals to electrical representative
signals (e.g., in voice mode control of Fig. 3, converting the
conditioned signals for into voice templates).
• “--processing of voice commands--
Referring generally to Figure 3, to operate the present
invention by use of voice commands the user selects one of
three voice keys (shown as 33, 34, or 37 in Fig. 5A)
depending on the desired function.” (Id., 33:19-23)

24
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “[--voice recognition software operation--]


The conditioned voice signal is … converted by ADC 24
into a digital voice signal. (Id., 32:17-18)
• “The digital voice signal is converted into a voice template
that is compared against previously stored voice templates
of the user's voice. The program then decodes the voice
templates as explained further below.” (Id., 12:7-10)
• “[A] processor … for generating a first digital voice
template from the first digital voice signal….” (Id., 48:19-22
(claim 1))
• See also id., 32:13-32 (describing converting the
conditioned voice signal into a digital voice signal, sampling
the digital voice signal, and using a transform process to
convert each spoken word to a fixed length sequence of
normalized frequency spectra (the voice template)).
• “Figure 3 shows some of the functions that can be operated
by voice commands; obviously, others could be
implemented as required by any particular application.” (Id.,
31:4-7)
• See Fig. 3, annotated to show the steps of the diagram
involving voice mode control (id., 6:12-14)

25
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Miyazawa discloses or at least renders obvious a low power (a


sleep mode in which CPU 10 is in an inactive power-saving
state) sound activation mode for detecting the presence of said
electrical signals (in the sleep mode, detecting the presence of
input sound signal power above a threshold before activating
the CPU 10).
• “CPU 10 is in an inactive, power-saving state during the
sleep mode, and only sound signal capture unit 1 and input
sound signal power detector 9 are in the active state. In
conventional speech recognition devices, power

26
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

consumption by an active mode CPU 10 is substantial in


order to handle complex speech recognition tasks. By
contrast, the power consumption by sound signal capture
unit 1 and input sound signal power detector 9 is relatively
quite small. In the present invention, CPU 10 momentarily
wakes up when a sound signal of at least a certain level is
input….Therefore, power consumption during the sleep
mode can be kept at an extremely low level, even in noisy
environments.” (Ex. 1008, 11:50-67)
• “[V]oice is input from sound signal input unit 1 and that the
sound signal waveform shown in FIG. 3A, for example, is
output by sound signal capture unit 1. Based on this, input
sound signal power detector 9 detects the input sound signal
power as explained below. Control begins at step s1, as
shown in FIG. 4. In step s1, input sound signal power
detector 9 determines whether or not the power of the input
sound signal is greater than a preset threshold th1, and
outputs a signal indicating that a sound signal has been input
when the power of the input sound signal becomes greater
than threshold th1 (at time t1 FIG. 3A). In such manner the
input signal power detector continues to monitor for sound
input signals until one meeting or exceeded threshold th1 is
encountered. (Id., 10:3-16)
1[C] Bissonnette discloses or at least renders obvious decoding said
decoding said electrical representative signals (e.g., microprocessor 21
electrical decodes and analyzes voice templates) and generating control
representative signals (e.g., IR remote control codes from RAM 23) for
signals and controlling one or more appliances (e.g., for controlling
generating component systems).
control signals • “Microcontroller 20 then processes the digital voice signal
for controlling by means of microprocessor 21 and a voice recognition
one or more software routine that is part of a control program stored in
appliances, ROM 22. The digital voice signal is converted into a voice
template that is compared against previously stored voice
templates of the user's voice. The program then decodes the
voice templates….” (Ex. 1006, 12:3-10)

27
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “[T]he voice recognition software within the control


program processes and decodes the spoken word at 311,
performs a device select at 312, and then outputs the
appropriate IR remote control code at 313.” (Id., 38:21-25)
• “When the user enters (…via a voice command) a particular
component command corresponding to one of the
component functions (such as volume for example),
processor 21, automatically selects both the appropriate
component and the remote control code corresponding to the
component function based on the component command
entered and the configuration data stored in reference RAM
23. This remote control code is then transmitted by IR
output 12 to the component to perform the function
corresponding to the component command.” (Id., 25:19-30)
• “[T]he present invention automatically outputs the correct
sequence of remote control codes required to effectuate a
particular system function. For example, when the user
enters (…via a voice command) a particular system
command (such as PLAY for example to play a
videocassette on a VCR), processor 21 automatically
generates a sequence of remote control codes based on the
… command and the configuration data described above.
This sequence would include remote control codes for
setting the TV to the appropriate channel depending on the
configuration data and playing a videocassette in the VCR[.]
IR output 12 then transmits the sequence of remote control
codes to the TV/cable box and VCR.” (Id., 29:16-27.)
• “a processor … for executing the control program to
determine whether the first digital voice template is
substantially equivalent to one of the reference digital voice
templates, and for selecting one of the remote control codes
based on the first digital voice template” (Id., 48:19-28
(claim 1))
• See also id., 34:22-25 (saying “START” to start a recording
process by outputting the necessary IR remote control
codes), 37:15-31 (saying a string of commands, with each

28
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

command being translated into corresponding remote


control codes)
1[D] Bissonnette discloses or at least renders obvious that, in the
wherein in said speech recognition mode (e.g., the voice operation mode/voice
speech mode control), the processor (e.g., microprocessor 21 and
recognition voice recognition software stored in ROM 22 of Fig. 2)
mode said decodes the electrical representative signals (e.g., decodes the
processor voice templates).
decodes said • “Microcontroller 20 then processes the digital voice signal
electrical by means of microprocessor 21 and a voice recognition
representative software routine that is part of a control program stored in
signals and ROM 22. The digital voice signal is converted into a voice
wherein in said template that is compared against previously stored voice
sound templates of the user's voice. The program then decodes the
activation voice templates….” (Ex. 1006, 12:3-10)
mode said • “[T]he voice recognition software within the control
processor is in program processes and decodes the spoken word at 311,
a low power performs a device select at 312, and then outputs the
state, appropriate IR remote control code at 313.” (Id., 38:21-25)
• “a processor … for executing the control program to
determine whether the first digital voice template is
substantially equivalent to one of the reference digital voice
templates” (Id., 48:19-26 (claim 1))
As discussed above, Bissonnette, in view of Miyazawa,
discloses the sound activation mode. (See, e.g., claim 1[B]
above.) Miyazawa discloses or at least renders obvious that the
sound activation mode includes a low power state of the
processor (a non-active, inactive, power-saving, low power
consumption, sleep mode state).
• “[P]ower detector includes processing circuitry for forcing
the mechanism to selectively enter or terminate a low-power
sleep mode.” (Ex. 1008, 3:18-20)
• “CPU 10 is placed in a non-active state or sleep mode, as is
well known in the computing arts.” (Id., 9:67-10:2)
• “CPU 10 is in an inactive, power-saving state during the
sleep mode, and only sound signal capture unit 1 and input

29
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

sound signal power detector 9 are in the active state. In


conventional speech recognition devices, power
consumption by an active mode CPU 10 is substantial in
order to handle complex speech recognition tasks. By
contrast, the power consumption by sound signal capture
unit 1 and input sound signal power detector 9 is relatively
quite small. In the present invention, CPU 10 momentarily
wakes up when a sound signal of at least a certain level is
input….Therefore, power consumption during the sleep
mode can be kept at an extremely low level, even in noisy
environments.” (Id., 11:50-67)
1[E] Bissonnette discloses or at least renders obvious switching
said speech from an activation mode (idle mode 200) to the speech
recognition recognition mode (e.g., the voice operation mode/voice mode
system control).
configured to • “[T]o operate the present invention by use of voice
automatically commands the user selects one of three voice
switch from keys…[¶]…the control program of the present invention is
said sound normally in idle mode 200.” (Ex. 1006, 33:20-29)
activation • See Fig. 3, showing an example flow diagram for the voice
mode to said operation mode software routine, which includes switches to
speech and from idle mode 200 (idle mode 200 shown in red). (Id.,
recognition 6:12-14)
mode as a
function of the
amplitude of
said electrical
signals; and

30
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Miyazawa discloses or at least renders obvious automatically


switching from the sound activation mode to the speech
recognition mode (switching from the low-power sleep mode,
in which only the input sound signal power detector 9 is active,
to the speech recognition mode, in which CPU 10 wakes to
perform speech analysis processing) as a function of the
amplitude of the electrical signals (automatically switching as
a function of the amplitude of the input sound signal exceeding
a threshold).

31
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “[I]nput sound signal power detector in communication with


at least the sound signal input unit and the interaction
controller for detecting the volume, magnitude or amplitude
of input sound signals based on sound signal waveforms
perceived by the sound input unit or capture device.” (Ex.
1008, 3:13-17, 4:10-13)
• “In step s1, input sound signal power detector 9 determines
whether or not the power of the input sound signal is greater
than a preset threshold th1, and outputs a signal indicating
that a sound signal has been input when the power of the
input sound signal becomes greater than threshold th1 (at
time t1 FIG. 3A). In such manner the input signal power
detector continues to monitor for sound input signals until
one meeting or exceeded threshold th1 is encountered.
Thereafter, control passes to step s2 in which CPU 10 wakes
up (step s2), and after a specified amount of time following
t1, (in about 10 msec) speech analyzer 2 thereof performs
speech analysis processing based on the signal captured by
sound signal capture unit 1 (step s3).” (Id., 10:8-21,
Abstract)
1[F] Bissonnette discloses or at least renders obvious an appliance
an appliance control circuit including a transmitter (e.g, transistors driving
control circuit a transmitter IR Output 12 having three LEDs) that receives the
which includes control signals from the speech recognition system (receives
a transmitter, the remote control codes as driven by Microcontroller 20) and
said appliance generates and automatically transmits appliance control
control circuit signals to the appliances (e.g., driving the output of IR remote
configured to control codes to the components).
receive said • “[A] Voice Controller 1 embodying the present invention
control signals consists generally of … an IR Output 12 and a
from said Microcontroller 20,” (Ex. 1006, 7:4-12)
speech • “The IR remote control codes from the user’s separate
recognition remote controllers are stored by the Microcontroller 20 until
system and such time as the user desires to control one of the separate
generate and components in their system. The IR remote control codes are
automatically output by IR Output 12. IR Output 12 … is also
transmit one or conventional, and consists of a well-known circuit including
more appliance
32
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

control signals three (3) infra-red light emitting diodes (LEDs) driven by
to said one or two transistors which in turn are driven from an output port
more of the Microcontroller 20.” (Id., 8:34-9:2)
appliances. • “When the user enters (…via a voice command) a particular
component command corresponding to one of the
component functions (such as volume for example),
processor 21, automatically selects both the appropriate
component and the remote control code corresponding to the
component function based on the component command
entered and the configuration data stored in reference RAM
23. This remote control code is then transmitted by IR
output 12 to the component to perform the function
corresponding to the component command.” (Id., 25:19-30)
• See also 29:16-27 (transmitting IR remote control codes
based on a sequence of codes automatically generated by
processor 21); 34:20-35:4 (outputting IR remote control
codes to control multiple appliances as necessary), 37:32-
39:26 (outputting IR remote control codes to multiple
components).
Claim 2 Bissonnette and Miyazawa
The audio Bissonnette discloses or at least renders obvious the
signal activated transmitter (IR Output 12) configured to wirelessly (e.g., via
control system IR or RF signaling) transmit the appliance control signals
as recited (e.g., IR remote control codes) to the one or more appliances
in claim 1, (components, e.g., TV, VCR, cable box).
wherein said • “While one embodiment of IR output 12 has been shown, a
transmitter is number of equivalent circuits which output IR remote
configured to control codes could be used in place of the circuit shown.
wirelessly Moreover, as with the remote control capture circuit
transmit said described above, the remote control output circuit (IR output
one or more 12) of the present invention could also easily be
appliance implemented by a skilled artisan to accommodate RF rather
control signals than IR signals.” (Ex. 1006, 9:34-10:4)
to said one or
more • See also id., 7:4-12; 8:34-9:2; 25:19-30; 29:16-27; 34:20-
appliances. 35:4; and 37:32-39:26 (all describing wireless IR output of
the IR remote control codes).

33
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 3 Bissonnette and Miyazawa


The audio Bissonnette discloses or at least renders obvious the
signal activated transmitter is an infrared transmitter (e.g., IR output 12). (Ex.
control system 1006, 9:34-35; see citations above regarding Claim 2)
as recited
in claim 2,
wherein said
transmitter is
an infrared
transmitter.
Claim 4 Bissonnette and Miyazawa
The audio Bissonnette discloses or at least renders obvious the
signal activated transmitter is an RF transmitter (e.g., output circuit
control system implemented to accommodate RF). (Ex. 1006, 9:34-10:4)
as recited
in claim 2,
wherein said
transmitter is
an RF
transmitter.

34
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 7 Bissonnette and Miyazawa


The audio As discussed above, Bissonnette, in view of Miyazawa,
signal activated discloses or at least renders obvious the low power sound
control system activation mode including the low power state of the processor.
as recited (See, e.g., claim 1[D] above.) Miyazawa further discloses or at
in claim 1, least renders obvious that the low power state of the processor
wherein said is a sleep state (e.g., a non-active, inactive, power-saving, low
low power power consumption, sleep mode state).
state of said • “[P]ower detector includes processing circuitry for forcing
processor is a the mechanism to selectively enter or terminate a low-power
sleep state. sleep mode.” (Ex. 1008, 3:18-20)
• “CPU 10 is placed in a non-active state or sleep mode, as is
well known in the computing arts.” (Id., 9:67-10:2)
• “CPU 10 is in an inactive, power-saving state during the
sleep mode, and only sound signal capture unit 1 and input
sound signal power detector 9 are in the active state. In
conventional speech recognition devices, power
consumption by an active mode CPU 10 is substantial in
order to handle complex speech recognition tasks. By
contrast, the power consumption by sound signal capture
unit 1 and input sound signal power detector 9 is relatively
quite small. In the present invention, CPU 10 momentarily
wakes up when a sound signal of at least a certain level is
input….Therefore, power consumption during the sleep
mode can be kept at an extremely low level, even in noisy
environments.” (Id., 11:50-67)

35
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 9 Bissonnette and Miyazawa


The audio Bissonnette discloses or at least renders obvious that the
signal activated speech recognition system switches from a speech recognition
control system mode (e.g., the voice operation mode/voice mode control) to a
as recited low power activation mode (idle mode 200) after, e.g.,
in claim 1, outputting IR remote control codes, as shown in annotated Fig.
wherein said 3 below.
speech
recognition
system is
further
configured to
automatically
terminate said
speech
recognition
mode and
automatically
switch to said
low power
sound
activation
mode.

• “[T]he control program of the present invention is normally


in idle mode 200.” (Ex. 1006, 33:27-29)
Miyazawa discloses or at least renders obvious the system
automatically terminating the speech recognition mode and
automatically switching to the low power sound activation
mode (e.g., when a non-keyword is detected or if there is a

36
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

duration of no input, the system returns to the low-power sleep


mode, in which only the input sound signal power detector 9 is
active).
• “After the sleep mode is initially set, the stuffed toy is
placed in the sleep mode after conversation with the toy is
finished. In this particular embodiment, if the speaker is
quiet for a specified duration or if the speaker says ‘Be
quiet’ or ‘Good night’ to the stuffed toy, the device
interprets this as a sleep mode request, sets the sleep mode
flag and places itself in the sleep mode.” (Ex. 1008, 9:52-58)
• “If the input sound signal is determined not to contain a
keyword such as ‘Power On’; ‘Hello’ ‘Good Morning’ or
the like, the input sound signal is judged to be noise or some
other speech signal not intended for the stuffed toy, so
control passes to step s7 CPU 10 goes inactive, and
processing returns to step s1, wherein the device again
listens for sound signal input.” (Id., 10:41-47)
• “If the speaker says ‘Be quiet’ or ‘Good night’ to the stuffed
toy or if the speaker is quiet for a specified duration, the
device interprets this as a sleep mode request….[I]f either of
these sleep mode requests is present,…the device and CPU
10 go into the sleep mode (step s15) and returns to the state
in step s1 again.” (Id., 11:18-29)
• “[B]y keeping the device … in the sleep mode, the speaker
need not bother to turn on the switch when he or she wants
to converse with the stuffed toy, and can simply speak to the
toy using a recognizable phrase, yet economical battery
drain characteristics can be preserved.” (Id., 11:33-42)
• “[R]eturning the speech recognition CPU to the sleep mode
if the detected first sound fails to contain the at least one
keyword recognized by the speech recognition CPU as an
activation command.” (Id., 14:55-58 (claim 1))

37
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 10 Bissonnette and Miyazawa


The audio Miyazawa discloses or at least renders obvious that conditions
signal activated for automatically terminating the speech recognition mode (as
control system discussed in claim 9 above) include detecting a predetermined
as recited period of time (e.g., a specified duration with no input).
in claim 9, • “[T]he stuffed toy is placed in the sleep mode after
wherein said conversation with the toy is finished. In this particular
speech embodiment, if the speaker is quiet for a specified
recognition duration…, the device interprets this as a sleep mode
system is request, sets the sleep mode flag and places itself in the
configured for sleep mode.” (Ex. 1008, 9:53-58)
automatically
terminating • “If … the speaker is quiet for a specified duration, the
said speech device interprets this as a sleep mode request….[T]he device
recognition and CPU 10 go into the sleep mode (step s15) and returns to
mode after a the state in step s1 again.” (Id., 11:18-29)
predetermined
period of time.

38
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 12 Bissonnette and Miyazawa


The audio Miyazawa discloses or at least renders obvious that conditions
signal activated for automatically terminating the speech recognition mode (as
control system discussed in claim 9 above) are based upon results of the
as recited decoding of the electrical representative signals (e.g., decoding
in claim 9, explicit voice commands (“Be quiet”) or errors (no keywords
wherein said result from decoding).
speech • “If the input sound signal is determined not to contain a
recognition keyword such as ‘Power On’; ‘Hello’ ‘Good Morning’ or
system is the like, the input sound signal is judged to be noise or some
configured for other speech signal not intended for the stuffed toy, so
automatically control passes to step s7 CPU 10 goes inactive, and
terminating processing returns to step s1, wherein the device again
said speech listens for sound signal input.” (Ex. 1008, 10:41-47)
recognition
mode based • “If the speaker says ‘Be quiet’ or ‘Good night’ to the stuffed
upon the toy…, the device interprets this as a sleep mode
results of said request….[T]he device and CPU 10 go into the sleep mode
decoding of (step s15) and returns to the state in step s1 again.” (Id.,
said electrical 11:18-29)
representative
signals.

39
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 14 Bissonnette and Miyazawa


The audio Bissonnette discloses or at least renders obvious a memory
signal activated (e.g., RAM 23, ROM Code Library 7) for storing data
control system associated with appliance control signals (e.g., voice
as recited templates, IR control codes).
in claim 1, • “This device includes a voice input circuit for receiving the
wherein said voice command and generating a voice signal based on the
control system voice command; an analog to digital converter … for
further converting the voice signal into a first digital voice signal; a
comprises a reference memory (a RAM) for storing a plurality of
memory for reference digital voice templates and the remote control
storage of data codes; a program memory (a ROM) for storing a control
associated with program; a processor … for converting the first digital voice
said appliance signal into a first voice template and for executing the
control signals. control program to determine whether the first voice
template is substantially equivalent to one of the reference
voice templates, and for selecting one of the remote control
codes based on the first voice template; and finally, a remote
control code transmitter coupled to the processor for
transmitting the selected remote control code.” (Ex. 1006,
3:2-20)
• “In order to recognize a spoken word or phrase, this voice
template is compared to the reference voice templates
previously trained and stored in RAM 23 which represent
the vocabulary of word commands used in the system.” (Id.,
32:32-36)
• “Microcontroller 20 works out of its high speed internal
memory RAM 23 any operations requiring high speed
memory access such as for voice recognition or IR code
capture or transmission. In addition to External DRAM 8 the
present invention accommodates an additional ROM Code
Library 7 to support a code library of known manufacturer
IR remote control codes. ROM Code Library 7 allows the
user to select an IR code associated with their equipment
without having to learn the remote control code through IR
Input 5.” (Id., 15:25-35)

40
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “The present invention includes sufficient RAM capacity to


learn and store the IR codes from several separate remote
control units.” (Id., 7:18-21)
• “a reference memory for storing a plurality of reference
digital voice templates and the remote control codes” (Id.,
48:14-16 (claim 1))
Claim 16 Bissonnette and Miyazawa
The audio Bissonnette discloses or at least renders obvious a memory
signal activated (e.g., RAM 23, ROM Code Library 7) for storing data
control system associated with a plurality of appliance control signals (e.g.,
as recited voice templates, a library of IR control codes) for controlling a
in claim 1, plurality of the appliances (e.g., TV, VCR, cable box, etc.; the
wherein said memory has capacity for storing IR codes from several separate
control system remote control units).
further • “Microcontroller 20 works out of its high speed internal
comprises a memory RAM 23 any operations requiring high speed
memory for memory access such as for voice recognition or IR code
storage of data capture or transmission. In addition to External DRAM 8 the
associated with present invention accommodates an additional ROM Code
a plurality of Library 7 to support a code library of known manufacturer
said appliance IR remote control codes. ROM Code Library 7 allows the
control signals user to select an IR code associated with their equipment
for controlling without having to learn the remote control code through IR
a plurality of Input 5.” (Ex. 1006, 15:25-35)
said
appliances. • “This circuit is used to capture IR remote control codes for
later control of each unit of the user’s equipment, such as
their TV, VCR, cable box, satellite dish receiver and the
like. The present invention includes sufficient RAM
capacity to learn and store the IR codes from several
separate remote control units.” (Id., 7:15-21)
• Id., 3:2-20 and 32:32-36 (describing RAM for storing
reference digital voice templates and remote control codes,
and describing the selection of remote control codes based
on matched voice templates).

41
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 21 Bissonnette and Miyazawa


21[P] To the extent the preamble is considered limiting, Bissonnette
An audio discloses or at least renders obvious an audio signal control
signal activated system (including voice controller 1) for controlling appliances
control system (electronic equipment and electronic components). (Ex. 1006
for controlling Abstract; 2:34-3:2; annotated Fig. 1 in claim 1[P])
one or more To the extent the preamble is considered limiting, Miyazawa
appliances, discloses or at least renders obvious an audio signal activated
said control control system (a voice activated interactive speech
system mechanism) for controlling appliances (electronic appliances
comprising: and similar devices). (Ex. 1008, 2:56-3:7, 14:8-14, 14:23-26;
see citations above regarding claim 1[P].)
21[A] Bissonnette discloses or at least renders obvious a microphone
a microphone (e.g., analog voice input 2 that can be or include a microphone)
for receiving for receiving audio signals (e.g., from the user’s voice) and
audio signals converting the audio signals to electrical signals (e.g.,
and converting conditioned signals for microcontroller 20). (Id., 10:8-19;
said audio annotated Fig. 1 in claim 1[A]; 10:30-11:22; 32:15-17; 48:7-9
signals to (claim 1); see citations above regarding claim 1[A].)
electrical
signals;
21[B] Bissonnette discloses or at least renders obvious a speech
a speech recognition circuit that includes a processor (e.g.,
recognition microcontroller 20 that includes microprocessor 21 and voice
circuit recognition software stored in ROM 22 of Fig. 1). (Id., 14:25-
including a 27; 12:3-10; annotated Fig. 1 in claim 1[B]; see citations above
processor and regarding claim 1[B])
having a Bissonnette discloses or at least renders obvious a plurality of
plurality of modes, including a low power … mode (low power idle mode
modes of 200) and a speech recognition mode for converting said
operation electrical signals to electrical representative signals (e.g., in
including a voice mode control of Fig. 3, converting the conditioned
speech signals for into voice templates). (Id., 33:27-29; 11:28-31;
recognition 33:19-23; 32:17-18; 12:7-10; 48:19-22; 32:13-32; 31:4-7; 6:12-
mode and a 14; annotated Fig. 3 in claim 1[B]; see citations above
low power regarding claim 1[B])
sound

42
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

activation Miyazawa discloses or at least renders obvious a low power (a


mode, sleep mode in which CPU 10 is in an inactive power-saving
state) sound activation mode for detecting the presence of said
electrical signals (in the sleep mode, detecting the presence of
input sound signal power above a threshold before activating
the CPU 10). (Ex. 1008, 11:50-67, 10:3-16; see citations above
regarding claim 1[B])
21[C] wherein Bissonnette discloses or at least renders obvious the speech
in said speech recognition mode for converting said electrical signals to
recognition electrical representative signals (e.g., in voice mode control of
mode said Fig. 3, converting the conditioned signals for into voice
speech templates). (Id., 33:19-23; 32:17-18; 12:7-10; 48:19-22; 32:13-
recognition 32; 31:4-7; 6:12-14; annotated Fig. 3 in claim 1[B]; see
circuit converts citations above regarding claims 1[B]-1[C])
said electrical Bissonnette discloses or at least renders obvious the speech
signals to recognition circuit, including the processor, decoding said
electrical electrical representative signals (e.g., microprocessor 21
representative decodes and analyzes voice templates). (Ex. 1006, 12:3-10;
signals and 38:21-25; 48:19-28; see citations above regarding claim 1[C])
said processor
decodes said
electrical
representative
signals and
21[D] wherein As discussed above, Bissonnette, in view of Miyazawa,
in said sound discloses or at least renders obvious the sound activation mode
activation is a low power sound activation mode. (See, e.g., claims 1[B]
mode said and 21[B] above.) Miyazawa discloses or at least renders
processor is obvious that the low power sound activation mode includes a
placed in a low low power state of the processor. (Ex. 1008, 3:18-20, 9:67-
power state; 10:2, 11:50-67; see citations above regarding claim 1[D])
21[E] a sound Bissonnette discloses or at least renders obvious switching
activation from an activation mode (idle mode 200) to the speech
circuit recognition mode (e.g., the voice operation mode/voice mode
configured for control). (Ex. 1006, 33:20-29; 6:12-14; annotated Fig. 3 in
determining if claim 1[E]; see citations above regarding claim 1[E])
the amplitude
of said
43
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

electrical Miyazawa discloses or at least renders obvious a sound


signals exceeds activation circuit (e.g., input sound signal power detector 9)
a configured for determining if the amplitude of the electrical
predetermined signals exceeds a predetermined threshold (e.g., whether or
threshold and not the power of the input sound signal is greater than a preset
causing said threshold th1) and causing the speech recognition circuit to
speech switch automatically from the sound activation mode to
recognition another mode (e.g., switch to the speech recognition mode, the
circuit to voice operation mode of Bissonnette). (Ex. 1008, 3:13-17,
switch 4:10-13, 10:8-21, Abstract; see citations above regarding claim
automatically 1[E])
from said
sound
activation
mode to
another of said
plurality of
said modes of
operation;
21[F] said Bissonnette discloses or at least renders obvious the speech
speech recognition circuit generating first control signals (remote
recognition control codes used to drive the output of IR remote control
circuit codes) in the speech recognition mode (the voice operation
configured for mode/voice mode control) if the electrical signals represent
generating first one or more predetermined audible commands (user voice
control signals commands correspond to component functions).
in said speech • “When the user enters (…via a voice command) a particular
recognition component command corresponding to one of the
mode if said component functions (such as volume for example),
electrical processor 21, automatically selects both the appropriate
signals component and the remote control code corresponding to the
represent one component function based on the component command
or more entered and the configuration data stored in reference RAM
predetermined 23. This remote control code is then transmitted by IR
audible output 12 to the component to perform the function
commands, corresponding to the component command.” (Id., 25:19-30)
• “IR Output 12 … is also conventional, and consists of a
well-known circuit including three (3) infra-red light
44
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

emitting diodes (LEDs) driven by two transistors which in


turn are driven from an output port of the Microcontroller
20.” (Id., 8:34-9:2)
• See also 29:16-27 (transmitting IR remote control codes
based on a sequence of codes automatically generated by
processor 21); 34:20-35:4 (outputting IR remote control
codes to control multiple appliances as necessary), 37:32-
39:26 (outputting IR remote control codes to multiple
components).
• See also citations and discussions above regarding claim
1[F].
21[G] said Bissonnette discloses or at least renders obvious that the
speech speech recognition circuit is configured for switching from
recognition the speech recognition mode (e.g., the voice operation
circuit mode/voice mode control) to another mode of the plurality of
configured for the modes of operation (to an idle mode 200), under
switching predetermined conditions (e.g., after outputting an IR code
automatically (steps 304, 309, 313)), as shown in annotated Fig. 3 below.
from said
speech
recognition
mode to
another of said
plurality of
said modes of
operation
under
predetermined
conditions; and

45
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “[T]he control program of the present invention is normally


in idle mode 200.” (Ex. 1006, 33:27-29)
Miyazawa further discloses or at least renders obvious, e.g.,
the system automatically terminating the speech recognition
mode and automatically switching from the speech
recognition mode to another of the plurality of modes of
operation (switching from the speech recognition mode, in
which CPU 10 is awake to perform speech analysis processing,
to the low-power sleep mode, in which only the input sound
signal power detector 9 is active) under predetermined
conditions, such as when a non-keyword is detected or if there
is a duration of no input.

46
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “If the input sound signal is determined not to contain a


keyword such as ‘Power On’; ‘Hello’ ‘Good Morning’ or
the like, the input sound signal is judged to be noise or some
other speech signal not intended for the stuffed toy, so
control passes to step s7 CPU 10 goes inactive, and
processing returns to step s1, wherein the device again
listens for sound signal input.” (Ex. 1008, 10:41-47)
• “If the speaker says ‘Be quiet’ or ‘Good night’ to the stuffed
toy or if the speaker is quiet for a specified duration, the
device interprets this as a sleep mode request….[I]f either of
these sleep mode requests is present,…the device and CPU
10 go into the sleep mode (step s15) and returns to the state
in step s1 again.” (Id., 11:18-29)
21[H] an Bissonnette discloses or at least renders obvious an appliance
appliance control circuit, including a transmitter (e.g, transistors driving
control circuit a transmitter IR Output 12 having three LEDs), that receives
configured for the first control signals from the speech recognition circuit
receiving said (receives the remote control codes as driven by Microcontroller
first control 20) and generates second control signals to cause appliances
signals from to perform one or more functions associated with the first
said speech control signals (e.g., driving the output of IR remote control
recognition codes to the components). (Ex. 1006, 7:4-12; 8:34-9:2; 25:19-
circuit and 30; 29:16-27; 37:32-39:26; see citations above regarding claim
generating 1[F])
second control
signals to cause
one or more
appliances to
perform one or
more functions
associated with
said first
control signals.

47
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 23 Bissonnette and Miyazawa


The audio Bissonnette discloses or at least renders obvious the appliance
signal activated control circuit (including IR output 12) configured to
control system wirelessly transmit (e.g., via IR or RF signaling) the second
as recited control signals (e.g., IR remote control codes) to the one or
in claim 21, more appliances (components, e.g., TV, VCR, cable box).
wherein said • “While one embodiment of IR output 12 has been shown, a
appliance number of equivalent circuits which output IR remote
control circuit control codes could be used in place of the circuit shown.
is configured to Moreover, as with the remote control capture circuit
wirelessly described above, the remote control output circuit (IR output
transmit said 12) of the present invention could also easily be
second control implemented by a skilled artisan to accommodate RF rather
signals to said than IR signals.” (Ex. 1006, 9:34-10:4)
one or more
appliances. • See also id., 7:4-12; 8:34-9:2; 25:19-30; 29:16-27; 34:20-
35:4; and 37:32-39:26 (all describing wireless IR output of
the IR remote control codes).
Claim 24 Bissonnette and Miyazawa
The control Bissonnette discloses or at least renders obvious transmitting
system as the second signals as RF signals (e.g., output circuit
recited in claim implemented to accommodate RF). (Ex. 1006, 9:34-10:4)
23, wherein
said second
control signals
are RF signals.
Claim 25 Bissonnette and Miyazawa
The control Bissonnette discloses or at least renders obvious transmitting
system as the second signals as infrared signals (e.g., IR output 12). (Ex.
recited in claim 1006, 9:34-35; see also citations above regarding claim 23)
23, wherein
said second
control signals
are infrared
signals.

48
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 26 Bissonnette and Miyazawa


The audio Miyazawa discloses or at least renders obvious the sound
signal activated activation circuit (e.g., input sound signal power detector 9)
control system causing the speech recognition circuit (e.g., Bissonnette’s
as recited microprocessor 21 and voice recognition software stored in
in claim 21, ROM 22 of Fig. 1) to switch automatically from the sound
wherein said activation mode (e.g., the low power idle/sleep mode) to
sound another of the plurality of modes, such as the speech
activation recognition mode (e.g., switch to the speech recognition mode,
circuit is the voice operation mode of Bissonnette). (Ex. 1008, 3:13-17,
configured to 4:10-13, 10:8-21, Abstract; see citations above regarding claim
cause said 1[E] and 21[E])
speech
recognition
circuit to
switch
automatically
from said
sound
activation
mode to
another of said
plurality of
said modes of
operation,
wherein
another of said
plurality of
said modes of
operation is
said speech
recognition
mode.

49
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 28 Bissonnette and Miyazawa


The audio As discussed above, Bissonnette, in view of Miyazawa,
signal activated discloses or at least renders obvious the low power sound
control system activation mode including the low power state of the processor.
as recited (See, e.g., claim 21[D] above.) Miyazawa further discloses or
in claim 21, at least renders obvious that the low power state of the
wherein said processor is a sleep state (e.g., a non-active, inactive, power-
low power saving, low power consumption, sleep mode state). (Ex. 1008,
state of said 3:18-20, 9:67-10:2, 11:50-67; see citations above regarding
processor is a claim 7)
sleep state.

50
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 29 Bissonnette and Miyazawa


The audio Bissonnette discloses or at least renders obvious that the
signal activated speech recognition circuit (e.g., microcontroller 20 that
control system includes microprocessor 21 and voice recognition software
as recited stored in ROM 22) is configured to automatically switch from
in claim 21, the speech recognition mode (e.g., the voice operation
wherein said mode/voice mode control) to another of the plurality of the
speech modes, including the idle mode (an idle mode 200), under
recognition predetermined conditions (e.g., after outputting an IR code
circuit is (steps 304, 309, 313)), as shown in Fig. 3 below). (Ex. 1006,
configured for 33:27-29)
switching
automatically
from said
speech
recognition
mode to
another of said
plurality of
said modes of
operation
under
predetermined
conditions,
wherein
another of said
plurality of
said modes of
operation is
said sound
activation
mode.

Miyazawa further discloses or at least renders obvious, e.g.,


the system automatically switching from the speech

51
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

recognition mode to another of the plurality of modes of


operation, including the sound activation mode (switching
from the speech recognition mode, in which CPU 10 is awake
to perform speech analysis processing, to the low-power sleep
mode, in which only the input sound signal power detector 9 is
active) under predetermined conditions, such as when a non-
keyword is detected or if there is a duration of no input.
• “If the input sound signal is determined not to contain a
keyword such as ‘Power On’; ‘Hello’ ‘Good Morning’ or
the like, the input sound signal is judged to be noise or some
other speech signal not intended for the stuffed toy, so
control passes to step s7 CPU 10 goes inactive, and
processing returns to step s1, wherein the device again
listens for sound signal input.” (Ex. 1008, 10:41-47)
• “If the speaker says ‘Be quiet’ or ‘Good night’ to the stuffed
toy or if the speaker is quiet for a specified duration, the
device interprets this as a sleep mode request….[I]f either of
these sleep mode requests is present,…the device and CPU
10 go into the sleep mode (step s15) and returns to the state
in step s1 again.” (Id., 11:18-29)
• See also id., 9:53-58 (describing putting the device in sleep
mode when conversation is finished).
Claim 30 Bissonnette and Miyazawa
The audio Miyazawa discloses or at least renders obvious that
signal activated predetermined conditions for automatically terminating the
control system speech recognition mode and automatically switching to the
as recited low power sound activation mode include a predetermined
in claim 21, period of time (e.g., a specified duration with no input).
wherein said • “If … the speaker is quiet for a specified duration, the
predetermined device interprets this as a sleep mode request….[T]he device
conditions and CPU 10 go into the sleep mode (step s15) and returns to
include a the state in step s1 again.” (Ex. 1008, 11:18-29)
predetermined
period of time. • See also id., 9:53-58 (describing putting the device in sleep
mode when conversation is finished).

52
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 31 Bissonnette and Miyazawa


The audio Miyazawa discloses or at least renders obvious that
signal activated predetermined conditions for automatically switching to the
control system low power sound activation mode include a period of time that
as recited is based upon the results of determining if the electrical
in claim 21, signals represent the predetermined audible commands (e.g.,
wherein said when in the speech recognition mode (i.e., after a keyword of
predetermined sufficient amplitude has been detected, switching back to the
conditions low power sound activation mode if a specific duration of time
include a passes with no input).
period of time • See Ex. 1008, Fig. 4 (below, annotated flow of steps s6, s10,
that is based s11, and s15).
upon the
results of • “[I]f the input sound signal is determined to be a keyword in
determining if step s6, … sleep mode flag is cleared for shifting the device
said electrical from the sleep mode to the active mode…. [I]f it is
signals determined that the input speech signal has been completed
represent said for a single interaction, the processing shifts to the
predetermined determination of whether or nor a sleep mode request is
audible present (step s11)…. If … if the speaker is quiet for a
commands. specified duration, the device interprets this as a sleep mode
request. If neither of these sleep mode requests is present,
the device assumes that the conversation is continuing….
On the other hand, if either of these sleep mode requests is
present, the device issues a shutdown response as needed
(step s13)….[T]he device and CPU 10 go into the sleep
mode (step s15) and returns to the state in step s1 again.”
(Id., 11:18-29)

53
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

54
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 39 Bissonnette and Miyazawa


The audio Bissonnette discloses or at least renders obvious a signal
signal activated circuit (e.g., A/D 24) that enables the electrical signals (from
control system analog voice input 2) to be in communication with the system,
as recited including the speech recognition circuit (microcontroller 20
in claim 21, that includes microprocessor 21 and voice recognition software
wherein said stored in ROM 22).
control system • “[T]he voice signal output of the microphone is fed into an
further analog input section of Analog Voice Input 2. The signal is
comprises a then conditioned by well known electronic circuits that
signal circuit amplify and filter the voice input signal from the
for enabling microphone prior to going to an analog to digital converter
said electrical (ADC) 24 in Microcontroller 20.” (Ex. 1006, 10:30-36)
signals to be in
communication • See annotated Fig. 1:
with said
speech
recognition
circuit in said
speech
recognition
mode and for
enabling said
electrical
signals to be in
communication
with said sound
activation
circuit in said
sound
activation
mode.
As discussed above, Bissonnette, as modified by Miyazawa,
includes a sound activation circuit (e.g., input sound signal
power detector 9 in Miyazawa) in addition to the existing
circuit for speech recognition (e.g., microcontroller 20 that
includes microprocessor 21 and voice recognition software
stored in ROM 22 in Bissonnette) as recipients of sound input

55
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

signals based on the mode of the device. See annotated Fig. 2


of Miyazawa, showing communication with each circuit:

Therefore, in the sound activation and speech recognition


modes, the signal circuit enables the electrical signals of
Bissonnette, modified by the teachings of Miyazawa, to be in
communication with each of the sound activation and speech
recognition circuits, respectively. In other words, for electrical
signals to be transmitted from the voice/sound input circuit and
then be processed or analyzed by the sound activation circuit
and speech recognition circuits, communications between the
circuits must be enabled.

56
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Claim 41 Bissonnette and Miyazawa


41[P] To the extent the preamble is considered limiting, Bissonnette
A method for discloses or at least renders obvious a method for providing
providing hands-free control of appliances.
hands-free • “[T]here is provided a portable remote control device
control of operated by a voice command for controlling one or more
appliances, electronic components each which in turn is responsive to a
comprising: set of respective remote control codes.” (Ex. 1006, 2:34-3:2;
see also Abstract)
• See Fig. 3, showing an example flow diagram for
Bissonnette’s voice operation mode software routine. (Id.,
6:12-14)

57
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

41[A] Bissonnette discloses or at least renders obvious providing a


providing a speech recognition system having a processor (e.g.,
speech microcontroller 20 that includes microprocessor 21 and voice
recognition recognition software stored in ROM 22 of Fig. 1, annotated
system having below) that has a speech recognition mode (e.g., processing of
a speech voice commands in the voice operation mode/voice mode
recognition control).
mode wherein • “To maximize on battery life, Analog Voice Input 2 is also
a processor powered up only when the user activates one of three voice
recognizes keys described below to speak into the unit.” (Id., 11:28-31)

58
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

representations • “--processing of voice commands--


of audible Referring generally to Figure 3, to operate the present
signals as invention by use of voice commands the user selects one of
corresponding three voice keys (shown as 33, 34, or 37 in Fig. 5A)
to depending on the desired function.” (Id., 33:19-23)
predetermined • See annotated Fig. 1, showing microcontroller 20 and
audible microprocessor 21:
commands;

• See Fig. 3, annotated to show the steps of the diagram


involving voice mode control (id., 6:12-14; 31:4-7)

59
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Bissonnette discloses or at least renders obvious the processor


recognizing representations of audible signals (e.g., voice
templates representing audible signals from the user’s voice) as
corresponding to predetermined audible commands (e.g.,
determining corresponding components and remote control
codes).
• “An Analog Voice Input 2 converts the audio information in
the user's voice to an analog electrical signal and also
conditions this electrical signal for processing by

60
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

Microcontroller 20….Analog Voice Input 2 includes a


microphone….” (Id., 10:8-19; see also Fig. 1)
• “The digital voice signal is converted into a voice template
that is compared against previously stored voice templates
of the user's voice. The program then decodes the voice
templates as explained further below.” (Id., 12:7-10)
• “When the user enters (…via a voice command) a particular
component command corresponding to one of the
component functions (such as volume for example),
processor 21, automatically selects both the appropriate
component and the remote control code corresponding to the
component function based on the component command
entered and the configuration data stored in reference RAM
23. This remote control code is then transmitted by IR
output 12 to the component to perform the function
corresponding to the component command.” (Id., 25:19-30;
see also 29:16-27; 37:32-39:26.)
• “[A] processor … for generating a first digital voice
template from the first digital voice signal….” (Id., 48:19-22
(claim 1))
41[B] Bissonnette discloses or at least renders obvious configuring
configuring the speech recognition system to have a low power mode
said speech wherein the processor is in a low power state (e.g., idle mode
recognition 200).
system to have • “Beginning at the top of the flow diagram in Figure 3, the
a low power control program of the present invention is normally in idle
sound mode 200.” (Id., 33:27-29)
activation • “To maximize on battery life, Analog Voice Input 2 is also
mode wherein powered up only when the user activates one of three voice
said processor keys described below to speak into the unit.” (Id., 11:28-31)
is in a low
power state; Miyazawa discloses or at least renders obvious using a low
power (a sleep mode in which CPU 10 is in an inactive power-
saving state) sound activation mode wherein the processor is
in a low power state (in the sleep mode, detecting the presence
of input sound signal power above a threshold before activating
the CPU 10).

61
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

• “CPU 10 is placed in a non-active state or sleep mode, as is


well known in the computing arts.” (Id., 9:67-10:2)
• “CPU 10 is in an inactive, power-saving state during the
sleep mode, and only sound signal capture unit 1 and input
sound signal power detector 9 are in the active state. In
conventional speech recognition devices, power
consumption by an active mode CPU 10 is substantial in
order to handle complex speech recognition tasks. By
contrast, the power consumption by sound signal capture
unit 1 and input sound signal power detector 9 is relatively
quite small. In the present invention, CPU 10 momentarily
wakes up when a sound signal of at least a certain level is
input….Therefore, power consumption during the sleep
mode can be kept at an extremely low level, even in noisy
environments.” (Ex. 1008, 11:50-67)
41[C] As discussed above, Bissonnette, in view of Miyazawa,
monitoring discloses or at least renders obvious the low power sound
said audible activation mode and the speech recognition mode. (See, claims
signals to 41[A] and [B] above.)
detect if the Miyazawa discloses or at least renders obvious enabling the
amplitude of speech recognition system to switch automatically from the
said audible sound activation mode to the speech recognition mode
signals exceeds (switching from the low-power sleep mode, in which only the
a input sound signal power detector 9 is active, to the speech
predetermined recognition mode, in which CPU 10 wakes to perform speech
threshold; analysis processing) if an amplitude of the audible signal
enabling said exceeds a predetermined threshold (automatically switching as
speech a function of the amplitude of the input sound signal exceeding
recognition a threshold).
system to • “[I]nput sound signal power detector in communication with
switch at least the sound signal input unit and the interaction
automatically controller for detecting the volume, magnitude or amplitude
from said of input sound signals based on sound signal waveforms
sound perceived by the sound input unit or capture device.” (Ex.
activation 1008, 3:13-17, 4:10-13)
mode to said • “In step s1, input sound signal power detector 9 determines
speech whether or not the power of the input sound signal is greater

62
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

recognition than a preset threshold th1, and outputs a signal indicating


mode if said that a sound signal has been input when the power of the
amplitude input sound signal becomes greater than threshold th1 (at
exceeds said time t1 FIG. 3A). In such manner the input signal power
predetermined detector continues to monitor for sound input signals until
threshold; one meeting or exceeded threshold th1 is encountered.
Thereafter, control passes to step s2 in which CPU 10 wakes
up (step s2), and after a specified amount of time following
t1, (in about 10 msec) speech analyzer 2 thereof performs
speech analysis processing based on the signal captured by
sound signal capture unit 1 (step s3).” (Id., 10:8-21;
Abstract)
41[D] Bissonnette discloses or at least renders obvious generating
generating and representations of the audible signals (e.g., converting into
decoding voice templates) and decoding representations of the audible
representations signals (e.g., decoding the voice templates) in the speech
of said audible recognition mode (the voice operation mode/voice mode
signals in said control functions to determine appropriate IR remote control
speech codes for output).
recognition • “Microcontroller 20 then processes the digital voice signal
mode; and by means of microprocessor 21 and a voice recognition
software routine that is part of a control program stored in
ROM 22. The digital voice signal is converted into a voice
template that is compared against previously stored voice
templates of the user's voice. The program then decodes the
voice templates….” (Ex. 1006, 12:3-10)
• “[T]he voice recognition software within the control
program processes and decodes the spoken word at 311,
performs a device select at 312, and then outputs the
appropriate IR remote control code at 313.” (Id., 38:21-25)
• “When the user enters (…via a voice command) a particular
component command corresponding to one of the
component functions (such as volume for example),
processor 21, automatically selects both the appropriate
component and the remote control code corresponding to the
component function based on the component command
entered and the configuration data stored in reference RAM

63
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

23. This remote control code is then transmitted by IR


output 12 to the component to perform the function
corresponding to the component command.” (Id., 25:19-30;
see also 29:16-27.)
41[E] Bissonnette discloses or at least renders obvious generating
generating and and transmitting one or more appliance control signals for
transmitting controlling one or more appliances (e.g., remote control codes)
one or more if the decoded representations of the audio signals represent
appliance one or more predetermined audible commands (e.g., if the
control signals decoded voice templates correspond to an IR remote control
for controlling code).
one or more • “When the user enters (…via a voice command) a particular
appliances if component command corresponding to one of the
said decoded component functions (such as volume for example),
representations processor 21, automatically selects both the appropriate
of said audio component and the remote control code corresponding to the
signals component function based on the component command
represent one entered and the configuration data stored in reference RAM
or more said 23. This remote control code is then transmitted by IR
predetermined output 12 to the component to perform the function
audible corresponding to the component command.” (Id., 25:19-30)
commands.
• See also 29:16-27 (transmitting IR remote control codes
based on a sequence of codes automatically generated by
processor 21); 37:32-39:26 (outputting IR remote control
codes to multiple components).
Claim 43 Bissonnette and Miyazawa
The method for As discussed above, Bissonnette, in view of Miyazawa,
providing discloses or at least renders obvious the low power sound
hands-free activation mode including the low power state of the processor.
control of (See, e.g., claim 41[B] above.) Miyazawa further discloses or
appliances as at least renders obvious that the low power state of the
recited in claim processor is a sleep state (e.g., a non-active, inactive, power-
41, wherein saving, low power consumption, sleep mode state). (Ex. 1008,
said low power 3:18-20, 9:67-10:2, 11:50-67; see citations above regarding
state of said claims 7 and 28.)
processor is a
sleep state.

64
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

(Ex. 1003 ¶¶ 41, 55.)

B. Ground 2: Claims 17-18 are rendered obvious by Bissonnette in


view of Miyazawa and Salazar under 35 U.S.C. § 103(a)

Claims 17 of the ’186 patent recites a programming mode that enables a user

to define a control signal associated with an audible command for use in the speech

recognition mode. Claim 18 recites similar limitations as claim 17 but differs,

primarily, by reciting a plurality of control signals (see bold emphasis in chart

below).

Claim 17 of the ’186 patent Claim 18 of the ’186 patent


The audio signal activated control The audio signal activated control
system as recited in claim 1, wherein system as recited in claim 1, wherein
said speech recognition system further said speech recognition system further
includes a programming mode to includes a programming mode to
enable a user to define a control signal enable a user to define a plurality of
to be associated with a defined audible control signals to be associated with a
command, wherein in said speech defined audible command, wherein in
recognition mode said speech said speech recognition mode said
recognition system generates said speech recognition system generates
defined control signal for controlling said defined plurality of control
one or more appliances if said electrical signals for controlling said one or
signals represent said defined audible more appliances if said electrical
command. signals represent said defined audible
command.

Bissonnette, in view of Miyazawa, discloses the audio activated control

system of claim 1. See Section VII.A.

Bissonnette discloses that system configuration data is used by the system to

determine which appliances to perform different functions (Ex. 1006, 23:13-25:36)


65
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

and that voice commands can be used to control a plurality of appliances in sequence.

(See, e.g., id., 37:32-38:12 (“PLAY” command outputs control signals to a TV and

VCR).) Miyazawa discloses that recognition target phrases are created and

registered in memory in advance. (Ex. 1008, 6:58-7:6.) (Ex. 1003 ¶¶ 56-59.)

Bissonnette and Miyazawa do not explicitly disclose a programming mode

that enables a user to define the control signals associated with the audible

commands.

Salazar is titled “Real-Time Reconfigurable Adaptive Speech Recognition

Command and Control Apparatus and Method,” and discloses a mobile device that

uses voice recognition to control other devices/equipment using voice commands.

(Ex. 1009, Abstract.) Salazar discloses a macro programming mode that enables

users to create and modify commands. (Ex. 1009, 12:1-2, 18:4-11.) For example, a

macro command word can be used to generate and output a list of commands to a

plurality of camera appliances. (Id., 15:51-58.) “Macro command processes …

allows recognition of one spoken word to execute a series of discrete commands.”

(Id., 1:26-28.) (Ex. 1003 ¶¶ 60-61.)

A POSITA would have been motivated to include a programming mode to

enable a user to define control signals associated with audible commands, wherein

the speech recognition system generates defined control signals for controlling one

or more appliances upon receipt of the defined corresponding audible commands, as

66
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

taught by Salazar, in the system of Bissonnette and Miyazawa. While Bissonnette

and Miyazawa disclose that commands and control signal data were stored in the

system, they do not explain that the user was enabled to define and associate controls

with audible commands. However, including Salazar’s programming mode would

have provided the benefit of user customizability, allowing the user to create macro

commands to generate a plurality of control signals for a defined audible command,

allowing the user to create predefined scenes based on a single spoken word (e.g.,

Bissonnette’s PLAY command generating the multiple commands to control the TV

and VCR for the activity of playing a tape). (Ex. 1003 ¶ 62.)

A POSITA would have had a reasonable expectation of success in

incorporating a programming mode in the system of Bissonnette and Miyazawa.

Such modification would have been implemented via user prompts (to gather desired

input data from the user) and modifications to memory (to store the commands and

control signals). Such basic user interfaces and data access techniques would have

been well-known and within a POSITA’s ability to apply in the electronic devices

of Bissonnette and Miyazawa, which already had various modes and programming

functions. (See, e.g., Ex. 1006, Figs. 2-4.) (Ex. 1003 ¶ 63.)

67
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

C. Ground 3: Claims 19-20 are rendered obvious by Bissonnette in


view of Miyazawa and McCall under 35 U.S.C. § 103(a)

Claims 19 and 20 of the ’186 patent recite limitations that require, inter alia,

decoding the electrical representative signals to define an appliance type and

determining if the electrical representative signals correspond to functions

associated with the appliance type.

Claim 19 of the ’186 patent Claim 20 of the ’186 patent


The audio signal activated control The audio signal activated control
system as recited in claim 1, wherein system as recited in claim 19, wherein
said speech recognition system wherein said speech recognition system in said
in said speech recognition mode is speech recognition mode is further
further configured for configured for
decoding said electrical representative decoding said electrical representative
signals to define an appliance type, said signals to determine if said signals
appliance type defining said appliance represent predetermined audible
to which said appliance control signals commands corresponding to control
are transmitted by said appliance functions associated with said defined
control circuit. appliance type.

Bissonnette, in view of Miyazawa, discloses the audio activated control

system of claim 1. See Section VII.A.

Bissonnette discloses that its system includes multiple controllable appliance

types (e.g., TVs, VCRs, and cable boxes; Ex. 1006, 3:32-4:15). Bissonnette

discloses entering various voice control modes for the different appliances (i.e.,

pressing a VOICE VCR key 33 to control VCR playback functions by voice,

68
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

pressing a VOICE CHANNEL key 37 to set a channel number by voice; 37:5-14,

38:13-25). (Ex. 1003 ¶¶ 64-67.)

Bissonnette does not explicitly disclose that the audio activated control

system decodes electrical representative signals to define an appliance type (claim

19) and further decodes electrical representative signals to determine commands

corresponding to functions associated with the defined appliance type (claim 20).

McCall discloses voice control

techniques for use in the medical arts and

analogous fields of endeavor—“anywhere

there is a need for speech recognition

control by a human operator over a plurality

of integrated voice-controllable devices.”

(Ex. 1007, Abstract, 1:15-20.) As shown in

annotated Fig. 2 (right), McCall discloses a

voice activated device selection command

step 104 (which determines which of a

plurality of devices are to be controlled by

voice) before proceeding to a function

selection command step 120 and further command steps 152 and 154. (Id., 7:49-56.)

(Ex. 1003 ¶¶ 68-69.)

69
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

A POSITA would have been motivated to use a voice control system by

decoding signals to define an appliance type and determine commands

corresponding to control functions associated with the defined appliance type, as

taught by McCall, to control the plurality of appliances in Bissonnette. As explained

by McCall, the user is assured that commands intended for one appliance are not

confused with commands for any other appliance. (Ex. 1007, 7:57-8:5.) Thus, for

example, where Bissonnette discloses a TV, VCR, and cable box (which may have

common commands; see Ex. 1006, 4:20-22), McCall would have added the benefit

of controls directed at the appliance of choice. (Ex. 1003 ¶ 70.)

A POSITA would have had a reasonable expectation of success in

incorporating the hierarchical commands of McCall in the system of Bissonnette in

view of Miyazawa. In particular, Bissonnette and Miyazawa each already disclose

the basic functions of receiving, processing, decoding, and outputting voice

commands. See Section VII.A. The added function of having one voice command

lead to another (i.e., identifying the appliance and then commanding the appliance)

would have been like implementing categories in a menu of commands—a basic

logical structure. As evidenced by Bissonnette above, classes of commands (i.e.,

VCR commands, CHANNEL commands) had already been implemented by others

in the art. It would have been well within the skill of a POSITA to simply access

70
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

those commands by the well-known input disclosed by all of Bissonnette,

Miyazawa, and McCall—voice. (Ex. 1003 ¶¶ 26-28, 71.)

VIII. CONCLUSION

Petitioner has shown a reasonable likelihood of prevailing with respect to the

Challenged Claims and respectfully requests institution of trial and cancellation of

the Challenged Claims.

Respectfully submitted,
ROPES & GRAY LLP

Dated: December 27, 2018 /Scott A. McKeown /


Scott A. McKeown
Reg. No. 42,866

71
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

CERTIFICATE OF COMPLIANCE

Pursuant to 37 C.F.R. § 42.24(a) and (d), the undersigned hereby certify that

the Petition For Inter Partes Review complies with the type-volume limitation of

37 C.F.R. § 42.24(a)(i) because, exclusive of the exempted portions, it contains

13,883 words as counted by the word processing program used to prepare the

paper.

DATED: December 27, 2018 ROPES & GRAY LLP


/Scott A. McKeown/
Scott A. McKeown

72
U.S. Patent No. 6,397,186
Petition for Inter Partes Review

CERTIFICATE OF SERVICE

The undersigned certifies service pursuant to 37 C.F.R. §§ 42.6(e) and

42.105(b) on the Patent Owners by Express Mail of a copy of this Petition for Inter

Partes Review and supporting materials at the correspondence address of record for

the ‘186 patent:

FITCH EVEN TABIN & FLANNERY, LLP


120 South LaSalle Street
Suite 2100
Chicago, IL 60603-3406

Sean A. Luner
Dovel & Luner, LLP
201 Santa Monica Blvd., Suite 600
Santa Monica, California 90401

Dated: December 27, 2018 /Jonathan Bradford/


Jonathan Bradford

73