Hotword Detection

[alanjames1987]

I would like to have hotword detection availible in Annyang, basically allowing me to tell Annyang to not "activate" until a certain hotword is spoken. Essentally speech recognition would be working the entire time but only start caching results returned from the webkitSpeechRecognition from the first instance of the spoken hotword.

Similar to how Okay Google works, but without the plugin.

[TalAter]

Closing as a duplicate of #18

[alanjames1987]

I'm not meaning a command prefix, I'm meaning a string that when detected will trigger a callback function so that the user can be visually informed speech input has been activated.

[TalAter]

Hello again @alanjames1987, sorry for the late response on this issue.

Here's an idea of how to do this, using the new regular expression support available in v2.0.0

// run this after hotword was detected to register the "real" commands
var hotWordDetected = function() {
  annyang.removeCommands();
  annyang.addCommands({
    'hello': function() { alert('Hello world!'); }
  });
}

// initial command to listen for the hotword
var hotwordCommand = {
  'hotword': {'regexp': /shenanigans/, 'callback': hotWordDetected}
}

annyang.addCommands(hotwordCommand);
annyang.start();

If what you had in mind was that the user would have to always say the hotword before each command, you can do something like:

var hello = function() {
  alert('Hello world!');
}

var goodbye = function() {
  alert('Goodbye world!');
}

annyang.addCommands({
  'hello':    {'regexp': /.* shenanigans hello/, 'callback': hello},
  'goodbye':  {'regexp': /.* shenanigans goodbye/, 'callback': goodbye}
});
annyang.start();

[alanjames1987]

I will try that out but that might be a good solution. It still seems slightly like a hack.

I was hoping this could be built into annyang. The code to interact with it might look something like this.

function hotwordDetectionHandler() {

    // code to trigger a sound
    // or update interface to show it's listening

}

function hotwordTimeoutHandler() {

    // code to trigger a sound
    // or update interface to show it's not listening

}

var hotwords = {
    '(hey) computer': hotwordDetectionHandler,
    '(hey) hal': hotwordDetectionHandler,
    '(hey) jarvis': hotwordDetectionHandler,
};

var commands = {
    'show me *term': showFlickr,
    'calculate :month stats': calculateStats,
    'say hello (to my little) friend': greeting
};

annyang.hotwords(true);

annyang.addHotwords(hotwords);

annyang.hotwordTimeout(1000); // <-- if a sentence isn't started within the time a deactivation function is called
annyang.hotwordTimeoutHandler(hotwordTimeoutHandler); // <-- function to run after timeout

annyang.addCommands(hotwordCommand);
annyang.start();

[TalAter]

That's an interesting idea... and a very well thought out API!
I wish all issues were posted like this 👍 Thanks

How do you see the importance of allowing separate hotwordDetectionHandlers? Why allow just one hotwordTimeoutHandler but multiple hotwordDetectionHandlers?

Is there a specific common use case that requires multiple ones?
Would it be good enough to just return the captured hotword as a parameter to the hotwordDetectionHandler?

This would allow us to simplify the API to something like:

var hotwords = [
    '(hey) computer',
    /(hey|hello) hal/
];

[alanjames1987]

I don't think multiple hotwordDetectionHandlers is very important. I added it in there because it was in line with the commands are currently added to annyang and I was trying to keep a similar API.

I think the idea of sending the spoken hotword to the hotword handler is great.

[TalAter]

Sounds good.

Would you like to give this a shot and send me a pull request?

[alanjames1987]

I will look into this as soon as I can, hopefully this weekend. I know I will have to use interim results, so I will be enabling that.

[TalAter]

Enabling interim results seems like a very drastic change to how annyang works, and doesn't really seem required for hotword detection.

Is there a reason this feature can't be enabled without enabling interim results?

[alanjames1987]

I can only see real time hotword detection being added if we have real time results using interim results.

There might be a better way. If you think there is I would love to hear it.

[4tee]

Hi there, I am wondering if this feature has been added.

[alanjames1987]

It hasn't been added. I have had no time to work on this yet.

[unbolt]

Any plans on a timeframe for getting this feature implemented?