Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length.

[flashtwo]

Hi,

Has anyone dabbled with voice command of their model boat?

Having got my steam boat telemetry system up and working (still to be proved on the water !), it has given me great flexibility in sending data to the boat. See:
http://www.modelboatmayhem.co.uk/forum/index.php/topic,48575.25.html

I've discovered this speech recognition board which would easily interface with my telemetry system and opens up all sorts of applications. See:
http://www.rapidonline.com/electronic-components/mikroe-speakup-click-board-73-5244

The only trouble is that other Mayhemers  might have to take the sacred vow of silence at the pond side so as not to confuse the poor device!

Apparently it can be trained to respond in any language!

Thoughts/jokes would be much appreciated.

Ian

[TailUK]

To save all the shouting, screaming and pleading from the pond side.  How about this as the next big thing in R/C boats!

http://news.nationalgeographic.com/news/2013/08/130829-mind-brain-control-robot-brainwave-eeg-3d-printing-music/

[steamboatmodel]

I have heard attempts at voice control of model boats at many a regatta all without much success. Maybe with a microcontroller and voice recognition it would work. I shall watch with interest.
 Regards,
 Gerald.

[TugCowboy]

I have done some work on similar systems, although never actually got one working into a boat's steering/propulsion.

With a decent noise cancelling headset good results can be achieved.
Using an Arduino board and a speech unit then some servo functions are actually really easy to control, things like lights, ramps, pumps and general toys are great.

There's a few ways of doing it, either on the fly interpretation or against a pre-defined speech pattern - the on the fly solution with something like an Arduino board can cause a slight delay while it works it out, the pre-defined speech pattern can be quicker but relies on a match good enough to work.

[McGherkin]

Real ships have this already. Here are some example basic commands.

Helm, Steer 280 degrees

Helm, Port 10

Helm, Amidships

Engine speed 1500 revolutions

Make speed 10 knots

Etc Etc Etc.

[Mad Scientist]

Flashtwo (Ian): IIRC, the voice recognition system needs to be 'trained' to respond to your voice. I suggest that you do the training outdoors on a breezy day, to give the system a good idea of what to expect.
Now, for the humourous side: Imagine two dozen boat operators at pondside all talking at once, with two dozen Arduinos trying to make sense of it all! Ah well, that's why we call it Mayhem! - Tom

[flashtwo]

Well, I've received the "Speakup" board and, using a head mounted microphone, I have been practising recording my voice (indoors at the moment).

It is very easy to set up and teach, not only single command words (left, right, stop etc), but also phrases like "Ok HAL" and "Open the pod doors HAL". It does make you think about your own voice and it is a two-way teaching process.

The red LED indicator light gives a short blink if it doesn't "understand" the command, or a longer 0.5s blink if it "comprehends" the command. For practical use, perhaps a audio signal could be sent to the headphones confirming a "comprehended" command.

If the word is repeated with inflections (caused by anger?!) it tends not to respond, therefore you must be polite!

For "panic" commands like "STOP!", it is possible to record several versions of it (including expletives!), which can be comprehended and then assigned the same logical output for all versions of "STOP!" There are 200 "commands" available, so repeated versions shouldn't be a problem.

The next stage of development is to link its serial output to a PIC based control system. The PIC will be programmed to convert the coded spoken messages into PWM signals for the rudder and steam engine reversing gear servos.

Having got the PIC system working, I will then introduce the XBee radio transmitter/receiver to give remote voice control of the boat.

Other possible developments could include pre-defined manoeuvrering displays programmed into the PIC, but selected by voice commands (model boat equivalent of Crufts!)

I think I will have my Futaba transmitter close to hand to take over control in emergencies!

Ian

[john44]

Flashtwo (Ian): IIRC, the voice recognition system needs to be 'trained' to respond to your voice. I suggest that you do the training outdoors on a breezy day, to give the system a good idea of what to expect.
Now, for the humourous side: Imagine two dozen boat operators at pondside all talking at once, with two dozen Arduinos trying to make sense of it all! Ah well, that's why we call it Mayhem! - Tom

It would be like sheep dog trials


John

[flashtwo]

Here's a blast from the past for those of a certain age -

http://russelldavies.typepad.com/planning/2007/01/in_praise_of_re.html


Like the author, I think I must have unknowingly been inspired by General Jumbo!

Ian

[g6swj]

Hello Ian,
What a great project! I am following your progress with interest
I would love to know the latency involved with successful voice commands also what xbees do you use?


Imagine what u could do with google glasses or the like if you could access the control messages!


Regards
Jonathan

[flashtwo]

Hi,

The Voice Control System (VCS) is progressing well.

I’m now using an Andrea ANC-700L Anti-noise headset (following TugCowboy’s advice) that has two microphones – one pointing to the voice and the other “listening” to the background noise, resulting in the noise being subtracted from the voice.

I’ve used it very successfully with a radio as the background “noise”.

Using an oscilloscope on the UART serial output, I can relate the different voiced commands to the bit pattern which is useful in determining repeatability of the those commands.

By using the PC based “Speakup” editor (free download), each voice command can be linked to any of the twelve digital outputs and all commands can be linked to the same output, if so desired.

The digital output can be switched on or off, toggled or pulsed. The pulsed one is particularly useful, since it can be used to generate an audible tone (continuous or for a period) that is fed to the headphone of the headset to indicate that the spoken command has been recognised.

Each command can generate a different tone from the same output to give double confirmation that a command has been understood. To be practical, I would envisage commands being grouped to a limited range of tones to avoid trying to remember the meaning of each tone.

The actual digital output is fed through an additional 74HC00 NAND gate, which acts as an amplifier with its open collector output connected, via a variable potentiometer volume control, to the headphone.

I’ve been using the “International Radiotelephony Spelling Alphabet” – Alpha, Bravo etc, in addition to Port, Starboard, Left, Right etc. as a comparison of repeatability.

I’ve found that a command with more vowels in it is more reliable, though I draw the line at calling out “Romeo – Romeo” at the pond side knowing the reply I would get from the majority of Mayhemers!

Regarding latency -  I’ve found the system quite responsive to the voice and it will be fine for most scale model boat control, but it wouldn’t match the response of a joystick simply because the voice is slower than the hand. On the other hand it promises to give a much broader range of controls for auxiliarys such as lamps, pumps etc.

The next stage is to link the UART serial output to a PIC to decode the commands into PWM signals for the rudder etc. I’ve already got working UART code from a previous project (see http://www.modelboatmayhem.co.uk/forum/index.php/topic,48575.25.html ), so I’m more than halfway through the VCS project.


Ian

[steamboatmodel]

I can just see it "Romeo Romeo where forth art though" and the lights on the model start flashing whistle starts blowing and the model sends its GPS location back to the transmitter.  Great work Ian.
Regards,
Gerald.

[flashtwo]

Hi,

I've managed to link the UART serial data to a 16F877 PIC and decode the digital commands into operating the rudder servo with the following voice commands:-

LEFT, RIGHT, MIDSHIPS (to recentre the rudder), HARD-LEFT and HARD-RIGHT; please excuse the non-maritime terminology - it can be made more nautical!

The LEFT and RIGHT commands currently "inch" the rudder servo in 5% steps (= 50us ), though it can be any value.

I am wondering whether to give the rudder a "degrees" command, e.g. "10deg PORT".

So far, the servo is responding very well (it seems really weird just commanding it with the voice).

I have incorporated an external PWM input, which will come from the normal Futaba receiver. This signal will be from a spare switched channel which will enable the Futaba transmitter to hand over control to the voice command system.

Having "cracked" the bulk of the code, I shall consider things like rudder trim.

SWMBO has heard me repeating all these LEFT and RIGHT commands and is issuing her own versions  ......"The kitchen bin needs emptying"..........."The kitchen bin needs emptying"..........."The kitchen bin needs emptying".....

oh well........

Ian

[steamboatmodel]

Be glad that She is only issueing the one command.

[flashtwo]

Hi Gerald,

Yes, SWMBO wonders why I can't be fitted with a voice command box - then she would only have to ask me once without repeating herself!

Meanwhile, back in the software "development lab", I've progressed to having four servo outputs and five outputs that will be operating relays. Once you have one servo operating its just a matter of cut-and-pasting the code for extra ones.

I'm using the channel 5 ("landing gear") switch of my Futaba 6EX as the command to swop between normal radio control and the Voice Command System. The Channel 5 PWM signal is checked by the software code and controls the setting of a digital output, which will switch a 74hc257 logic chip. The 74hc257 is Quad 2-input multiplexer, i.e. a 2-way 4-pole switch, which will select normal or voice commands (from the PIC) for the servos. I initially thought of putting the normal signals through the 16f877 PIC, but it would have required too many interrupts to be handled and used up the I/O pins needlessly.

Using Channel 5 will also require having the Futaba transmitter on, i.e. it will always be there for backup if required.

I've incorporated rudder trim (TRIM LEFT, TRIM RIGHT), which gives far better resolution than the normal Futaba trim lever - down to 0.1% if you would ever want that.

The servo PWM signals now also have maximum and minimum limits to prevent repeated voiced commands from driving the servos too far in either direction (SWMBO mod coming up!).

Having all the code now in place, I'm now designing the final hardware to go in the boat and will be ordering the XBee radio transmitter and receiver.

Code from my other projects has been "lifted" and modified, which has help speed this project along.

Ian

[flashtwo]

Hi,

The Voice command System is virtually complete.

The shore-based transmitter has been fitted (squeezed) into a see through case small enough to fit into the top pocket of a shirt, although the side entry audio plugs will require a hole to be made in the pocket! The status LEDs can be observed through the case.

The boat based receiver incorporates the existing Futaba receiver and the new XBee Pro S1 device, which is being used in receive mode only.

The Mikroe “Speakup” device codes the voice commands which are then transmitted to the boat, where the PIC 16F877 decodes them into PWM values for the Rudder, Valve gear and boiler feed water demand values.

The Voice PWM and the Futaba PWM signals then go to the logic switch 74HCT257.

Channel 5 (landing gear) of the Futaba system also goes to the PIC, where it’s on/off position is detected and is used to select whether to use the Futaba or Voice PWMs for controlling the servos.

The select signal goes to the 74HCT257 chip, where either the Futaba or Voice PWMs can be selected for outputs to the servos.

For testing I’ve been looking at the Rudder servo PWM on the oscilloscope (it does work with a real servo) and had my daughter observing the value as I’ve walked out into the garden. Left, Right, Hard left, Hard Right, Midships, Trim left and Trim right all worked as expected.

I had to repeat a few of the commands, since I didn’t hear the confirmation bleep in the headset to indicate the correct voice command had been detected.

A couple of hic-cups in getting the system working....................
 
One missing wire between the Futaba Channel 5 and the PIC.

The other more serious was the 0 to 3 volt Futaba output not triggering the interrupt in the PIC. The External Interrupt input requires at least 4 volts to work and during testing I had used my PWM simulator which operates over the 0 to 5 volt range.

I looked back at previous projects, which used the Futaba signal and realised I had forgotten to design in a comparator chip to raise the 3 volt PWM signal to 5 volts.
Disaster…….I didn’t have enough room on the board for an additional chip!

Luckily, looking at the PIC data sheet, I found that an interrupt input (RB4) was available that would operate at the Futaba signal levels – it was a TTL rather than a Schmidt trigger input.
So, deftly “adjusting” the code, I managed to solve the signal level problem without having to install a “dongle”.

As a side issue, the 74HCT257 chip was originally a 74HC257, which although it worked, was on the border line of being compatible with the Futaba signal levels.

Next step: installing it on the boat for testing.


Ian.

[flashtwo]

Hi,

The Voice Command System has now been extended to include voice control of the MP3 player ( see thread http://www.modelboatmayhem.co.uk/forum/index.php/topic,50200.0.html ) as well as the boat operation.

The PIC USART Transmit TX output was hardwire linked to the MP3 player Receive RX input.

The voice decoding software in the PIC had extra lines added to handle the voice commands STOP/START and NEXT TRACK, which were then transmitted serially to the MP3 player.

The voice commands can now specify which track out of 255 to play, volume and start /stop, this is in addition to hardwire triggers for specific tracks be it music, engine sounds or verbal "GET OUT OF THE WAY!" type messages (any ideas thankfully welcome!).

The only initial hic-cup this time was not having matching transmit and receive baud rates of 9600.

I can now ask for my favourite Marie Lloyd track or that of famous Australian, Florrie Forde..... plus the Welsh National Anthem for Stavros!

Ian

[Mad Scientist]

An idea for your sound system: 5 or more 'short blasts' of the horn, as prescribed by INTERCO, followed immediately by that voice shouting 'GET OUT OF THE WAY!' That ought to get their attention!

Tom

[flashtwo]

Thanks Tom - I had a look at the INTERCO codes - all those complications as to whether you're a tug towing or not!

Well, the latest change to the Voice command project is the addition of a compass that can feed directional data into the navigation system.

This will enable voice commands such as Head North, South etc plus manoeuvres like "U-turn", "Figure of Eight";  oh, what ambitions!

The compass is a CMPS11 ( http://www.hobbytronics.co.uk/sensors/magnetic/cmps11-tilt-compass ); it is very much over the top with all its gyroscopes and accelerometers, but it does all the number crunching to gives a proper 0 to 3600 output to go straight into the navigation code; it is an amazing bit of kit no bigger than a postage stamp.

The software for the compass was very much an uphill struggle, since it used a comms protocol called "I2C". Most people use the readily available code and rely on programme delays to talk to the device, but with the control system working in "real time" I wrote my code in assembler utilising the interrupts rather than delays.

As a temporary display method, I'm using the compass bearing to control the pulse width of one of the servo outputs, then on the 'scope I can see the pulse width change in line with the direction the compass is pointing.

I'll have to be careful in mounting the compass on board the launch because of stray magnetic fields.

The well known complication of compass readings, is that for a small change in direction about North the value switches value between 0 and 360, not going via 270deg.

This used to be a right nuisance with the old paper chart recorders with a northerly wind, since then ink pen would keep skipping across the whole width of the paper roll leaving it in a very soggy condition. The software will have to deal with this problem.

On a lighter note, having a pint with Boneash the other day chatting about the Voice Command system controlling the MP3 player, we came up with the crazy idea (must have been the third pint) of a dog's bark translator!

If we could get the dog to bark consistently, we could then decode and play back an MP3 message (yes, I know Disney have already done it!), but this might work (or not). Some may remember the Far Side cartoon of a dog's bark translator - all the dogs were actually saying was "Hi!" ,"Hi"..............

I must get the launch back on the water - all this software is no good for me!

Ian

 

[flashtwo]

Hi Mayhemers,

The Voice Command System now has the compass fully incorporated into the system allowing compass headings to be commanded by voice.

As a start, I can command north, east, south and west commands.

To avoid changing direction by more than 180deg, the control system determines whether to turn left or right,  e.g. if the launch is heading 30deg and I command NORTH it won't do a 330deg turn to the right (starboard) to achieve the direction, but it decides the "quickest" direction will be to the left.

When the direction gets within +/-2deg (adjustable), the controls revert to "normal" voice commands left, right etc. If the launch direction overshoots the required direction the rudder will still be adjusted automatically until the direction is achieved.

At any time, the command "MIDSHIPS" will drop it out of Compass heading control.

Additional commands include "U-TURN-RIGHT" and "U-TURN-LEFT", which carry out a 180deg manoeuvre from whatever the direction the launch is heading.

A very tricky "FIGURE-OF-8" command is also available in which the control system makes a note of the current direction, does a 270deg turn to the left, then a 270deg turn to the right until it is back on its original heading.

I've just got to install the system in the launch and "swing" the boat around in the garden to find magnetic north and test the system before going on the water.

Ian

[dreadnought72]

I'm enjoying this thread very much.

Martin - any chance of a split in the black arts section: basic electrics and one for electronics/programming? There's much more to come here I'm sure!

Andy

[flashtwo]

Thanks Andy.... things can only get more interesting!

The Voice Command System (VCS) has been installed in the steam launch and the servo plugs swopped over from the original Futaba receiver to the VCS.

I've "swung" the launch around on its trolley (it displaces 34kg) and successfully tested all the rudder commands including the Compass heading commands, but haven't tried the "Figure of 8" yet.

I was going to check the engine's valve gear voice commands, then realised I hadn't written the code - so back to the coding shed!

I must also write the boiler filling code and carnival lights code.

Mayhem member, Boneash, has suggested adapting the system for real boats using the standard navigation interface - hmmmmm!

Ian.

[flashtwo]

Hi Mayhemers,

I've written all the Voice Command code for navigation, carnival lighting, boiler filling and MP3 player, but when testing it had some of the dreaded servo jittering (not much, but irritating) especially on the Valve Gear Servo.

I finally traced it to interrupts clashing between the Servo PWM code, the Compass data request code and the navigation scheduler code.

On the 'scope, you could see the PWM jitter was caused when two interrupts occurred very close to each other, which resulted in the servo pulses (PWMs) being delayed in switching off causing the pulse to be lengthened - hence the jittering.

The Compass data request now has a "window" of operation after the last servo pulse has completed, thus ensuring the Compass interrupt avoids the PWM code.

The navigation scheduler (which runs every 0.25s) has had its interrupt removed (under anaesthetic!) and is now scheduled by a counter rather than an interrupt timer.

I've just replaced the piston rings in the Stuart D10 and repacked the glands and will soon be back on the water.

.... and, oh yes, the MP3 player now has Josh MacRae's song "Messing about on the River" downloaded - (Voice Command "Gramophone" + "Messing about").

Any additional musical (boating type etc) suggestions will be welcome...........

Ian

[flashtwo]

Hi Mayhemers,

The Voice command system was tested successfully today at the Bradbourne Ponds, Sevenoaks
.
Despite the floating weeds, all the voice commands were tested, although the “FIGURE OF EIGHT” was terminated early due to the weed – the first 270deg turn was successful and the second turn was started automatically before approaching the weed caused me to retake control.

The Compass functioned correctly with the launch finding the commanded headings.

The boiler control commands were used OK, from enabling the gas valve to open with “OPEN GAS” through to “MORE STEAM”, “LESS STEAM” and finally “CLOSE DOWN”.

Due to its importance there are three voiced versions of “CLOSE DOWN”, plus a final “PLEASE STOP!”

I’ve taken Mad Scientist’s advice and trained the system outside using a laptop with all the background “mush” and bird song present.

I did have problems getting the MP3 “gramophone” responding, especially “PUGWASH” (the children’s TV program Captain Pugwash theme music). I gave up on the fourth attempt of calling out “PUGWASH” across the pond!

I was about to pack up when a member of the public became interested in the goings on, so I did another couple of rounds on the pond.

Then disaster struck! All servos stopped operation when it was doing a hard to starb’d turn – would it come back to shore – oh no! Around and around it went.
Luckily my daughter’s boyfriend helped with the “Tennis ball on the fishing line” emergency kit and managed to hook the launch and bring it to shore.

On investigation, it was found the steam engine’s valve gear servo had developed an internal fault and caused the servo fuse to blow.

This was annoying since I had only just installed it last week due to the previous one burning out. The previous one had burnt out because, during setting up of the Voice Command System, I had accidently left it switched on in the stalled condition while I had lunch. The previous one I had been using for about five years with no trouble.

I think I’ll give the servos separate fuses to limit the consequences of just one failing, and also consider control of the servo speed to reduce stress on them.

Apart from the servo problem, I was very pleased with the Voice Command System.

Ian.

[flashtwo]

Hi Mayhemers,

Here's a final (?) Steam Launch Control system Schematic showing the relationship between all the various modules.....

what next - Automatic Collision Avoidance? I think that can wait for a rainy day!

Ian