Voice Activated Robotic Arm

This project will utilize the ELECHOUSE Voice Recognition Module to enable voice commands for controlling a robotic arm. Users can instruct the robotic arm to perform various tasks by recording and recognizing seven distinct voice commands. The project integrates the voice recognition module with an Arduino UNO, Arduino Mega 2560, and the Braccio Robotic Arm equipped with a motor shield. 

Components Required:

Project Workflow:

Use case

The Voice Activated Robotic Arm project offers an interactive and intuitive way to control a robotic arm using voice commands. By combining the ELECHOUSE Voice Recognition Module with Arduino microcontrollers and the Braccio Robotic Arm, users can create a versatile and hands-free robotic system capable of performing various tasks. This project serves as an excellent introduction to voice-controlled automation and robotics, providing opportunities for further exploration and customization. 

System Descriptions

The system in this project has two major sub-systems (blocks).

These two system block diagrams illustrate the flow of signals and components involved in transmitting voice commands wirelessly from the transmission block to the receiving block, ultimately controlling the movements of the Braccio robotic arm.

System Block Diagram

1. Transmission Block:

2. Receiving Block:

Note: 

This project uses the Braccio Robotic Arm kit, which includes the Arduino Mega and the motor shield from the previous project. Detailed instruction for the Braccio robotic arm and Arduino Mega assembly is available in the previous project. Please refer to the "Braccio Robotic Arm" section of this website.

Voice Recognition Module

The Elechouse Voice Recognition Module is a versatile breakout PCB board designed to facilitate the recognition of pre-recorded voice commands. This module offers users the capability to train and upload voice commands, which can then be recognized and executed upon detection. Here's an overview of its features and functionality:

VR module 

Key Features:

Training Process:

First, we will train the VR module to recognize seven voice commands. Then, we will import these commands to the recognizer which means seven commands are ready to use.​ 

Training the VR module:

Now, start training the VR module by sending sigtrain command followed by indexNumber and signitureName. For example, send: sigtrain 0 (red light on the VR module will be illuminated for short time) .

Then, the serial monitor prints “Speak now”, say a word, something like “move”. The serial monitor will print “speak again”, you will need to repeat the same word again..."move". If you obtain “success”, these two recordings are match and "record 0" is trained. If not, repeat the command until you succeed.

To train another voice, send sigtrain 1 . Let's say we want to record the word "Stop" command to train record 1 with signature "Stop". Repeat the above steps for all the seven signatures. 

Note: "Move" and "Stop" are used for demonstration purpose, you can use any meaningful word you choose.

Continue training up to seven voices (index 0 to 6).

Once the training is done, export these records to the recognizer using the "load" command followed by the indices. For example, to send all the above signatures, type  "load 0 1 2 3 4 5 6 " , ​this will send all the seven records to the recognizer.

To see all the available commands, type help ant any time and the serial monitor will print all the commands with their syntaxes.

Voice Recognition V3 Module "Train" Sample

Purpose:

The primary purpose of the "Train" sample is to guide users through the process of training the Voice Recognition V3 Module to recognize and differentiate between distinct voice commands. By following this sample, users can create a customized set of voice commands tailored to their specific application or project requirements.

Steps Involved:

Voice Recognition V3 Module "train" sample.

The "Train" sample for the Voice Recognition V3 Module provides a practical demonstration of how to train the module to recognize specific voice commands. This sample serves as a foundational step in configuring the module for subsequent voice command recognition tasks. Let's delve into the details of the "Train" sample:

Usage Guidelines:

Connecting the APC220 Module to Arduino 

The APC220 module is a versatile RF (Radio Frequency) communication module that allows wireless communication between Arduino boards or other microcontrollers. Connecting the APC220 module to an Arduino UNO board enables wireless data transmission and reception, opening up numerous possibilities for remote control, telemetry, and sensor data acquisition. Here's how to connect the APC220 module to an Arduino UNO:

Additional information about the APC220 interface can be obtained from the DFROBOT wiki site. 

Components Required:

APC220 connection to Arduino UNO and Mega
APC220 Radio Communication Modules

Block 1 - UNO connection Steps:

A. UNO to APC220

RGB                          Arduino pwm pin (~)

Note: I added the RGB indication for troubleshooting purposes only. This will help us validate the recode voice recognition when a voice is generated. 

Arduino UNO to APC220 connection

B. UNO to VR module

Connection steps between the Arduino UNO and the Voice Recognition Module:(this was done during the Training Process above)

Arduino UNO to VR module connection

Block 2 - Mega and Braccio connection Steps:

Here is the detailed mapping of the connectors to their respective pins on the shield and the Arduino board:

The detailed instructions for the Braccio robotic arm and Arduino Mega can be found in the previous project. Please refer to the "Braccio Robotic Arm" section of this website.

Problem

The APC220 RF module, akin to a Bluetooth device, interfaces with Arduino through its TX/RX pins. However, this conventional connection poses a significant issue during code uploading. When attempting to upload code to the Arduino, the TX/RX port, shared with the APC220, conflicts, rendering the upload unsuccessful. Consequently, users are compelled to disconnect the APC220 module from the Arduino every time they intend to upload new code, leading to inconvenience and workflow disruption.

Solution:

To address these challenges comprehensively, we propose the following solutions:

Likewise, I have designed a PCB that will serve as communication shield for Arduino Mega 2560 as shown in the figure below. The PCB is designed in Eagle CAD and printed by JLCPCB company. We have also done visual test, continuity test, and soldering. 

Communication shield for Arduino Mega 2560. 

Robotic Manipulator

This project uses the Braccio robotic arm to perform a few tasks based on a received voice command. However, we will not discuss the Braccio Robotic Arm here. This information can be obtained from the “Braccio Robotic Arm” section of this website. Here, we will only utilize the 6-DOF (degree-of-freedom) advantage of the Braccio robotic arm and the voice recognition module. 

Here, we simply need to add the APC 220 module to the Braccio Robotic Arm controlled by the Arduino Mega, as described in the "Braccio Robotic Arm" section of this website.

Next, upload the code below onto the Arduino Uno and replace all the voice signatures with the ones you have trained and recorded. Remember to modify the `savedVoices()` function at the end of the code.

Use the microphone attached to the VR module to call one of the signatures you recorded. Observe the RGB LED light color change. For example, if you call index "0", you should see a RED light. If this works, you are ready to proceed to part two of this project (Braccio robotic arm).

[Voice recognition code]


/* - This code is a modified version of the Arduino example sketch

 * - Receives voice command from the VR module

 * - Indicator light turn on (optional)

 * - Transmits serial data to the Robotic Arm thru the APC220 RF module

 * You can replace these voice signatures with yours

 * This project doesn't use the serial monitor and some of the serial print lines are commented out

 * Replace the signatures on "savedVoices()" function at the end of this code

 * RGB LED light can be omitted

 */

#include <SoftwareSerial.h>

#include "VoiceRecognitionV3.h"

// 11:RX 12:TX, you can choose your favorite pins.

VR myVR(11, 12);   

uint8_t records[7]; // save record

uint8_t buf[64];

//RGB light

int red = 3;

int green = 5;

int blue = 6;

// Voice signitures in Amharic (Ethiopian language)

//      Signiture (VR index)

#define Shero     (0)   // braccio's name

#define Tezegaje  (1)   // get ready

#define Anesa     (2)   // pick up

#define Wedkegne  (3)   // to the left

#define Wedgera   (4)   // to the right

#define Melese    (5)   // put it back

#define Beka      (6)   // Done

void setup()

{

  /** initialize */

  myVR.begin(9600); // serial com for the VR

  /* APC220 is connected to RX/TX pins of the Arduino

   * Disconnect the module before uploading code to Arduino

   */

  Serial.begin(9600); // RF com with Braccio

  pinMode(red, OUTPUT);

  pinMode(green, OUTPUT);

  pinMode(blue, OUTPUT);

  led_all_color_test ();//lamp test

  delay (500);

  RGB_off ();

  savedVoices(); // do not modify

}

void loop()

{

  int ret;

  ret = myVR.recognize(buf, 50);

  if (ret > 0) {

    switch (buf[1]) {

      case Shero: // braccio's nick name

        Serial.write('0');   // Send data over APC220

        analogWrite(red, 255);

        analogWrite(green, 0);

        analogWrite(blue, 0);

        delay (5000);

        RGB_off ();

        break;

      case Tezegaje: //Voice command "get ready"

        Serial.write('1');

        analogWrite(red, 0);

        analogWrite(green, 255);

        analogWrite(blue, 0);

        delay (5000);

        RGB_off ();

        break;

      case Anesa: //Voice command "pick it up"

        Serial.write('2');

        analogWrite(red, 0);

        analogWrite(green, 0);

        analogWrite(blue, 255);

        delay (5000);

        RGB_off ();

        break;

      case Wedkegne: //Voice command "to the right"

        Serial.write('3');

        analogWrite(red, 0);

        analogWrite(green, 255);

        analogWrite(blue, 0);

        delay (500);

        analogWrite(red, 0);

        analogWrite(green, 0);

        analogWrite(blue, 255);

        delay (1000);

        RGB_off ();

        break;

      case Wedgera: //Voice command "to the left"

        Serial.write('4');  

        analogWrite(red, 255);

        analogWrite(green, 0);

        analogWrite(blue, 0);

        delay (500);

        analogWrite(red, 0);

        analogWrite(green, 255);

        analogWrite(blue, 0);

        delay (1000);

        RGB_off ();

        break;

      case Melese: //Voice command "return"

        Serial.write('5');  

        led_all_color_test ();

        delay (200);

        RGB_off ();

        break;

      case Beka: //Voice command "end"

        Serial.write('6');  

        for (int i = 10; i <= 100; i++)

        {

          analogWrite(green, i);

          delay (50);

        }

        RGB_off ();

        break;

      default:

        Serial.println("Record function undefined");

        break;

    }

    /** voice recognized */

    printVR(buf);

  }

}

// FK functions

void led_all_color_test () {

  //Red, bright

  analogWrite(red, 255);

  analogWrite(green, 0);

  analogWrite(blue, 0);

  delay(200);

  // blue, bright

  analogWrite(red, 0);

  analogWrite(green, 0);

  analogWrite(blue, 255);

  delay(200);

  //Green, bright

  analogWrite(red, 0);

  analogWrite(green, 255);

  analogWrite(blue, 0);

  delay(200);

  // Cyan, bright

  analogWrite(red, 0);

  analogWrite(green, 255);

  analogWrite(blue, 255);

  delay(200);

  // magenta, bright

  analogWrite(red, 255);

  analogWrite(green, 0);

  analogWrite(blue, 255);

  delay(200);

  // Yellow, bright

  analogWrite(red, 255);

  analogWrite(green, 255);

  analogWrite(blue, 0);

  delay(200);

  // white, bright

  analogWrite(red, 255);

  analogWrite(green, 255);

  analogWrite(blue, 255);

  delay(200);

}

void RGB_off () {

  //Red, bright

  analogWrite(red, 0);

  analogWrite(green, 0);

  analogWrite(blue, 0);

  delay(200);

}

void RGB_white () {

  //Red, bright

  analogWrite(red, 255);

  analogWrite(green, 255);

  analogWrite(blue, 255);

  delay(200);

}

//VR functions

void savedVoices() {

  if (myVR.clear() == 0) {

    Serial.println("Recognizer cleared.");

  } else {

    Serial.println("Not find VoiceRecognitionModule.");

    Serial.println("Please check connection and restart Arduino.");

    while (1);

  }

  // add instruction to print here

  if (myVR.load((uint8_t)Shero) >= 0) {

    Serial.println("Shero loaded");

  }

  if (myVR.load((uint8_t)Tezegaje) >= 0) {

    Serial.println("Tezegaje loaded");

  }

  if (myVR.load((uint8_t)Anesa) >= 0) {

    Serial.println("Anesa loaded");

  }

  if (myVR.load((uint8_t)Wedkegne) >= 0) {

    Serial.println("Wedkegne loaded");

  }

  if (myVR.load((uint8_t)Wedgera) >= 0) {

    Serial.println("Wedgera loaded");

  }

  if (myVR.load((uint8_t)Melese) >= 0) {

    Serial.println("Melese loaded");

  }

  if (myVR.load((uint8_t)Beka) >= 0) {

    Serial.println("Beka loaded");

  }

}

//

void printSignature(uint8_t *buf, int len)

{

  int i;

  for (i = 0; i < len; i++) {

    if (buf[i] > 0x19 && buf[i] < 0x7F) {

      //Serial.write(buf[i]);

    }

    else {

      // Serial.print("[");

      // Serial.print(buf[i], HEX);

      // Serial.print("]");

    }

  }

}

//

void printVR(uint8_t *buf)

{

  //Serial.println("VR Index\tGroup\tRecordNum\tSignature");

  //Serial.print(buf[2], DEC);

  //Serial.print("\t\t");

  if (buf[0] == 0xFF) {

    //Serial.print("NONE");

  }

  else if (buf[0] & 0x80) {

    //Serial.print("UG ");

    //Serial.print(buf[0] & (~0x80), DEC);

  }

  else {

    //Serial.print("SG ");

    //Serial.print(buf[0], DEC);

  }

  //Serial.print("\t");

  //Serial.print(buf[1], DEC);

  //Serial.print("\t\t");

  if (buf[3] > 0) {

    printSignature(buf + 4, buf[3]);

  }

  else {

    //Serial.print("NONE");

  }

  //Serial.println("\r\n");

}

[Robotic Manipulator Code]


​/*  Braccio Voice Activated from Arduino Uno

    By FK Debebe, California State University

    Robotic Arm movement on Ardunio Mega 2560

*/


#include <BraccioRobot.h>

#include <Servo.h>


// M1=base degrees. Values from 0 to 180 degrees

Servo base; 

// M2=shoulder degrees. Values from 15 to 165 degrees

Servo shoulder; 

// M3=elbow degrees. Values from 0 to 180 degrees  

Servo elbow;   

// M5=wrist rotation. Values from 0 to 180 degrees   

Servo wrist_rot;  

// M4=wrist vertical. Values from 0 to 180 degrees

Servo wrist_ver;  

// M6=gripper degrees. Values from 10 to 73 degrees.

Servo gripper;    

int data;

int buzzer = 4;   // for both UNO and Mega 2560

int ledPin =  51; //only for Mega 2560

Position myInitialPosition(90, 45,  185, 0, 180,  70);


void setup() {

  // Serial.begin(9600); // Serial monitor

  Serial1.begin(9600); // voice Rcognition

  pinMode(ledPin, OUTPUT);

  pinMode(buzzer, OUTPUT);

  InworkIndicator ();

  BraccioRobot.init(myInitialPosition);// check this

  delay (1000);

  OffworkIndicator ();

  Serial1.flush ();

  delay (50);

}


Position pos;


void loop() {

  // wait for voice command

  delay(5);

  if (Serial1.available() > 0)

  {

// Read data received from UNO

    data = Serial1.read();   

    Serial1.print (data);

    if (data == '0') {

      InworkIndicator ();

// Braccio's nick name, call it what ever you want!

      shero1 (); 

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

    if (data == '1') {

      InworkIndicator ();

      goHome2 ();

      delay (2000);

//get ready function call

      tezegaje1 (); //get ready function call

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

    // Pick an object

    if (data == '2') {

      InworkIndicator ();

// Pick an object function call

      anesa ();

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

    if (data == '3') {

      InworkIndicator ();

// Move right function

      wedkegne1 (); 

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

    if (data == '4') {

      InworkIndicator ();

// Move left function

      wedgera1 (); 

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

    if (data == '5') {

      InworkIndicator ();

// Put it back function

      melese1 (); 

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

    if (data == '6') {

      InworkIndicator ();

      // beka1 ();

      delay (1000);

      endTaskSafePosition ();

      delay (1000);

      OffworkIndicator ();

      Serial1.flush();

    }

  }

}



// ====Functions ================================

void InworkIndicator () {

  digitalWrite(ledPin, HIGH);   // turn LED on

}


void OffworkIndicator () {

  digitalWrite(ledPin, LOW);    // turn LED on

}


void comCheck () {

  tone(buzzer, 200);            // Send 0.21KHz sound signal...

  delay(500);                   // ...for half a sec

  noTone(buzzer);               // Stop sound...

  delay(1500);                   // ...for 1.5 sec

}


void goHome () { // *** do not modify

  // go home

  BraccioRobot.moveToPosition(pos.set(90, 45,  185, 0, 180, 70), 50);

}


void goHome2 () { // *** do not modify

  // go home2

  BraccioRobot.moveToPosition(pos.set(90, 135,  0, 185, 180, 70), 50);

}


void endTaskSafePosition () {  // *** do not modify

  // go to a safe position for power off

  BraccioRobot.moveToPosition(pos.set(90, 45,  185, 0, 180,  70), 50);

}


// voice control

void shero1 () {

  // Yes

  BraccioRobot.moveToPosition(pos.set(90, 45,  185, 45, 100, 20), 30);

  delay (1000);

  BraccioRobot.moveToPosition(pos.set(90, 45,  185, 0, 180, 70), 30);

  delay (1000);

}


// get ready

void tezegaje1 () {

  delay (1000);

  BraccioRobot.moveToPosition(pos.set(90, 85,  80, 90, 178, 70), 30);

  delay (1000);

}


//pick up

void anesa () {

  delay (1000);

  // go to the box

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 10), 20);

  delay (1000);

  // pick the box

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 50), 20);

  delay (1000);

  // lift

  BraccioRobot.moveToPosition(pos.set(90, 85,  80, 90, 178, 50), 20);

  delay (1000);

  }


void wedgera1 () {

  delay (2000);

  BraccioRobot.moveToPosition(pos.set(10, 85,  80, 0, 178, 50), 20);

  delay (1000);

}


void wedkegne1 () {

  delay (2000);

  BraccioRobot.moveToPosition(pos.set(178, 85,  80, 0, 178, 50), 20);

  delay (1000);

}


void melese1 () {

  delay (1000);

  // go to the box

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 50), 20);

  delay (1000);

  // drop the box

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 10), 20);

  delay (1000);

  // clear

  BraccioRobot.moveToPosition(pos.set(90, 90,  20, 20, 178, 10), 20);

  delay (1000);

  }


// more fun

void Tegna1 () { // Sleep

  delay (2000);

  BraccioRobot.moveToPosition(pos.set(90, 90,  12, 2, 90, 73), 20);

  delay (2000);

}


// Show time

void TakeAbox () {

  // go to box

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 10), 20);

  delay (1000);

  // pick the box

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 50), 20);

  // show the box

  BraccioRobot.moveToPosition(pos.set(90, 90,  90, 90, 90, 50), 20);

  // show to the right

  BraccioRobot.moveToPosition(pos.set(0, 90,  90, 45, 90, 50), 20);

  // show to the left

  BraccioRobot.moveToPosition(pos.set(180, 90, 90, 45, 90, 50), 20);

  // return to dropoff

  BraccioRobot.moveToPosition(pos.set(90, 90,  0, 0, 178, 50), 20);

  // drop the box

  BraccioRobot.moveToPosition(pos.set(90, 95,  0, 0, 178, 10), 20);

  delay (2000);

 //

}


This video demonstrates a variation of this project with a different application compared to the previous example, but it uses the same hardware: the Braccio robotic arm, Arduino UNO, Mega 2560, VR recognition module, and two APC 220 RF modules.


»»» END of  Voice Activated Robotic Arm »»»