Engineer	School	Area of Interest	Grade
Tuhin M	Bellarmine College Preparatory	Electrical Engineering and Computer Science	Rising Sophomore

Gesture Controlled Robot Car

First Modification: 7/8/24

Figure 1 -- Gesture Controlled Robot Car with Ultrasonic Sensor

Figure 2 -- Gesture Controlled Robot Car with Gesture Controller

• My first modification was to add an ultrasonic sensor to the gesture controlled robot that would detect objects and prevent the robot from colliding into other surronding things. I was mainly focusing on software and coding for this modification because processing the data from the sensor and linking up the sensor to the robot controls through bluetooth would be the main challenges of this undertaking.

• I accomplished this modification by completing the hardware first and then working on the software. First, I looked up documentation on the ultrasonic sensor and how it is used. Then, I wired and added the sensor on a spare breadboard in order to test its capabilities. Once the sensor was assembled, I ran some code to understand what kind of data I was able to recieve from the sensor and how I could use that data. Following the testing of the sensor, I wired it onto the main body of the robot and ran the same test code to test if the sensor would work in tandem with the robot. Then, I edited and modified the code by creating variables and flags to complete the logic required to extract the data from the sensor and apply it to the controls. I used conditionals and loops to continuosly check the data from the sensor and apply that data to the robot’s motor controls.

• The first challenge I faced was that the sensor itself had some difficulty in generating data when an object was too close. I solved this problem by adding an extra parameter to check whether or not an object was in the technical blind spot of the sensor. Another challenge was that the data I recieved from the sensor would often clog the serial monitor, preventing the basic motor controls from working. I solved this problem by adding delays and conditionals. Lastly, the ultrasonic sensor would sometimes draw power away from the bluetooth module, causing the car to lose connection wtih the controller. I solved this problem by using a breadboard and electrical taping the wires down to ensure that they would pull the right voltage from the battery.

• Going forward, I plan to complete add some LEDs to the robot body that could serve as a visual display of the direction that the robot is moving in. For example, if the motor controller was told to spin the motors forward, an LED would blink at a pre-designated pace. Thus, the user would be able to identify what information the motor controller is recieving from the gesture controller.

Final Milestone 6/19/24

Images

Figure 3 -- Gesture Controlled Robot Car and Controller without Modifications

Figure 4 -- Gesture Controller Robot Car and Controller without Modifications

• My final milestone was to succefully connect the gesture controller to the robot car and ensure that all functions were working properly. I was mainly focusing on the software section for this milestone. I accomplished this by first uploading the robot code to the robot and then the hand gesture code to the robot controller. Following that, I uploaded code to each bluetooth module in order to pair them together

• The technical side of what I wrote as part of my code is as follows. First, on the hand gesture side, I printed the accelerometer values from the bluetooth module into the Serial monitor. The accelerometer only gave XYZ values and I needed to convert those values into something the bluetooth module at the other end could use. I accomplished this issue by using if statements. I set a certain angle value as a direction. (eg. 45 degrees forward tilt equals R) As a result, instead of worrying about the three different values, I only had to worry about one character. I did this four times, for each direction. Next, I uploaded code to the robot car that recieved data from the controller, and then activated motors using if statements in order to determine the direction.

• THE BIGGEST HURDLE OF THE PROJECT: Bluetooth Connectivity Issues… Getting the bluetooth connection between the robot module and the breadboard module was one of the most difficult aspects of this projects. In order to get the two modules paired, I had to research a lot about bluetooth modules, Arduino Nanos, and Arduino UNOs. Eventually, by crossreferencing two sources, I was able to succesfully set the gesture module as the master controller, and the robot module as the recipient of the data. Once I was able to do this, I was able to seamlessly integrate both the robot code and the gesture controller code into one master project. This Bluetooth journey exposed me to Serial Monitors and AT mode. These two attributes are the ones that determine master controllers and pairing for HC05 Bluetooth modules. After following a longwinded process of figuring out how to use AT mode and the Serial Monitor, I was able to succesfully pair the modules together.

• Going forward, I plan on adding some modifications to my project. One idea I have is to add a sensor that will detect whether or not the robot will run into obstacles. Then, the sensor will trigger an autonomous override in the code which will stop the robot from crashing. Another idea I had was to add LED headlights to the robot that would be toggleable or automatically turn on in the dark.

Second Milestone 6/18/24

Images

Figure 5 -- Gesture Controller without Modifications

Figure 6 -- Gesture Controller without Modifications

• My second milestone was to assemble the electrical and mechanical parts of the gesture controller. I wanted to build the parts that ensured that the gesture controller was completely sound and would not give me any failures. I accomplished this by first attaching the Arduino Nano to a breadboard and connecting it to the accelerometer. Basically, the accelerometer is a component that detects the X, Y, and Z positions of the breadboard. This is vital in order to determine the gestures of the controller. Next, I connected the bluetooth module to the Arduino Nano, so that I could send the values from the accelerometer. Finally, I connected the Arduino Nano to a 9 Volt battery in order to power the whole board.

• The technical side of what is happening within the controller is as follows. Power from the 9V battery goes to the Arduino Nano. The Nano in turn powers on the accelerometer through its wiring. The accelerometer picks up the data (XYZ position) of the breadboard, and then sends the data to the Bluetooth Module. The Bluetooth module then sends the data to the other bluetooth module on the robot car itself.

• Some challenges I faced was powering the module. When I attempted to power the module for the first time, I discovered that instead of turning the module on, all the happened was that the battery overheated. I solved this problem when I discovered that the solder joint wasn’t properly created. As a result, I resoldered the connection, and the device started working.

• Going forward, I plan to code the robot software and the gesture controller software. By doing this, I will be able to succesfully transfer the data and get the robot car to move based on the gestures. Once I complete that milestone, I plan on adding a sensor to the robot that will detect obstacles and overide manual controls to prevent the robot car from crashing.

First Milestone: 6/14/24

Images

Figure 7 -- Gesture Controlled Robot without Modifications

Figure 8 -- Gesture Controlled Robot without Modifications

• My first milestone was purely mechanical and hardware based. I wanted to completely build all of the electrical and mechanical parts of the robot car. In order to accomplish this, I first attached four motors to a baseplate and wired them to the motor controller. Then, I connected the motor controller to the Arduino Uno and connected the battery pack to the Arduino Uno. Then, I connected all of the motor wires to the Arduino Uno and the motor controller. Once I did that, I attached the wheels and fastened the top plate to the robot.

• The technical side of what is happening within the robot is that the power from the battery pack travels to the Arduino Uno which provides power to the motor controller. Then, the motor controller provides power to the motors, enabling them to run.

• Some challenges I faced was that the motors had some trouble providing enough torque to spin the wheels. I solved this problem by letting them warmup before I attached the wheels. Another problem I faced was that one of the motors wasn’t spinning. After debugging for a while, I discovered that it was because the motor wires weren’t soldered properly together. Once I fixed this, I ran some expiremental code to test whether the motors would spin and the all succesfully spun.

• Going forward, I plan to complete all of the hardware for the gesture controller. Then, I want to run the code and test whether or not the car will move according to the gesture detection. Once I complete all of these steps and ensure that the robot is fully functional with the gesture controller, I plan to add some extra features like side to side movement.

Schematics

Figure 9 -- Schematics from Hackster.io

Figure 10 -- Schematics for the Gesture Controller

Flowchart for Code Process

Figure 11 -- Flowchart of the Code Process

Code

Test Code for Running Motors

int In1 = 7;
int In2 = 8;
int In3 = 4;
int In4 = 12;
int ENA = 5;
int ENA2 = 6;
int SPEED = 210;

void setup()
{

pinMode(In1,OUTPUT);
pinMode(In2,OUTPUT);
pinMode(In3,OUTPUT);
pinMode(In4,OUTPUT);
pinMode(ENA,OUTPUT);
pinMode(ENA2,OUTPUT);

digitalWrite(In1,HIGH);
digitalWrite(In2,LOW);
digitalWrite(In3,HIGH);
digitalWrite(In4,LOW);

analogWrite(ENA,SPEED)
analogWrite(ENA2,SPEED);

}

void loop()
{

}

Code for the Robot Car

#include <SoftwareSerial.h>
SoftwareSerial BT_Serial(2, 3);

#define enA 10
#define in1 9
#define in2 8
#define in3 7
#define in4 6
#define enB 5

char bt_data;
int Speed = 150;

void setup() {

Serial.begin(9600);
BT_Serial.begin(9600); 

pinMode(enA, OUTPUT); 
pinMode(in1, OUTPUT);
pinMode(in2, OUTPUT); 
pinMode(in3, OUTPUT); 
pinMode(in4, OUTPUT); 
pinMode(enB, OUTPUT); 

delay(200);
}
void loop(){
if(BT_Serial.available() > 0){    
bt_data = BT_Serial.read(); 
Serial.println(bt_data);          
}
  
     if(bt_data == 'f'){forward();  Speed=180;}  
else if(bt_data == 'b'){backword(); Speed=180;} 
else if(bt_data == 'l'){turnLeft(); Speed=250;} 
else if(bt_data == 'r'){turnRight();Speed=250;}
else if(bt_data == 's'){Stop(); }

analogWrite(enA, Speed); 
analogWrite(enB, Speed);

delay(50);
}

void forward(){
  digitalWrite(in1, HIGH);
  digitalWrite(in2, LOW);
  digitalWrite(in3, LOW);
  digitalWrite(in4, HIGH);
}

void backward(){ 
  digitalWrite(in1, LOW);
  digitalWrite(in2, HIGH);
  digitalWrite(in3, HIGH);
  digitalWrite(in4, LOW);
}

void turnRight(){
  digitalWrite(in1, LOW);  
  digitalWrite(in2, HIGH); 
  digitalWrite(in3, LOW);
  digitalWrite(in4, HIGH);
}

void turnLeft(){
  digitalWrite(in1, HIGH);
  digitalWrite(in2, LOW); 
  digitalWrite(in3, HIGH);
  digitalWrite(in4, LOW); 
}

void Stop(){
  digitalWrite(in1, LOW); 
  digitalWrite(in2, LOW);
  digitalWrite(in3, LOW);
  digitalWrite(in4, LOW);
}

Code for the Gesture Controller

#include <SoftwareSerial.h>
SoftwareSerial BT_Serial(3, 2);

#include <Wire.h>

const int MPU = 0x68;
int16_t AcX, AcY, AcZ;

int flag=0;

void setup () {

  Serial.begin(9600);
  BT_Serial.begin(9600); 

  Wire.begin();
  Wire.beginTransmission(MPU);
  Wire.write(0x6B);
  Wire.write(0);
  Wire.endTransmission(true);

  delay(500); 
}

void loop () {
  Read_accelerometer();

  if (AcX<60  && flag==0) {
    flag=1;
    BT_Serial.write('f');
  }
  if (AcX>130 && flag==0) {
    flag=1;
    BT_Serial.write('b');
  }
        
  if (AcY<60  && flag==0) {
    flag=1;
    BT_Serial.write('l');
  }
  if (AcY>130 && flag==0) {
    flag=1;
    BT_Serial.write('r');
  }
    
  if((AcX>70) && (AcX<120) && (AcY>70) && (AcY<120) && (flag==1)) {
    flag=0;
    BT_Serial.write('s');
  }

  delay(100);  
}

void Read_accelerometer(){
  Wire.beginTransmission(MPU);
  Wire.write(0x3B);
  Wire.endTransmission(false);
  Wire.requestFrom(MPU, 6, true);

  AcX = Wire.read() << 8 | Wire.read();
  AcY = Wire.read() << 8 | Wire.read();
  AcZ = Wire.read() << 8 | Wire.read();

  AcX = map(AcX, -17000, 17000, 0, 180);
  AcY = map(AcY, -17000, 17000, 0, 180);
  AcZ = map(AcZ, -17000, 17000, 0, 180);

  Serial.print(AcX);
  Serial.print("\t");
  Serial.print(AcY);
  Serial.print("\t");
  Serial.println(AcZ); 
}

Code for the Bluetooth Connection

#include <SoftwareSerial.h>

SoftwareSerial Bluetooth(3, 2);

void setup() 
{
  Serial.begin(9600);
  Bluetooth.begin(38400);
}

void loop(){
  if (Bluetooth.available())
    Serial.write(Bluetooth.read());
  
  if (Serial.available())
    Bluetooth.write(Serial.read());
}

Bill of Materials

Part	Note	Price	Link
Arduino UNO	Used to control the robot movement	$25.81	Link
Arduino Nano R3	Used for the hand mounted part	$23.23	Link
Inertial Measurement Unit (IMU) (6 deg of freedom)	Used for sensing changes and movement	$5.99	Link
SparkFun Dual H-Bridge motor drivers L298	Used to control the motors	$7.21	Link
Solderless Breadboard Half Size	Used to hold the pins and devices for the controller	$4.96	Link
HC-05 Bluetooth Module	Used for connecting the robot car to the controller on the wrist	$10.39	Link
Male/Male Jumper Wires	Used to connect electronics to the boards	$2.10	Link
Male/Female Jumper Wires	Used to connect electronics to the boards	$4.11	Link
DC Motor, 12 V	Used to make the robot move	$2.10	Link
Pimoroni Maker Essentials - Micro-motors & Grippy Wheels	Used to translate the motor power to movement	$33.79	Link
Rocker Switch, SPST	Used to switch the robot and controller on and off	$1.26	Link
9V Battery Clip	Used to connect to the battery pack	$0.309	Link
9V battery (generic)	Used to power the robot	$1.54	Link
Battery Holder, 18650 x 2	Used to hold the batteries	$9.48	Link
Arduino IDE	Used to code the functionalities of the robot	$0.00	Link

Other Resources/Examples

• Team PSI
• Hackster.io
• Instructables

Arduino Starter Project: 6/12/24

• The Arduino Starter Project provided a basic foundation for learning soldering and coding with Arduino. As part of the project, I built a Proto Shield for the Arduino UNO. This Proto Shield featured a reset button, resistors, LEDs, and rails for wire inputs. Using the Proto Shield, the Arduino UNO, and breadboard I built a circuit with a LED that was controlled by a button. I coded this button and LED using Arduino IDE, an application that uploaded code to the Arduino UNO. I used conditional statements to detect button presses and translate the result gained into a signal to turn the LED on or off.

Code

#define LED_PIN 8
#define BUTTON_PIN 7

void setup() {
  pinMode(LED_PIN, OUTPUT);
  pinMode(BUTTON_PIN, INPUT);
}
void loop() {
  if (digitalRead(BUTTON_PIN) == HIGH) {
    digitalWrite(LED_PIN, HIGH);
  }
  else {
    digitalWrite(LED_PIN, LOW);
  }
}

Bill of Materials

Part	Note	Price	Link
Arduino Uno	Control the circuit power and ground	$24.50	Link
USB A -> B cable	Connection to computer	$9.89	Link
Arduino Proto Shield	Used to make easier wire connections	$9.95	Link
Breadboard	Used to hold LED, Button, and wiring	$2.90	Link
Male to Male Connectors	Used to connect electronics	$2.10	Link