All posts by Andre Haland

Smart Mirror – Last update from software

Link to GitHub:

https://github.com/erlendhel/smartmirror

In hindsight, the software part of the project has mostly gone according to plan, the biggest turnaround we had to do, was to change from a touch screen to speech recognition. This was mainly due to the fact that the foil we decided to use to provide the mirror effect wouldn’t support touch. This meant that we had to implement the speech recognition function, which in turn meant that we had to facilitate the smartmirror app for touchless navigation throughout. The user will be displayed a progress bar at the bottom of the screen, indicating when voice is being recorded. By saying a valid command in the timeinterval it takes the bar to go from 100% to 0%, the application will perform the chosen command.

Speech Recognition

In order to implement speech recognition, we used the python package SpeechRecognition, which has support for the Google Cloud Speech API, which is the one we ended up using. The decision of using the Google Cloud Speech API was made because the interface seemed the easiest and also since the services provided from the Google API are well tested. Since we had a limited budget ($1 soundcard and $3 microphone), we tried to limit the negative factors by choosing safe.

How we used the API

Our initial plan for the speech functionality for the mirror was to have a thread which continously listened for commands, and then returned them to the main program, which would handle the different command. While testing on Windows, the listen() function, which is a part of the SpeechRecognition worked fine, and was able to process the commands we gave it while running continously. When we started testing the code on the Raspberry Pi, the functionality was weak at best. Usually it would listen endlessly without handling the commands it got, while it worked fine one or two times. This resulted in a change of design of the original plan.

Instead of using a listener, we decided to use recordings which were timed to three seconds, for some reason, the recordings were able to process the speech. The short answer to how this works, is that the smartmirror will record three seconds of audio, send it to the Google API for processing, get a return value from the API which should be a word/command, and then use the command to navigate the smartmirror.

Database of words

As mentioned earlier, this smartmirror has been made on a tight budget. Low-cost hardware, coupled with not so perfect pronounciation gave some weird returns from the Google API. In order to be able to work around this, and provide a ‘smooth as possible’ flow for the user. We had to create our own database of keywords for the different commands used in the smartmirror. The more we tested, the more words we got. Although this might be a way of ‘cheating’ the system, the structure of the smartmirror and the set number of commands allowed us to use similar words to represent our actual commands (an example is ‘Logan’ for ‘login’).

To use the database of words which we created throughout the testing period. The smartmirror sends the value which it gets returned from the API to functions which in turn iterate through lists of ‘allowed keywords’ for a given command. When it finds a keyword representing a command, it returns the command to the main program.

Face Recognition

The way we decided to differentiate between users of the smartmirror, was to use Face Recognition. We decided to use the OpenCV library together with a RaspiCam to provide this feature. In order to be able to use the feature as we wanted in the smartmirror app, we had to do a lot of testing in terms of speed and accuracy to get to a level we felt was satisfactory.

Finding the right algorithm

The first thing we experienced when using OpenCV together with our RaspiCam, was that it was very inaccurate in it’s predictions. After testing FisherFaces, EigenFaces and LBPH (Local Binary Pattern Histograms), we opted for using the LBPH. Using this we had a very high rate of success when recognizing different users compared to the other two. Especially FisherFaces seemed to be very sensitive to light, which basically meant we had to provide new training pictures for the face recognizer every time the lighting changed. In addition, both EigenFaces and FisherFaces use the entire database of images for a given user when searching for a match, whereas the LBPH algorithm only uses one at a time, this eases the processing time.

LBPH works in the way that it produces a matrix representing the face of a user/subject. The matrix consists of binary-numbers representing different areas of the face. When setting up a profile of a given face, and when comparing to the image fed to the algorithm by the smartmirror. LBPH looks at relative difference between neighbouring tiles of the matrix in order to provide a final number representing the accuracy of the prediction.

When starting out with FisherFaces, we were happy with accuracy-numbers in the ranges of 750-900. At the end of the project with LBPH, we set the threshold for ‘accepted accuracy’ to 50 (Lower numbers indicate higher accuracy).

Connecting with the database

As the OpenCV library works, the smartmirror has to provide training data for it to work. The training data are the pictures of users of the smartmirror, which is stored locally on the Raspberry Pi. In order to be able to make a prediction when given a face, the algorithm has to have a way to sort the different faces of the database. Locally on the Raspberry, the images taken during registration (which are used in the face recognition) are stored in folders named s1, s2, s3, etc. When training, the algorithm will iterate through the folders and read the integers and at the same time process the images stored in the folders in order to bind all images in s1 to the index of 1, all images in s2 to the index of 2 and so on. This will produce two vectors which will represent a profile of a face found in a image (which will depend on the algorithm used), and also a reference to the indexing number of the folder. If during login, the face which is processed corresponds to a profile indexed to 1, the face recognition knows that the face trying to logon belongs to index number 1.

We also decided to use this index as a reference to our database’s Primary Key. If a face is recognized, the index is stored to a variable and used as a referencing variable in the database throughout the smartmirror.

Database

To store the different users and preferences we’re using a SQLite3 database which is stored locally on the Raspberry Pi. The ‘users’ table consists of a ID (Primary Key), name, path to images, three news sources, a users preferred destination and also preferred type of travel. As mentioned earlier, the referencing from the smartmirror to the database is done using the ID returned from the face recognition at login.

Other modules

In addition to the named modules, we have utilized NewsAPI, WeatherAPI, and Google API for travel. These are used as described in earlier blog posts.

User specific data

When the application is started, the user can see a startup screen with three
options: sign in, register and guest. By saying something along the lines
“sign in” or “log in”, the smartmirror will check if the face is a registered
user in the database. If a known face is found within 10 seconds, the user is
met with a welcome screen, and then redirected to the main screen.

Not user specific data

In the main screen, the user can see various personal information fetched from
the database. The navigation part of the main screen will
display the users chosen destination and travel type, and tell the user the
time it will take to get there. The news part will show the users chosen
preferred news and display the news icons. By saying the name of the source,
e.g. “BBC News”, the user will be redirected to the spoken source. Here the
user will be displayed with the 10 latest titles from the news soruce.

Regardless of which user is logged in, the main screen will always show the
date and time. The main screen will also show the current temperature in Kongsberg,
as well as a image describing the weather, e.g. sunny. By saying the word weather,
the user will be redirected to the weather screen. Here more detailed weather data
will be displayed. This contains a daily forecast, a weekly forecast, as well as a
more detailed current weather description.

Brushing your teeth

Anytime when in the main screen, the user can say the word ‘hello’ to get
handed a toothbrush with toothpaste on.

Navigating backwards

In any screen, the user can go back to the screen he/she came from by saying
something along the lines of ‘go back’, ‘return’ or ‘previous’. When in the
main screen, the user can also log out by saying for instance ‘log out’, ‘sign
out’ or ‘return’. The application will then go back to the startup screen.

Not finished modules

If we would have more time in the project, we would continue on the
registration and setting modules. The registration is one of the choices in
the startup screen, and allows for a new user sign up by spelling their name,
as well as letting the mirror take pictures of their face to use for the login.
The setting module would allow for a already registered user to adjust their preferences.
This would include changing their chosen destination and travel type, as well
as mixing up their preferred news sources.

Smart Mirror – Arduino: Update on the code

//////////////////Relay//////////////////////////////////
const int relay = 5;
int relayVal;
//\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
//////////////////loadCell///////////////////////////////
#include “HX711.h”
#define zero_factor 555230
#define DOUT 3 //gelb / yellow
#define CLK 2 //orange
float calibration_factor = ((200000 * 0.00045359237) + 52400);
HX711 scale(DOUT, CLK);

float valueReadLoadCell;
float valueLimit;
//\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
//////////////////Servo//////////////////////////////////
#include <Servo.h>
Servo myservo360; // continious rotation servo
Servo myservo180; // 180 servo
int direction;
int pos3 = 0;
//int pos1 = 110;
const int servPin360 = 9;
int speed = 94; // number < 90: movement clockwise; number > 90: movement counterclockwise; the actual number controles the speed of rotation
const int servPin180 = 8;
//\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
////////////////Button///////////////////////////////////
const int butPin = 12;
int butVal;
//\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
///////////////Raspberry/////////////////////////////////
char receivedChar; //
boolean newData = false;
//\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
long firstMeasure = 0;
long secondMeasure;
long interval = 7000;

unsigned long previousMillis = 0;

//_________________________________________________________________________________________
//——————————————-SETUP—————————————–

void setup() {
pinMode(butPin, INPUT);
pinMode(relay, OUTPUT);

Serial.begin(9600);

scale.set_scale(calibration_factor); // when u open the serial monitor the value should be between -0.5 and + 0.5: for setting it to 0 we will have to mount it to the box
scale.set_offset(zero_factor); // Zero out the scale using a previously known zero_factor
Serial.println(“Reading: “);
delay(5000);
valueReadLoadCell = (scale.get_units()); // get the initial value of the load cell into the variable. has to happen once in the setup otherwise the first calculation will be 0 + the interval
}

//________________________________________END of SETUP_____________________________________
//——————————————-LOOP—————————————–

void loop() {
recvInfo();
Serial.print(“Loop: “);
Serial.println(scale.get_units()); // keeps spamming the actual value of the loadCell. should be commented out if the connection is through the pi

checkButton(); // function to check if the button is pressed
if (butVal == HIGH) {
Serial.println(“called through button”);
cButton();
}
if (newData == true) { // if there is the right input data received from the raspberry PI, then start running the system
Serial.println(“called through PI”);
cPI();
}
else {
digitalWrite(relay, HIGH); // disconnect powersupply
relayVal = HIGH; // relay works inverse: HIGH = power gets disconnected from system; only the red led on relay is on
}
}
//_____________________END of LOOP_____________________________________________
//———————FUNCTIONS———————————————–
void recvInfo() { // checks if data is received from the raspberry pi
if (Serial.available() > 0) {
receivedChar = Serial.read(); // puts the value into ther variable receivedChar
if (receivedChar = ‘s’) { // if the value equals s (start)
newData = true; // set new data to true, so that the function of starting the system in the loop can get called
}
else {
newData = false; // if its not a s (start) then do nothing
}
}
}

void cButton() {
wholeSystemFunction(valueReadLoadCell, valueLimit ); // calls the function of starting the sytem started by the button
}

void cPI() {
wholeSystemFunction(valueReadLoadCell, valueLimit); // calls the function of starting the sytem started by the pi
newData = false; // after running the system, the variable has to be set to false again
delay(1000); // give time to process: found out during testing that without its more liable for mistakes
}

void wholeSystemFunction(float valueReadLoadCell, float valueLimit) {
delay(1000);
myservo360.attach(servPin360); // attach the servo that presses the toothpaste
delay(500);
digitalWrite(relay, LOW); // relay works inverse: LOW = connected to external powersupply; red and green led on relay are on
relayVal = LOW; // set the value to its start state with which the servos gets controlled

checkLoadCell(); // check the measured value on the load cell
calcNewValue(valueReadLoadCell, valueLimit); // call the function to calculate a new value

//PUSH TOOTHPASTE
if (relayVal == LOW) { // run following if the relay is on
delay(1000);
Serial.print(“In whole System Function: “);
Serial.println(valueLimit);
previousMillis = millis();
while (scale.get_units() <= valueLimit && millis() – previousMillis <= 8000) { // keep the servo running as long as the measured value, which is changing all the time, is smaller then the interval (one portion of toothpaste). stop moving if the weight is not reached within 8 seconds.
servo360(speed, valueReadLoadCell); // call function to move the servo
}
while (scale.get_units() <= valueLimit && millis() – previousMillis >= 8000) {
return;
}
myservo360.detach(); // detach the servo to stop it from twitching
delay(100);
//OPEN DOOR
myservo180.attach(servPin180); // connect the second servo
servo180(); // call the function of the second servo to open the door
myservo180.detach(); // detach the servo to stop it from twitching
}
}

int checkButton() { // gets called in the loop
butVal = digitalRead(butPin); // put the state of the button into the butVal variable
return butVal; // return the value
}

int checkLoadCell() { // function gets called in wholeSystem function
valueReadLoadCell = (scale.get_units()); // stores the value of the load cell in this variable
Serial.print(“Measured Value in loadCell Function: “);
Serial.println(valueReadLoadCell);
return valueReadLoadCell; // returns the value. this is now a fixed value, which is used in the next function
}

float calcNewValue (float & valueReadLoadCell, float & valueLimit) { //calculate a new variable depending on the measured value + a preset interval
valueLimit = valueReadLoadCell + 0.3; // this variable is very important. the load cell is very responsive, so from one go to the next the number levels in at different values. befor the measured value was compared to a fixed number but since the measured value was not stable at all that was a huge liability and the limit value had to be changed all the time. now the servo rotates on base of a not changing interval and not on a uncontrolable difference.
Serial.print(“Value calculated in calc Function: “);
Serial.println(valueLimit);
return valueLimit;
}

void servo360(int speed, float valueReadLoadCell) {
myservo180.attach(servPin180); // connect the door opener. necesserary because it always twitches open
myservo180.write(110); // set him to 110 degree, which is the closed position. could be set as a variable
delay(100); // give him time to do that
myservo180.detach(); // detach him to stop it from twitching
delay(1000); // give him time to do that
myservo360.write(speed); // start to push toothpaste
}

void servo180() {
open(); // call function to open the box
while (scale.get_units() < 15) { // do nothing as long as the weight recognised is smaller then 50 (we still need to find out which number actually makes sense through making a few measurements when the toothbrush gets put into the bracket, thats the number we need here)
Serial.println(scale.get_units());
}
while (scale.get_units() > 15) { // call function close, when toothbrush gets put into the bracket / box
close();
}
}

void open() {
delay(5000); // this delay gives the toothpaste a little bit time to settle onto the toothbrush
direction = 1; // 1 equals opening direction = clockwise
if (direction == 1) { // if it is one
for (int i = 115; i >= 30; i–) { // move the servo from its starting position to the open position at 30 degree. can be opened further but 30º is enough
myservo180.write(i); //
delay(50); // delay determines the speed of opening the door
}
}
}

int close() {
delay(5000); // this delay is very important, it should be as accurate as possible and be the average time that a person needs to put the toothbrush back into the bracket, and remove his hand, so that the door doesnt close if the hand is still in the door
direction = 0; // 0 equals closing direction = counter clockwise
if (direction == 0) { // if it is zero
for (int i = 30; i <= 115; i++) { // move the servo from its opening position to the close position to 115 degree. the closing position was determined through various tests
myservo180.write(i);
delay(50); // delay determines the speed of opening the door
}
}
return newData = false; // set newData to false, again. probably redundant
delay(5000); // give time to settle
}

 

Smart Mirror – Electronics: Update

pastedGraphic.png

In order to make the movement in the mechanical arms we used an Arduino microcontroller attached with 2 engines and a load cell to control the amount of toothpaste to put in the toothbrush.

This Arduino device is controlled by a relay, which is further connected to a power bank to provide 5V on it, and also a raspberry pi which is also attached to a 5V source. This last source is the power transformed from the power plug.

We plugged the raspberry pi in the power plug because e it is a fragile system that should receive constant clean power. We also should say that the relay we used is a switch that opens and close the main circuit composed by the Arduino and the engines and load cell.

It is also important to say that in the test phase while we were trying to complete this task we used a button in order to start the whole system. This means that this button should initiate the load cell and the first engine. After the enough amount of paste was weighed the system controlled by the Arduino should stop the first engine and the load cell and run the second engine which brings the toothbrush. Just after the load cell receives the message this engine should bring the toothbrush inside again and stop the circuit.

Smart Mirror – Mechanical: Change in Design and Parts

Changes in design

 The toothpaste movement was first designed with a “crankshaft” movement. The 3D parts for the tooth paste dispenser were printed with a tight fit that the small engines wouldn’t be able to force the piston into the cylinder. The design is changed to a linear movement met gear and rack. The engine has a small gear to press the tooth paste out as slowly as possible till the load cell sensor shuts the engine down.

3d printed parts

The 3d printed parts came out bigger than expected, the project group members grinded the parts to the size that was designed.

New parts

The box 

The box itself is made of 6mm plywood plates, wood is chosen as material because it is cheap and strong enough. They are connected by a linear pattern of cut-outs. The 3D tool indent gives the opportunity to get the pattern of the cut-outs into the other plate. Eventually everything fits together and can be glued together.

  Door movement

The door movement is done by a 180 degrees servo engine. The engine opens the door with a connected double arm. One arm is attached to the engine and one arm is attached bolted to the door. The arms are connected by a M4 bolt and separated by a two millimetres thick washer out of plexiglass.  The arms are both made of three millimetres thick plywood. Plywood is chosen because it’s strong enough with a low thickness and because it’s a cheap material.

The door with the mounts

The door plate is designed out of 6mm plywood with cut outs for mounts that are applied on the inside of the door, we used 6mm because we needed a strong basement for the load cell. The sensor will give wrong signals when the mounts are too flexible. The mounts are made in a way that they get strength out of 2 ways in the door plate, horizontal and vertical. The mounts are connected to get more strength out of the design. The mounts that opens the door are only have horizontal connections in the door because the force is also applied in that direction.

Toothbrush

The toothbrush is made of wood because it’s still a prototype and this was the cheapest way to simulate a tooth brush. The toothbrush is based on two layers of six millimetres plywood and glued together to the final product.

Toothbrush mounts

There were 2 ideas at the beginning, first one with a U form profile and a pressing plate attached to a bolt that can hold the toothbrush. The second idea was to use the 3D printer with a clamp that was based on flexibility.  The decision is based on the curiousness to the flexibility of the 3D printed parts and to try something new.

The mounts of the toothbrush are based on the flexibility of the material that is used by the 3d printer.  The mounts are locking the toothbrush by the edges of 1 mm that are holding the upperpart of the toothbrush. The mounts open when there is pressure applied to the sides of the mount.

Engine mounts

The engine mounts are cut out of 6 mm plexiglass because there were holes required in the layer. Wood will be torn apart and plexiglass will be a possibility.  After that there are holes drilled in the plexiglass to connect it to the base plate of the box. This was a really hard process because the plexiglass melts or cracks fast. Eventually the task is completed with the technic of pecked drilling.

Cylinder mounts 

The cylinder mounts are also cut out of 6 mm plexiglass. The cylinder mounts are made a little bit smaller than the cylinder is to work as a clamp to hold the cylinder into place when the engine pushes.

  The cover 

The cover plate of the box is cut out of six millimetres plywood as a base that can hold the weight of the ipad. The cover and one plate of the box is milled on two areas to get a engravement for the hinges. The engravements are used to prevent that the cover edges and the box are scratching when the cover is open.

The inner cover parts 

The front face of the mirror and the walls are cut out of 3 mm to make it as light as possible. The walls are 5 centimetres high to give space inside for the mount of the iPad and the iPad itself. The cut out in the front face is for the camera. The cut outs in the walls are for wiring and a bolt connection together with a mount that will be able to hold the cover in a certain angle.

The parts to hold the cover under an angle 

The mount is made of 6 mm and will be grounded in the walls of the inner cover part. The two arms are connected by a m4 bolt and a fly nut to fix and loose them in an easy way. The mount is connected to the inner plate that will be attached above all the working parts in the box.

The cover and the inner cover parts is connected with mounts in the form of a L cut out 6 mm plate, this mount will be attached on 16 places.

  iPad mount

The iPad mount is also 3D printed. The first try failed, we tried to guess the thread of the original mount, it didn’t fit at all. The second try was successful, we tried it without the thread and designed only the mount plate with the arms to connected in the original mount. The only thing that was wrong that the mount plate was too long. It fitted after a cut and some grinding work.

Engine to gear connector

The engine to gear connector is a 3D printed part. The holes had to be drilled again but there was no grinding necessary in this part. This part connects the servo engine to the cross bar that is connected to the gear.

M4 nut locker

The nut locker is a square piece of 3 mm thick plywood with the form of the nut inside. The nut locker is glued to the wooden baseplate of the in between plate.

Operational descriptions

Box with cover plate

The box plates must be glued together with wood glue. The plate that is attached to the cover and the cover itself have to be milled first with a 10mm mill and finished by a 2 mm mill. The hinges have to be placed at the areas of the milling. The hinges are connected by m4x10 bolts with a conical head. The hinges are locked by M4 nuts with washers.

Tooth paste movement

Piston and cylinder

First attach the cylinder to the mounts and screw the bolt till the cylinder is not able to move in the mounts anymore. Put the O-ring on the piston and connect the piston to the rack with a M4x12 bolt and lock it with a M4 nut. Place the piston into the cylinder. Glue the cylinder in place according to the placement drawing.

Engine and support

Attach the engine to the mount by four M3x12 bolts and lock them with a washer and a M3 nut. After this take the gear connection and attach it to the engine with 8 M2x6 bolts. Connect the gear to the connector with the cross bar. Glue the mounts of the engine in place according to the placement drawing.

The support wheel relates to Lego materials. Put the wheel on the crossbar and move it into the right position in the hole pattern, lock it with the yellow locking parts. Glue the support wheel mount under the hole pattern and let it rest for a minute. Glue the mounts of the support wheel in place according to the placement drawing.

Door movement

Glue the mounts in place according to the assembly pictures. Screw the load cell to the lower door part with 3mm x 8mm screws, screw it with the load cell on the edges of the door part. Glue the tooth brush mounts to the load cell and the left door part, use the edges again to outline the parts.

Engine and the mounts

Drill the holes into the mount with a 2,8 mm bore and the speed as low as possible to prevent the production of heat in the plastic. Screw the engine on both sides with 2 M3x8 screws.  Mount the engine connection with 8 M2x6 bolts to the engine arm. The engine arm is connected to the arm with a M4x16 with a in between bush, locked by a M4 nut. The arm is connected to the door with a M4x16 and that one is also locked by a M4 nut.

Cover and inside parts

Glue the inside plates together with wood glue. Glue the ipad mount into place and slide the ipad holder in the ipad mount. Put the ipad in place. Glue the screen layer of plexiglass onto the wooden plate on the inside cover. Glue the camera also in place at the top part of the cover. Screw the raspberry pi on the located place according to the wiring diagram.  Glue the L links to the inside plates of the cover and to the cover itself to connect. 3 on the top and lower part and 5 on the sides. Divide the L links on the side parts with a distance around 7 cm.

Cover hold up mounts and locker

Place the mount of the in between plate holder on the location according to the location drawing. Bolt the in between plate to the mount. Glue the M4 nut lockers around the nuts and to the wooden plate of the mount. This will keep the nuts locked when you unscrew the bolts from the top. Bolt the mounts of the cover arms on the located places with m4x12, locked by m4 nuts. Bolt the arm to the mount and bolt the first arm to the second arm by a M4 bolt and a M4 fly nut. Bolt the upper arm to the side of the cover and the mount that is still left. Put glue under the mount and connect it to the cover plate.

Smart mirror: Sofware update

Smart mirror – Software update

Last tuesday (31.10.2017) we recieved the soundcard, and configured the Raspberry Pi to use it as default sound card. Today we ordered a mirror foil, expected to be delivered between 16th and 22th of November.

We made good progress on the software part of the smart mirror today. The face recognition module have been integrated with the main application and runs smoothly. Going forward we are going to integrate the registration module, as well as the speech recognition module.

We also decided to add a new option to the start-up screen, giving the user the ability to login as a guest. This means that there is no need for facial recogniton and no user data will be used, but instead some preset options will be used for both the news module, and the navigation module. The figure below shows a updated flowchart of the application.

Smart Mirror: CW 42

Electrical/Mechanical:

This is the first and well working draft of our arduino program to control the motors.

/* Connect 5V on Arduino to VCC on Relay Module
Connect GND on Arduino to GND on Relay Module
Connect Power on Arduino to the Common Terminal (middle terminal) on Relay Module.
*/

//////////////////Relay///////////////////////////
const int relay = 5;
// int relayVal;

//////////////////loadCell///////////////////////////
#include “HX711.h”
#define zero_factor 145230
#define DOUT 3 //gelb / yellow
#define CLK 2 //orange
float calibration_factor = ((200000 * 0.00045359237) + 52400);
HX711 scale(DOUT, CLK);

//////////////////Servo///////////////////////////
#include <Servo.h>
Servo myservo360; //continious rotation servo
Servo myservo180; // 180 servo
int direction = 1;
int pos3 = 0;
int pos1 = 0;
const int servPin360 = 9;
const int servPin180 = 8;

 

////////////////Button///////////////////////////////
const int butPin = 12;
int butVal;

 

void setup() {
pinMode(butPin, INPUT);
pinMode(relay, OUTPUT);

Serial.begin(9600);

scale.set_scale(calibration_factor);                    // when u open the serial monitor the value should be between -0.5 and + 0.5: for setting it to 0 we will have to mount it to the box
scale.set_offset(zero_factor);                    //Zero out the scale using a previously known zero_factor
Serial.println(“Reading: “);
close();
}

void loop() {
Serial.println(scale.get_units(), 1);

checkButton(); // function to check if the button is pressed
if (butVal == HIGH) {
myservo360.attach(servPin360);
digitalWrite(relay, LOW); //relay works inverse: LOW = connected to external powersupply

//push the toothpaste out of the cylinder
while (scale.get_units() <= 10) { // when the weight is more then 10, call the function
servo360(); // of the first servo
}
myservo360.detach();
myservo180.attach(servPin180); // connect the second servo

// open the door of the box, so that the user can grab the toothbrush
servo180(); // call the function of the second servo to open the door
myservo180.detach();
}
else {
digitalWrite(relay, HIGH); // disconnect powersupply
}
}
//——-End of Loop——————————-

//——-Start of Functions————————
int checkButton() {
butVal = digitalRead(butPin);
return butVal;
}

void servo360() {
direction = 100; // number < 90: movement clockwise; number > 90: movement counterclockwise; the actual number controles the speed of rotation
myservo360.write(direction);
//to be attached here: stop the movement when the toothpaste is empty. so we need to know the maximum
// in centimeters and calculate it back to the rotation, best is probably to measure it
}

 

void servo180() {
open(); // call function to open the box
while (scale.get_units() < 50) { // do nothing as long as the weight recognised is smaller then 50 (we still need to find out which number actually makes sense through making a few measurements when the toothbrush gets put into the bracket, thats the number we need here)
}
while (scale.get_units() > 50) { // call function close, when toothbrush gets put into the bracket / box
close();
}
}

void open() {
delay(2000); // this delay gives the toothpaste a little bit time to settle onto the toothbrush
direction = 1;
if (direction == 1) {
myservo180.write(140);
delay(100);
}
}
void close() {
delay(2000); // this delay is very important, it should be as accurate as possible and be the average time that a person needs to put the toothbrush back into the bracket, and remove his hand, so that the door doesnt close if the hand is still in the door. maybe we have to use a sensor to make it nicer
direction = 0;
if (direction == 0) {
myservo180.write(0);
delay(1000); // give to servo time to do the movement
}
}

Software:

The software side of the project is moving forward in terms of getting our program to the Raspberry Pi that will be used to provide information to the screen of our smartmirror.

Since we discovered that making a touch-interface would be to expensive, we’ve decided to control the smartmirror using simple voice-commands. User identification is done using facial recognition. The backbones for both the voice- and facial recognition are done, the remaining work for these parts are the configurations needed to get them working on the Pi. We also need to integrate them into our main program which is made in Kivy.

As a screen we have decided to port the Raspberry Pi to an iPad through the built-in VNC functionality of the Pi. Below is a video of the current concept.

 

Smart Mirror: KW 41

With some help we were now able to connect the two required motors to the arduino.

The hardware works in the following manner:

  1. A pushbutton activates the system
  2. the relay switches so that the motors are connected to the powersupply
  3. the 360º servo starts rotating: it presses the toothpaste out and onto the toothbrush. the toothbrush is attached to a loadcell
  4. the loadcell measures the weight and gives a signal back to the arduino
  5. if the weight reaches a certain limit, the first servo stops moving,
  6. the second 180º servo starts and opens the door so that the user can grab the toothbrush
  7. as soon as the user puts the toothbrush back into its bracket the door closes and the process is finished

The code is not yet finished completely. 

 

Smart Mirror: KW 39

Progress:

Design: 

Changes:

The mirror will no longer be usable by touch. The touch overlays are just too expensive. We are currently considering which alternative is the best, if its buttons ore some kind of a mouse interface or something completely different.

Mechanical:

The first parts are finished. After 3D printing them they needed a little finishing as the piston and the cylinder didn’t fit as smoothly as they should. So Youp grinded the it by hand into our required measures.

To make the project a little nicer we decided to use two different kinds of movement for the toothpaste dispenser (TD) and the door opener. At first it was planed to use a 180º servo for the TD. Now we will use a 360º servo. This gives us the opportunity to move the piston more precise. With our first approach the pushrod hat to be all the way lowered within 180º so we were very limited.

More details will follow.

Arduino (Bene):

Fitting the relay was more complicated then thought in the beginning. At first I had to get more theoretical knowledge about how it is working. Now with some help I was able to add it to the circuit so that we can start it by pressing a button.

Due to the decision of changing the movement for the TD the coding has to be changed as well.