Contact: kathryn.lepine1@ucalgary.ca
Project 2: Unlocking Phone
Project 2 focuses on creating a unique way to unlock a phone. The first section of our project centers around unlocking a phone via sensor. By leveraging the front camera and speakers of the phone our idea allows visually impaired individuals to unlock their phone well simultaneously learning how to perform certain hand signals. The second section of our project centers around multi touch. Our idea puts a fun gamified twist to unlocking a phone requiring the user to pop a random number of balloons in a random amount of time.
10 Sketch Ideas: Sensor Input
Each member of our team came up with ten sketch ideas using a sensor to unlock a phone. These ideas used a range of sensors available on our phones including a light sensor, camera, accelerometer, gyroscope, etc. My personal idea came from the themes of staying active, implementing positive healthy life habits, and quick fun easy tasks. On Sunday October 9, 2022, we had a team meeting where all members came with their ten sketches ready to present. A few of the ideas that stood out were Alexis’s finger number unlock with machine learning, Vardhini’s break the glass with voice, and Kathryn’s mirror selfie with camera. We conducted a vote; the results were all in favour of Alexis’s finger number unlock with machine learning. This idea stood out to us because we were all eager to learn about machine learning. Additionally, our phones currently use face recognition to unlock, and this idea presented as a similar method that would be both effective and creative at the same time. Click here to see the google forms used to conduct the vote. Click here to see the google forms used to conduct the vote.
Hold a squat for 10 seconds to unlock phone. I thought of this because it really hard to hold pause squats at the gym.
Make a big teeth smile at the camera to unlock the phone. I thought of this because everyone should be smiling more during the day.
Take a mirror selfie to unlock. I thought of this idea because I love taking mirror selfies especially in different mirrors.
Hold a squat for 10 seconds to unlock phone. I thought of this because it really hard to hold pause squats at the gym.
10 Detailed Sketch Ideas: Sensor Input
After our meeting on October 7, 2022, we all left ambitious of what we could achieve with our new decided idea to use number gestures to unlock phones. A strong theme in the sketches we created was learning, including math, binary number system, roman number system, and a new language. On Tuesday October 11, 2022, we had a meeting to present the ten detailed sketches each of us had created. A few of the top ideas we presented were Kathryn’s binary converter idea, Alexis’s guess the number in the song idea, and Vardhini’s math jokes idea. After conversation there was strong enthusiasm towards Alexis’s guess the number present within the song idea. We all loved this idea because it brought a fun trivia game component to the application. It appealed to us that we wanted to do a “fun” task to unlock our phone rather than something that was intensive to learning.
User is given a decimal number. They must convert the number into binary and gesture with zero and ones to input the correct binary number. I thought of this idea because I am a computer science major so converting decimal to binary is a useful skill.
A passcode is displayed on the screen then shortly disappears. The user must memorize the passcode then use finger gestures to repeat the passcode to unlock. I thought of this idea because I always forget the text verifications numbers and need improve my short term memorization.
Code is shown on the screen which the user must memorize and hand gesture repeat. I picked this design because I lack in memorization so this a great source to strengthen memorization skills.
User is given a decimal number. They must convert the number into binary and gesture with zero and ones to input the correct binary number. I thought of this idea because I am a computer science major so converting decimal to binary is a useful skill.
Refined Detailed Sketch: Sensor Input
While our group had originally chosen the 8: Song Number Titles idea to implement, after a discussion with the TA, we have decided to see if we could generate more creative ideas based on the concept that we have chosen. Keeping the concept of putting together audio and hand gesture at the center of our idea generation, we have thought about how we could create a representation of this concept in a more meaningful or metaphorical context. The inspiration for our refined idea was based on Huawei’s Facing Emotions app, which used artificial intelligence to inform the visually impaired of the emotion on the face of someone they are talking to by translating it into sound. A snippet of our idea generation process is presented below as the second image. For choosing which hand gestures to include in the application, we decided to select a handful from the hand gesture emojis. Another aspect to consider in the design process was to ensure that all the instructions on how to proceed through the application were spoken aloud. Thus, we figured that speech-to-text functionality API would have to be utilized for this purpose.
Sources
-
Each description of a gesture contains information on – 1) the meaning of the gesture, 2) when the gesture is typically used, and 3) how to perform the gesture. The scripts for the audio files I have generated are from this source.
-
The machine learning model was trained using Teachable Machine.
-
The audio files were generated using this text-to-speech application.
User is given audio describing a hand gesture. The user must then correctly perform the hand gesture described to unlock the phone.
Idea generation process.
User is given audio describing a hand gesture. The user must then correctly perform the hand gesture described to unlock the phone.
Video Presentation: Sensor Input
Link to Website and Github: Sensor Input
10 Sketch Ideas: Multi Touch Input
Each member of our team came up with ten sketch ideas using multi touch input to unlock a phone. The ideas stemmed from our day to day lives and what each one of us thought would be an innovative and productive task to continuously perform throughout the day. My personal sketches surrounded themes of improving math and geography skills, learning CPR, etc. On Sunday October 9, 2022, we had a team meeting during which all members of the team presented their ten sketch ideas surrounding using a sensor to unlock their phone. There were many excellent ideas, including Alexis’s guess the time zone idea, Vardhini’s pop balloon idea, and Kathryn’s math is fun idea. From the seven sketches that stood out we took a vote and recognized Vardhini’s balloon pop idea as our favorite. Popping balloons appealed to all of us because it is satisfying, and many different variations could be implemented keeping our creative juices flowing. Click here to see the google forms used to conduct the vote.
User taps their initials in Morse code to unlock. Thought of this because morse code was used in the war which is interesting.
User taps and holds two specifics spots on the screen that only they know where they are. I thought of this idea because its simpler then current digit codes would be fasters and less to remember.
Tap the rate of doing CPR for one minute to unlock. This is 100-120 taps / compressions per minute. I thought of this idea because when I was learning CPR it is much faster than I thought.
User taps their initials in Morse code to unlock. Thought of this because morse code was used in the war which is interesting.
10 Detailed Sketch Ideas: Multi Touch Input
After our meeting on October 7, 2022, we were excited to look forward to implementing our detailed sketches and see what new innovative ideas we could produce. Each one of us took the idea of popping balloons to unlock and put our own unique spin on the idea. Some of our ideas focused on the learning aspect implementing simple math, well others took a more gamified approach to balloon popping. On Tuesday October 11, 2022, we had a meeting to present our ten detailed sketches and carve a path forward. Ideas that stood out during the meeting were Kathryn’s pop in order of rainbow, Alexis’s time limit to pop a certain number of balloons, and Vardhini’s pop balloons based upon emotion. After much discussion we decided pursue Alexis’s time limit to pop a certain number of balloons mixed with Kathryn’s and Vardhini’s idea of popping balloons based upon colour. We liked the idea of a gamified tasks to unlock the phone. Moreover, by combining our ideas together we were able to produce a unique vision where each of our contributions were displayed.
Pop the balloons in the order of the rainbow starting with red to unlock. I thought of this idea because I actually do not know the order of the rainbow without thinking super hard.
User must pop the balloon only of a certain colour. I choose this because my boyfriend is colour blind so then he could not unlock my phone.
Using either a “bump left and right” or a “hold and drag” action, the user will have to put all of the balloons into the bin within a given amount of time. This idea was inspired by a similar task on a game application “Among Us”.
Pop the balloons in the order of the rainbow starting with red to unlock. I thought of this idea because I actually do not know the order of the rainbow without thinking super hard.
Refined Detailed Sketch: Multi Touch Input
On October 7, 2022, after our team meeting, we attended class tutorial seeking feedback on our recently decided design. One positive piece of feedback we received was the time crunch to pop the balloons was a good idea because it created a game for the user where a small degree of urgency and stress was created. The negative feedback received was the idea was not interesting enough. As our group worked towards comprehending why the idea was not interesting enough, we developed a few more design ideas. The first element we decided to add was instead of tapping the balloons to pop them the user would be given a dart to throw (in a swiping motion) towards the balloons. This swiping motion gives the user a different interaction with the phone then the classic tap used for almost all other interactions. Furthermore, we thought by using a dart and popping the balloons, the user is essentially opening the balloon up revealing the insides, very similarly to how the user is breaking into the phone to reveal what is inside. Lastly, we felt as though, are original idea could be thought of as a game rather than a way to unlock a phone. Thus, we decided to implement a small layer of security in our final design. The idea is to show both red and blue balloons but if the user pops a blue balloon they will be locked out. We thought this idea was good because security is enhanced but not at the expense of the game aspect for the user because they still have plenty of different balloons to pop.
User swipes dart to pop the number of balloons required. There is a timer to pop all the required balloons within the time limit. The user must only pop red balloons as a security component.
User swipes dart to pop the number of balloons required. There is a timer to pop all the required balloons within the time limit. The user must only pop red balloons as a security component.
Video Presentation: Multi Touch Input
Link to Website and Github: Multi Touch Input
My Contributions
I attended all team meetings and collaborative sessions, during these time periods I actively participated to help group progress move forward. I drew and presented 10 initial sketch ideas for both sensor input and multitouch input. I drew and presented 10 detailed sketches for both sensor input and multitouch input. I wrote the project write ups for the two paragraphs corresponding to the 10 initials sketches. I wrote the project write ups for the two paragraphs corresponding to the 10 detailed sketches. I wrote the paragraph description and drew the sketch for the second detailed multi touch idea. I designed, developed, and tested the entirety of the balloon multi touch program in JavaScript and HTML. I helped Alexis to implement a small component of the text to speech API call of the gesture program on JavaScript. I film the screen recording videos used to describe the balloon multi touch program. I was a character acting in both the balloon video and the gesture video, also helping film clips. I made the introduction montage for the gesture video. I helped to create the video script, storyline, and necessary components for the balloon video. I made some final finishing edits to the balloon video.