Here is the final documentation on the PComp final project. Our piece has a lovely name now courtesy of Sisa!
Concept: Stitching Stories is a tale of a seamstress and her boss, but the user controls the non-linear narrative with a physical controller. Each function reveals a different part of the story that relates to the function in some way.
Materials: Hardware - Bop It, Arduino Micro, rainbow cable and other misc electronics
Software: Arduino and Processing
Our initial concept was to fabricate a custom controller similar to a cube, but we found that that that would have been expensive and limited in terms of interactions. I pitched the idea of hacking a Bop It, a popular handheld party game from the 90's with multiple controls, to control the video. We hacked one open and discovered that the Bop It was basically controlled by a bunch of individual push buttons, which we could use to hook up to an Arduino that we could program ourselves by resoldering the buttons to our wires.
At one point we thought about hacking up the Bop It since we were using an Arduino Uno and a breadboard, but we opted to keep the Bop It intact as a controller rather than build a custom one thanks to Xuedi's input. She directed us to replace the Bop It's original contents with an Arduino Micro (thanks to Joe Mango for lending us his!) and a ribbon cable, which would help us contain all of the Bop It's wiring in a clean way.
So I gutted the Bop It of its innards to give us a blank slate to work with:
Sisa had created a prototype circuit to test the functions, which became the basic template for each function's circuit. Originally we planned to use all six functions: Bop It, Twist It, Flick it, Twist It, Spin It, and the mysterious purple button that I don't know the function of; but since we had limited footage in our narrative, we opted to forgo the center button - I stuffed it with foam to maintain the integrity of the controller while rendering the button immovable so the user can't press it on accident.
I also hacked the purple pushbutton to prevent it from catching - as seen in the original picture above, the purple button stays pushed down in its original state. To pop it back up, you have to push on it again - this caused some problems with the way the video event handlers were written in Processing. I stuffed it with foam so that the user doesn't have to push twice to restore the button back to its original state
It took us a few days to put the hardware together (achievement unlocked: learned to solder!), and it was interesting trying to put it back in. Luckily the Bop It had built in hooks to thread the cable through to hold it out of the way.
Here is the final product. There are little tape tags labelled to show which button goes to which function, which made my life a lot easier when I reassembled the Bop It
Additional photos and videos can be found in my Google Plus album, the link to which is in the Resources section of this post.
The software was used to play the video; as the user played with the physical controller, the video on the screen would switch to show a different clip based on what the user selected.
Since switching to the Bop It for our controller, we thought we'd have to use something like Unity or Max MSP to get this done, but the time limit forced me to write a Processing sketch to connect to the Arduino. This actually worked in our favor - Processing did the job quite well with the exception of some performance issues where the video would stutter. I attribute this to a couple of things: lag from the serial communication with the Arduino and from the computer running too many things at once. I was able to minimize this effect by closing down all windows before running the program. It also helped to have smaller videos.
I used Processing's video library to load the videos, track the function from the Bop It, and change the video whenever a change was made. I also was able to implement a 'Plan B' that Sam Slover recommended - the code would also work based on some keyboard inputs in case the Bop It failed.
I've posted the code to Github, which I link to at the Resources part below
- Worship the Residents - The residents are amazing. Bless the residents, in particular Sam and Xuedi. They were so helpful and I don't think we could have done this without them.
- Keep Calm and Change On - this was the big takeaway for me. This project has evolved so much from the concept that Sisa pitched in class and that I got interested in and took us down many paths in regards to the technology used to implement it as well as the interaction model (since we changed the controller entirely). There was a lot of stress and pressure, but I think that Sisa and I had a good deal of patience that helped us get this done well and produced a working project.
- Take Notes - documenting all the meeting notes in a shared Google Drive was so helpful because we could both see what was going on and share resources easily. We kept on track pretty well when we were taking notes to have something to go back to.
- Practice - I was much more comfortable with the concepts I learned in this class going through this project. Putting the circuit together was a great way of relearning the basics of electronics. I also was able to get a good handle on how to do serial communication via Processing, which was something that I had done with Will in the midterm as well.
- Don't Fear the Solder - this was my first time soldering, so I was a little nervous about it. Even though we made a few mistakes, it worked out well in the end! I enjoyed it immensely