Yoon Kim
Concerts of Ions: Turning Neural Activities into Music
Last September, I was carefully looking for a course that could fulfill my WIM (Writing In the Major), which was an essential graduation requirement. Facing my supposedly last year at Stanford and graduate school application, I wanted to fill every single course in my schedule meaningful. All of a sudden, a course introduction of Bio 196: The Senior Reflection came up. Actually, I had not heard anything about the course before, but the fact that Susan McConnell, definitely one of the best neurobiologists I met at Stanford, and Andrew Todhunter, a renowned writer, filmmaker and new media consultant were teaching the class captured my attention. I carefully read the description. The core concept of integrating science and creativity to invent a project fascinate me. Without any hesitation, I signed up to join this three quarter project.
Before delving into my reflection about the project, I want to thank all the people I met throughout the course. When I first joined the project, I knew nothing, literally, about creativity or creative thinking. However, by meeting with Sue and Andrew every week and receiving their guidance, and by discussing creative projects with other TSR students and sharing our thoughts and comments, I learned how to think creatively and regained confidence that I can do things that I was not even aware of. I was really lucky to join this year’s TSR and all the experience had been a great asset to me.
I thank Prof. Bruce MacIver, my undergraduate research advisor, who has been my scientific mentor. His research focuses on interpretation of brain-slice EEG signals and one of the new projects in the lab involves novel representation method of EEG. Also, my creative mentor Prof. Takako Fujioka, a renowned professor in CCRMA, has been a great help in my inspiration throughout the project.
I also thank Stanford music community who has provided me enough support and resources for the project. Jieun Oh, a Ph.D candidate from the Department of Computer Music. Majoring in Computer-Based Music Theory and Acoustics, has given me great supporter to me. She was my Music 19 TA and was familiar with my musical preferences and creative style. She has experience both in music and in computer programming. Although she was too busy to devote her time as my primary creative mentor, she had provided a great help.
Motivation
Being born into a family with an older brother diagnosed, from age two, with autism, I have been inspired to find answers to mysteries of neuroscience. To be more specific, my first motivation came from neurosurgeons who treated my brother. While treating him, they emphasized how important studying neuroscience is, and I came to wonder how I might possibly contribute to this fascinating field. Since then, getting involved in neuroscience research was always my top priority. Throughout my undergraduate years, I have worked as a research assistant on projects in neuroscience laboratories. The experiments I’ve done have ranged from measuring the action potential to visualizing synapse proteins. Every time I did these experiments successfully, I was filled with joy. It was just amazing. Wow, I thought, how can these neurons work so beautifully? All together, those magical movements, mechanisms and manipulations increased my fascination with the world of neuroscience.
My motivation for doing this project was simple and clear. I would like to convey the passion I feel about neuroscience to more and more people who are unaware of this form of science. Setting aside my personal goal of pursuing career as a neuroscientist, introducing the very basic concept of the field in creative ways would contribute to greater accessibility in neuroscience research.
Theme
My specialization is in the electrophysiology of brain slices and molecular analysis of chemical synapses. Every time I measure action potentials from a rat brain, or take a picture of a particular protein in a neural synapse, I feel a pure intellectual pleasure of actually doing science--which is the largest reinforcement to my continuing research. With my creative project, I want to make a product that could represent this form of neuroscience in a friendly and non-scientific manner.
One of my hobbies is listening to diverse kinds of music. I listen to almost all spectra of music, but my favorite kinds of musical forms are those with distinct components of sounds, such as classical, rock, and electrical house music. I enjoy the way sound elements combine together and harmonize into a whole piece.
Combining my different interests, I propose to create a series of musical pieces that can efficiently represent neural activities The key to this project’s success lies my ability to efficiently depict and choose appropriate form of music and strongly represent and highlight neuroscience in music.
Neuroscience
Neurons, which are units in the nervous system, send signals to each other electrochemically. This means that chemicals cause electric signals. Chemicals that are “electrically charged” are called ions. The four important ions in the nervous system are sodium, potassium, calcium and chloride. Sodium, potassium and calcium ions are positively charged, whereas chloride ions are negatively charged. Nerve cells are surrounded by a membrane that allows some ions to pass through and blocks the passage of other ions. It is easy to understand when you think of a formidable fortress where no one but only selected people can pass through the “gates.” Those gates are called “ion channels.” Usually, there are more sodium ions outside the neuron and more potassium ions inside the neuron.
Neurons do not always send out a signal, and they are said to be at “rest” when they are in such states. At rest, all channels except a small portion of potassium channels are closed, so only a few potassium ions can cross through the membrane easily. Inside the membrane, negatively charged ions are present and this separation of charges creates a voltage difference across the membrane. In other words, the inside of the neuron has accumulated an excess of negative charges relative to the outside.
Action potential is an explosion of electrical activity created by a concert of the flow of these ions. When a signal arrives to a neuron, it first opens sodium channels. Biologically, it is natural for ions to move from a positive to a negative side, so sodium ions starts to slip into the neuron. Since sodium ions are positive, they cause the voltage inside the neuron to become more positive. After the voltage reaches a critical level, which is called a “threshold” for firing, most of the sodium channels are triggered to open up all at once, causing a surge of sodium influx, and making the voltage reach its peak.
Meanwhile, it takes longer for potassium channels to open. As the voltage peaks, the rest of the potassium channels are triggered to open. Potassium ions rush out of the cell and decrease the number of positively charged ions inside the neuron. Simultaneously, sodium channels start to close. This causes the action potential to go back toward rest. The action potential actually goes past resting energy because the potassium channels stay open longer. Gradually, the ion concentrations are restored to resting levels. This whole process is called an “action potential,” and it represents the fundamental mechanism that allows an electric signal to traverse the length of a nerve cell.
This action potential was the main concept of the music composition. Rhythmical repetitions of ion movements and their electric characteristic induced me into several different concepts and I started to search adequate music that fits my thinking.
Music Composition
A method for using non-speech audio to convey information or perceptualize data is called sonification. There are already hundreds of unique attempts, even in neuroscience, to sonify complex data such as EEG signals. For my project, I planned to first sonify different mechanisms of neural activities and unify them to make unique music.
I also audited a course called MUSIC 220B: Compositional Algorithms, Psychoacoustics, Computational Music course provided at winter quarter. During the audition, I learned a programming language called ChucK, a programming language for real-time sound synthesis and music creation. By using ChucK, I was able to convert simple movements of ions into delicate sound formats that became building blocks of my music pieces.
I chose Electric House music as my genre. Electric house is characterized by its repetitive rhythmical progress, composed of several different sounds including drums, cymbals, and bass lines. To me, it is the most suitable music to represent neural activities since it is electronic, minimalistic, and repetitive, like neural activity itself. For example, electric house music starts with a basic heavy beat with a snare in the 2nd and 4th beats, which can be analogous to the resting potassium conductance in neuron. Also, house music varies a lot in style and influence, ranging from deep house to the more minimalistic microhouse.
The First Piece
Firstly, I was stuck for more than one month before staring this project. Having so many other works to do, and not having any prior experience with this artistic field, I was nervous about everything. It was again, Sue and Andrew, and other TSR members who encouraged and endorsed me to start this project. First several weeks before I started my workshop, I could hear how other people were starting their rudimentary works and how they set up their plans and follow that schedule systematically.
I first used Guitar Pro 4 for composition. I picked the simple action potential piece and assigned different sounds to different ions for conversion. It was basic, but it was successful to me. I presented it during the workshop, and people gave positive and helpful comments about the project. I was really happy, not only because I composed my first music piece, but also because I found my hidden talent in music.
Next step was transforming one whole electric signal into music. Among the myriads of data I received from Professor Bruce MacIver, I chose an EEG signal from a sleeping rat. Its dynamic movement of ions was exciting enough to be converted into house music. It was really inspiring. The first piece is based on a rapid paced bass and snares to represent the resting membrane potential. Following comparatively quiet prelude is the breakout of sodium and chloride channels, which is depicted by burst of different kinds of sounds. I put a little glitch in the transition part, intended to give a hint to audiences about the characteristic of original signal.
Second Piece
Markedly more advanced and dynamic than the first piece, the second piece was made out from EEG of an exercising rat, whose enthusiastic movement seems a lot more grandiose. The most complex and, to some, the most solemn of the three pieces, this song captures the energetic and forceful nature of action potential signals. Considering different experimental context of this EEG, sets of sounds used to present different ions are completely different from those used in the first piece. This song is special to me since it is my second piece of composition, and it made me to openly communicate with other people about my composition.
Aptly composed, second piece is considerably more complex but is equally pleasant, when compared to its predecessor. The basic notes are a combination of electric snare, bass, and twisted glitch—including the airplane sound. The starting notes are powerful and sublime, but simple, the solemn feeling of EEG signals are conveyed throughout the whole piece. As the actual action potential starts, a lighter but richer combination of sounds dominates the overall tone. Personally, this is the most complex and emotionally satisfying of the three pieces.
Third Piece, and a Video
The third piece is made from a signal of rat with severe epilepsy. It was the most ambitious one among the three, since the signal is gained from experiment during my undergraduate honors project. However, due to technical difficulties and time limit, the third piece is left incomplete. Song composition programs that I am able to use are all intended to make a single-paced piece. However, epilepsy is characterized by its abrupt bursting of seizure movements, which break out in much faster speed than the normal signal. Unfortunately, I was not able to learn more complicated program, since the learning curve jumped to a professional level. I made a prelude, which is equally pleasant and complex as the other pieces, and I hope to finish this one, whenever possible.
During the whole project, one of the comments that I have heard from colleagues is the presence of visual assistances. I picked the Pikachu the electric rat, Nintendo’s main cartoon character, for my video piece. In the video, I tried to make Pikachu walk, sleep, jump and run, corresponding to the EEG signal. The music video is humorous, entertaining, but also is intended to show audiences what is going on throughout the music itself.
Conclusion
This project was meaningful to me in many ways. It was my first opportunity to approach science in an unusual way and produce something completely novel. This was one of my greatest challenges at Stanford, since I have never done this kind of project that involves two completely different areas. However, I ended up producing a lot of video and audio pieces, which did turned out to interest some people and drive them into what I have been doing. I was so glad every time I present my works to people, and I was really happy when I presented my work to people in the exhibition.
From next quarter, I am starting Ph.D program in Bioengineering here at Stanford. Having flourished from three quarters of most challenging but enjoyable experience, I am confident that I have become a different person and at least have achieved something. I am happy from what I have done, and I believe I can successfully finish the training starting from next year.