Computing
Arts (CS 498) is part of a new breed of computer science courses that
brings together students from both sides of Green Street, an
imaginary line that separates the engineering from the rest of
campus. The course is part of a new Department of Computer Science
initiative, the
Siebel Center Cultural Computing Program (CCP). The program is
aimed at enhancing collaborations between the departments of
computer science and those within the arts and humanities areas.
Computing Arts II
The Intelligent Performance Space will be offered spring,
2005.
Guy Garnett is the instructor for CS 498 and is also the
co-director, with
Roy Campbell, of CCP. He is an associate professor in the School
of Music and co-leader of the University of Illinois’
Seedbed Initiative for
Transdomain Creativity. His interests include composition,
interactive computer performance, music theory, analysis,
aesthetics, and their confluences. He is currently working on a “cyberopera”,
The Death of Virgil, that incorporates singers and instrumentalists
along with technology in a meditation on life, art, and love based
on the novel by Hermann Broch.
Is this the first time the course has been offered?
I taught a previous course for music students and one for music/art
students. It was difficult because there was a lot of technical
stuff they have to get through first. In this course, I was pretty
impressed by the CS students’ projects. They did in two weeks what
we normally did in 4-6 weeks. Now we have to see what kind of art
they do – they have some good ideas. There are ten computer science
and five music students. At the end of the semester students will be
presenting their final projects on and around the video wall in the
Siebel Center lobby.
Is there a difference teaching this class versus one for just
music students?
We are moving through the material faster. The material we have
covered in these three weeks was spread over the entire semester in
the previous two music courses. We will be covering video processing
and gesture tracking which I have usually covered at the end of the
semester if there was time. For this class we are doing everything
we can right up front, and cover a little bit on each topic and then
turn it over to projects and see what we get. The idea is to do a
performance with video, music, gesture tracking and coordinating it
with multiple computers, projectors and video screens and who knows
what else – we will see.
Is there a need for this type of class in computer science?
I think it serves a lot of different purposes. As part of the
Cultural Computing Program, it is a way for computer scientists to
look more broadly at the culture around them and where they can
serve a real need. That might be in making computers faster or
networking better, but I think a larger and larger portion of
resources will be devoted toward things that are considered leisure
pursuits, like computer gaming – it’s a huge industry. At first it
was for fun but now you see the Army developing computer games
mainly as a means for training. All of these things interrelate. So
we are working with the arts in general and hoping to branch out
from the performing arts to other forms of art and culture.
We are using the performing arts to get the students excited about
what the possibilities are, and to give them practical, real-world
problems to solve. We also want to get music and CS students to
communicate better – open up the community more, see how different
people work. You can go through the CS program inside this building
and hardly ever meet someone who speaks a different academic
language. We are encouraging them to work together. That is the kind
of thing that will happen when they go out to get a job.
It is expanding their experiences into different arenas and
hopefully providing some new challenges. The kinds of projects you
can do when you look at it in this way are pretty open ended. Some
of the same students who were really excited about Roy Campbell’s
gaming project from last semester are closely associated with this
course. The students got a great experience and wanted to do more.
They learned an awful lot not just about nuts and bolts of CS but
how to work in a team, how to make things happen on schedule. I
think this practical experience will help students out when they are
looking for a job.
Is this the jumping off point for other classes?
Hopefully. This is kind of a test to see how it works. There are
lots of ideas – doing something with more of a gaming thrust to it,
but there are lots of possibilities to explore. More humanities,
library access tools, digital culture, etc. We hope to be bringing
in guest speakers next spring when we are fully up and running.
This is our first hub site under the seedbed initiative. We plan to
set up several hub sites around campus that will keep in close
contact with each other but will also attract new people into places
like Siebel Center, bringing in people from music or art to work
with people in CS. Another hub site might be in humanities with
people from CS working there. The initiative is a means to get more
interchange across campus and break down some traditional barriers.
We see this as a way to tackle very different kinds of projects than
those found on campus today.
What will be in the Cultural Computing Lab?
The lab is dedicated to research and advanced education in digital
media, new interfaces, and new tools for the arts and humanities.
Jonathan Fineberg, a professor of art history, is also working on
projects in the lab. The lab has an Avid video editing suite, a
motion tracking system, and several large displays. We are seeking
grants for more equipment. One of the things we are interested in,
is developing new ways to perform and to re-conceive the nature of
performance. We are very interested in developing technology to
support real-time control of a variety of media (sound, video,
light) through non-traditional interfaces, interfaces that are more
akin to musical instruments than to a mouse and keyboard.
There seems to be a lot of HCI activity. How does this tie into
it?
One issue you are always faced with is managing complexity. Trying
to come up with what I think is an interesting offshoot of this is,
can we design a set of gestures that people can use easily, remember
easily and yet can have a powerful effect on the computer? Can we
come up with a paradigm for how one moves around with their body,
track that and use that instead of a mouse and keyboard for
interfacing a computer?
Right now my thrust is defining those gestures and using them for
performance art. That has other extendable aspects too. I have been
talking with Professor Brian Bailey in CS who does a lot of
interesting work in interface design. We are considering a number of
ways to collaborate. One idea we’ve discussed is to find a way to
develop new interfaces for writing music scores following some of
his story boarding techniques.
How did you get interested in this?
I was getting my doctorate in music at Columbia University. One of
the requirements was to take a couple of courses in electronic
music. This was the 80s and they had an analog tape studio, where
you recorded onto magnetic tape, and then used a razor and a ruler
to cut and splice the tape into a piece of music. We had to take
courses there and I started getting a little frustrated with the
cumbersomeness of the working methods. I started looking around a
bit and realized that some people were using computers to do some of
this stuff. You could synthesize sounds by computer and then control
them. So I went down the hall and started using main frame computers
writing code in Fortran.
I still do a lot of acoustic music, or pieces where acoustic
instruments are combined with electronics. My most recent piece is a
“cyberopera” for spring 2006. It is operatic because people are on
stage singing, but their voices are also getting processed by
computers. The opera will use free-form gestures to control video
playback; possibly synchronizing the video to conductor. Usually in
movies the conductor watches the movie and synchronizes the music to
it. What I am trying to do is have the video follow the conductor.
We will use the tracking to smoothly manipulate the video so that it
synchronizes with the tracked gestures.
About CS 498 Computing Arts
The advanced-level course covers computing principals, tools, and
applications focusing on the performance arts with emphasis on
music, video and gesture tracking. Students learn to program and
debug a real-time audio-video processing system, and program in a
robust, fault-tolerant manner. They will be creating an Intelligent
Performance Space to be used for interactive multimedia performance
events in Siebel Center.
Computing Arts II: The Intelligent
Performance Space, Spring 2005
Professors Campbell and Garnett
CS 498 GG, Fridays, 3:00-5:00, Siebel Center
Labs will take place in the Siebel Center and elsewhere on campus.
Prerequisite: Computer Arts I, or permission of instructor
This is a project class to design and build an intelligent space
that will support and respond to the performing arts and extend
their capabilities. This includes using location awareness,
tracking, and other sensors to coordinate people and their
activities and to control video, lighting, scenography, and sound.
We will design and build the computing infrastructure for the future
of performance including: developing CORBA-based tools for
communication among diverse platforms and applications, developing
unified gesture and control paradigms for all the performing arts
(including dance, music, theater, gaming, interactive arts, etc.),
developing artist-oriented user tools for real-world conditions.
We are looking for students with skill or interest in music/audio,
dance, theater, video/graphics, industrial/fine arts design,
networking, HCI, real-time systems, sensors, control, multimedia,
etc.
Interested students can contact Prof. Campbell or Garnett for more
information.
About Siebel Center Cultural
Computing Program
With significant amounts of multimedia equipment that includes a
video wall, plasma displays, touch panels, Siebel Center promotes a
view of digital technology being seamlessly integrated in the daily
fabric of the department. The program will allow faculty from the
arts and humanities to spend time in Siebel Center, use its
infrastructure in support of their research, and collaborate with CS
faculty in joint research and education.
About the Seedbed Initiative for
Transdomain Creativity: Exploring Human Experience through Art and
Technology
This cross-campus, cross-disciplinary initiative provides a
comprehensive and flexible framework for the exploration of
technology-enhanced art and aesthetic experience. It will not only
allow creation of new work and new forms of work in new ways, but
also serve to elevate the discourse of ideas surrounding the
creation and experience of art. |
