Interview with Ranjith Hegde, Developer of MayaFlux.
- Jul 9
- 25 min read
Updated: Jul 29
Ranjith Hegde traces his journey from late-beginning violinist to creating production audio for Metro VR Awakening and developing MayaFlux: a radical open-source framework merging audio, visuals, and control without traditional interfaces. This interview by Anugrah, a Research Associate at TheISRO, reveals
how gesture-tracking, live coding, and embracing computational indeterminacy reshaped his artistic practice across Sonology, corporate game audio, and interdisciplinary performance.

Anugrah: First off, what initially hooked you on to sound in your early years, starting from studying violin and then moving into electroacoustic experimentation and programming? What specifically sparked that transition to working with code?
Ranjith: Compared to most people who study violin, it came very late in my life. Regardless of whether you study Indian classical, European classical, or whatever else, the idea is usually that you start younger so that your body develops with the instrument. When you're older, you have both physical and psychological baggage that you need to undo if you want to begin seriously with an instrument. If the goal is simply to have fun with the instrument or play with friends, that doesn't matter as much. But if you want to take it professionally, those things become important. I began with sound almost completely by accident. I was put into the marching band when I was in boarding school, and they made me learn trumpet. The school had these small tubes with notations written down, and there were about twelve different instruments in the marching band. I was fascinated by the idea that I could read something and then play it back, so I just learnt all the instruments they had to offer. Violin came much later. I was around sixteen, maybe sixteen and a half, when I decided to see whether violin could be interesting.
There was no special reason. I just picked it up, tried it for a while, and felt that this was something I could spend a lot of time with. It did not feel like something that would be exhausted once I had played the tunes I already knew. It felt like something that could keep me invested for a very long time. After college, I went to Chennai to study music directly. I joined a conservatory there that had just started, and I was in its second year. But after a year, I realised it was not for me, because it was focused primarily on the film industry rather than on music exploration as I imagined it. So I found tutors around Chennai instead. My main instructor was a German musician called Holger Jetter, and we have stayed in touch for many years. He taught me not only sound, but also aesthetics, philosophy, and art more broadly. Coding was not part of my life for a long time. When I went to the Netherlands to explore what electricity and electronics could do for my artistic practice, I found the tools on the computing side interesting, but in the beginning I was more drawn to the electrical side: circuit boards, oscillators, filters, and things like that. You could build the circuits you wanted and create something that you simply could not produce acoustically. That was very fascinating. Coding happened differently. I thought, if I now move into computers, what else can they do? To my surprise, the move from acoustic to electric had felt much more liberating than the move from electric to electronic. The same liberation was not there, because the tools I encountered in electronic music, or at least the ones the community at large tended to use, felt like very direct mimics of electric or acoustic systems. That is when I thought that maybe, if I learnt programming, I could push beyond that. That is how I ended up in computing. It was almost completely accidental.
A: So when gesture tracking and audio code first made sounds feel truly alive and responsive, what was the biggest surprise you encountered during that process?
R: This was part of the journey I mentioned. In Chennai, I was part of a group called Basement 21, and the primary idea there was that we explored together regardless of what discipline we came from. So you might have a painter working with a sound artist, or a sound artist working with a dancer or movement artist. The effort was always to make sure that, if I entered the space, I was not entering simply as a sound person. I had ideas in sound because that was my training, but the point was not to remain inside the things I already knew. I wanted to learn from the other person. If I was working with a dancer, for example, they understood space in a way I could never understand purely through sound. For me, space in sound has often been about placement. There is a lot of research that makes this more nuanced, of course, but broadly it is still about placement. For a dancer, space can be about equilibrium, spatial tension, weight, relation, and many other things. Learning how a dancer thinks, to the best of my ability, or learning how a painter thinks about composition and placement, opened up ways of thought that I could not reach through violin alone. When I began exploring computation, I found that limiting in one sense. I was asking myself how to read those kinds of insights computationally. If I had learnt something from watching a dancer, how could I translate that into electrical or electronic systems? That was tricky. At the same time, the cameras available for tracking did not initially seem very interesting, because they only did, badly, what my eyes could already do. They did not offer new information. So I started asking what I could do that I could not get before. My eyesight already tells me where a person is in space. If a camera only reproduces that, it is not yet very useful to me unless my eyesight is poor. Then I encountered tools like Kinect and Leap Motion. They build skeletal structures of the body and give you data based on that. That interested me more. Even then, I thought, this still gives me position, which I can already perceive. It was not enough yet, but it was the right direction, because now I had data my eyes could not measure precisely.
From there, I started asking what I could do that vision alone could not do. Suppose a dancer I work with has certain tendencies. They tilt their head a lot, or they favour one corner of the space. Computationally, I can track that. I can set a timer and ask how often they go there, or how long they remain there. Then I can connect that with another parameter, like how often they turn their head. Computation allows me to combine those values into a new relationship, something like a two-dimensional value or an emergent pattern, and work artistically with a relationship that did not exist for me before. If that relationship only exists in my head, it is not very interesting. But if I can record it and use that data to generate sound, visuals, or some other material, then it becomes powerful. That was the important aspect of gesture tracking for me. It made many parts of a person's behaviour, and many aspects of their tendencies, available for artistic interpretation in ways that are very difficult, or practically impossible, to achieve physically.
A: It is very interesting how you seem to embrace complexity in order to open up possibilities
for your craft and practice. Moving to one of your projects, Metro VR, how did creating movement, sound effects, and music for that differ from your earlier experiences in improvisation and other group work?
R: It was an entirely different experience because, for the first time, I was in the corporate world and had a career role as an artist within production. That was not part of my training. I had done performances for corporate contexts before, but those were always limited in time. You have a contract, one night, or one event. It is not the same as being part of a production team. In a production environment, especially a corporate one, the group is not just you and the people you are artistically collaborating with. It also includes the people who pay for it, the people who fund it, the marketing team, and many others.
So understanding that the task in front of me already had its limitations and possibilities mapped out was the first thing I had to come to terms with. At the same time, it was fascinating. I had always felt games were an interesting area to explore, because they contain many aspects that other art forms do not hold together at once. With painting, you have time in the sense that you can spend as long as you like with the work, but the work itself does not change over time. Your interpretation changes. In dance or movement, you do not have the same freedom to freeze time and explore a moment in depth. With sound, too, you can place a speaker in space, but you cannot easily move through that space in the same way unless a specific technology allows for it. Games offer all of this together. You can spend as much time as you want with one aspect, combine it with others, and move within a designed environment that still allows for exploration. It can be both curated and open-ended at the same time, and that is difficult in other domains or domain combinations. When I was offered the chance to work on a game, I did not know at first which game it would be. At that point I was working with physical modelling, and the company thought what I was doing there might be useful. When I joined, I realised it was a VR game, and they wanted both physical modelling and spatial thinking because, in VR, immersion is fundamental. Without immersion, it becomes difficult to justify the medium. If all you are doing is what a normal game already does, but with something heavier on your head, then why would anyone care? My focus became: which sounds are necessary, and how can they remain convincing within the limitations of the platform? VR headsets have a very small computational budget. In fact, many mobile phones have more powerful CPUs than VR headsets. So optimisation becomes crucial.
In an ideal case you might want hundreds of sounds at once to make an underground environment feel alive: train tracks screeching, people speaking in the background, fragments of conversation, ambient textures, and so on. But you cannot have all of that because of memory and processing limits. So a large part of the work was figuring out which sounds could be reduced while still remaining recognisable, and how many could exist at one point in time. Alongside that, there were the more concrete interaction questions: how do footsteps become more interesting, how should objects sound when picked up, and so on. In VR, people will pick up objects, so those tactile interactions matter. On the product side, you are constantly trying to understand the whole experience. You keep playing the game over and over in different environments, with every new feature, to understand what effect it has. On the engineering side, we also had to work a great deal, because Unreal Engine's built-in audio tools are only serviceable. The assumption is often that people will import middleware and handle the audio there, so the internal audio pipeline is not especially strong. We had to write many new systems to make the results we wanted possible. One of the things we implemented was dynamic music, where every player action that affected the game also changed the music in some way. I do not mean swapping in entirely new compositions. I mean adding new layers to the music based on what was happening in the game. Because the game is not deterministic and we do not know what the player will do next, you need many systems sitting there, ready, listening and polling for possible actions, so that they can trigger the appropriate musical or sonic events.
So it was very different. I had to think much more directly as an engineer and only indirectly as an artist, though of course I was still contributing to the larger artistic world of the game. It was educational, sometimes frustrating, but overall very worthwhile.
A: And in another project, MayaFlux, where audio and visuals mix seamlessly without traditional cables, knobs, and similar interfaces, how did that free up, or perhaps constrain,
the sounds you were able to create?
R: It is interesting because MayaFlux itself was almost accidental. What exists now is what I have been working towards for about fifteen years, but the specific path that led to it was a fluke, to be honest. I was teaching a class at a school, and a few students wanted to explore audio beyond what we were already teaching. They knew my practice and wanted to go further. So I was teaching them OpenFrameworks outside the course. OpenFrameworks is another toolkit, similar in some ways to what I am building, but at that point it was already becoming dated because many of the tools it relied on were deprecated or disappearing. While teaching it, the students kept saying that audio was too difficult.
Graphics felt easier because tools like p5.js had made them accessible, whereas do-it-yourself audio still felt hard. So I thought, let me do a live stream over a weekend and build an entire audio engine from scratch in one day, just so people can see that if I can build a whole engine in one day, then students can spend a few months exploring what they want. It is not that audio is inherently too difficult. It is just that it has not been explored and taught in the same way.
Building that engine in one day gave me the idea that C++ had improved quite a lot in terms of usability compared to when I first learnt it. That is when I thought, let me write something. But once I started, I did not write anything simple for a couple of months, because I realised this was an opportunity to build everything I had ever wanted from computing. That is where the current effort really began. One of the core ideas was to explore what the digital truly is. This is something I discussed in the C++ conference talk I gave. One thing I said there, which was not especially well received because it is somewhat inflammatory, is that computing as we teach it is primarily vocational. You are taught to use tools. You are rarely taught to question them. You are not taught to think about computing itself.
If you look at physics, you are not simply taught to use a machine. You are taught to consider the properties of physical systems, to question them, and to experiment. There are many recognised avenues of exploration. Computing is generally not taught that way. Usually someone gives you a brief and asks you to build a tool. So there is very little widespread inquiry into what computing could do that we do not already have. That also means we have not spent enough time engaging with what the digital is aesthetically, interactively, or conceptually. If you think about a piano and ask how to make it more visual, you quickly run into the instrument's physical reality. The more you open it up or modify it visually, the less it remains sonically what it was. That limitation is real. In some cases it is interesting, but it is still a limitation.
In computing, by contrast, a video is a large number of pixels updated at a rate that makes our eyes think there is motion. Sound is a stream of pulses sent to a speaker so that the air vibrates and we hear it. All of these things can be generated at the same time. One function can generate both sound and visuals. So the reason we separate them is primarily because we are mimicking the camera, the microphone, and the screen. We are not thinking about what the digital can be in itself.
We have spent enormous effort recreating the limitations of physical devices in digital environments without asking what digital systems can do beyond those limitations. If a physical camera cannot do something, there is a good chance your camera app cannot either, not because the computer is limited, but because nobody has thought beyond the physical model. For me, that was the opportunity: if we stop recreating what we know and instead begin from zero, what becomes possible? In order to get sound and image, yes, eventually I need to send twenty-four frames to the screen and forty-eight thousand samples to the audio card, or something in that range. But that is a constraint of output. That decision should happen at the end, when something goes to the screen or the speaker. It does not need to structure the beginning of creation.
If you look at something like Processing or p5.js, they have a draw call, a function that runs at a frame rate. The problem is that the needs of the screen become the user's problem. But the user should not be responsible for providing twenty-four frames every second. The user should be able to provide what they want, and the computer should present whatever is appropriate. That was one of the primary things I wanted to explore. If I am not forcing computing to look like what its outputs expect, what becomes possible?
At the end you decide which subsystem runs a particular function. If the video subsystem runs it, maybe the data needs to be interpreted in a form the video system understands. But that decision belongs to the presentation layer, not the artistic layer, not the creation side. That is where my focus is at the moment: you write a simple function, and then you decide later what to do with it.
A: Across your projects and the experience you have gained, could you point to one or two techniques, perhaps physical modelling, gesture-to-sound work, or vowel extraction, that have most reshaped the way you listen to the world around you, and why they had that impact on your listening practice?
R: What I am about to say is really an evolution of something I had observed since I was younger, but when I was young, I did not have the tools to articulate it, or even to know that it was worth investigating. Over the last five or ten years I have become able to put it into words, and what struck me most is that everything around us is not linear. Unless we are in controlled environments, almost nothing is linear. Things are always asynchronous. More importantly, things do not operate through the assumption that everything has a linear interaction. But we build the world around us expecting one hundred per cent controllable, fully automated linearity, and that causes many failures.
I explored sound in the same way. If I stop worrying about the linearity of a particular musical tradition, I see that even what we think of now as a fixed musical moment within traditional music is actually part of a continuum developed over centuries. We isolate it and study it linearly, but if you allow it to remain what it is, you absorb much more. You open up new ways of conversation and interaction. I had the same experience while working with dancers. If I am too bothered by the idea that I am entering as a sound artist, then I am not doing justice to the collaboration. Dancers think through things that I cannot think through with my instrument, or at least not easily. If I am open to that, if I am open to the possibility that what I know is only a subset of things and not something comprehensive, then new ways of thinking open up. That also applies to identity. If my identity is not simply, "I come from this, therefore I am this", then I can explore more. I stop saying, "This is not my domain" or "This is not my concern".
So overall, the important thing for me has been an appreciation not just of asynchrony, but of what we might call indeterminacy. Truly believing that things are random, and that we form patterns because as humans we are good at forming them. Those patterns make us comfortable, but they are not necessarily really there. If you realise that, so much of the world opens up. That has been the one constant throughout everything I have done. Accepting and appreciating indeterminacy, noise, randomness, whatever term one uses, opens the world in a way that expecting linearity never can.
That idea carried me through different projects, different mindsets, everything I explored. Once you open yourself to it, you make connections that are not otherwise possible. You may suddenly feel that what you are doing as a programmer smells like something you ate yesterday, or resembles some childhood experience. There are too many connections to make if you see everything as part of a continuum rather than as isolated categories. Those connections may only be temporary. Tomorrow morning you may think the idea was foolish, and that is fine. But being open to the possibility that things are connected, whether temporarily or otherwise, is central to how I work, how I create, and how I think about tools.
Anugrah: In terms of creating sound, crafting soundscapes, sound code, and electroacoustic work, how has that practice evolved for you through your experiences in Chennai, Den Haag, and TheISRO? What is the philosophy behind the way you approach making sound?
Ranjith: I think most contemporary artists go through a similar journey. You begin with what surrounds you. Traditional music, popular music, whatever is around you, you absorb it, and your
aesthetics are shaped by that. If you are thinking about aesthetics at all, they are usually just variations of what is already in front of you. What causes people to break out of that is usually an encounter with something that does not fit. It
need not even be sound. You might watch a film that has nothing to do with anything you know, and that can be enough. Once there is an injection of something truly different, you may begin to
investigate. That is what happened to me.
I was deeply invested in traditional art, both Indian and European, not because I was a traditionalist, but because that was what I grew up with. It was simply what existed in front of me. But the moment something else entered, I realised that scales, modes, harmonies, and all the things we know are only part of the picture. Once you have the courage to explore the idea that the known world is not all there is, you enter a much larger space. For me, that happened through free jazz, especially as a violinist learning to appreciate free improvisation and, in particular, the aesthetics of American jazz from the forties to the sixties. That opened me up a great deal. It was about raw energy, about pushing sound out of the instrument and shaping it as you go. You are not aiming for a fixed rhythm, melody, or harmonic structure. Those things might arise, and you may engage with them, but you are not restricted to them.
My exploration of different kinds of sound began there, listening to free jazz. Very quickly that extended into working with dancers. I noticed that, for dancers, dynamics do not just mean louder or softer. They can also mean speed, density, how many people are on stage, how often a particular idea changes. Once they were thinking like that, I began asking what happens to my
sound if there are two people making the same gesture at different points on the stage, or if they are moving while I respond. What happens if I shape sound according to the decisions a dancer is making? At that point it became less about the violin itself and more about using any and every object around me to make sound, and also using the violin in ways that were not traditionally accepted. I had built up a large arsenal of extended techniques. That is also why I eventually got the violin I use now. It has seven strings, it is fully electric, and the body is effectively absent or skeletal rather than traditionally resonant. That allowed me to explore things like multiphonics more
directly. Multiphonics means producing more than one sound at the same time from a single sounding body or instrument. You do that by changing pressure and bringing out different harmonics and
vibration behaviours. The technical details vary, but the point is that I became interested in immediacy: if I feel something, can I push that sound through me, and does the instrument allow
me to do that? That became my practice for a long time, and it is also what led me into electroacoustics. For the first time I felt my output need not be limited by what my hands, feet, or body could physically facilitate. I wanted to know what might happen if there were many more things than I could physically do, but which could still exist through speakers, circuits, oscillators, and so on.
So the evolution of creating sound for me has really been about the quality of sound, what it sounds like, how it feels in different positions in space, and how it works when I am creating with different people. I have the same way of thinking when I work with visuals or other artistic forms. It all began from the idea that the injection of new ideas, even strange or temporary ones, can open new avenues of thought. It happened largely because I was listening to jazz, but not the more formulaic jazz of the twenties and thirties. It was the jazz of the fifties and sixties, people like Ornette Coleman and John Coltrane, where there is much more emphasis on raw sound, open exploration, and political force. Their sound represented aggression, anger, empathy, all of it, and not in a cryptic way. Much music carries emotion in a coded form. If you know the aesthetic, you can decode the sadness or tension in a melody. But in a lot of that jazz, the emotional force was much more direct. If someone was angry, the sound was louder, harsher, more forceful. Meaning came not through decoding structure but through the immediacy of expression.
A: You do move away from a traditional background and foundation, but are there elements of that past, or that practice, which have persisted in your current work with MayaFlux or other projects? For example, have aspects of your education in violin, trumpet, or other instruments remained in your present practice, perhaps transformed in some way?
R: It depends on what we mean by "traditional". If we mean the rules that come with tradition, then not really. But if we mean aesthetics, then absolutely. One thing I particularly enjoyed as a
violin player, especially within European classical music, was the fugue. A fugue is made of independent voices coming together. In some cases they align, in other cases they do not. They do not need to have the same pauses, the same lengths, or the same notes at every point. They are independent, and yet they form a structure together. That pattern exists in a lot of other music too, including free jazz and contemporary free improvisation.
The idea that many independent things can coexist, sometimes meeting and sometimes not, is something I explore everywhere. It is not directly part of my programming process, because
development itself is not real-time in the same sense. You build something, compile it, observe the effect, and then build further. So there is no literal fugue in programming. One can always force an interpretation, but it is not truly the same.
Still, fugue is one of the things from traditional practice that remains important to me. Certain rhythmic ideas remain too. By that I mean structures placed in chronological order, where points
of meeting recur, something like what in English we call a downbeat, where one cycle and the next meet. Even if the system is not strictly cyclical, the idea that there are certain structures that are, to some extent, non-negotiable still matters. My interest is often in how to work around those
structures rather than against them.
A: In your experience of crafting visuals and sound in live coding, when code suddenly sparks sound and visuals around you, how does that experience feel in the moment, both while writing
the code and while seeing the audience respond?
R: In the beginning, it was terrifying. I realised that I was not only at the mercy of what I could do, but also at the mercy of the quality of my tools. A bad violin still functions; it just sounds bad. It does not usually collapse in front of you. The same is true of most physical instruments. They may not allow certain possibilities, but they continue to function. That is not true of code. If you are experimenting with tools you are building yourself, everything can break at any moment. For a long time that was very frightening, not only because a performance might go badly, but because it could break the flow entirely. So much of performance is about not falling into panic, about staying inside the moment and giving something meaningful to the audience. One of the powerful aspects of performance is allowing vulnerability and fragility to be visible, allowing the audience to see the risks you are taking. But that only works when the risks belong to the artistic act. It is less interesting if the risk is simply that the whole thing might crash because the system is unstable.
So, at first, working with computation meant living with the sense that everything might fail and I might have to begin again from nothing. That feeling still exists to some extent, but I have learnt to work with it, and it has shaped my aesthetic. There was a period in American experimental music and composition, roughly from the fifties to the eighties, where some composers became interested in not polishing things too much, in presenting them more as they are. That idea stayed with me. Because computers can break, I became increasingly pushed towards getting ideas out as quickly as possible before the system collapsed, and towards playing with what was in front of me rather than refining it into something conventionally neat or enjoyable. That left a mark on my aesthetic. My live coding has become rough and raw, not only because that is my preference, but also because it emerged from the fear that everything might collapse at any moment.
The responsive part is also difficult. Many tools claim to be responsive, but they usually are not, at least not in the way an instrumentalist understands responsiveness. If I am used to playing hundreds of notes quickly on the violin, I cannot get that same immediacy from a computer, no matter how fast the interface claims to be. So I had to work around that. I had to accept that what I want might happen, but not immediately. That led me towards ideas where I plan for things to emerge rather than happen the instant I press something. That is also where systems thinking becomes important. You are no longer thinking purely in terms of direct reaction. You are thinking in structures and emergence.
A literal one-to-one mapping, like moving your hand up and having the sound go up, is interesting only if you have never done it before. After five minutes it becomes boring. But if moving your hand up makes the sound go sideways for half an hour and then come back, suddenly you have something to investigate. Then you have something to question. So I became interested in working with systems where new interactions emerge over time. Some of the performances I have done were based on exactly that. I write something, wait for it to unfold, and while it is unfolding I prepare the next thing. It is more like baking in that sense. You layer things and build on what already exists. It is less about immediate response and more about the shaping of emergence.
A: As you move through this intersection of sound, coding, and performance, are there any figures or moments, contemporary or historical, that continue to stay with you? You have already mentioned jazz and related movements, but are there other people, movements, or projects that have shaped you, or that you are watching closely today?
R: Apart from people in history, I have been lucky that many of the contemporary people who caught my attention are people I have actually met, and in some cases worked around, if not directly with. My violin teacher had enormous impact on me, partly because he himself was impressive, but also because he was radically different from my surroundings. In the beginning, that difference was important. He was not local, and that meant that almost every idea he had felt foreign to me, whether good or bad. It was not only sound. It was what he ate, when he ate, how he thought, what made him laugh, how he taught. Every aspect of life felt shaped by a different world. That helped me a lot. He was also deeply philosophical and political, so our violin lessons were often less about violin practice and more about discussing ideas, having coffee, arguing about something interesting or pointless. It was a form of exploration.
Another artist who had a tremendous impact on me is Padmini Chettur, a choreographer based in Chennai. Her work has influenced a great deal of dance in the country, especially contemporary dance. Her philosophy of extreme minimalism, of exploring form, equilibrium in space, deliberation, mass movement, entry and exit points, and the effect of very slow or even excruciatingly slow action on an audience, had a major effect on me. That way of thinking allowed attention to settle on different kinds of experience.
The community I was part of also mattered greatly. It was not only that there were artists from different disciplines, but that we explored different performance structures. In one performance, the opening act involved parkour artists scaling a building in relation to the sound we were making. In another, we had a quartet of musicians moving around the city with headphones, listening to each other while playing. We were experimenting with every aspect of presentation: how music is presented, how dance is presented, where something happens, how long it lasts. Some pieces lasted two or three hours. Some lasted fifteen seconds.
When I was in Delhi exploring on my own, I was also influenced by films and by the confidence of certain artists in pursuing exactly what they wanted. One thing that increasingly frustrated me about Delhi was a social obsession with having a "nice evening" and pursuing only pleasant experiences. It felt to me as though that narrowed the world to a very tiny part of human possibility. So some of us began making performances that were funny, cheeky, unpleasant, obnoxious, or difficult to sit through, precisely because anger, nausea, discomfort, and disorientation are just as real and valid as pleasure. We did not want everything to be filtered into neat categories of pleasantness. Surprise, disorder, and friction are as valuable as comfort.
Another hugely important figure for me is the composer Richard Barrett (research supervisor and mentor). His work, with its complexity, political thinking, and global range, influenced me very strongly. I was very fortunate that he eventually became my research supervisor in the Netherlands. To have someone I respected so much become a supervisor had a major effect on me.
Maarten Visser has also been important to me, both as a partial mentor and a friend. In terms of current work, one game designer who matters to me a great deal is Hideo Kojima. That is of course not an obscure name, but his work is important to me because he keeps rethinking what interaction in games can be. His recent games, especially Death Stranding and its continuation, are interesting because they are not primarily about domination or destruction. They are about nurturing a world, building a sense of asynchronous community, and interacting with the traces other people leave behind. You do not necessarily see other players directly, but you experience the effects of their labour. Someone built part of a bridge, someone left something behind, and you enter into relation with that. It is a way of being together asynchronously. That was something many of us also felt during lockdown, though usually through messaging platforms. His work explores that in game form. It is also full of adventure, surrealism, life, awkwardness, and, yes, some deliberate cringe. But it feels honest. Things are there not because they make the product marketable, but because they belong to the work. That kind of honesty is extremely rare in mainstream media.
A: Are you currently chasing any new ideas or projects? Is there anything people can look forward
to in your present work?
R: At this point it is difficult to separate one project from another, because almost everything I am doing goes directly into MayaFlux. I am primarily working towards rethinking many aspects of computation through it. With every layer I add, the challenge remains the same: I do not want to do things simply because that is how people have thought about them before. I would rather rethink them from the beginning. If that fails, that is fine. I will at least have learnt something. At the moment I am working on visual layers for my artworks, where one might place a window, a graph, a slide, or something similar. But I am also trying to build those differently, because I personally dislike calling them "UI". UI literally means user interaction. The handle on my chair is user interaction because it lets me change the height. When I am cooking and add ingredients, that too is interaction.
The fact that interaction has been reduced, in computing education, to cables and dials is extraordinary to me. Everywhere one studies UI, it is treated that way. I am trying to build these elements differently, as just another kind of signal. They should be responsive, but not authoritative. That means, for example, that a slider can itself be moved from elsewhere. The slider can move, or can be transformed into something else, and not merely through some abstract conversion where a floating-point value changes another system. The slider itself can go to the GPU and become something else visually. Every aspect of computing can be inspected, altered, destroyed, and recreated.
So what people call widgets or interface layers are, in the end, just signals. Numbers. The same number can become sound, visual, control, text, or something else. The computer knows bits. Everything else is human interpretation. That also means that whenever something seems impossible, often it is only because we have not done it yet, not because the tool is truly limited. I have also been building a mesh layer, but again I want to approach it differently. A mesh is, fundamentally, just pointers and series of data. So you should be able to create a mesh from sound, from a dictionary, from a phone directory, from anywhere. It should not matter where the data comes from. They are all ultimately subsets of numbers and bits.
If you treat them that way, then you can create horrible shapes, beautiful shapes, unexpected shapes, without first deciding that they must fit into a fixed concept such as a vertex buffer of a known type. That decision can happen later, during upload or presentation, and even then it does not need to dominate the whole process. I have a workshop coming up in two weeks that is also based on this idea. I will be teaching systems that are already functional, and the whole point for the students will be to break the system completely. Then, once they break it, to break it differently again. Instead of learning one layer at a time in a controlled way, the idea is to learn through destabilising the system and understanding what that reveals.
One of the main goals there is to become comfortable with not knowing immediately, and comfortable with not producing something recognisably impressive straight away. Give it time. Let it evolve. Then you may arrive at something more valuable than an immediate result where you press a button and something amazing happens.



Comments