Planet Music

Planet Music is a sound visualizer on the moon! The current version has a set playlist of 6 songs that can be activated with the media player buttons. The space scene reacts to the sound using a fast Fourier transform (FFT). Here is a sample video:

Try it for yourself here (Note: please allow ~20 seconds to load. Currently not optimized for mobile)

See the code here

Inspiration, Credit, and Copyrights:

Planet music was inspired by the game Hello Kitty Dance Party, a game where you can play songs, tap along to the beat, and explore the dance party world.

I was able to make this thanks to teaching from Allison Parrish via Computational Media class, Dan Schiffman’s set of p5.js instructional videos, and the reference resources pages of p5js.org

Also a big thanks to my ITP classmates for all their feedback and help along the way.

Copyrights for the used in the sketch, specifically:

1. ‘Don’t Mind’ by Kent Jones, under the labels Epidemic, We the Best and Epic
2. ‘Escape Me’ by Tiesto ft. C. C. Sheffield, under the labels Musical Freedom and Black Hole
3. ‘March Madness’ by Future, under the labels A1, Freebandz and Epic
4. ‘Seeing What’s Next’ byHollywood Principle, mixed by Mike Ault for Rocket League
5. ‘Lock Doh’ by Giggs et al, under SN1 Records
6. ‘Keep it Melo’ by Omar LinX and Marshmello, under the Joytime Collective label

Making Planet Music

Planet Orientation

To achieve this “around the world” effect, use the translate and rotate function. Directly in the draw loop of the sketch you will find…

translate(cvX / 2, cvY);

… which orients the world in the middle of the x-axis of the screen, and the bottom of the y-axis. This is best demonstrated by the planet, which is an image oriented at (0,0) but is not in the upper-lefthand corner as usual.

Rotate comes into play in the .show() functions of each object you will find:

rotate(-a);

…which provides a rotation relative to this new origin point. Working like this is very disorienting; I had to draw out what I meant to happen a few times and also consider the point at which the object origin is defined

Programming Principles, Techniques and Concepts

Once the orientation was set, I tried to explore some ideas from introduction to computational media class, with varied levels of success and efficiency. They were:

• Commenting code
• Universal sketch proportions
• Modular functions
• Sound analysis
• Array manipulation and calculations
• Object-oriented programming

Commenting Code

I really really really tried to comment my code well so that you can understand it easier now, and I can understand it easier later. This was a difficult practice. Once something such as a function finally worked I was always tempted to move on; however I took the time to make some comments. Hopefully it helps!

Universal Sketch Proportions

The big idea was to make everything proportional and avoid ‘hard-coding’ any numbers into the drawing. In other words, whether your screen is a huge monitor or just and iPad, I wanted the scene to look normal. If you open the sketch on a smaller device or with a custom window size, you can see this kind-of works

The key to achieving this was defining an aspect ratio for the sketch from the get-go. In this case, I defined a 16:9 aspect ratio. I set this ratio with some variables that you will see EVERYWHERE in the sketch: cvX and cvY.

cvX = windowWidth;

cvY = cvX * 0.5625;

cvX is set based on the width of the window, and cvY is defined based on this width in the 16:9 aspect ratio (16 ÷ 9 = 0.5625). All other values for size were intended to be set based on this value.

push()

push() and pop() were also essential to this style and are in pretty much every function and object as well. Praise be to push() and pop() in p5!

pop()

Modular Functions

The big idea was to make functions that just-plain-work, without need to be changed if some other parts of the code changes. You will see in the main sketch.js file, that basically everything is a function – e.g. loadSongs(), loadImages(), setStandards(), createPallat(), and createMediaPlayer(). All of these functions can be found in the sketch-functions.js file.

I like working like this because it helps keep my code organized, simpler to digest, and easier debug. It also allowed me to move some code out of the main sketch.js file, in order to simplify the overall information flow/hierarchy.

…yet I may have gone a bit overboard.

I often found myself turning thee or four lines of code into a function, sometimes detaching its meaning from what it is doing. For example, look at runEnergyAnalyzer().

//define specific "energy levels" based on the sound playing
function runEnergyAnalyzer() {
 spectrum = fft.analyze();
 treblergy = map(fft.getEnergy('treble'), 0, 1254, 0, 256);
 bassergy = map(fft.getEnergy('bass'), 512, 1254, 0, 256);
 midergy = map(fft.getEnergy('mid'), 128, 1254, 0, 256);
}

But whatever, thats my style and it made me happy to box up those little lines. ?

Sound Analysis

As mentioned in the proposal, the core part of this project was visualizing sound. Boy, is it is a confusing world. This video actually helped me out a lot:

I worked intimately with the p5 FFT analysis, and relied heavily on the getEnergy() function to analyze “amplitude across the frequency domain”. In human speak this means how loud certain pitches (like bass and treble) are in the current moment of the sound that is playing.

Can you tell what objects are mapped to what values? Most object sizes change based on these values. The spaceman rotates based on one of these values as well.

The tube that connects the spaceman to the spaceship is also based on sound analysis, but not in the same way. The tube is a histogram of a specific band on the frequency spectrum (i.e. one specific pitch). I was curious about how this would look and it became a deep, all-consuming black hole…

Array Manipulation and Calculations

Check out the drawAirTube() function to see what went into it. This tube – a histogram – required an array to store the historic values of the frequency. This array is called tubeParts. The first part of the drawAirTube() function fills 0 values into every space of the array in order to connect the spaceman and the spaceship.

From then on, old values are spliced out of the array as new values are pushed into it. Hooray for Arrays.

tubeLength = dist(spaceman.x, spaceman.y, ship.x, ship.y) + sq(curvature + 2);
 //adjust the size of the array to match the length of the tube
 if (tubeParts.length > tubeLength) {
 tubeParts.splice(0, tubeParts.length - tubeLength);
 }
 if (tubeParts.length < tubeLength) {
 for (i = tubeParts.length; i < tubeLength; i++) {
 tubeParts.push(0);}
 }
 tubeParts.splice(0, 1); //remove the least recent value in the array
 tubeParts.push(spectrum[frequency]); //push a new value to it based on one band of the frequency

However, these values jumped up and down too quickly, making for a jagged looking shape – definitely not a tube in space. In order to smooth these values, I made a moving average, which can be modified to different factors/periods based on input to the function. I am very proud of this simple math:

avgFactor = factor;
    subAvg = 0;
    for (j = 0; j < avgFactor; j++) {
      subAvg = subAvg + tubeParts[i + j];
      average = subAvg / avgFactor;
    }

This ‘average’ value was what was used instead of the raw value at each point in the tube. This made for smoother curves, but less waviness.

BUT WAIT, there’s more math. Because the spaceman and spaceship are both moving around in x and y terms, the tube needed to move in both ways. Thanks to the first part of this function, the x-axis distance is already calculated.

How many math problems start with “well, since we know what x is…” ?

It turns out this was a time for trigonometry. Something I had completely forgotten since leaving high school. Remember SOHCAHTOA? Neither did I.

It took some practice problems, but the code is math; please just trust me.

 //determine where the unvaried y position of each tube would be
 angle = atan(abs(spaceman.y - ship.y) / abs(spaceman.x - ship.x));
 y = tan(angle) * i - spaceman.y + spaceman.size / 2 + sqrt(i * 2);
 y = -y + (average / 3.5) + sq(curvature);

Finally, to add curvature to the tube I used shearY()…

shearY(PI / cvY * curvature);

…and drew a tiny circle at each point in the array to form the tube.

ellipse(spaceman.x + i - spaceman.size / 10, y, 4, 4);

I hope you like the air tube, because it took forever to make.

Object Oriented Programming

The tube also relied on the fact that the spaceman and spaceships were objects. All the objects I made are in the space-objects.js file. Besides using these values for the airTube, I also understood that making them objects would allow me to easily make more of them!

Thanks again to ITP professors Allison Parrish and Dan Schiffman for their work helping me understand this stuff. I see how powerful it is!

The spaceMan and spaceShip objects have two functions for their (class?): show() which makes then actually appear, and floating(), which makes them float around in a contained imaginary square.

As for the StarrySky object(s) – its a mess.

The arrays they reference and modify got very complicated, very quickly. I definitely did not use the object attributes right and probably have a redundant array or three because of it. In future I hope to clean this up. But for now, enjoy planet music v1!

Potential future improvements

• Add more assets with more uses of the FFT sound analysis, perhaps across the time domain instead of the frequency domain
• Host it somewhere else besides the p5 editor: I had to decrease the size of the music files in order for them to upload properly. The difference in quality is noticeable
• Allow the user to play music of their choice, perhaps from the Spotify API
• Make it more interactive, with the ability to:
  • click and move the assets around
  • change the planet in some ways
  • change the colors of the stars
  • control some of the inputs to the functions and objects I worked so hard to make
• Refine proportions to make it mobile friendly, or make a mobile version with tap instead
• Add a short cinematic introduction to planet music