I am using processing and the minim library and trying to create a 3D real-time visualisation for live audio input.
i have boxes drawn and is responding to the kick, snare , and hi hats of the audio input. I am looking to make these boxes rotate also responding to the kick ect. how could i make these boxes rotate?
if ( beat.isKick() ) kickSize = 200;
if ( beat.isSnare() ) snareSize = 250;
if ( beat.isHat() ) hatSize = 200;
translate ( width/4, height/4);
box(kickSize);
translate( - width/4, - height/4);
translate ( width/2, height/3);
sphere(snareSize);
translate( - width/2, - height/3);
translate ( 3*width/4, height/4);
box(hatSize);
translate( - 3*width/4, - height/4);
kickSize = constrain(kickSize * 0.95, 1, 32);
snareSize = constrain(snareSize * 0.95, 1, 32);
hatSize = constrain(hatSize * 0.95, 1, 32);
Use pushMatrix();popMatrix(); calls to isolate coordinate systems for each object:
pushMatrix();
translate ( width/4, height/4);
box(kickSize);
popMatrix();
pushMatrix();
translate ( width/2, height/3);
sphere(snareSize);
popMatrix();
pushMatrix();
translate ( 3*width/4, height/4);
box(hatSize);
popMatrix();
Have a look at the 2D transformations Processing tutorial for more details.
The same principle applies to 3D
Related
I have my shader setup to generate rays from a view and projection matrix coming from GLM, the GLSL function looks like this:
Ray Camera_getRay(inout uvec4 useed) {
vec3 rayNDC = vec3(
((2.0f * fragCoord.x) / iResolution.x - 1.0f),
(1.0f - (2.0f * fragCoord.y) / iResolution.y),
1.0f
);
vec4 rayClip = vec4( rayNDC.x, rayNDC.y * -1, -1.0f, 1.0f );
vec4 rayCamera = inverse(projection) * rayClip;
rayCamera.z = -1.0f, rayCamera.w = 0.0f;
vec3 direction = normalize((inverse(view) * rayCamera).xyz);
return Ray(position, direction);
}
I am trying to add the randomized offset as described in the Defocus Blur chapter.
I have gotten this far:
float aperture = 0.1;
float lens_radius = aperture / 2;
vec3 rd = lens_radius * sampleUnitDisk(useed);
But I am not sure what to do with rd next, do I simply add it to the ray's position? do I also need to add it to the ray direction? Do I alter the view matrix? Any help is appreciated.
In my ray tracer I add the (2D) randomized offset to the eye position (ensuring that the movement is in the plane parallel to the virtual view plane) and then calculate the ray's direction through the originally intended projection point.
https://github.com/raybellis/RRT/blob/master/rrt/camera.cpp#L66
I'm working on an audio visualisation that's basically supposed to be a circular spectrogram. I have a graph that shows the frequency already and an arc, that evolves based on the time passed. Now I would like to fill the arc with white points based on the amplitude of each frequency, much like here: https://vimeo.com/27135957. Apparently, I need to make a PGraphics that is filled with points, which change from white to black based on the amplitude. Then I need to texture the arc with this graphic. Does anyone know how to do this?
import ddf.minim.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;
import ddf.minim.signals.*;
import ddf.minim.spi.*;
import ddf.minim.ugens.*;
Minim minim;
AudioPlayer song;
FFT fft;
PGraphics pg;
PShape arc;
float deg = 90;
float rad = radians(deg);
void setup()
{
size(1000, 1000);
minim = new Minim(this);
song = minim.loadFile("Anthology.mp3");
song.play();
fft = new FFT(song.bufferSize(), song.sampleRate());
pg = createGraphics(width, height);
}
void draw()
{
background(0);
fft.forward(song.mix);
for (int i = 0; i < fft.specSize(); i++)
{
pushMatrix();
stroke(255);
line(i, height, i, height - fft.getBand(i)*0.5);
popMatrix();
println(fft.getBand(i));
//Map Amplitude to 0 → 255, fill with points and color them
float brightness = map(fft.getBand(i), -1, 1, 0, 255);
pg.beginDraw();
pg.endDraw();
fill(255, 255, 255,);
noStroke();
float evolution = radians(map(song.position(), 0, song.length(), 90, 450));
//texture(pg);
arc(height/2, height/2, height-100, height-100, rad, evolution, PIE);
}
}
There are few concepts that appear unclear based on your code:
If you plan to render the arc within the pg PGraphics instance access pg with . notation and call drawing functions in between beginDraw()/endDraw() calls. At the moment there is nothing rendered in pg and pg isn't rendered anywhere using image(). For more details see the createGraphics() reference, run the sample code/ tweak it/ break it/fix it/understand it
Similarly PShape arc is created but is not used
There is a commented attempt to use pg as a texture however the texture mapping is unclear
If using both PGraphics and PShape is confusing you can achieve a similar effect with PGraphics alone: simply render some larger gray dots instead of arcs. It won't be an identical effect but it will have a very similar look with less effort.
Here's a variant based on your code:
import ddf.minim.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;
import ddf.minim.signals.*;
import ddf.minim.spi.*;
import ddf.minim.ugens.*;
Minim minim;
AudioPlayer song;
FFT fft;
PGraphics pg;
void setup()
{
size(600, 600, P2D);
minim = new Minim(this);
song = minim.loadFile("jingle.mp3", 1024);
song.loop();
fft = new FFT(song.bufferSize(), song.sampleRate());
// optional: use logarithmic averages: clover to how we perceive sound
fft.logAverages( 30, 6 );
// setup pg graphics layer disable fill, make points stroke thick
pg = createGraphics(width, height);
pg.beginDraw();
pg.strokeWeight(3);
pg.noFill();
pg.endDraw();
}
void draw()
{
background(0);
image(pg, 0, 0);
// perform FFT on stereo mix
fft.forward(song.mix);
// center coordinates
float cx = width * 0.5;
float cy = height * 0.5;
// count FFT bins
int fftSpecSize = fft.specSize();
// calculate the visual size for representing an FFT bin
float sizePerSpec = (height * 0.5 ) / fftSpecSize;
stroke(255);
noFill();
// start #editing# the pg layer (once
pg.beginDraw();
// start the FFT graph shape
beginShape();
// for each FFT bin
for (int i = 0; i < fftSpecSize; i++)
{
// get the vands in reverse order (low frequencies last)
float fftBand = fft.getBand(fftSpecSize - i - 1);
// scale FFT bin value to pixel/render size
float xOffset = fftBand * 10;
// map FFT bins to 0-255 brightness levels (note 35 may differ
float brightness = map(fftBand, 0, 35, 0, 255);
// draw the line graph vertex
vertex(cx + xOffset, cy + sizePerSpec * i);
// map song position (millis played) to 360 degrees in radians (2 * PI)
// add HALF_PI (90 degrees) because 0 degrees points to the right and drawing should start pointing down (not right)
//float angle = map(song.position(), 0, song.length(), 0, TWO_PI) + HALF_PI;
// as a test map it to a lower value
float angle = (frameCount * 0.0025) + HALF_PI;
// map radius from FFT index
float radius = map(i, 0, fftSpecSize - 1, 0, width * 0.5);
// use mapped brightness as point stroke
pg.stroke(brightness);
// use polar coordinates mapped from the centre
pg.pushMatrix();
pg.translate(cx,cy);
pg.rotate(angle);
pg.point(radius,0);
pg.popMatrix();
// alternatively use polar to cartesian coordinate conversion
// x = cos(angle) * radius
// y = sin((angle) * radius
// cx, cy are added to offset from center
//pg.point(cx + (cos(angle) * radius),
// cy + (sin(angle) * radius));
}
// finish FFT graph line
endShape();
// fnish pg layer
pg.endDraw();
}
Note
you may want to change jingle.mp3 to your audio file name
for the sake of a test with a short track I used an arbitrary mapping of the angle (same as evolution in your code): there is a commented version that takes the track duration into account
the grayscale point position is rendered using coordinate transformations. Be sure to go through the 2D Transformations tutorial and bare in mind the order of transformations is important. Alternatively there is a versioon that does the same using the polar (angle/radius) to cartesian (x,y) coordinate system transformation formula instead.
P.S. I also wondered how to get nice visuals based on FFT data and with a few filtering tricks results can be nice. I recommend also checking out wakjah's answer here.
I use the following code to display OpenCV's IplImage. It works well for the RGB image.But for the gray image (only one channel), the image is upside down (inverted).
if( m_img && m_img->depth == IPL_DEPTH_8U )
{
uchar buffer[sizeof(BITMAPINFOHEADER) + 1024];
BITMAPINFO* bmi = (BITMAPINFO*)buffer;
int bmp_w = m_img->width, bmp_h = m_img->height;
FillBitmapInfo( bmi, bmp_w, bmp_h, Bpp(), m_img->origin );
from_x = MIN( MAX( from_x, 0 ), bmp_w - 1 );
from_y = MIN( MAX( from_y, 0 ), bmp_h - 1 );
int sw = MAX( MIN( bmp_w - from_x, w ), 0 );
int sh = MAX( MIN( bmp_h - from_y, h ), 0 );
SetDIBitsToDevice(
dc, x, y, sw, sh, from_x, from_y, from_y, sh,
m_img->imageData + from_y*m_img->widthStep,
bmi, DIB_RGB_COLORS );
}
I looked at the function in link here and showed that lpvBits is for RGB color. How can I modify for gray image?
Thanks
Use GDI+
Load the colored Bitmap
Create a ColorMatrix with 30% Red 59% Green
11% Blue
Create an ImageAttributes
Set the ColorMatricx into the ImageAttributes
Create a new bitmap
Use DrawImage with the ImageAtributes and draw the colored bitmap into the new bitmap.
There is a nuch of C# code samples, but the GDI+ interface in C+ is nearly identical.
I would like draw sphere in pure OpenGL ES 2.0 without any engines. I write next code:
int GenerateSphere (int Slices, float radius, GLfloat **vertices, GLfloat **colors) {
srand(time(NULL));
int i=0, j = 0;
int Parallels = Slices ;
float tempColor = 0.0f;
int VerticesCount = ( Parallels + 1 ) * ( Slices + 1 );
float angleStep = (2.0f * M_PI) / ((float) Slices);
// Allocate memory for buffers
if ( vertices != NULL ) {
*vertices = malloc ( sizeof(GLfloat) * 3 * VerticesCount );
}
if ( colors != NULL) {
*colors = malloc( sizeof(GLfloat) * 4 * VerticesCount);
}
for ( i = 0; i < Parallels+1; i++ ) {
for ( j = 0; j < Slices+1 ; j++ ) {
int vertex = ( i * (Slices + 1) + j ) * 3;
(*vertices)[vertex + 0] = radius * sinf ( angleStep * (float)i ) *
sinf ( angleStep * (float)j );
(*vertices)[vertex + 1] = radius * cosf ( angleStep * (float)i );
(*vertices)[vertex + 2] = radius * sinf ( angleStep * (float)i ) *
cosf ( angleStep * (float)j );
if ( colors ) {
int colorIndex = ( i * (Slices + 1) + j ) * 4;
tempColor = (float)(rand()%100)/100.0f;
(*colors)[colorIndex + 0] = 0.0f;
(*colors)[colorIndex + 1] = 0.0f;
(*colors)[colorIndex + 2] = 0.0f;
(*colors)[colorIndex + (rand()%4)] = tempColor;
(*colors)[colorIndex + 3] = 1.0f;
}
}
}
return VerticesCount;
}
I'm drawing it with using next code:
glDrawArrays(GL_TRIANGLE_STRIP, 0, userData->numVertices);
Where userData->numVertices - VerticesCount from function GenerateSphere.
But on screen draws series triangles, these aren't sphere approximation!
I think, I need to numerate vertices and use OpenGL ES 2.0 function glDrawElements() (with array, contained number vertices). But series of triangles drawn on the screen is not a sphere approximation.
How can I draw sphere approximation? How specify order vertices (indices in OpenGL ES 2.0 terms)?
Before you start with anything in OpenGL ES, here is some advice:
Avoid bloating CPU/GPU performance
Removing intense cycles of calculations by rendering the shapes offline using another program will surely help. These programs will provide additional details about the shapes/meshes apart from exporting the resultant collection of points [x,y,z] comprising the shapes etc.
I went through all this pain way back, because I kept trying to search for algorithms to render spheres etc and then trying to optimize them. I just wanted to save your time in the future. Just use Blender and then your favorite programming language to parse the obj files that are exported from Blender, I use Perl. Here are the steps to render sphere: (use glDrawElements because the obj file contains the array of indices)
1) Download and install Blender.
2) From the menu, add sphere and then reduce the number of rings and segments.
3) Select the entire shape and triangulate it.
4) Export an obj file and parse it for the meshes.
You should be able to grasp the logic to render sphere from this file: http://pastebin.com/4esQdVPP. It is for Android, but the concepts are same.
Hope this helps.
I struggled with spheres and other geometric shapes. I worked at it a while and created an Objective-C class to create coordinates, normals, and texture coordinates both using indexed and non-indexed mechanisms, the class is here:
http://www.whynotsometime.com/Why_Not_Sometime/Code_Snippets.html
What is interesting to see the resulting triangles representing the geometry is to reduce the resolution (set the resolution property before generating the coordinates). Also, you can use GL_LINE_STRIP instead of GL_TRIANGLES to see a bit more.
I agree with the comment from wimp that since calculating the coordinates generally happens once, not many CPU cycles are used. Also, sometimes one does want to draw only a ball or world or...
I have successfully drawn circles and applied translations and scaling on them. When I rotate a circle by only 1 degree (or any degree), a half circle is drawn. I am using an ortho perspective. Why is this?
translateX = (float) (ratio * (xCoor - windowWidth / 2)) / (windowWidth / 2);
translateY = (float) (-(yCoor - windowHeight / 2)) / (windowHeight / 2);
gl11.glBindBuffer(GL11.GL_ARRAY_BUFFER, vertexPointerCircleHR);
gl11.glBindBuffer(GL11.GL_ELEMENT_ARRAY_BUFFER, indexPointerCircleHR);
gl11.glVertexPointer(3, GL10.GL_FLOAT, 0, 0);
circleRotation++;
gl11.glPushMatrix();
gl11.glColor4f(...);
gl11.glScalef(...);
gl11.glTranslatef(translateX, translateY, 0);
gl11.glRotatef(1,translateX,translateY, 0); //or circleRotation
gl11.glDrawElements(GL11.GL_TRIANGLE_FAN, verticesCircle,
GL11.GL_UNSIGNED_SHORT, 0);
gl11.glPopMatrix();
Thanks for reading.
glRotate receives the angle and a 3d vector that will be the axis of rotation (Axis-angle representation).
If you're using orthogonal projection I assume you're doing a 2D rotation in front of the camera, and normally that's around the Z axis (0, 0, 1) or maybe (0, 0, -1).
You should probably replace your call with glRotatef(circleRotation, 0, 0, 1).
Obviously without a texture or different vertex colors you won't notice a thing.