C#/XNA/HLSL - Applying a pixel shader on 2D sprites affects the other sprites on the same render target - colors

Background information
I have just started learning HLSL and decided to test what I have learned from the Internet by writing a simple 2D XNA 4.0 bullet-hell game.
I have written a pixel shader in order to change the color of bullets.
Here is the idea: the original texture of the bullet is mainly black, white and red. With the help of my pixel shader, bullets can be much more colorful.
But, I'm not sure how and when the shader is applied on spriteBatch in XNA 4.0, and when it ends. This may be the cause of problem.
There were pass.begin() and pass.end() in XNA 3.x, but pass.apply() in XNA 4.0 confuses me.
In addition, it is the first time for me to use renderTarget. It may cause problems.
Symptom
It works, but only if there are bullets of the same color in the bullet list.
If bullets of different colors are rendered, it produces wrong colors.
It seems that the pixel shader is not applied on the bullet texture, but applied on the renderTarget, which contains all the rendered bullets.
For an example:
Here I have some red bullets and blue bullets. The last created bullet is a blue one. It seems that the pixel shader have added blue color on the red ones, making them to be blue-violet.
If I continuously create bullets, the red bullets will appear to be switching between red and blue-violet. (I believe that the blue ones are also switching, but not obvious.)
Code
Since I am new to HLSL, I don't really know what I have to provide.
Here are all the things that I believe or don't know if they are related to the problem.
C# - Enemy bullet (or just Bullet):
protected SpriteBatch spriteBatch;
protected Texture2D texture;
protected Effect colorEffect;
protected Color bulletColor;
... // And some unrelated variables
public EnemyBullet(SpriteBatch spriteBatch, Texture2D texture, Effect colorEffect, BulletType bulletType, (and other data, like velocity)
{
this.spriteBatch = spriteBatch;
this.texture = texture;
this.colorEffect = colorEffect;
if(bulletType == BulletType.ARROW_S)
{
bulletColor = Color.Red; // The bullet will be either red
}
else
{
bulletColor = Color.Blue; // or blue.
}
}
public void Update()
{
... // Update positions and other properties, but not the color.
}
public void Draw()
{
colorEffect.Parameters["DestColor"].SetValue(bulletColor.ToVector4());
int l = colorEffect.CurrentTechnique.Passes.Count();
for (int i = 0; i < l; i++)
{
colorEffect.CurrentTechnique.Passes[i].Apply();
spriteBatch.Draw(texture, Position, sourceRectangle, Color.White, (float)Math.PI - rotation_randian, origin, Scale, SpriteEffects.None, 0.0f);
}
}
C# - Bullet manager:
private Texture2D bulletTexture;
private List<EnemyBullet> enemyBullets;
private const int ENEMY_BULLET_CAPACITY = 10000;
private RenderTarget2D bulletsRenderTarget;
private Effect colorEffect;
...
public EnemyBulletManager()
{
enemyBullets = new List<EnemyBullet>(ENEMY_BULLET_CAPACITY);
}
public void LoadContent(ContentManager content, SpriteBatch spriteBatch)
{
bulletTexture = content.Load<Texture2D>(#"Textures\arrow_red2");
bulletsRenderTarget = new RenderTarget2D(spriteBatch.GraphicsDevice, spriteBatch.GraphicsDevice.PresentationParameters.BackBufferWidth, spriteBatch.GraphicsDevice.PresentationParameters.BackBufferHeight, false, SurfaceFormat.Color, DepthFormat.None);
colorEffect = content.Load<Effect>(#"Effects\ColorTransform");
colorEffect.Parameters["ColorMap"].SetValue(bulletTexture);
}
public void Update()
{
int l = enemyBullets.Count();
for (int i = 0; i < l; i++)
{
if (enemyBullets[i].IsAlive)
{
enemyBullets[i].Update();
}
else
{
enemyBullets.RemoveAt(i);
i--;
l--;
}
}
}
// This function is called before Draw()
public void PreDraw()
{
// spriteBatch.Begin() is called outside this class, for reference:
// spriteBatch.Begin(SpriteSortMode.Immediate, null);
spriteBatch.GraphicsDevice.SetRenderTarget(bulletsRenderTarget);
spriteBatch.GraphicsDevice.Clear(Color.Transparent);
int l = enemyBullets.Count();
for (int i = 0; i < l; i++)
{
if (enemyBullets[i].IsAlive)
{
enemyBullets[i].Draw();
}
}
spriteBatch.GraphicsDevice.SetRenderTarget(null);
}
public void Draw()
{
// Before this function is called,
// GraphicsDevice.Clear(Color.Black);
// is called outside.
spriteBatch.Draw(bulletsRenderTarget, Vector2.Zero, Color.White);
// spriteBatch.End();
}
// This function will be responsible for creating new bullets.
public EnemyBullet CreateBullet(EnemyBullet.BulletType bulletType, ...)
{
EnemyBullet eb = new EnemyBullet(spriteBatch, bulletTexture, colorEffect, bulletType, ...);
enemyBullets.Add(eb);
return eb;
}
HLSL - Effects\ColorTransform.fx
float4 DestColor;
texture2D ColorMap;
sampler2D ColorMapSampler = sampler_state
{
Texture = <ColorMap>;
};
struct PixelShaderInput
{
float2 TexCoord : TEXCOORD0;
};
float4 PixelShaderFunction(PixelShaderInput input) : COLOR0
{
float4 srcRGBA = tex2D(ColorMapSampler, input.TexCoord);
float fmax = max(srcRGBA.r, max(srcRGBA.g, srcRGBA.b));
float fmin = min(srcRGBA.r, min(srcRGBA.g, srcRGBA.b));
float delta = fmax - fmin;
float4 originalDestColor = float4(1, 0, 0, 1);
float4 deltaDestColor = originalDestColor - DestColor;
float4 finalRGBA = srcRGBA - (deltaDestColor * delta);
return finalRGBA;
}
technique Technique1
{
pass ColorTransform
{
PixelShader = compile ps_2_0 PixelShaderFunction();
}
}
I would be appreciate if anyone can help solving the problem. (Or optimizing my shader. I really know very little about HLSL.)

In XNA 4 you should pass the effect directly to the SpriteBatch, as explained on Shawn Hargreaves' Blog.
That said, it seems to me like the problem is, that after rendering your bullets to bulletsRenderTarget, you then draw that RenderTarget using the same spriteBatch with the last effect still in action. That would explain why the entire image is painted blue.
A solution would be to use two Begin()/End() passes of SpriteBatch, one with the effect and the other without. Or just don't use a separate RenderTarget to begin with, which seems pointless in this case.
I'm also very much a beginner with pixel shaders so, just my 2c.

Related

GLSL shader circle is wobbling as it changes size in LibGDX

In LibGDX, I created a shader to do a transition effect where a circle forms over top of the image, where everything outside the circle is black, and everything inside the circle shows up as normal. This circle starts out big and shrinks to nothing. But I have a problem, as the circle is shrinking, it wobbles around. I also found I had this problem when using a ShadeRenderer to create a circle that also shrank over time, before using a shader. I think the problem as something to do with floating point numbers, or the way that circles are rendered. Anyway, I need to know, how do I fix it, so that I can get the circle to shrink smoothly?
Here is a gif demonstrating my problem of the wobbling circle (which I need to be smooth): my wobbling circle
Here is my Fragment Shader Code:
varying vec4 v_color;
varying vec3 v_position;
varying vec2 v_texCoord0;
uniform vec2 u_resolution;
uniform sampler2D u_sampler2D;
uniform float u_radius;
uniform int u_hasCircle;
void main() {
vec4 color = texture2D(u_sampler2D, v_texCoord0) * v_color;
vec2 relativePosition = gl_FragCoord.xy / u_resolution - .5;
relativePosition.x *= u_resolution.x / u_resolution.y;
float len = length(relativePosition);
if (u_hasCircle == 1 && len > u_radius) {
color = vec4(0, 0, 0, 1);
}
gl_FragColor = color;
}
And here is my vertex shader code that runs before that:
attribute vec4 a_color;
attribute vec3 a_position;
attribute vec2 a_texCoord0;
uniform mat4 u_projTrans;
uniform vec3 u_distort;
varying vec4 v_color;
varying vec3 v_position;
varying vec2 v_texCoord0;
void main() {
v_color = a_color;
v_position = a_position;
v_texCoord0 = a_texCoord0;
gl_Position = u_projTrans * vec4(a_position, 1.0);
}
When I want to transition to run, u_hasCircle is passed 1, otherwise it is passed 0. When the transition is running, I start with passing u_radius 1, and then I gradually decrease the value to 0, using a FloatAction in LibGDX.
I send these values to the shader once per frame.
Here is the relevant Libgdx Java code that interacts with the shader:
public class PlayWorld extends Group implements InputProcessor, Disposable
{
//various members
private PlayScreen screen;
private OrthographicCamera camera;
private FloatAction transitionToBattleAction;
private final float TRANS_TO_BATTLE_DURATION = 10f;
private float circleSize;
private boolean hasCircle = false;
public PlayWorld(PlayWorld playWorld) {
this.playWorld = playWorld;
camera = new OrthographicCamera();
tiledMap = new TiledMapActor(camera);
addActor(tiledMap);
transitionToBattleAction = new FloatAction();
}
//function that triggers transition
public void enterBattle() {
transitionToBattleAction.reset();
transitionToBattleAction.setStart(0);
transitionToBattleAction.setEnd(1);
transitionToBattleAction.setDuration(TRANS_TO_BATTLE_DURATION);
addAction();
}
// this function gets called every frame
#Override
public void act(float delta) {
super.act(delta);
if (transitionToBattleAction.getValue() == 0) {
//this function is defined in code shown below
tiledMap.unsetCircleSize();
} else if (transitionToBattleAction.getValue() < 1) {
//this function is defined in code shown below
tiledMap.setCircleSize(
1 - transitionToBattleAction.getValue());
} else if (transitionToBattleAction.getValue() == 1) {
//this function is defined in code shown below
tiledMap.setCircleSize(
1 - transitionToBattleAction.getValue());
transitionToBattleAction.restart();
screen.getGame().setScreen("battle");
} else if (transitionToBattleAction.getValue() > 1) {
//this function is defined in code shown below
tiledMap.unsetCircleSize();
transitionToBattleAction.restart();
}
}
//this gets called whenever the window resizes
public void resize(int width, int height) {
// this function is defined in code shown below
tiledMap.resize(width, height);
}
//various other methods
}
public class TiledMapActor extends Actor implements InputProcessor, Disposable
{
//various variables
private ShaderProgram shader;
private OrthographicCamera camera;
private TiledMap map;
private OrthogonalTiledMapRenderer renderer;
private float UNIT_SCALE = 1 / 16f;
public TiledMapActor(OrthographicCamera camera) {
super();
this.camera = camera;
map = new TmxMapLoader().load("myMap.tmx");
renderer = new OrthogonalTiledMapRenderer(map, UNIT_SCALE);
shader = new ShaderProgram(
Gdx.files.internal("shaders/myshader.vsh"),
Gdx.files.internal("shaders/myshader.fsh");
System.out.println(
shader.isCompiled() ?
"shader compiled" : shader.getLog());
renderer.getBatch().setShader(shader);
shader.begin();
shader.setUniformi("u_hasCircle", 0);
shader.end();
}
// this is called every time the window changes size
// from the PlayScreen class, see code above
public void resize(int width, int height) {
camera.viewportWidth = width;
camera.viewportHeight = height;
camera.update();
shader.begin();
shader.setUniformf("u_resolution", (float)width, (float)height);
shader.end();
}
//this method is called from code above, seen PlayScreen class code
//
public void setCircleSize(float circleSize) {
this.circleSize = circleSize;
hasCircle = true;
shader.begin();
shader.setUniformf("u_radius", circleSize);
shader.setUniformi("u_hasCircle", 1);
shader.end();
}
//this method is called from code above, seen PlayScreen class code
//
public void unsetCircleSize() {
hasCircle = false;
shader.begin();
shader.setUniformi("u_hasCircle", 0);
shader.end();
}
// Various other methods
}
So I found the problem, and stupid me! I was re-entering the transition scene every frame, once it started. I fixed it by using a boolean flag to tell me if the transition had started, and only starting the transition if that boolean flag wasn't set yet. Then later, at a point, after the transition had completed, I set that boolean flag back to false, so the transition could occur again!

Color of texture skybox unity

I'm working with google cardboard in unity.
In my main scene I have a skybox with an image as texture.
How can I get the color of the pixel I'm looking?
The skybox is an element of mainCamera, that is child of "Head".
I put also GvrReticle as child of head; is it useful for my purpose?
Thanks
Basically you wait for the end of the frame so that the camera has rendered. Then you read the rendered data into a texture and get the center pixel.
edit Be aware that if you have a UI element rendered in the center it will show the UI element color not the color behind.
private Texture2D tex;
public Color center;
void Awake()
{
StartCoroutine(GetCenterPixel());
}
private void CreateTexture()
{
tex = new Texture2D(1, 1, TextureFormat.RGB24, false);
}
private IEnumerator GetCenterPixel()
{
CreateTexture();
while (true)
{
yield return new WaitForEndOfFrame();
tex.ReadPixels(new Rect(Screen.width / 2f, Screen.height / 2f, 1, 1), 0, 0);
tex.Apply();
center = tex.GetPixel(0,0);
}
}

Kinect & Processing - Convert position of joint as mouse x and mouse y?

I'm currently using an XBOX KINECT model 1414, and processing 2.2.1. I'm hoping to use the right hand as a mouse to guide a character through the screen.
I managed to draw an ellipse to follow the right hand joint on a kinect skeleton. How would I be able to figure out the position of that joint so that I could replace mouseX and mouseY if needed?
Below is the code that will track your right hand and draw a red ellipse over it:
import SimpleOpenNI.*;
SimpleOpenNI kinect;
void setup()
{
// instantiate a new context
kinect = new SimpleOpenNI(this);
kinect.setMirror(!kinect.mirror());
// enable depthMap generation
kinect.enableDepth();
// enable skeleton generation for all joints
kinect.enableUser();
smooth();
noStroke();
// create a window the size of the depth information
size(kinect.depthWidth(), kinect.depthHeight());
}
void draw()
{
// update the camera...must do
kinect.update();
// draw depth image...optional
image(kinect.depthImage(), 0, 0);
background(0);
// check if the skeleton is being tracked for user 1 (the first user that detected)
if (kinect.isTrackingSkeleton(1))
{
int joint = SimpleOpenNI.SKEL_RIGHT_HAND;
// draw a dot on their joint, so they know what's being tracked
drawJoint(1, joint);
PVector point1 = new PVector(-500, 0, 1500);
PVector point2 = new PVector(500, 0, 700);
}
}
///////////////////////////////////////////////////////
void drawJoint(int userID, int jointId) {
// make a vector to store the left hand
PVector jointPosition = new PVector();
// put the position of the left hand into that vector
kinect.getJointPositionSkeleton(userID, jointId, jointPosition);
// convert the detected hand position to "projective" coordinates that will match the depth image
PVector convertedJointPosition = new PVector();
kinect.convertRealWorldToProjective(jointPosition, convertedJointPosition);
// and display it
fill(255, 0, 0);
float ellipseSize = map(convertedJointPosition.z, 700, 2500, 50, 1);
ellipse(convertedJointPosition.x, convertedJointPosition.y, ellipseSize, ellipseSize);
}
//////////////////////////// Event-based Methods
void onNewUser(SimpleOpenNI curContext, int userId)
{
println("onNewUser - userId: " + userId);
println("\tstart tracking skeleton");
curContext.startTrackingSkeleton(userId);
}
void onLostUser(SimpleOpenNI curContext, int userId)
{
println("onLostUser - userId: " + userId);
}
Any kind of links or help will be very appreciated, thanks!
In your case I would recommend, that you use the coordinates of the right hand joint. This is how you get them:
foreach (Skeleton skeleton in skeletons) {
Joint RightHand = skeleton.Joints[JointType.HandRight];
double rightX = RightHand.Position.X;
double rightY = RightHand.Position.Y;
double rightZ = RightHand.Position.Z;
}
Be aware of the fact that we are looking at 3 dimensions so you will have a x,y and z coordinate.
FYI: You will have to insert these lines of code in the event handler SkeletonFramesReady.
If you still want the circle around it have a look at the Skeleton-Basics WPF Example in the Kinect SDK's.
Does this help you?
It's slightly unclear what you're trying to achieve.
If you simply need the position of the hand in 2D screen coordinates, the code you posted already includes this:
kinect.getJointPositionSkeleton() retrieves the 3D coordinates
kinect.convertRealWorldToProjective() converts them to 2D screen coordinates.
If you want to be able to swap between using kinect tracked hand coordinates and mouse coordinates, you can store the PVector used in the 2D conversion as a variable visible to the whole sketch which you updated either by kinect skeleton if it is being tracked or mouse otherwise:
import SimpleOpenNI.*;
SimpleOpenNI kinect;
PVector user1RightHandPos = new PVector();
float ellipseSize;
void setup()
{
// instantiate a new context
kinect = new SimpleOpenNI(this);
kinect.setMirror(!kinect.mirror());
// enable depthMap generation
kinect.enableDepth();
// enable skeleton generation for all joints
kinect.enableUser();
smooth();
noStroke();
// create a window the size of the depth information
size(kinect.depthWidth(), kinect.depthHeight());
}
void draw()
{
// update the camera...must do
kinect.update();
// draw depth image...optional
image(kinect.depthImage(), 0, 0);
background(0);
// check if the skeleton is being tracked for user 1 (the first user that detected)
if (kinect.isTrackingSkeleton(1))
{
updateRightHand2DCoords(1, SimpleOpenNI.SKEL_RIGHT_HAND);
ellipseSize = map(user1RightHandPos.z, 700, 2500, 50, 1);
}else{//if the skeleton isn't tracked, use the mouse
user1RightHandPos.set(mouseX,mouseY,0);
ellipseSize = 20;
}
//draw ellipse regardless of the skeleton tracking or mouse mode
fill(255, 0, 0);
ellipse(user1RightHandPos.x, user1RightHandPos.y, ellipseSize, ellipseSize);
}
///////////////////////////////////////////////////////
void updateRightHand2DCoords(int userID, int jointId) {
// make a vector to store the left hand
PVector jointPosition = new PVector();
// put the position of the left hand into that vector
kinect.getJointPositionSkeleton(userID, jointId, jointPosition);
// convert the detected hand position to "projective" coordinates that will match the depth image
user1RightHandPos.set(0,0,0);//reset the 2D hand position before OpenNI conversion from 3D
kinect.convertRealWorldToProjective(jointPosition, user1RightHandPos);
}
//////////////////////////// Event-based Methods
void onNewUser(SimpleOpenNI curContext, int userId)
{
println("onNewUser - userId: " + userId);
println("\tstart tracking skeleton");
curContext.startTrackingSkeleton(userId);
}
void onLostUser(SimpleOpenNI curContext, int userId)
{
println("onLostUser - userId: " + userId);
}
Optionally, you can use a boolean to swap between mouse/kinect mode when testing.
If you need the mouse coordinates simply to test without having to get in from of the kinect all the time, I recommend having a look at the RecorderPlay example (via Processing > File > Examples > Contributed Libraries > SimpleOpenNI > OpenNI > RecorderPlay). OpenNI has the ability to record a scene (including depth data) which will make it simpler to test: simply record an .oni file with the most common interactions you're aiming for, then re-use the recording when developing.
All it would take to use the .oni file is using a different constructor signature for OpenNI:
kinect = new SimpleOpenNI(this,"/path/to/yourRecordingHere.oni");
One caveat to keep in mind: the depth is stored at half the resolution (so the coordinates with need to be doubled to be on par with the realtime version).

TextureRegion cuts texture in a wrong way

I want to have a background texture with 3 rectangles and i want to create animation with them, - texture
But first rectangle cuts in a proper way and two others are cut in a dumb way
Proper way,
Dumb way #1,
Dumb way #2
Here is my code.
public class MainMenu implements Screen{
Texture background_main;
TextureRegion[] background_textures;
Animation background_animation;
SpriteBatch batch;
TextureRegion current_frame;
float stateTime;
BobDestroyer game;
OrthographicCamera camera;
public MainMenu(BobDestroyer game){
this.game = game;
}
#Override
public void show() {
camera = new OrthographicCamera();
camera.setToOrtho(true, Gdx.graphics.getWidth(), Gdx.graphics.getHeight());
batch = new SpriteBatch();
background_main = new Texture(Gdx.files.internal("main_menu_screen/Background.png"));
background_textures = new TextureRegion[3];
for(int i = 0; i<3; i++){
background_textures[i] = new TextureRegion(background_main,0, 0+72*i, 128, 72+72*i);
}
background_animation = new Animation(2f,background_textures);
}
#Override
public void render(float delta) {
Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT | GL20.GL_DEPTH_BUFFER_BIT);
stateTime += Gdx.graphics.getDeltaTime();
current_frame = background_animation.getKeyFrame(stateTime, true);
batch.begin();
batch.draw(current_frame,0, 0,Gdx.graphics.getWidth(),Gdx.graphics.getHeight());
batch.end();
}
}
If I understand you correctly, are you trying to create 3 TextureRegions of the same width/height? If yes, your issue may be with:
new TextureRegion(background_main,0, 0+72*i, 128, 72+72*i)
I think you'd want:
new TextureRegion(background_main,0, 0+72*i, 128, 72)
As the 128x72 is the width/height (not x/y positions) of your next TextureRegions, and do you not want them all the same height (72) as opposed to varying heights (72+72*i)?

Transparent objects covering each other

For an academic project I'm trying to write a code for drawing billboards from scratch; now I'm at the point of making them translucent. I've managed to make them look good against the background but they still may cover each other with their should-be-transparent corners, like in this picture:
I'm not sure what I'm doing wrong. This is the effect file I'm using to draw the billboards. I've omitted the parts related to the vertex shader, which I think is irrelevant right now.
//cut
texture Texture;
texture MaskTexture;
sampler Sampler = sampler_state
{
Texture = (Texture);
MinFilter = Linear;
MagFilter = Linear;
MipFilter = Point;
AddressU = Clamp;
AddressV = Clamp;
};
sampler MaskSampler = sampler_state
{
Texture = (MaskTexture);
MinFilter = Linear;
MagFilter = Linear;
MipFilter = Point;
AddressU = Clamp;
AddressV = Clamp;
};
//cut
struct VertexShaderOutput
{
float4 Position : POSITION0;
float4 Color : COLOR;
float2 TexCoord : TEXCOORD0;
};
//cut
float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
{
float4 result = tex2D(Sampler, input.TexCoord) * input.Color;
float4 mask = tex2D(MaskSampler, input.TexCoord);
float alpha = mask.r;
result.rgb *= alpha;
result.a = alpha;
return result;
}
technique Technique1
{
pass Pass1
{
VertexShader = compile vs_2_0 VertexShaderFunction();
PixelShader = compile ps_2_0 PixelShaderFunction();
AlphaBlendEnable = true;
SrcBlend = SrcAlpha;
DestBlend = InvSrcAlpha;
}
}
I've got two textures, named Texture and MaskTexture, the latter being in grayscale. The billboards are, most likely, in the same vertex buffer and are drawn with a single call of GraphicsDevice.DrawIndexedPrimitives() from XNA.
I've got a feeling I'm not doing the whole thing right.
You have to draw them in order. From farthest to closest.
I have found a solution. The shaders are fine, the problem turned out to be in the XNA code, so sorry for drawing your attention to the wrong thing.
The solution is to enable a depth stencil buffer (whatever it is) before drawing the billboards:
device.DepthStencilState = DepthStencilState.DepthRead;
It can be disabled afterwards:
device.DepthStencilState = DepthStencilState.Default;

Resources