I want to make an application for cardboard with a sony actioncam. I noticed that we can get the video preview by a SurfaceView method. But I remember that the only way to have a split screen with the camera device is by a SurfaceTexture.
So my question is, is there a way with one actioncam to have 2 video previews simultaneously ( split screen ) ??
thanks
EDIT:
okay so I went ahead and bought an action cam AS200, the sdk sample worked perfectly and i was able to get the video preview very quickly. I tried to duplicate the SimpleStreamSurfaceView with no success as expected. Now I m trying to copy the byte array in order to have two previews in one array. First I have tried to simply create an arraybuffer where i put two times the bitmap array, just to see what changes ... and I was surprised to see that nothing changed ... Here is the code
`
while (mWhileFetching) {
try {
byte[] jpegData = mJpegQueue.take();
ByteBuffer test=ByteBuffer.allocate(jpegData.length *2);
test.put(jpegData);
test.put(jpegData);
frameBitmap = BitmapFactory.decodeByteArray(//
test.array(), 0, test.array().length, factoryOptions);
//frameBitmap.setWidth(frameBitmap.getWidth()*2);
} catch (IllegalArgumentException e) {
if (mInMutableAvailable) {
clearInBitmap(factoryOptions);
}
continue;
} catch (InterruptedException e) {
Log.i(TAG, "Drawer thread is Interrupted.");
break;
}
if (mInMutableAvailable) {
setInBitmap(factoryOptions, frameBitmap);
}
drawFrame(frameBitmap);
}
if (frameBitmap != null) {
frameBitmap.recycle();
}
mWhileFetching = false;
}
};
mDrawerThread.start();
return true;
}
`
Of course I wasnt expecting a great result but why nothing changed ??
I solved it. I just had to draw two times in the canvas to different rectangles.
Related
I am using Android camera2 to create a custom camera. The cameraDevice.close() method is slow and it makes UI freeze for 1 sec. I put it in another thread and it seems to work just fine. I want to know if this will cause some serious problem and whether there is another way to achieve this. Here is my closeCamera method:
private void closeCamera() {
boolean release = false;
try {
mCameraOpenCloseLock.acquire();
release = true;
} catch (InterruptedException e) {
release = false;
}
try {
preparing = true;
if (mCaptureSession != null) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M && mCaptureSession.isReprocessable()
|| validCameraSession) {
mCaptureSession.close();
}
mCaptureSession = null;
validCameraSession = false;
}
} catch (IllegalStateException e) {
mCaptureSession = null;
} catch (Exception e) {
mCaptureSession = null;
}
try {
new Thread(new Runnable() {
#Override
public void run() {
if (mCameraDevice != null) {
if (openCamera) {
mCameraDevice.close();
mCameraDevice = null;
}
}
}
}).start();
} catch (IllegalStateException e) {
Log.e(TAG, "closeCamera: mCaptureSession - ", e);
} catch (Exception e) {
Log.e(TAG, "closeCamera: mCaptureSession - ", e);
}
if (release) {
if (mCameraOpenCloseLock != null) {
int lock = mCameraOpenCloseLock.availablePermits();
if (lock > 1) mCameraOpenCloseLock.release(lock - 1);
else if (lock == 0) mCameraOpenCloseLock.release();
}
}
}
I think it may cause crash when mCameraDevice has not been closed but user open camera again. But it is rare case, and I am thinking of putting another check before open camera again. I don't want my UI to freeze 1 sec for it to close, is there any other way I can achieve that except putting it in seperate thread?
As Alex Cohn mentions, the recommended practice is to do all camera-related work on a separate thread from the UI.
It also takes a long time to open the camera, or create a capture session, relatively speaking, so doing those operations not on the UI thread is also a good idea.
That said, as long as you're not losing track of your own app state (so that you don't try to use a camera device you've already closed by accident, for example), there's no reason you can't mix calls to the camera device or capture session from multiple threads. The classes themselves are thread-safe.
As far as I know, such freeze with cameraDevice.close() happens on some unfortunate devices, and sometimes is cured by performing a normal system upgrade.
But this is a little consolation if this happens to you, on your device. Actually, you are kind of lucky that you can prepare a fix for that. The end-users of your app will benefit from your misfortune.
Your code looks OK, if it delivers desired improvements for you. As I explained, it may be hard to reproduce this problem on another device.
I would rather put all closeCamera() logic on the same background thread. If you provided a Handler to openCamera(), as in the official example,
manager.openCamera(mCameraId, mStateCallback, mBackgroundHandler);
then I would suggest posting all closeCamera() sequence to this mBackgroundHandler.
Recently I am writing an app on My PC and lumia 640, everything seems good when it runs under Debug mode in VS 2015, but when I use it on my phone and tap a button to open a FileOpenPicker to choose picture from PicturesLibrary, it just crash...
When I try connecting my phone and use VS to find the problem, it never shows up any more...Also, when I run my app on my PC, whether under Debug mode or not, the problem also never happens.
The Listener of the button looks like this:
private async void OnSnap(object sender, RoutedEventArgs e)
{
try
{
FileOpenPicker fileOpenPicker = new FileOpenPicker();
fileOpenPicker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;
fileOpenPicker.FileTypeFilter.Add(".jpg");
fileOpenPicker.FileTypeFilter.Add(".png");
fileOpenPicker.ViewMode = PickerViewMode.Thumbnail;
var inputFile = await fileOpenPicker.PickSingleFileAsync();
if (inputFile == null)
{
// The user cancelled the picking operation
return;
}
else
{
Windows.Storage.AccessCache.StorageApplicationPermissions.FutureAccessList.Add(inputFile);
Frame frame = Window.Current.Content as Frame;
frame.Navigate(typeof(PictureChoosePage), await inputFile.OpenAsync(FileAccessMode.Read));
}
}
catch (Exception exception)
{
Debug.WriteLine(exception.Message);
var msgDialog = new MessageDialog(exception.Message) { Title = "Unkown Error" };
msgDialog.Commands.Add(new Windows.UI.Popups.UICommand("OK", uiCommand => {
Frame frame = Window.Current.Content as Frame;
frame.Navigate(typeof(MainPage));
}));
msgDialog.Commands.Add(new Windows.UI.Popups.UICommand("Cancel", uiCommand => { }));
await msgDialog.ShowAsync();
return;
}
}
One more thing to mention is that when my App crash, the bottom bar of the FileOpenPicker shows up...like this:
The required permissions should have been declared, and if there is a UnathorizedAccessException thrown, it should be caught by my code...So I'm thinking if this is the bug of the OS since the windows 10 mobile on my lumia 640 is 10.0.14295.1000, just a preview version.
I have a problem handling the end of file state of the axWindows Media Player and then open another frame in Visual C++
Here is my function :
private: System::Void axWindowsMediaPlayer1_PlayStateChange(System::Object^ sender, AxWMPLib::_WMPOCXEvents_PlayStateChangeEvent^ e) {
if (e->newState == 8) {
this->Hide();
MainForm^ frame = gcnew MainForm();
frame->ShowDialog();
}
}
If I build the project, the media player render the movie, plays it and when it reaches the end of file it's called the Hide function but the frame didn't show up. I receive no error.
What should I do or what I'm doing wrong ?
In this kind of situation you should try to catch an exception, and read its message - it's very helpful, and helps spare some time in the future.
try
{
...
}
catch (System::Exception^ e)
{
System::Console::WriteLine(e->Message);
}
In fact it is a wild mix of technologies, but the answer to my question (I think) is closest to Direct3D 9. I am hooking to an arbitrary D3D9 applications, in most cases it is a game, and injecting my own code to mofify the behavior of the EndScene function. The backbuffer is copied into a surface which is set to point to a bitmap in a push source DirectShow filter. The filter samples the bitmaps at 25 fps and streams the video into an .avi file. There is a text overlay shown across the game's screnn telling the user about a hot key combination that is supposed to stop gameplay capture, but this overlay is not supposed to show up in the recoreded video. Everything works fast and beautiful except for one annoying fact. On a random occasion, a frame with the text overaly makes its way into the recoreded video. This is not a really desired artefact, the end user only wants to see his gameplay in the video and nothing else. I'd love to hear if anyone can share ideas of why this is happening. Here is the source code for the EndScene hook:
using System;
using SlimDX;
using SlimDX.Direct3D9;
using System.Diagnostics;
using DirectShowLib;
using System.Runtime.InteropServices;
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
[System.Security.SuppressUnmanagedCodeSecurity]
[Guid("EA2829B9-F644-4341-B3CF-82FF92FD7C20")]
public interface IScene
{
unsafe int PassMemoryPtr(void* ptr, bool noheaders);
int SetBITMAPINFO([MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 1)]byte[] ptr, bool noheaders);
}
public class Class1
{
object _lockRenderTarget = new object();
public string StatusMess { get; set; }
Surface _renderTarget;
//points to image bytes
unsafe void* bytesptr;
//used to store headers AND image bytes
byte[] bytes;
IFilterGraph2 ifg2;
ICaptureGraphBuilder2 icgb2;
IBaseFilter push;
IBaseFilter compressor;
IScene scene;
IBaseFilter mux;
IFileSinkFilter sink;
IMediaControl media;
bool NeedRunGraphInit = true;
bool NeedRunGraphClean = true;
DataStream s;
DataRectangle dr;
unsafe int EndSceneHook(IntPtr devicePtr)
{
int hr;
using (Device device = Device.FromPointer(devicePtr))
{
try
{
lock (_lockRenderTarget)
{
bool TimeToGrabFrame = false;
//....
//logic based on elapsed milliseconds deciding if it is time to grab another frame
if (TimeToGrabFrame)
{
//First ensure we have a Surface to render target data into
//called only once
if (_renderTarget == null)
{
//Create offscreen surface to use as copy of render target data
using (SwapChain sc = device.GetSwapChain(0))
{
//Att: created in system memory, not in video memory
_renderTarget = Surface.CreateOffscreenPlain(device, sc.PresentParameters.BackBufferWidth, sc.PresentParameters.BackBufferHeight, sc.PresentParameters.BackBufferFormat, Pool.SystemMemory);
} //end using
} // end if
using (Surface backBuffer = device.GetBackBuffer(0, 0))
{
//The following line is where main action takes place:
//Direct3D 9 back buffer gets copied to Surface _renderTarget,
//which has been connected by references to DirectShow's
//bitmap capture filter
//Inside the filter ( code not shown in this listing) the bitmap is periodically
//scanned to create a streaming video.
device.GetRenderTargetData(backBuffer, _renderTarget);
if (NeedRunGraphInit) //ran only once
{
ifg2 = (IFilterGraph2)new FilterGraph();
icgb2 = (ICaptureGraphBuilder2)new CaptureGraphBuilder2();
icgb2.SetFiltergraph(ifg2);
push = (IBaseFilter) new PushSourceFilter();
scene = (IScene)push;
//this way we get bitmapfile and bitmapinfo headers
//ToStream is slow, but run it only once to get the headers
s = Surface.ToStream(_renderTarget, ImageFileFormat.Bmp);
bytes = new byte[s.Length];
s.Read(bytes, 0, (int)s.Length);
hr = scene.SetBITMAPINFO(bytes, false);
//we just supplied the header to the PushSource
//filter. Let's pass reference to
//just image bytes from LockRectangle
dr = _renderTarget.LockRectangle(LockFlags.None);
s = dr.Data;
Result r = _renderTarget.UnlockRectangle();
bytesptr = s.DataPointer.ToPointer();
hr = scene.PassMemoryPtr(bytesptr, true);
//continue building graph
ifg2.AddFilter(push, "MyPushSource");
icgb2.SetOutputFileName(MediaSubType.Avi, "C:\foo.avi", out mux, out sink);
icgb2.RenderStream(null, null, push, null, mux);
media = (IMediaControl)ifg2;
media.Run();
NeedRunGraphInit = false;
NeedRunGraphClean = true;
StatusMess = "now capturing, press shift-F11 to stop";
} //end if
} // end using backbuffer
} // end if Time to grab frame
} //end lock
} // end try
//It is usually thrown when the user makes game window inactive
//or it is thrown deliberately when time is up, or the user pressed F11 and
//it resulted in stopping a capture.
//If it is thrown for another reason, it is still a good
//idea to stop recording and free the graph
catch (Exception ex)
{
//..
//stop the DirectShow graph and cleanup
} // end catch
//draw overlay
using (SlimDX.Direct3D9.Font font = new SlimDX.Direct3D9.Font(device, new System.Drawing.Font("Times New Roman", 26.0f, FontStyle.Bold)))
{
font.DrawString(null, StatusMess, 20, 100, System.Drawing.Color.FromArgb(255, 255, 255, 255));
}
return device.EndScene().Code;
} // end using device
} //end EndSceneHook
As it happens sometimes, I finally found an answer to this question myself, if anyone is interested. It turned out that backbuffer in some Direct3D9 apps is not necessarily refreshed each time the hooked EndScene is called. Hence, occasionally the backbuffer with the text overlay from the previous EndScene hook call was passed to the DirectShow source filter responsible for collecting input frames. I started stamping each frame with a tiny 3 pixel overlay with known RGB values and checking if this dummy overlay was still present before passing the frame to the DirectShow filter. If the overlay was there, the previously cached frame was passed instead of the current one. This approach effectively removed the text overlay from the video recorded in the DirectShow graph.
I am trying to write code in J2ME for the Nokia SDK (S60 device) and am using Eclipse.
The code tries to play some wav files placed within the "res" directory of the project. The code is as follows:
InputStream in1 = null;
System.out.println("ABout to play voice:" + i);
try {
System.out.println("Getting the resource as stream.");
in1 = getClass().getResourceAsStream(getsound(i));
System.out.println("Got the resouce. Moving to get a player");
}
catch(Exception e) {
e.printStackTrace();
}
try {
player = Manager.createPlayer(in1, "audio/x-wav");
System.out.println("Created player.");
//player.realize();
//System.out.println("Realized the player.");
if(player.getState() != player.REALIZED) {
System.out.println("The player has been realized.");
player.realize();
}
player.prefetch();
System.out.println("Fetched player. Now starting to play sound.");
player.start();
in1.close();
int i1 = player.getState();
System.out.println("Player opened. Playing requested sound.");
//player.deallocate();
//System.out.println("Deallocated the player.");
}
catch(Exception e) {
e.printStackTrace();
}
}
Where the function getSound returns a string that contains the name of the file to be played. It is as follows:
private String getSound(int i) {
switch(i) {
case 1: return "/x1.wav";
case 2: return "/x2.wav";
}
}
My problem is this:
1. When I try to add more than 10 sounds, the entire application hangs right before the prefetch() function is called. The entire system slows down considerably for a while. I then have to restart the application.
I have tried to debug this, but have not gotten any solutions so far. It would be great if I could get some help on this.
The problem lies in the emulator being used for the project. In the emulation tab in the Run Configurations window, the following Device must be selected:
Group: Nokia N97 SDK v1.0
Device: S60 Emulator
Changing to the above from Devices listed under the Sun Java Wireless Toolkit solved the problem.