My team is tasked with building a full screen, kiosk-style application for playing back media files. Initially we need to support WMV / MP4 as well as some images in full 1080p, although down the line we will need to extend this to cover other formats (different videos formats as well as display of HTML, SWF, etc).
The application also contains a decent chunk of business logic relating to scheduling, logging, performance monitoring as well as network code to talk to a central server through web services (or maybe TCP) and potentially act as a server itself.
For our WMV / MP4 video playback, hardware acceleration will be a massive bonus. The targetted hardware has weak CPUs but strong graphics cards.
Here's the kicker: we're a .NET shop (our existing application is a WinForms smart client) and extremely experienced and productive in C# and the .NET stack. The app will initially be targetting Windows Embedded (.NET 3.0), but we will quickly need a Linux version as well. Between us we have some C/C++ experience and some Linux experience but we do not anticipate good productivity on that platform.
So I am soliciting recommendations specfically on the following points:
Video. On Windows we have seen good success using DirectShow.NET. On capable hardware, the WPF MediaElement also seems to perform well. What should we be using on Linux? libavcodec seems like a common choice. Is it hardware accelerated on NVidia graphics cards on Linux? What other options do we have on Linux? Is there something cross-platform that I could consider?
Stack.
a) Ideally we could write the whole thing in .NET and then run under Mono on Linux. The video playback and presumably some other components (like performance monitoring) would not be supported on Mono. I guess we could rewrite these elements in, say, C++; but I'm guessing that most stuff on the business logic side would work.
b) Maybe it's better to forfeit our up-front productivity on the Windows version for something that's cross platform out of the gate. What about Java? Do we have different options when it comes to video there? How about another framework? Something like QT? Can anyone else suggest something cross platform that would be relevant?
Broadly speaking, given the requirements, what would you use?
I appreciate any anwsers you might have.
My suggestion is that you use Fluendo's GStreamer components for the video playback as it has support for hardware acceleration where available and fully licensed codecs.
You can look at the Banshee media player which support video playback if you have the Fluendo/GStreamer packages installed. Get OpenSUSE 11.2 which contains everything you need to try it and develop, and then buy and install the Fluendo codecs.
Source code wise, Banshee does the video display from C#, look here:
The C# source code consuming GStreamer and doing the video rendering is here:
http://git.gnome.org/browse/banshee/tree/src/Extensions/Banshee.NowPlaying/Banshee.NowPlaying
The C supporting library to call into Fluendo is available here:
http://git.gnome.org/browse/banshee/tree/libbanshee
For testing Banshee, you do not need to buy anything, but your video codecs will be limited to Ogg/Theora encoded videos. Once you get Fluendo's codecs you will be able to play WMV files.
One option would be to use Silverlight, and explore Moonlight as an option for the linux version. My understanding is that Moonlight has several media/codec plugins (I believe ffmpeg is the main provider) and can additionally use the MS codec pack to give you support for things like WMV/MP4.
You can use ffmpeg in mono and .net. This may or may not include video display - ffmpeg usually just provides you with a decoded bitmap that you can do whatever you want with, be it display it in a window, save it in a file, whatever. If you use ffmpeg-sharp the same code should work on Windows or Linux. Really, putting the bitmap in a window is the easy part.
Moonlight offers two codecs: (a) A fully licensed version that comes straight from Microsoft and requires no further negotiation with the MPEG-LA and other patent holders, or (b) an ffmpeg backend that requires you to negotiate with the patent stock holders if you plan on using.
You could build a Silverlight-based application, the trick to get access to the local system is very simple: you run a local web server that exposes those services.
You can still use C#/Sqlite or VistaDB as your storage system as part of your Silverlight application.
You could host the silverlight app in http://localhost/App.xap and this app would gain local access to the machine by contacting a REST or SOAP web service on http://localhost/rest.ashx or http://localhost/soap.asmx
For example, if you needed to read some values from a scanner connected to the machine, you would issue this request:
http://localhost/scanner.ashx?operation=scan_badge
Then your scanner.ashx HttpHandler will do the actual scanning (this one has full system rights) and return the value to the Silverlight application.
Related
I recently purchased Micro:Bit. I've seen that micro-python and bluetooth cannot be used at the same time due to memory capacity.
Does anyone know if I would be able to build a decent application using the javascript block programming?
The app basically has to do the following:
Read data from acceleretometer.
Acumulate some accelerometer data.
Send the information to another device connected via bluetooth.
Yes, you should be able to write a program for the microbit that does this. the official documentation describes the services that are available. I also found an example which suggests that there is an app which you can use at the phone end if that's relevant to your application.
The micropython restriction is a combination of the BLE protocol stack requiring 12 kB of RAM, and python being interpreted (so having a high RAM requirement).
You can chose the block version or test javascript - and should be able to write reasonably complex programs (even if the text entry might be best done in an editor). As a final fall-back, you can fall back on C/C++ using the microbit DAL (which seems to be built on top of the mbed offline toolchain).
Upon considering the driver side implementation for DirectX API on windows systems for modern video cards I was wondering why this implementation is not available on non-windows system, most notably linux.
Since there is an obvious absence of this functionality I can only assume there is a good reason which I am blind to, but in my primitive understanding I simply see the DirectX calls as no more than function entry points on the hardware device. And for the record I am not referring to a compatibility layer (most notably WINE, a project I am amazed by every single day) but a library making direct hardware calls.
Is it possible to create an open source version of directx? Would it be possible but obscenely difficult?
AFAIK, the DirectX contains 3 parts
Vendor driver (provide low level API defined by DX)
DirectX library (use low level API defined by DX to access hardware, provide DirectX API)
Software (use DirectX API, ex. game)
There are no driver providing the low level API defined by DX on linux, so even it is possible to provide the DirectX library, but it won't work without proper vendor driver, and I can't see if there is any vendor is going to create one for their hardware platform.
Why not just use OpenGL? It supports all the functionality that DirectX does. Do you have a specific reason to use DirectX? As for a reason, Microsoft made DirectX and I guess they didn't see any need to allow it to run on Linux.
You can run DirectX games in Linux using Wine. You can program XNA on Linux using MonoGame. But all of these use OpenGL to provide hardware-accelerated rendering. AFAIK, OpenGL has been (and probably will be) the only option on Linux for the foreseeable future.
Is it possible to write your own DirectX implementation? Sure - but it involves writing drivers, proprietary knowledge and probably too much cost for anyone to REALLY benefit from it.
EDIT: These days, DXVK is a very real and performant way to run DirectX applications on Linux.
DirectX is a suit of API's:
Direct3D (drawing 3D graphics)
DirectX Graphics Infrastructure (enumerating adapters and monitors and managing swap chains)
Direct2D (drawing 2D graphics)
DirectWrite (fonts)
DirectCompute (GPU Computing)
DirectSound3D (playback of 3D sounds)
DirectX Media (DirectAnimation for 2D/3D web animation, DirectShow for multimedia playback and streaming media, DirectX Transform for web interactivity, and Direct3D Retained Mode for higher level 3D graphics)
DirectX Diagnostics (tool for diagnosing and generating reports on components related to DirectX, such as audio, video, and input drivers)
DirectX Media Objects (support for streaming objects such as encoders, decoders, and effects)
DirectSetup (installation of DirectX components, and the detection of the current DirectX version)
DirectX components deprecated, but still supported
DirectDraw
DirectInput
DirectPlay
DirectSound
DirectMusic
As you can see many parts of the DirectX suit of API's would need to be rewritten with functionality for Drivers written for Linux. Also some parts of the DirectX suit are likely going to make calls to the Windows OS and that code would have to rewritten for a Linux machine without infringement of the copy-written parts of either DirectX or the Windows OS.
Yes you could write a open source API suit which does the same kind of thing as DirectX for Linux, however without a great knowledge of sound and graphic drivers/support by their manufactures for Linux, it would be a very hard task to do.
However if you question was more of a "Can I get/write something to be able to run games and programs made with DirectX to run on Linux?" in it's meaning, in short no due to more likely copyright infringement.
Your question is wrong in 2019. DXVK is an implementation of D3D11 and D3D10 implementation for Linux using Vulkan and Wine
Also mesa has native Direct3D 9 implementation but in my knowledge, no one uses it except wine
The work to create a port for DirectX would create the same problems as you encounter in WINE. It will never be the same as on Windows. Derived from that principle you would have to look into OpenGL and related multi-platform libraries.
At this very moment WINE would be your closest medium if you want to accomplish something with DirectX code you already have. Then again, I'm not sure in what manner Visual Studio or plain typing would get you closer yo a fluent environment. The WINE libraries aren't that far from native, but there always exists emulation, what is acceptable to a certain point in my honest opinion.
If I look at how superbly Final Fantasy XIV ran on full settings on my iMac (2011) I think it's not that bad to rely on WINE's implementation. The game exactly rendered as on Bootcamp (Windows) on my iMac.
If you really want to make work on this front you could try to ask WINE Devs at their forums or mailing lists how you could maybe use the implementation of their DirectX compatibility only and use that in your project. Where you maybe can call libraries instead of requesting DirectX through Windows-emulated system calls.
Edit: I fully agree with user956030's answer too.
DirectX is a propriety products designed for Microsoft targets, so for this to happen would be very unusual
There are two other pieces of software that come to mind, SDL and OpenGL
OpenGL provides the hardware alternative to DirectX, and SDL works with OpenGL to provide the software support you would expect from DirectX
Since 2020, Microsoft has been working to port DirectX 12 to WSL2, which is being done in this repository. In a mailist, Steve Pronovost commented that they might work to port this driver to Linux to have it running natively.
However, in order to do this, they would need to implement the driver on top of the kernel's DRM stack, which itself needs to be modified if they want to keep the driver similar to the Windows one and have related APIs.
There is some effort being done in the Mesa library as well, to support D3D12 within itself, the main purpose being to optimize server-sided GPU calculations that are heavily used in machine learning algorithms.
Even though this is great, it's unlikely that we will ever see any advantage running Windows games on Linux with that driver. The open source community would only have the upper ground if more games start supporting Vulkan, which is faster and better than DirectX, there had been experiments where using DXVK on Windows would be faster than DirectX 9 or even 11, such as the one mentioned here, but DXVK tends to be less reliable and more prone to errors and crashes.
This is one of my coding projects. I'm fairly new to linux, so I need some pointers and thoughts from you guys, before I get started. I know there exists screen sharing software already, but I want to make my own! (=
Specifically, I want to clone my laptop screen to my TV over WLAN, via a linux box that is connected to a TV through a VGA cable:
Laptop streams it's screen
Linux box reads the stream
Linux box outputs the stream into the TV (through a VGA cable)
First of all, how do I record the screen and send the stream in real time in linux?
Secondly I must write a program that reads the stream being sent. The program must listen to some port, and collect the data being streamed from the laptop. Any thoughts?
Then I must output that data in real time to the TV. Do you how any ideas on how to solve this?
Thanks!
Edit: Regarding programming languages, I'm most comfortable with python.
Sharing your screen can be done via the various flavors of VNC (ie. RealVNC, TightVNC, UltraVNC, etc.). Most of them are Open Source, you might want to:
Stick with the VNC protocol for later compatibility
Take example of how the established solutions does for screen-hooking.
In Linux, the graphics are all processed by Xorg (new version of X Server), which was developed with networking embedded. This explains why you can ssh -X into a machine, execute a graphical interface on it and see it on your remote computer. I recommend you to read about hooks on Xorg to achieve your needs.
You need a client-server topology to achieve your needs. You are not talking about any programming language you forecast to use, though. Some languages may be harder than some to start with. Furthermore, this kind of code is already really well understood under every major programming language. You should try to at least use a framework that simplifies your networking portion of the project.
Sharing a screen on the TV can be done by your video card driver in Linux. Just check on your Desktop Environment (KDE and Gnome offers video configuration panels, for example) or in your video card configuration (nVidia and ATI Linux drivers offers multiple screen support)
It seems to me like you're trying to reinvent the wheel and are not too sure about how to begin. I recommend you to begin simple with one of the already proven VNC software and see how it goes from there. If a feature is missing, you've got the source code of the server and the client, so you can continue development of these projects. Once you've got your setup working, start thinking about replacing a single piece of the puzzle by your own code, and see how it goes.
Do not expect good (full HD, for instance) video quality on your TV without some very capable CPU/GPU and a 802.11n wireless network empty of users and be ready to accept some lag for the codecs to kick in.
You should try to take as small steps as possible. If I were taking up such a project, my first step would be to try to implement a solution using standard unix tools (e.g. netcat or socat for the network part, mplayer or vlc for the playback and maybe ffmpeg for the capture)? Then, replace each component with custom-written ones if needed.
I am working on a browser-based application that is essentially a Digital Audio Workstation. I've seen one out there (at http://www.indabamusic.com), and it looks like it is a Java applet. Is a Java applet the best way to do this?
I've read that C++ is generally more widely used for audio programs, and I've looked at Wt ( http://www.webtoolkit.eu/wt/) for a web interface option but it doesn't seem to be meant for this kind of use (correct me if I'm wrong).
I have almost no experience with C++, so I might just be biased.
You either want a java applet or to do it in Flash (perhaps Flex). It is generally messy to get C++ bits deployed broadly in a browser.
Is it possible to write a mobile application with J2ME and whenever we want to implement a functionality not offered by J2ME call native mobile API ? (kind of like what is done with .NET, whenever you need something not provided, you just call the Win32 API from the .NET platform).
While JNI is not officially supported in J2ME, the Symbian implementation of J2ME obviously uses something very similar.
Since most J2ME APIs have a corresponding Symbian OS C++ API, what Symbian did (a lot) is develop adaptation layers between the 2.
Between that and the new modular architecture of the IBM J9 virtual machine (from Series60 3rd edition feature pack 2 onward), it is technically possible to add a new API to the virtual machine without actually needing to recompile most of a handset firmware.
However, third party developers simply are not allowed to do it without a lot of help from the handset manufacturer.
A loopback socket connection is the solution that people have tipically implemented to transfer data accross the language barrier. Large-scale deployment is not the easiest task, though, as you technically need to install 2 applications on the device. Once you understand the threading model of the Symbian J2ME implementation, you also realize how ugly and inefficient a hack it is.
When you reach that kind of issue, it's time to learn Symbian OS C++ and do it right.
No. That's not possible with Java ME.
Only sort of. Actually it is possible but not directly. You can write a native server that listens on a local socket and send requests to it from your Java MIDlet. Someone even released a framework to do this called MIDP-JNI (as opposed to real JNI). Obviously the form of the solution imposes some limitations on what you can do with it but it isn't actually impossible.
Be aware that using this approach can make your application much more complex to debug.
If you are specifically looking out for Nokia/Symbian, then APIBridge http://wiki.forum.nokia.com/index.php/J2ME_API_Bridge_Interface could be a solution. Typically it is the 'official' version of 'native server on local socket' kind of thing. The APIBridge is 'extensible' & you can write your own 'plugins' for your suitable 'low-level' task.