Sorry for the rather long post.
I need some input regarding a project that I am going to undertake.
I am trying to make an application that collects kernel debugging information from a guest Linux OS, located inside a VmWare Virtual Machine, and send them to a host OS efficiently.
So far, I have found a similar project, but written for Windows[1].
The author of the project wrote a DLL that is loaded into memory, and replaces the implementation of the KdSendPacket and KdReceivePacket functions, to use the VmWare GuestRpc[2] mechanism, instead of the slow serial port.
The data are then send to a debugging application on the host(Kd or WinDbg) trough a named pipe.
The author claims that there is a speed-up up to 45%, by avoiding the serial port transmission.
I am trying to achieve something similar ,but for Linux, and try to make the debugging process a little faster, than using the serial port.
My concrete questions are :
Do any similar applications exist?
I didn't manage to find any.
Would such an application be worth it ,comparing its functionality to netconsole[3], for example?
What method of intercepting printk messages would you suggest ?
Is there an equivalent of KdSendPacket/KdReceivePacket on Linux ?
[1]. http://virtualkd.sysprogs.org/dox/operation.html
[2]. http://articles.sysprogs.org/kdvmware/guestrpc.shtml
[3]. http://www.kernel.org/doc/Documentation/networking/netconsole.txt
Using the serial port is really suboptimal.. even the (virtual) network would be preferable to that, but getting back to host-guest IPC channels, VMware's VMCI comes to mind.
many approaches can use to achieve your goal, below methods can be applied if network is connected:
use syslog service and transfer log though network to your server:
syslogd, syslogng seems support sending log to a log server with some filter critiera.
directly call tcp/udp socket functions in your kernel module to sends your collected data back to server.
other approaches, you may write application on host machine that calls hypervisor's share memory access function to read the memory buffer of your kernel module. However, the xen/kvm hypervisor both support these apis and i am not sure about weather vmware have this kind of library.
Related
I'm posing a question here directly in relation to this issue on github for node-serialport. In a nutshell something that used to work fine in v4.x of the library no longer works in v6.x of the library. I think it must have something to do with how the library is opening the COM port (options or something), and I suspect its artificially limiting the power delivered over USB in the current version of the library.
I wrote the simplest scripts that I could to reproduce the problem (scripts posted in the issue) using:
NodeJS and v4.x of the library [works]
NodeJS and v6.x of the library [fails]
Python and PySerial equivalent [works]
Following through on a recommendation by the repository maintainer, I researched and found a utility for windows called drstrace that allowed me to capture logs of the execution of each of these scripts executing for a period of time (these logs are posted as attachments in the referenced issue).
Now I'm stuck, as I don't know how to make heads or tails of the drstrace logs, though I feel confident that the difference is probably evident in comparing the three files. I just don't know enough about how to read the drstrace logs and windows drivers and system calls to break through.
I realized posting this question here is something of an act of desperation, but I figure it's worth a shot. Hopefully it is clear that I've not lacked in effort pursuing this on my own, I'm just over my head at this point, and could use help getting further. Any guidance would be appreciated. Most awesome would be someone who is versed in this level of diagnostics giving it a look and reading the tea leaves. It would be great to contribute back to such an important open source library.
Update 2017 Nov 10
I reached out to FTDI support asking:
I use the FT231X in many of my products. I need some help with
understanding how the Windows FTDI driver manages power. More to the
point, I'm hoping you can help me understand how to direct the driver
to allow the full 500mA allowed by USB to be delivered to my product
by a Windows computer.
The reply was:
Just use our FT_Prog utility to set the max VBUS current to 500
mA:
This drive current becomes available after the FT231X enumerates.
I haven't tried this advice yet, but I wanted to share it with anyone reading this. The fact remains that node-serialport 6.0.4 behavior differs from both node-serialport 4.0.7 behavior and pyserial behavior.
Here is an alternative theory you could look into:
Windows interacting with V6.x might be interacting with the flow control settings differently, which might be causing your device to respond with an unexpected state causing your test to fail.
I Read a bit more about windows drivers and how they manage that i only found out that its related to the hardware manufacturer i think its not a fail from serialport it self since its really using the drivers it self it adds no extras on that level.
i am Contributor of SerialPort and can tell you that it offers only bindings for the Operating System to node that means it don't does any actions it offers you only a API read the following from microsoft they say you should ask your hardware vendor
Power Management in Serial Port Drivers (Windows CE 5.0)
Windows CE 5.0
Send Feedback
The minimum power management that a serial port driver can provide is to put the serial port hardware into its lowest power consumption state with the HWPowerOff function, and to turn the serial port hardware fully back on with the HWPowerOn function. Both of these functions are implemented in the lower layer. Beyond this minimal processing, a serial port driver can conserve power more effectively by keeping the port powered down unless an application has opened the serial port. If there is no need for the driver to detect docking events for removable serial port devices, the driver can go one step further and remove power from the serial port's universal asynchronous receiver-transmitter (UART) chip, if no applications are using the port.
Most serial port hardware can support reading the port's input lines even without supplying power to the serial line driver. Consult the documentation for your serial port hardware to determine what parts of the serial port circuitry can be selectively powered on and off, and what parts must be powered for various conditions of use.
Source:
https://msdn.microsoft.com/en-us/library/aa447559.aspx
about changes from serialport v4 => 6
new Stream Interface
but nothing changed with the core opening method of the port.
also nothing changed in the bindings which open the port
node serialport is a collection of bindings written in c++
I am wondering if it is possible to write a program on Windows that communicates with a program within a Linux Virtualbox on the same machine. If this is possible, what is the best approach to doing this? Is there a way to do this without using the internet to communicate?
I found instructions showing how you could potentially use SSH, but I have never tried doing this before, so I do not know if using SSH to communicate would be the best option.
I was going to put this as a comment to a very vague question, but then it got too long.
It depends what you mean by "communicate"....
If the Windows machine should start a program on the Linux VM, you probably want plink.exe - see here.
If you want to transfer whole files, you probably want scp or FTP or FileZilla - see here.
If you want to send small messages occasionally, maybe netcat, also known as nc - see Netcat Cheatsheet here.
If you want full-on, high speed, continuous messages, maybe sockets or some messaging protocol like mqtt.
If you want to share data structures, like lists, queues or sets, you could allow both Windows and the Linux machine to access a shared Redis database - see here.
Or maybe it is enough to share a filesystem between the two machines - in which case you can make a Shared Folder in VirtualBox on your host and the VM can just mount that and read/write it. See diagram:
I'm searching for help and an opinion-advice for a network project, in which I'm working lately. This requires a Linux machine to be a passive network appliance.
Network packets come in from one network interface and come out from another interface ( net--eth0-->Linux PC--eth1-->net) without making any modifications on data.
The application, which is going to run on the Linux system, will change only the order of the packets. It is going to be a "silly" network emulator application.
The first implementation was made with RAW sockets, where read() is called every time a packet arrives to user space and write() is called when an Ethernet packet should be sent down to the NIC.
I would like to know if there is a more practical and direct way than RAW sockets, bypassing Linux's network stack.
If what you want is to bypass the kernel, DPDK in Linux and NetMap in FreeBSD are options to do just that.
Indeed this can be done in dpdk in Linux. There are l3fw and l2fwd sample applications in the examples folder of the dpdk tree, which may inspire you. Also consider using vpp, a fd.io project hosted by Linux Foundation, which can use dpdk.
Rami Rosen
I'm trying to run some network program (like nginx) on RISC-V.
I'm not aware of network devices available on RISC-V, and I don't want to implement it myself, so I'm looking for the other way.
First I tried running simple program that send HTTP request to the given address. It resolves the IP address using gethostbyname and sends a HTTP request.
I successfully ran riscv-linux on spike simulator, and compiled the program and ran it. gethostbyname returns Unknown server error, and when I use IP address directly, connect returns Network is unreachable error.
I found that the front end server fesvr can handle system calls that is forwarded from the RISC-V processor, and thought maybe it will handle network related system calls. I also found fesvr-eth and thought maybe it is something related to handling network services, but according to this link it is just for connecting fesvr running on PC to the development board by ethernet. Also fesvr-eth has been removed from the latest git repo.
So now I want to look and risc-v linux and see how it actually handle network system calls like connect. I'm thinking maybe if the network operations are only done inside localhost, it can be handled properly without a network device. Or there can be easy way to extend fesvr to handle network services.
Summarizing the questions:
is there a way to run network applications in RISC-V, that I missed?
can network services be actually handled if the request is from within the same host, even if there is no network devices?
any other comments or references that can be helpful?
At the moment, neither spike nor riscv-qemu provide models for network interfaces. With both of these tools, you will be able to use the loopback device (127.0.0.1) for networking among processes. Loopback networking is implemented directly by the OS kernel.
For other architectures, qemu provides a large number of networking options. It should not be too difficult to modify riscv-qemu/hw/riscv/riscv_board.c to instantiate one of these virtual network interfaces into an emulated risc-v system.
i am newbie to Linux platform, i am working on java technology.
what i have to do is : Having a program that running on mobile devices,that sends some data to my Linux machine, now i have to create a program in java that
listen to a particular port.
access data comes on that port(which is sending by mobile device)
save that data to the database.
response back to the mobile device.
i.e. i would make my Linux system as server that can listen from many clients(mobile devices), but not getting how to configure this environment... :(
i used cent OS 5.4 and
installed jdk1.6.0_24
any help would be appreciated.....
thanx in advance!
khushi
One of Java's greatest strengths is that you can pretty much ignore the host operating system as long as you stick to core Java features. In the case you're describing, you should be able to accomplish everything by simply using the standard Java networking APIs and either the JDBC to access an existing, external database or you could choose any number of embedded Java databases such as Derby. For your stated use case, that you'll be running the application on Linux is pretty much irrelevant (which should be good news... you don't need to learn a whole operating system in addition to writing your app ;-).
Here's a nice client/server tutorial, in that it is broken into steps, and adds each new concept in another step.
Here's another client/server tutorial with much more detail.
I would write it to accept one connection at a time. Once that works, I would study the new(ish) java.lang.concurrent classes, in particular the ExecutorService, as a way of managing the worker bee handling each connection. Then change your program to handle multiple connections using those classes. Breaking it up in two steps like that will be a lot easier.