We are currently evaluating our next-generation company-wide developer pc-configuration and have noticed something really weird.
Our rather large monolith has - on our current configuration a build time of approx. 4.5 minutes (no test, just compile).
For our next generation configuration we upgraded several components. A moderate increase in frequency and IPC with the processor, doubling the number of CPU cores and a switch from a small SATA SSD towards a NVMe SSD rated at >3GBps. Also, the next generation configuration switches from Windows 7 to Windows 10.
When executing the first tests, we noticed an almost identical build time (4.3 Minutes), which was a lot less improvement than we expected.
During our experiments we tried at one point to run the build process from within a virtual Linux machine running on the windows host. On the old configuration (Windows7) we saw a drop in build times from 4.5 to ~3.7 Minutes, on the Windows 10 Host, we saw a decrease from 4.3 to 2.3 minutes. We have ruled out things like virus scan.
We were rather astonished with these results and have tried to find another explanation than some almost-religious and insulting statements about different operation systems.
So the question is: What could we have possibly done wrong in configuring the Windows machine such that the speed is almost half of a Linux running virtualized in the very same windows host? Especially as all the hardware advancements seem to be eaten up by the switch from windows 7 to 10.
Another question is: How can we ace the javac process use up more cores, because right now, using Hotspot JDK 8 we can see at most two cores really used by the build. I've read about sjavac but that seems a rather experimental feature only available to OpenJDK9 onward, right?
After almost a year in experimenting we came to the conclusion, that it is indeed NTFS which is the evil-doer. If you have a ntfs user-partition with a linux host, you get somewhat similar results compared to an all-windows-setup.
We did benchmarks of gradle-build, eclipse internal build, starting up wildfly and running database-centered tests on multiple devices. All our benchmarks showed consistently a speedup of at least 100% when switching from Windows to Linux (sometimes, Windows takes 3x the amount of time in real world benchmarks than Linux, some artificial benchmarks had a speedup of 60!). Especially on notebooks we experienced much less noise, as the combined processor load of a complete build is substantial less than with windows.
Our conclusion was, to switch from Windows to Linux over the course of the last year.
Regarding the parallelisation thing, we realized, it was some form of code-entanglement. Resolving this helped gradle and javac to parallelise the build a lot (also have a look into gradle-composite-builds)
Related
I have two Linux server, and MATLAB is installed independently on each server.
One day, we have found that running many MALTAB jobs using SLURM on one machine(1) somehow slows down MATLAB on the other machine(2).
First, I thought it might be caused by SLURM, so I turned off SLURM on second machine. However, it did not help. I have checked CPU usage, but CPU usage for second machine has not changed while first machine is running. The two servers don't share memory or CPU. They do have some shared mounted directories, but I'm not sure if it can reduce speed of MATLAB this much.
Would there be any idea about what is going on and how I can examine this?
Thank you.
Added: There are a bit of updates. I have checked bench for MATLAB of second machine again with even more jobs running on first machine. The 2-D and 3-D worktime greatly increased on second machine. I hope this helps to find out the reason.
Default
After many MATLAB jobs are running on first machine
Our Dynamics CRM Solution is quite large, and it takes between 20 to 25 minutes to complete an import on our servers.
While testing the import process on a Dynamics CRM hosting provider, the import took significantly less time, around 8 minutes.
In an attempt to see if hardware can improve the import speed, I've setup a virtual machine with Dynamics CRM in VMware Workstation 8. The VM is on an SSD, 4 cores (from a 6 core i7-3930K), and has 12 GB RAM. It still took around 20 minutes. I tried SQL 2008 R2 SP2 and SQL 2012 with no noticeable difference.
How can I improve the import speed software-wise? Is there any information available that goes into detail on what the import process does, so we can optimize around those variables?
The solution contains 60+ entities, customizations to 40+ system entities, plugins, ribbon buttons, sitemap changes, processing steps, and a few hundred web resources. It's currently a little over 6MB.
Also, how can I know what hardware component is the biggest bottleneck for the import process? Perfmon showed the SSD idling away most of the import process, RAM was at 6.5 GB, only the processor showed relatively higher use, but not more than 30%-40%. Or, is VMware Workstation itself the bottleneck, and dedicated hardware, or ESX/Hyper-V, will improve this?
Even though it may be four cores, the import process itself is running on a single thread, so that might explain the relatively low CPU use - one core (plus a bit of another) is working really hard, which equals 30-40% total CPU. My bet is that you would see similar times (maybe a bit longer, but not much) even if you just gave the VM one core.
Your real problem here is that your solution is just too damn big. Break it into components. Test to see if you can narrow down the performance issues to certain customizations. I'd start removing all Web Resources, for example, and see how that effects the import.
Another approach would be to split it into two solutions - one that is "stable" and another that is under active development. Then you'd be importing a smaller solution more frequently.
You're guaranteed to not get any worse performance on real hardware. But since your VM isn't using everything that is has, you should be able to improve performance without going to that step just yet. Have you tired changing the IIS Settings in your VM?
I know that it is a pretty vague question but I was hoping to get some ideas about where to look as it is a little puzzling to me.
I have a web app that computes some value and returns it to the client (EJB remote calls). When I call my localhost from a main() test looping 10 times, it comes back within about 100 milliseconds. When I call the DEV machine following the same process, it is sometimes fast and sometimes really slow, like 4 seconds, which is a huge difference.
The weird thing is that my localhost is a 32 bit 1GB Jboss config but my DEV machine is a 64 bit 6GB Jboss config so if anything, I would expect my localhost to hang... not the the DEV machine.
Where would you suggest starting the troubleshooting process?
If I understood right, both calls are made from same computer? If that is the case, network between is much more likely source for response time differences than 32 vs. 64 bit.
If that is not the case, then monitor dev and check what is the difference in context (other applications etc.) between "fast" and "4 seconds" cases. Anyway, most likely difference in response times have nothing to with difference between 32 bit / 64 bit.
Some time ago I worked on application which was deployed on JBoss on two servers with exactly the same hardware configuration. The first server had CentOS and the second FreeBSD. Exactly the same hardware, the same network, similar load. From what I observed, application responses when it was running on FreeBSD was about 1.5 - 2 times faster. On the first sight, it was strange for me, but after week of tests differences in response times was confirmed.
Since that time I do not consider hardware configuration as so important as I thought before ;)
We resolved the issue after finding out that the install on the linux machine actually had two different instances of JBoss running on the VM therefore resulting in unpredictable behavior. The resources that were consumed were enormous, which did not make any sense based on the app that was deployed...
I am trying to compare how an i7 dual core 2.7Ghz would perform vs. an i7 quad core 2.0Ghz in a multitasking environment. The quad core scores at around 9000 while the dual comes in at around 7500 (for Geekbench). At the same time, Geekbench explicity specifies that the tests show the full performance potential of all the cores. However, in real world, everyday use, almost none of the application I would be running are multi-threaded (Ruby runtime, Java IDE, Windows VM on mac, app server).
This machine would server as a web development machine. Which cpu would be most "snappy" in terms of response time in this use case?
Results of a benchmark have any practical meaning only if the benchmark very closely approximates your typical workload.
You should consider whether your typical development environment regularly calls for parallelism. For example, if I develop a C/C++/Java app it's common that a header file (or Java source) change to cause several other files to be recompiled and a few binaries to be relinked - that's a highly parallel workload and many-core CPU may prove advantageous.
On the other hand, if I'm changing a few Python or Javascript sources, I doubt I will create any parallel workload when I try to execute and test the changes.
However, these are theoretical considerations.
I don't think the speed of the machine is a bottleneck in any development effort. The human is.
How does IcedTea 6's performance stand up against Sun's own HotSpot on linux systems? I tried searching Google but Phoronix's test is the best I got, which is almost a year old now. Hopefully things have improved since then.
Also, once Sun completely open sources the JVM, would it be possible to implement it for Linux platforms such that a main module (Quickstarter in the Consumer JRE) starts up with the OS and loads the minimal Java kernel, regardless of any Java apps running. And then progressively load other modules as necessary. Might improve startup times.
so it will be within the answer: http://www.phoronix.com/scan.php?page=article&item=java_vm_performance&num=1 and http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=1
I'd expect SUN's stuff to be faster, but it really depends on all kinds of optimizations, so one version might be faster doing operation X, but in the next version it might not be as fast..
EDIT:
regarding kernel preloading: on linux you may use preload or alternatives to speed up app loading, without affecting the overall system performance (loading a Quickstarter equivalent will keep memory occupied at all times). Also, as far as i know, java loads lots of shared libraries, that are shared between apps, so i don't really see the point of building in-kernel support for this thing. I guess its easy to make a simple app that loads some libraries and does nothing after that(quickstarter), but i dont see this doing a big difference when loading apps, and in some cases it might even slow down the system(i'm thinking about ram usage, and memory swapping)