Our hosting provider has set an instance of the Pimcore demo package as the basis for our website. There are two problems which may or may not be linked: excessive resource usage when visiting the home page (resulting in the provider limiting/denying access) and FFmpeg ("Sorry message" or thumbnail displays but video does not load). We'd like to solve both problems but access is the main one.
The providers are asking for more information without saying what they want, so I am wondering if the sequence of error log entries in this file rings any bells with you.
System cache write lock?
Thank you
Ian Russell
first of all, Mysqli is not supported anymore by Pimcore, please use PDO instead (https://github.com/pimcore/pimcore/issues/1236). This should take care of error messages you have in your error log.
Secondly, use Redis for caching, default mysql cache can be resource-heavy. You can read how to enable Redis here:
https://www.pimcore.org/docs/latest/Development_Tools_and_Details/Cache/Custom_Cache_Backends.html
In the case of ffmpeg - didn't it get answered in this google group post already? In short - make sure ffmpeg version is correct and Pimcore can find it's executable.
https://groups.google.com/forum/#!topic/pimcore/WxZ9OqbKf4k
If you can't get your hosting provider to update ffmpeg for whole system, try to install it in your home directory and point Pimcore to your ffmpeg binary.
Hope this helps,
kubaplas
Related
I am playing with node.js and angular for first time. I use shared server hosting space. I am trying to get some node.js tests running.
CPanel seem to provide interface to deploy node applications. Example:
application url: myurl.com
application root: node-hello-world
application startup file: app.js
This seems to create directory and some artifacts in
/home/myurl/nodevenv/node-hello-world/6/bin
I have (limited?) shell access through Cpanel emulation, however I get error on source command.
source activate
got error: error: jailshell: fork: Cannot allocate memory
Does this mean node.js is installed and ready to run? Do I have to upload project as well? Where to? Trying to find more info on process of deploying to this type of server if possible.
sorry for nooby question.
Googling your error message, I came across this thread -- which admittedly is very old, but it's from cPanel and has the following comment from an administrator at the time:
Jailshell is a constrained environment by design. It is not meant to be a replacement for a full-featured, unrestricted, shell environment, such as is provided by Bash. If your user's need such full-featured environments then perhaps they need full shell access, or another method whereby they can accomplish their goal.
That answer was given in 2006 (yes, 13 years ago) but I have to imagine the spirit of that response is still true.
To be perfectly honest, I'd be afraid to use any shared hosting provider that would give you more than very limited shell to use -- it opens the door to many security vulnerabilities, and if multiple customers are in the same runtime (i.e. shared hosting) it could be catastrophic. Maybe your host does allow this, or maybe what you're describing isn't actually the same thing I'm referring to... you didn't offer a lot of details on this point.
Back to your question: Does this mean node.js is installed and ready to run? Do I have to upload project as well? Where to?
If I had to guess, Node probably isn't installed (it isn't in most shared hosting providers) -- but I can't say for sure based on the information you provided. My recommendation would be to call their customer support. Or pay for a dedicated hosting account where you get root access. Or just use something like Heroku.
I'm trying to deploy a nodejs application to google cloud, I've posted several SO questions about the errors I've been getting and how to fix them. Let me illustrate my endeavours...
problem 1) I tried following the tutorial, the hello world app deployment. When I entered the command to deploy,
$ gcloud preview app deploy app.yaml --set-default
It gave me an error saying managed vms aren't available for your application's region. I contacted someone from Google, a technical support representative from google, said his name was Ling, and he said he "wasn't aware managed vms weren't supported in India" and he wanted me to point out where I found that information. So I sent him this, note one of the answers. OK.....
problem 2) So, I was stuck, didn't really know what to do. Searched "how to deploy nodejs app on google cloud" and found a video in which a google developer informed me that there are "two main ways to run your application in the cloud, infrastructure as a service and platform as a service". Running it on app engine was the platform as a service, and that was off the list so I went with my only remaining option. There is a tutorial, the bookshelf app tutorial, also by google, and the last part describes how to run your application on the compute engine. I tried to go through the tutorial, but when I entered the command
git push cloud 7-gce:master
to push my code to the repository, it's asking me for my username and password, It never mentioned that in the tutorial, I don't know what to enter. I found this question but I'm not sure it applies to my situation. So next I went through some tutorials suggested by Zachary Newman, and I wasn't able to follow the tutorial where they use the ubuntu because when they tell you to hello.js file, it's really weird because I had to ssh into the ubuntu image vm I created and everything was really slow, I'm not familiar with commands on linux so I had to look up how to "save and exit" after creating the hello.js file. I got the "web server" to run, it's the reverse proxy nginx server, but since I don't know what the APP_PRIVATE_IP_ADDRESS is, when I accessed the page, I could see the nginx page but not the "hello world" message that's supposed to display. Please see the Prerequisites heading to see what the tutorial was trying to teach me. OK....
problem 3) On the google developers console, once you log in, there is a navigation section on the left, there is a link to the "Cloud Launcher". In there I found a link to create a Virtual Machine with nodejs installed, it's created by Bitnami. I clicked it and my vm was running, I went to the external ip address provided, and saw a Congratulations (ugh...) page by Bitnami. I didn't really know what to do from there, so I posted another SO question. A bitnami developer told me I "need to copy your files to the remote repository, create the configuration files and after that you have to include that configuration in the configuration of Apache to serve the application". How?!?! He just posted a link as well to a bitnami wiki page. I used filezilla to connect to my vm, and I tried to upload my application files to the vm. I was able to upload everything except the node modules (they were taking too long to upload and for some reason the ftp client kept loosing the connection with the server after 10 minutes or so of uploading). So I thought I would just ssh into the instance and try to npm install my dependencies. But it gave me more errors...
problem 4) I don't know why this post was put on hold, but I found this. I can't find the exact page right now but I read somewhere in the google docs that we can't ssh into an instance as root/administrator by design, and that we have to perform some advanced configuration in order to be able to do this. THE PROBLEM is that I can't install the dependencies unless I ssh in as an administrator....:'( So I just uploaded my node modules with filezilla. Everything looks good to go, I ssh into my instance and I cd into my application directory and enter the command
node app.js
and almost as if to taunt me, it console logs out "connected to mongo...." and "Listening on port 3000.....". I get so excited, I think it's working.....
problem 4) The server never really started running I think, it wasn't able to connect to the mongodb database (another instance) that I was running, this was the command to connect to the mongodb instance in my app.js file
mongoose.connect('mongodb://****/myappdatabase');
where the **** indicates the external IP of my mongodb instances. After about 5 minutes, I get an error on the ssh browser
Error: connect ETIMEDOUT ****:27017
BUT WHO HAS TIME TO READ ALL OF THIS RIGHT?
I'm not a developer, I don't have much experience with programming (close to zero actually). I'm a hobbyist. I picked up a book on c programming by stephen g. kochan about 5 or 6 months ago, and since then, I've just been following tutorials on the internet, a programming in java book, and some reference material. I wanted to create a website.
I learned some html, css, javascript and jquery off of w3schools and went through some tutorials on creating a basic node server and serving some html pages, that's literally all my application does. That's all I want it to do.
I used the express framework for node, and mongoose to connect to my mongodb running locally. Socket.io for a little bit of networking. I created the website for fun and I would really like to host it and share it with my friends, I made it this far, it would be really embarrassing to quit now because I can't host the website. I went through a lot more than what I posted on this question by the way.
Now I would really appreciate some help, even some guidance as to what I need to know, maybe books I should read or documentation I should go through to become more familiar with website hosting would be great. I am going to offer this question up for a bounty in 2 days but until then, if there is anyone that went through this and would like to help for no personal gain of their own, your saving my life. I've been going at this for about 4 days with no luck. Thanks and let me know if there is anything you would like to know about my application. I mean seriously, how hard could it be, I'm doing something (or everything) terribly wrong.
Wow dude. I thought about flagging this, but you appear to genuinely want to learn to program and need help. And experienced coders teaching novice coders is what this site is all about. So, here we go.
First and foremost, Linux is your friend. Learn it. Use it. I don’t use Ubuntu, but you should. Use 14.04 because it’s a forgiving disto and comes cram-packed with things you need when you’re just learning.
Time to address your questions:
Problem 1: you are using a gcloud preview product. Those are awesome, but not for beginners because you don’t know what you’re doing. Just stand up an Ubuntu instance on an n1-standard and rock it. No bells. No whistles. Learn Ubuntu linux with the same passion that you taught yourself how to program a web site.
Problem 2: Don’t use git. I use git every day and it’s awesome, but you have a long way to go and git is hella confusing. Take your workspace, tar/gzip it and just ssh the whole shebang to cloud. Then unzip it and you’ll have your whole code right there on your server. Caveat: never do this professionally. But for someone just learning … small bites dude … small bites.
Problem 3: Don’t use Bitnami pre-built images. It robs you of the opportunity learning how to do this from scratch and that is an invaluable experience. Here are two commands you must learn:
apt-get update
apt-get install <insert thing you want to install>
That’s it. Not rocket science and you will discover the power of package management on linux.
Problem 4: You are over-thinking this. For real, learn apt-get. It will get you so far, you’ll curse yourself for not using it earlier. apt-get node. apt-get mongo. And slam it all on one instance. Is that what I do professionally? Hell no. But it’s all about baby steps when you combine dev-ops with programming and you need to take those steps.
Errata: You wanted someone to recommend a book or a website or guidance. All you need to know is already on Ubuntu. They are called man pages. It’s short for manual. You can buy all the books in the world off Amazon and watch infinite youtube pages. It won’t compare to simply maning commands. You gotta trust me on this.
In summary: Just create an Ubuntu instance. tar and gzip all your code and scp it to your instance. Learn apt-get and for real, you are set. You will totally destroy your instance many times. So just delete it, re-create it, and keep starting over. Each time, you will be wiser and smarter than the time before. I tutor novice developers all the time. You gotta start small and work your way up. Eventually you’ll understand why you’d want to use git. Why you want to scale. Why LXC is amazing. Why GCE’s managed instance and VMs are crazy baller, and why scripting your deployment is a best practice. But first you have to understand how to build an instance and get your code there in any way possible. You’ll grow from there. Good luck, buddy.
For the first time, I am writing a web service that will call upon external programs to process requests in batch. The front-end will accept file uploads and then place them in a queue. The workers on the backend will take that file, run it through ffmpeg and the rest of my pipeline, and send an email when the process is complete.
I have my backend process working on my computer (Ubuntu 10.04). The question is: should I try to re-create that pipeline using binaries that I've compiled from scratch? Or is it okay to use apt when configuring in The Real World?
Not all hosting services uses Ubuntu, and not all give me root access. (I haven't chosen a host yet.) However, they will let me upload binaries to execute, and many give me shell access with gcc.
Usually this would be a no-brainier and I'd compile it all from scratch. But doing so - not to mention trying to figure out how to create a platform-independent .tar.gz binary - will be quite a task which ultimately doesn't really help me ship my product.
Do you have any thoughts on the best way to set up my stack so that I'm not tied to a specific hosting provider? Should I try creating my own .deb, which contains Ubuntu's version of ffmpeg (and other tools) with the configurations I need?
Short of a setup where I manage my own servers/VMs (which may very well be what I have to do), how might I accomplish this?
The question is: should I try to re-create that pipeline using binaries that I've compiled from scratch? Or is it okay to use apt when configuring in The Real World?
It is in reverse: it is not okay to deploy unpackaged in The Real World IMHO
and not all give me root access
How would you be deploying a .deb without root access. Chroot jails?
But doing so - not to mention trying to figure out how to create a platform-independent .tar.gz binary - will be quite a task which ultimately doesn't really help me ship my product.
+1 You answer you own question. Don't meddle unless you have to.
Do you have any thoughts on the best way to set up my stack so that I'm not tied to a specific hosting provider?
Only depend on wellpackaged standard libs (such as ffmpeg). Otherwise include them in your own deployment. This problem isn't too hard too solve for 10s of thousand Linux applications over decades now, so it would probably be feasible for you too.
Out of the box:
Look at rightscale and other cloud providers/agents that have specialized images/tool chains especially for video encoding.
A 'regular' VPS provider (with Xen or Virtuozzo) will not normally be happy with these kinds of workload, but EC2, Rackspace and their lot will be absolutely fine with that.
In general, I wouldn't believe that a cloud infrastructure provider that doesn't grant root access will allow for computationally intensive workloads. $0.02
How come I always get
"GConf Error: Failed to contact configuration server; some possible causes are that you need to enable TCP/IP networking for ORBit, or you have stale NFS locks due to a system crash. See http://projects.gnome.org/gconf/ for information. (Details - 1: Failed to get connection to session: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.)"
when I start 'gedit' from a shell from my superuser account?
I've been using GUI apps as a logged-in user and as a secondary user for 15+ years on various UNIX machines. There's plenty of good reasons to do so (remote shell, testing of configuration files, running multiple sessions of programs that only allow one instance per user, etc).
There's a bug at launchpad that explains how to eliminate this message by setting the following environment variable.
export DBUS_SESSION_BUS_ADDRESS=""
The technical answer is that gedit is a Gtk+/Gnome program, and expects to find a current gconf session for its configuration. But running it as a separate user who isn't logged in on the desktop, you don't find it. So it spits out a warning, telling you. The failure should be benign though, and the editor will still run.
The real answer is: don't do that. You don't want to be running GUI apps as anything but the logged-in user, in general. And you never want to be running any GUI app as root, ever.
For some (RHEL, CentOS) you may need to install the dbus-x11 package ...
sudo yum install dbus-x11
Additional details here.
Setting and exporting DBUS_SESSION_BUS_ADDRESS to "" fixed the problem for me. I only had to do this once and the problem was permanently solved. However, if you have a problem with your umask setting, as I did, then the GUI applications you are trying to run may not be able to properly create the directories and files they need to function correctly.
I suggest creating (or, have created) a new user account solely for test purposes. Then you can see if you still have the problem when logged in to the new user account.
I ran into this issue myself on several different servers. It I tried all of the suggestions listed here: made sure ~/.dbus had proper ownership, service messagbus restart, etc.
I turns out that my ~/.dbus was mode 755 and the problem went away when I changed the mode to 700. I found this when comparing known working servers with servers showing this error.
I understand there are several different answers to this problem, as I have been trying to solve this for 3 days.
The one that worked for me was to
rm -r .gconf
rm -r .gconfd
in my home directory. Hope this helps somebody.
We received access to the environment, but I now need to go through the process of securing it so that the previous vendor can no longer access it, or the Web applications running on it. This is a Linux box running Ubuntu. I know I need to change the following passwords:
SSH
FTP
MySQL
Control Panel Admin
Primary Application Admin
However, how do I really know I've completely secured the system using best practices, and am I missing anything else that I need to do other than just changing passwords?
3 simple steps
Backup configurations / source files from HTTP / SQL tables
Reinstall operating system
Follow standard hardening steps on fresh OS
Regardless of who it was, they could have installed any old crap on there (rootkits) that you can't configure away.
You will probably get more responses at serverfault.com on these kinds of questions.
There are several things you can do to secure SSH by editing your sshd_config file which is usually in /etc/ssh/:
Disable Root Logins
PermitRootLogin no
Change the ssh port from Port 22
Port 9222
Manually specifying which accounts can login
AllowUsers Andrew,Jane,Doe
SecurityFocus has a good article about securing MySQL, although it's a bit dated.
The best thing you could do would be reinstall and make sure when you bring over files from the old system to the new that it is just data, and not executables that could be nasty. If this is to much, changing all the passwords, and watching the logs for a few weeks, as well as playing with iptables to block former vendor. Also given that it could have a rootkit at the kernel level its probably good idea to change that out, and also watch traffic coming out of the box fro something that might be going to the vendor. It really is a hassle to take someone else's machine and say that is safe now, I would go as far to say it is nearly impossible.
side note. This isn't really programming related so probably shouldn't be on this site.