I have my website available to public and there is Scrapyd running at port 6800 like http://website.com:6800/
I do not want anyone to see list of my crawlers. I know anyone can easily guess type up port 6800 and can see whats going on.
I have few questions, answer any of them will help me.
Is there way to password protect Scrapyd UI?
Can I password protect a specific Port on Linux? I know it can be done with IPTables to ONLY ALLOW PARTICULAR IPs but thats not a good solution
Should I make changes to Scrapyd's source-code?
Can I password protect a specific port only via .htaccess?
You should bind address of the machine that is going to make calls.
If its the localhost which is going to make calls to the endpoints just bind it to 127.0.0.1 and voila, the address doesn't work for external ips.
Use the latest version of scrapyd (1.2.1 when am wrting this). Scrapyd support Basic HTTP Auth. Inorder to enable it just add username and password to scrapyd.conf as below
pip install git+https://github.com/scrapy/scrapyd.git
[scrapyd]
eggs_dir = /var/lib/scrapyd/eggs
...
username = username_here
password = password_here
...
As of scrapyd version 1.2.0 the default bind address is 127.0.0.1
To add a password protection use this gist which uses nginx as a reverse proxy to add basic authentication to scrapyd.
You may also check scrapyd-authenticated repository.
Related
yesterday I installed gitlab on a vm of mine and configured everything to work with it.
Gitlab listens on port 8081 of my domain (e.g. domain:8081).
I have an apache instance which listens to port 80 and 443, so I did a forward there (e.g. domain/git).
Everything worked fine (except the css theme of domain/git, but thats no problem), but then I changed the root url (I think, I don't know how this settings is called) in the admin section directly in gitlab to http://domain/git to let gitlab show me directly this url if I want to copy a URL to clone.
Now I can't access my gitlab instance, because I do have an redirection loop.
I also can't find where the setting was done by gitlab itself, I guess it's stored in the database and not any file.
Can someone help me figure out how to change this particular configuration back to default?
Thanks in advance!
You likely changed the 'homepage url' used for redirecting logged out users. Instead of hitting the domain mainpage, hit /users/sign_in and you should be able to sign back in as your admin user. Go to the admin section, and clear out the setting.
You instead need to go into your config/gitlab.yml (source install) or /etc/gitlab/gitlab.rb (package install) and set the external_url to be the address you wanted.
Then restart/reconfigure the app to have it used in the git clone instructions.
In OpenCart v. 2.0.0 in /admin when I receive an order I should be able to change status of the order to let the customer know what is going on with her purchase. This functionality is in 'History' tab.
Unfortunately I can't change or modify any orders.
When I change the status and try to save it Openart gives me an error:
SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data OK
It happens even on clean install. Everything is set up correctly - the domain name is OK, the shop is not in maintenance mode, there is no password in .htaccess file (BTW even removing it doesn't help).
It looks like in the screenshot attached.
The one solution is to install the newest version of OpenCart which is in the moment I'm writing this post 2.0.1.1. It's probably the best idea.
My problem is that I have made quite a lot of modifications and that would be very difficult for me. Making my changes I didn't use vqmod which probably I should use (but nobody is perfect - isn't she?).
Or maybe you don't want to go into upgrade for a different reason?
Now, how to resolve this problem without actually changing the software itself?
I got the same error and I fixed it:
Go to Admin -> System -> Users -> API
Add a new API generate password, and enable it
Then go to Admin -> System -> Settings -> Edit -> Option(tab)
Select API user under Checkout section as your API user
Then save the changes
Then go to Sales -> Orders -> Edit
I guess that the error comes when a cURL method in your OC tries to get data from the OC API and gets blocked.
-- Quick Testing
Goto function info() in /admin/controller/sale/order.php
find the curl initialization inside info() and aim on $json = curl_exec($curl);
add after that:
if($json === false){
var_dump(curl_error($curl));
exit;
}
if there is an error, you should see it when you go to an order info from your admin panel.
You may have a refused connection msg on 80 or 443 port.
If yes, possible errors and solutions may be:
You are working in a protected directory.
This is a website message (like js alert with fields) that asks for user/pass.
You may turn it off and try again to add a history record.
In Plesk you can find it as "Password Protected Directory" in domain options.
If you recently modify your .htaccess, you should check it for auth statements causing the above symptoms.
PHP; Check if you have added any $_SERVER['REMOTE_ADDR'] into an if block to reject ips on purpose.
When using Plesk, have root access (ssh), and you have the same website domain as the Plesk admin panel hostname,
(assume that your domain is www.yourdomain.com)
you can check from your ssh, executing curl -vv "http://www.yourdomain.com" (or https) inside of your own server (localhost) which your domain is hosted.
If you get a refused connection on port 80, or 443 for https and have a success return msg on curl -vv "http://www.yourdomain.com:7080" OR curl -vv "http://www.yourdomain.com:7081" (virtualhost ports) then you can fix it as below. It is also strange to get a successfull connection msg on 80 & 443 when try it from other server!
So, you have to modify your server hosts (/etc/hosts) and add right to your public ip your full website domain (last line usually).
So, add to the public-ip line xxx.xxx.xxx.xxx www.yourdomain.com. You may have other hosts defined right on your public ip. Please, do not remove any. Add www.yourdomain.com in the last of the piblic-ip line. You can check again with curl, or adding an order history.
I may help,
giannisepp
After some hard time I managed to solve this problem and I want to share this solution with you:
In the admin panel of your shop go to:
Admin > Settings > Users > API
There should be one user with name like
XrpeYEWrFHOcqB1phjBXdUCRO1A3sCvDpgmTGBcJ7G6WuYIMKXCrIJUpzvFPfimWT6LHQLisTYz0nuOy7ZK
if there is not create one and give her reasonably complicated name like in the example.
Then you have to check out your database (using phpMyAdmin helps a lot) where keeps your store data.
find the api table which contains the API user and check the api_id of this user.
find table setting (by default OpenCart instalation it's named oc_setting) and find key config_api_id.
Set the value field to the same number as the api_id you had found in the api table.
Problem should be solved.
In my OpenCart 2.0.0 installation the value was set to 0 while the api_id was 2.
I had the same issue and tried all workarounds possible and imaginable.
In the end gave up and called my hosting provider. They found that OpenCart wont run properly on any php version above 5.4.33. Ergo: use php 5.4.33
This might not solve all issues, or might work only in conjunction with other fixes bove, but itÄs worth a try to check which php version your hosting is running by default.
If your server does not support HTTPS, please modify the https to http.
Open the admin/config.php file, modify the https to http.
define('HTTPS_SERVER', 'http://'.$_SERVER['HTTP_HOST'].'/admin/');
define('HTTPS_CATALOG', 'http://'.$_SERVER['HTTP_HOST'].'/');
Disabling maintenance mode did it for me.
In admin/controller/sale/order.php file line below 2438 you should see like this:
if ($store_info) {
$url = $store_info['ssl'];
} else {
$url = HTTPS_CATALOG;
}
As you can see CURL is allways use HTTPS_CATALOG defined constant which contain your shop SSL URL. Why? I don't now.
The solution: edit your config.php on shop root folder and change HTTPS_CATALOG defined constant value to non-ssl URL, simply delete "s" after end of https.
I have installed Jenkins Continues Integration system on my windows server successfully and it works without any errors. But I was unable to get Jenkins URL working with my host domain. The default Jenkins address which is http://localhost:8080 works well. My domain/server name is projectdev so I want to give Jenkins the http://projectdev/jenkins URL so that other developers in my network will be able to access Jenkins dashboard easily.
Although I added http://projectdev/jenkins as the Jenkins URL from the Jenkins configuration sections, it doesn’t work. I can’t access it from other computers in my network. But when I use http://localhost:8080 I can access the dashboard directly.
I also tried to add Jenkins as a web application on IIS and give it the address I want. But I don’t know what to provide as the physical path of Jenkins as it was installed using Jenkins.jar file.
It would be really great if someone can help me to get this done as I want.
Thank you.
You have to configure IIS to re-route requests to {domain}/jenkins to {domain}:8080/jenkins
check the info on:
http://www.iis.net/learn/web-hosting/microsoft-web-farm-framework-20-for-iis-7/setting-up-a-server-farm-with-the-web-farm-framework-20-for-iis
Try this address: http://projectdev:8080/jenkins If you type without 8080 port , the port will be 80 by default.
ok so i can access the dash board from the server by going to http://localhost/ccnet
what i need to do is to be able to view this page from another computer but typing that url in the browser. if any one knows how to do this please help
thanx
Put instead your localhost the IP address of server where you have ccnet
for example it would look something like this: http://192.168.1.66/ccnet
Of course I typed some default IP you need to check what's yours.
Also check if firewall is not blocking access.
You can use the server name to connect. http://ServerName/ccnet.
I'm just fooling around, learning the ins and outs of web development and I'd like to make my domain private until I'm ready to display it.
What's the best way to do this?
BTW, I have an Apache server on Debian.
use an .htaccess file to create a user/pass prompt (be warned that this can be bruteforced but it'll keep the general public out)
here's a good article on how to go about implementing this: http://www.freewebmasterhelp.com/tutorials/htaccess/3
You can do one of two things that I can think of:
Put it on a internal network that has no public access.
Require a password to see the the site and don't give it to anyone.
Other alternatives:
Set up a firewall rule to only allow your IP address(es) access to the site.
Use a subdomain that you don't tell anybody about (this is similar to the password and hidden directory options).
Quick and easy way would be to tell your computer's firewall to block port 80 (the port webservers use).
Also, depending on your webserver, you could configure it to only serve on localhost.
In Apache, for example, I believe you can edit the httpd.conf file and add Listen 127.0.0.1:80.
Put it in a hidden directory and have the toplevel site be an empty page. Unless someone knows or guesses the directory name, no one will see your actual site.