I want to redirect every post 301 redirect, but I have over 3000 posts.
If I list
Redirect permanent /blog/2010/07/post.html http://new.blog.com/2010/07/23/post/
Redirect permanent /blog/2010/07/post1.html http://new.blog.com/2010/07/24/post1/
Redirect permanent /blog/2010/07/post2.html http://new.blog.com/2010/07/25/post2/
Redirect permanent /blog/2010/07/post3.html http://new.blog.com/2010/07/26/post3/
Redirect per......
for over 3000 url redirect command in .htaccess would this eat my server resource or cause some problem? Im not sure how .htaccess work but if the server is looking at these lists each time user requests for page, I would guess it will be a resource hog.
I can't use RedirectMatch because I added date variable in my new url. Do you have any other suggestions redirecting these posts? Or am I just fine?
Thanks!
I am not an Apache expert, so I cannot speak to whether or not having 3,000 redirects in .htaccess is a problem (though my gut tells me it probably is a bad idea). However, as a simpler solution to your problem, why not use mod_rewrite to do your redirects?
RewriteRule ^/blog/(.+)/(.+)/(.+).html$ http://new.blog.com/$1/$2/$3/ [R=permanent]
This uses a regex to match old URLs and rewrite them to new ones. The [R=permanent] instructs mod_rewrite to issue a 301 with the new URL instead of silently rewriting the request internally.
In your example, it looks like you've added the day of the post to the URL, which does not exist in the old URL. Since you obviously cannot use a regexp to divine the day an arbitrary post was made, this method may not work for you. If you can drop the day from the URL, then you're good to go.
Edit: The first time I read your question, I missed the last paragraph. ("I can't use RedirectMatch because I added date variable in my new url.") In this case, you can use mod_rewrite's RewriteMap to lookup the day component of a post.
You have two options:
Use a hashmap to perform fast lookups in a static file. This means all your old URLs will work, but any new posts cannot be accessed using the old URL scheme.
Use a script to grab the day.
In option one, create a file called posts.txt and put:
/yyyy/mm/pppp dd
...for each post where yyyy is the year of the post, mm is the month, and pppp is the post name (without the .html).
When you're done, run:
$ httxt2dbm -i posts.txt -o posts.map
Then we add to to the server/virtual server config: (Note the path is a filesystem path, not a URL.)
RewriteMap postday dbm:/path/to/file/posts.map
RewriteRule ^/blog/(.+)/(.+)/(.+).html$ http://new.blog.com/$1/$2/${postday:$1/$2/$3}/$3/ [R=permanent]
In option two, use pgm:/path/to/script/lookup.whatever as your RewriteMap. See the mod_rewrite documentation for more info about using a script.
Doing the lookup in mod_rewrite is better than just redirecting to a script which looks up the date and then redirects to the final destination because you should never redirect more than once. Issuing a 301 or 302 incurs a round trip cost, which increases the latency of your page load time.
If you have some way in code to determine the day of a post, you can generate the rewrite on the fly. You can setup a mod_rewrite pattern, something like .html and set up a front controller pattern to calculate the new url from the old and issue the 301 header.
With php as an example:
$_SERVER['REQUEST_URI']
will contain the requested url and
header("Location: http://new.blog.com/$y/$m/$d/$title/",TRUE,301);
will send a redirect.
That's... a lot of redirects. But the first thing I would tell you, and probably the only thing I can tell you without qualification, is that you should run some tests and see what the access times for your blog are like, and also look at the server's CPU and memory usage while you're doing it. If they're fairly low even with that giant list of redirects, you're okay as long as your blog doesn't experience a sudden increase in traffic. (I strongly suspect the 3000 rewrites will be slowing Apache down a lot, though)
That being said, I would second josh's suggestion of replacing the redirects with something dynamic. Like animuson said, if you're willing to drop the day from the URL, it'll be easy to set up a RewriteRule directive to handle the redirection. Otherwise, you could do it with a PHP script, or generally some code in whatever scripting language you (can) use. If you're using one of the popular blog engines, it probably contains code to do this already. Basically you could do something like
RewriteRule .* /blog/index.php
and just let the PHP script sort out which post was requested. It has access to the database so it'll be able to do that, and then you can either display the post directly from the PHP script, or to recover your original redirection behavior, you can send a Location header with the correct URL.
An alternative would be to use RewriteMap, which lets you write a RewriteRule where the target is determined by a program or file of your choice instead of being directly specified in the configuration file. As one option, you can specify a text file that contains the old and new URLs, and Apache will handle searching the file for the appropriate line for any given request. Read the documentation (linked above) for the full details. I will mention that this isn't used very often, and I'm not sure how much faster it would be compared to just having 3000 redirects.
Last tip: Apache can be significantly faster if you're able to move the configuration directives (like Redirect) into the server or virtual host configuration file, and disable reading of .htaccess entirely. I would guess that moving 3000 directives from .htaccess into the virtual host configuration could make your server considerably faster. But even moving the directives into the vhost config file probably wouldn't produce as much of a speedup as using a single RewriteRule.
It's never a good idea to make a massive list of Redirects. A better programming technique is to simply redirect the pages without that date variable then have a small PHP snippet that detects if it's missing and redirects to the URL with it included. The long list looks tacky and slows down Apache because it's checking that URL (any every other URL that might not even be affected by this) against each line. If it were only 5 or so, I'd say fine, but 3,000 is a definite NO.
Although I'm not a big fan of this method, a better choice would be to redirect all those URLs normally using a single match statement, redirecting them to the page without the date part, or with a dash or something, then include a small PHP snippet to check if the date is valid and if not, rewrite the path again to the correctly formed URL.
Honestly, if you didn't have that part there before, you don't need it now, and it will probably just confuse the search engines changing the URL for 3,000 posts. You don't really need a date in the URL, a good title is much more meaningful not only to users, but also to search engines, than a bunch of numbers.
Related
I have these three links:
localhost/my_projects/my_website.php
localhost/my_projects/my_website.html
localhost/my_projects/my_website
The paths of the php and html files are as follows:
C:\xampp\htdocs\my_projects\my_website.php
C:\xampp\htdocs\my_projects\my_website.html
The link without an extension is "artificial" and I want to use said link:
localhost/my_projects/my_website
to get the contents of either of these links:
localhost/my_projects/my_website.php
localhost/my_projects/my_website.html
The reason for the two example files, instead of just one, is that I want to be able to switch between those two files when I edit the htaccess file. Obviously I only want to access one of those files at a time.
What do I need to have in my .htaccess file inside the my_projects folder to accomplish that? How can I make one specific link redirect to another specific link?
After reading your comment clarifying your folder structure I corrected the RewriteRule. (By the way, it would be best if you add that info to the question itself instead of in comments).
The url you want to target is: http://localhost/my_projects/my_website
http:// is the protocol
localhost is your domain (it could also be 127.0.0.1 or a domian name like www.example.com in the Internet)
I assume you are running Apache on port 80, otherwise in the url you need to also specify the port. For port 8086 for example it would be http://localhost:8086/my_projects/my_website.
The real path is htdocs/my_projects/my_website.php or htdocs/my_projects/my_website.html depending on your needs (obviously both won't work at the same time).
Here the my_projects in the "fake" url collides with the real folder "my_projects" so Apache will go for the folder and see there is no my_website (with no extension) document there (it won't reach the rewrite rules).
There is a question in SO that provides a work around for this, but it is not a perfect solution, it has edge cases where the url will still fail or make other urls fail. I had posted it yesterday, but I seem not to find it now.
The simple solution if you have the flexibility for doing it is to change the "fake" url for it not to collide with the real path.
One option is for example to replace the underscores with hyphens.
Then you would access the page as http://localhost/my-projects/my-website if you want to keep a sort of "fake" folder structure in the url. Otherwise you could simply use http://localhost/my-website.
Here are both alternatives:
# This is for the directory not to be shown. You can remove it if you don't mind that happening.
Options -Indexes
RewriteEngine On
#Rule for http://localhost/my-projects/my-website
RewriteRule ^my-projects/my-website(.+)?$ my_projects/my_website.php$1 [NC,L]
#Rule for http://localhost/my-website
RewriteRule ^my-website(.+)?$ my_projects/my_website.php$1 [NC,L]
(Don't use both, just choose one of these two, or use them to adapt it to your needs)
The first part the rewrite rule is the regular expression for your "fake" url, the second part is the relative path of your real folder structure upto the page you want to show.
In the regular expression we capture whatever what we assume to be possible query parameters after .../my_website, and paste it after my_website.php in the second part of the rule (the $1).
Later on if you want to point the url to my_website.html, you have to change the second part of the rule, where it says .php, replace it by .html.
By the way, it is perfectly valid and you'll see it in most SEO friendly web sites to write an url as http://www.somesite.com/some-page-locator, and have a rewrite rule that translates that url to a page on the website, which is what I had written in my first answer.
I'm kind of noob in the world of web so my apologies... I tried many things found on SO and elsewhere, but I didn't manage to do what I want. And the Apache documentation is... well too much complete.
Basically what I want to do is redirect my domain to a subfolder. I found easy solutions for this (many different actually).
http://www.foo.com/
http://foo.com/
should redirect to /bar and appear as http://foo.com/
Using the following I got the expected result :
RewriteEngine on
Options +FollowSymLinks
RewriteCond %{HTTP_HOST} ^www\.foo.com$
RewriteRule ^/?$ "http\:\/\/foo.com" [R=301,L]
RewriteRule ^((?!bar/).*)$ bar/$1 [NC,L]
But I also want the subfolder as well as filenames not to appear when explicitly entered, i.e :
http://www.foo.com/index.html
http://foo.com/index.html
http://wwww.foo.com/bar
http://foo.com/bar
http://wwww.foo.com/bar/index.html
http://foo.com/bar/index.html
Should all appear as
http://foo.com/
Is this possible ?
Obviously using .htaccess, since I'm on a virtual host.
Thanks
As Felipe says, it's not really possible, because you lose information when you do that R=301 redirect: a hard redirect like this starts a whole new request, with no memory of the previous request.
Of course, there are ways to do similar things. The easiest is to put the original request in the query string (here's a good rundown on how mod_rewrite works with query strings). Sure, the query string does show up in the URL, but most modern browsers hide the query string in the address bar, so if your goal is aesthethics, then this method would be workable.
If you really don't want to show any of the original query in the URL, you might use cookies by employing the CO flag (here are some very good examples about cookie manipulation). At any rate, the information about the original request must somehow be passed in the hard redirect.
But anyhow, and most importantly, why would you want to do something like this? It's bound to confuse humans and robots alike. Great many pages behaved like this back when frames were fashionable, and it was pretty terrible (no bookmarking, no easy linking to content, Google results with the snippet "your browser cannot handle frames", no reloading, erratic back button, oh boy, those were the days).
Speaking of which, if your content is html, you may just use a plain old iframe to achieve the effect (but I'd sincerely advise against it).
Short Version:
I wrote the question, and realized most people wouldn't want to read that much text. Consider the below reference, here's the TL;DR:
I need to 301 redirect this url http://app.com/search/foo-bar/
to this url http://app.com/#!/search/foo-bar/
and send this: /foo-bar/, or anything else past /search/ to a server side script. In this case, it's written in php.
Edit for clarity:
Current answers seem to focus on the rewrite to hashbang. That part is not the problem. The problem is that I lose any associated data when rewriting to a hashbang url, as the server side will see app.php as the location, not app.php/#!/foo-bar/ - So I need to capture foo-bar, and send it to the server somewhere other than in the URL. The rewrite works, and is not the issue. Thanks for your answers though!
Long Version:
Ok, so I have an interesting issue that has been tough for me to figure out.
The Scenario:
I have a backbone.js app that uses the hashbang for state:
app.com/#!/search/search-term/key-value/foo-bar/
In addition, I have google traffic coming to the site from the previous version that will be hitting "pretty url" style urls:
app.com/search/search-term/key-value/foo-bar/
I use an .htaccess mod_rewrite to swap the old url out for a hashbanged one if a user hits the legacy url.
I recently introduced a javascript-less bootstrapped version of the site that the site will be built on top of to gracefully downgrade and support crawlers. This is written using php.
For the php site to work, I need to pass in the values past the hashbang to the server side script, so I can figure out what to display.
The Problem:
When I transform a url and add an anchor, everything past the anchor (hashbang) is no longer sent to the request, so I don't have access to it in php.
RewriteRule search/?(.*) #!/search/$1 [R=301,NC,L]
My options for sending things to the server side then are reduced to:
1. Query String
2. Environment Variables
3. Headers
So, I tried sending things via the query string
RewriteRule search/?(.*) #!/search/$1?filter=$1 [R=301,NC,L]
Obviously that didn't work (the query is behind the anchor), so I tried it in front of the hashbang
RewriteRule search/?(.*) ?filter=$1/#!/search/$1 [R=301,NC,L]
That works, but is hideous and redundant to the end user. So, I thought I might try using environment variables.
RewriteRule search/?(.*) /!#/search/$1 [R=301,NC,L,E=FILTER:$1]
That failed, because environment variables aren't preserved through a redirect (duh). I turned to using headers:
RewriteRule search/?(.*) /#!/search/$1 [R=301,NC,L,E=FILTER:$1]
Header set filterParams "%{FILTER}e"
But for some reason, the headers aren't received by the page through the redirect. That seemed to make sense (although I've now stepped well outside of my comfort level with apache directives), so I tried echoing the header, in hopes that it would be passed, received by the second rewrite (that didn't find search), and echoed out.
RewriteRule search/?(.*) /#!/search/$1 [R=301,NC,L,E=FILTER:$1]
Header set filterParams "%{FILTER}e"
Header echo filterParams
Nada - the filter doesn't exist, so although it makes it to the server, it is null. My next thought was to attempt to employ some sort of conditional. Here was my attempt:
RewriteRule search/?(.*) legacy.php/#!/search/$1 [R=301,NC,L,E=FILTER$1]`
<FilesMatch "legacy.php">
Header set filterParams "%{FILTER}e"
</FilesMatch>
Header echo filterParams
That didn't seem to work either, so I'm stumped. I realize that I've spent so long on this that I probably have the solution within my grasp and I'm just tired of looking at it, or it's not even remotely possible, even with gross header hacking.
Anyone have a clue how to to this?
rfc1738.txt says # is not a valid url character
additionally the apache docs says # signals a comment in apache config files.
short answer is your solution is broken not your implementation
AFAIK, there's no good way to preserve variables through redirect without sticking them in the query string...
I figured out how to redirect someone, if they happen across one particular filename:
Redirect /index.php http://www.website.com/#myaccount
As you can see, I'm pretty much redirecting that visitor to the same page, which doesn't work. It's an endless look, regardless of the slight/minuscule change.
I want to force someone to see a part of the page, by adding the hash. (it's a little weird, I know)
I'm guessing this may be a time to use regex in the htaccess file.
But I thought I'd ask if there's a simpler way to do this from the htaccess file.
The fragment of an URI (the part after the first #) is not sent to the server. That means you cannot use server side techniques to test if the fragment has a certain value of does exist at all; only client side techniques like JavaScript can do that.
At my work we have various web pages that, my boss feels, are being ranked lower than they should be because "mywebsite.org/category/" looks like a different URL to search engines than "mywebsite.org/category/index.php" does, even though they show the same file. I don't think it works this way but he's convinced. Maybe I'm wrong though. I have two questions:
How do i make it so that it will say "index.php" in the address bar of all subcategories?
Is this really how pagerank works?
Besides changing all the links everywhere, a simpler solution is to use a rewrite rule. Make sure it is a permanent redirect, or Google will keep using the old link (without index.php). How you do this exactly depends on your web server, but for Apache HTTPd it looks something like the example given below.
Yes. Or so I've heard. Very few people know for sure. But Google mentions this guideline (as "Be consistent"). Make sure to check out all of Google's Webmaster guidelines.
Apache config for rewrite rule:
# in the generic config
LoadModule rewrite_module modules/mod_rewrite.so
# in your virutal host
RewriteEngine On
# redirect everything that ends in a slash to the same, but with index.php added
RewriteRule ^(.*)/$ $1/index.php [R=301,L]
# or the other way around, as suggested
# RewriteRule ^(.*)/index.php$ $1/ [R=301,L]
Adding this code to the top of every page should also work:
<?php
if (substr($_SERVER['REQUEST_URI'], -1) == '/') {
$new_request_uri = $_SERVER['REQUEST_URI'].'index.php';
header('HTTP/1.1 301 Moved Permanently');
header('Location: '.$new_request_uri);
exit;
}
?>
You don't tell us if you're using straight PHP or some other framework, but for PHP, probably you just need to change all the links on your site to "mywebsite.org/category/index.php".
I think it's possible that this does affect your search engine rank. However, you would be better off using only "mywebsite.org/category" rather than adding "index.php" to each one.
Bottom line is that you need to make sure all your links in your website use one or the other. What actually gets shown in the address bar is unimportant.
A simple solution is to put in the <head> tag:
<link rel="canonical" href="http://mywebsite.org/category/" />
Then, no matter which page the search engine ends up on, it will know it is simply a different view of /category/
And for your second question--yes, it can affect your results, if Google thinks you are spamming. If it wasn't, they wouldn't have added support for rel="canonical". Although I wouldn't be surprised if they treat somedir/index.* the same as somedir/
I'm not sure if /category/ and /category/index.php are considered two urls for seo, but there is a good chance that it will effect them, one way or another. There is nothing wrong with making a quick change just to be sure.
A few thoughts:
URLs
Rather than adding /index.php, you will be better off making it so there is no index.php on any of them, since the keyword 'index' is probably not what you want.
You can make a script that will check if the URL of the current page ends in index.php and remove it, then forward to the resulting URL.
For example, on one of my sites, I require the 'www.' for my domain (www.domain.com and domain.com are considered two URLs for search purposes, though not always), so I have a script that checks each page and if there is no www., it ads it, and forwards.
if (APPLICATION_LIVE) {
if ( (strtolower($_SERVER["HTTP_HOST"]) != "www.domain.com") ) {
header("HTTP/1.1 301 Moved Permanently"); // Recognized by search engines and may count the link toward the correct URL...
header("Location: " . 'www.domain.com/'.$_SERVER["REQUEST_URI"] );
exit();
}
}
You could mode that to do what you need.
That way, if a crawler visits the wrong URL, it will be notified that it was replaced with the correct URL. If a person visits the wrong URL, they will be forwarded to the correct URL (most won't notice), and then if they copy the url from the browser to send someone or link to that page, they will end up linking to the correct url for that page.
LINKING URLS
They way other pages link to your pages is more important for seo. Make sure all your in-site links use the proper URL (without /index.php), and that if you have a 'link to this page' feature, it doesn't include the /index.php part. You can't control how everyone links to you, but you can take some control over it, like with the script in item 1.
URL ROUTING
You may also want to consider using some sort of framework or stand-alone URL rerouting scheme. It could make it so there were more keywords, etc.
See here for an example: http://docs.kohanaphp.com/general/routing
I agree with everyone who's saying to ditch the index.php. Please don't force your visitor to type index.php if not typing it could get them the same result.
You didn't say if you're on an IIS or Apache server.
IIS can be set to assume index.php is the default page so that http:// mywebsite.org/ will resolve correctly without including index.php.
I would say that if you want to include the default page and force your users to type the page name in the url, make the page name meaningful to a search engine and to your visitors.
Example:
http://mywebsite.org/teaching-web-scripting.php
is far more descriptive and beneficial for SEO rankings than just
http://mywebsite.org/index.php
Might want to take a look at robots.txt files? Not quite the best solution, but you should be able to implement something workable with them...