Error in htaccess rewrite rule - .htaccess

Question

I am trying to make sure that if HTTPS is used outside of the secure subdomain it gets redirected to HTTP.
This is what I have in the root .htaccess file:
# Redirect HTTPS requests for non-SSL pages back to HTTP. (Note that shared objects
# such as images are excluded from this rule)
RewriteCond %{HTTPS} =on
# my.EXAMPLE.com is the secure subdirectory.
RewriteCond %{HTTP_HOST} !^my.EXAMPLE.com [NC]
RewriteCond $1 !\.(gif|jpe?g|png|ico|css|js)$
RewriteRule ^(.*)$ http://www.EXAMPLE.com/$1 [R=301]
Put simply:
if HTTPS
if not in my.example.com
if NOT an image/css/js file
redirect to HTTP
But this is not working as expected, instead if I try to access a page outside of the my.example.com sub-directory via HTTPS I get a 404 Not Found error. Accessing the same page via HTTP has no problems, it works fine.
Any idea why this rule may not be working?
EDIT
Here's the entire .htaccess file:
# Don't display .htacess files in directory listings.
IndexIgnore .htaccess
Options +FollowSymLinks
RewriteEngine on
# Password protected for now
AuthType Basic
AuthName "EXAMPLE"
AuthUserFile "/home/EXAMPLE/.htpasswds/public_html/passwd"
require valid-user
# Redirect HTTPS requests for non-SSL pages back to HTTP. (Note that shared objects
# such as images on both HTTP and HTTPS pages are excluded from this rule)
RewriteCond %{HTTPS} on
RewriteCond %{HTTP_HOST} !^my\.EXAMPLE\.com$ [NC]
RewriteCond %{REQUEST_FILENAME} !\.(gif|jpe?g|png|ico|css|js)$
RewriteRule ^(.*)$ http://www.EXAMPLE.com/$1 [R=301]
# Redirect non-www requests to www
RewriteCond %{HTTP_HOST} ^EXAMPLE.com$
RewriteCond %{HTTP_HOST} !^my\.EXAMPLE\.com [NC]
RewriteRule ^(.*)$ "http\:\/\/www\.EXAMPLE\.com\/$1" [R=301]
# Prevent direct access to the WHMCS folder must be accessed through secure subdomain
RedirectMatch 301 ^/WHMCS/(.*)$ https://my.EXAMPLE.com/$1
ErrorDocument 404 /404.php

Answer 1

Try this :
RewriteCond %{HTTPS} on
RewriteCond %{HTTP_HOST} !^my\.EXAMPLE\.com$ [NC]
RewriteCond %{REQUEST_FILENAME} !\.(gif|jpe?g|png|ico|css|js)$
RewriteRule ^(.*)$ http://www.EXAMPLE.com/$1 [R=301]

Answer 2

The problem is that if you don't have an SSL vhost setup for www.example.com, and point its document root to the same place that the non-SSL vhost for www.example.com is, when someone goes to https://www.example.com/, your htaccess file is never read. I'm willing to bet that you need to put your HTTPS->HTTP rules in the htaccess file of your SSL vhost (where my.example.com's document root is).
What you have seems to be:
http://www.example.com/ -> /home/EXAMPLE/ (or whatever your document root is)
https://my.example.com/ -> /home/EXAMPLE/subfolder/
So without a vhost setup for https://www.example.com, the htaccess file in /home/EXAMPLE/ is never accessed when you go to the SSL example.com.