SignedXml.CheckSignature(key) always throws error - c#-4.0

Question

I am new to C#.NET programming. With help of online references, I have written the code below to verify signature of a SAML assertion (generated by a server API).
My Env:
VS 2010 Ver4.0
Win XP SP3
SAML Assertion token looks something like this:*
<saml:Assertion xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" Version="2.0" ID="idGYEv...USb8GfnqF" IssueInstant="2012-12-05T14:13:39.00Z">
<saml:Issuer Format="urn:oasis:names:tc:SAML:2.0:nameid-format:entity">http://noszti...xyz.com</saml:Issuer>
<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
<SignedInfo>
<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
<Reference URI="">
<Transforms>
<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
</Transforms>
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
<DigestValue>uDeAjOE/iCa6Pfz5oOjaOMtAQe4=</DigestValue>
</Reference>
</SignedInfo>
<SignatureValue>IGjZX...LaEMzA=</SignatureValue>
<KeyInfo>
<X509Data>
<X509Certificate>MIIE...cg6A==</X509Certificate>
<X509SubjectName>emailAddress=xmlsec#aleksey.com,CN=Aleksey Sanin,OU=Test Root Certificate,O=XML Security Library (http://www.aleksey.com/xmlsec),ST=California,C=US</X509SubjectName>
</X509Data>
</KeyInfo>
</Signature>
<saml:Subject>
<saml:NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified">admin</saml:NameID>
<saml:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">
<saml:SubjectConfirmationData NotOnOrAfter="2012-12-05T14:19:39.00Z" Recipient=""/>
</saml:SubjectConfirmation>
</saml:Subject>
<saml:Conditions NotBefore="2012-12-05T14:13:39.00Z" NotOnOrAfter="2012-12-05T14:19:39.00Z">
<saml:AudienceRestriction>
<saml:Audience/>
</saml:AudienceRestriction>
</saml:Conditions>
<saml:AuthnStatement AuthnInstant="2012-12-05T14:13:39.00Z">
<saml:AuthnContext>
<saml:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:Password</saml:AuthnContextClassRef>
</saml:AuthnContext>
</saml:AuthnStatement>
</saml:Assertion>
My code snippet to verify signature:
//Load the SAMLAssertionToken in XML Document
XmlDocument xDoc = new XmlDocument();
xDoc.PreserveWhitespace = false;
xDoc.LoadXml(SAMLAssertionToken); //SAMLAssertionToken above
//Retrieve the public key from certificate available to end user
X509Certificate2 X509Cert = new X509Certificate2("D:/Schemas/X509Certificate.cer");
RSACryptoServiceProvider rsaKey = (RSACryptoServiceProvider)X509Cert.PublicKey.Key;
//Signature Verification Starts. Find the Signature element
XmlNamespaceManager xMan = new XmlNamespaceManager(xDoc.NameTable);
xMan.AddNamespace("ns", "urn:oasis:names:tc:SAML:2.0:assertion");
xMan.AddNamespace("ns1", "http://www.w3.org/2000/09/xmldsig#");
XmlElement SigElm = (XmlElement)xDoc.SelectSingleNode("//ns:Assertion//ns1:Signature", xMan);
//Create SignedXml object and load signature for verification
SignedXml sig = new SignedXml(xDoc);
sig.LoadXml(SigElm);
bool verified = sig.CheckSignature(rsaKey);
if (verified)
{
Console.WriteLine("Signature verified successfully");
}
else
{
Console.WriteLine("Signature not valid");
}
On running the code, it throws an error "SignatureDescription could not be created for the signature algorithm supplied."
on line: bool verified = sig.CheckSignature(rsaKey);
On debugging, Signature is correctly assigned to SigElm.
Note: the certificate retrieved from "X509Certificate.cer" is exactly the same as that displayed in element of the signature (in SAML Assertion). So it looks like a valid and matching certificate. The certificate in SAML assertion token is signed with a private key; so I am using the public key from "X509Certificate.cer" (certificate available to end users) to verify the signature (in SAML assertion).
I also tried to verify the signature using:
bool verified = sig.CheckSignature(X509Cert, true);
but it throws the same error.
I tried few approaches (using online references for this error) but unable to figure out the problem.
Pls advise...