Im trying to POST request using Python to retreive a specific File. Since the URL is behind a server with authorized access theres no use posting it here
However the form data contains a field called base64 and lengthy which I cant figure out if its a form data value or base64 encoding of post request
Here are browser parameters
General:
Request URL: http://exampleapi.com/api/Document/Export
Request Method: POST
Status Code: 200 OK
Remote Address: XX.XXX.XXX.XX:XX
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Access-Control-Allow-Origin: http://example.com
Cache-Control: no-cache
Content-Disposition: attachment; filename=location-downloads.xlsx
Content-Length: 7148
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Date: Tue, 23 Jul 2019 21:00:18 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Request Headers :
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 10162
Content-Type: application/x-www-form-urlencoded
Cookie: abcConnection=!UA7tkC3iZCmVNGRUyRpDWARVBWk/lY6SZvgxLlaygsQKk+vuwA1NxvhwE9ph4i+3NZlKeepIfuHhUvyQjl68fhhrT9ueqMx/3mBKUDcT
DNT: 1
Host: exampleapi.com
Origin: http://example.com
Referer: http://example.com/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Form Data:
fileName: location-downloads.xlsx
contentType: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
base64: UEsDBAoAAAAAAAh4904AAAAAAAAAAAAAAAAJAAAAZG9jUHJvcHMvUEsDBAoAAAAIAAh490(shortened for simplicity)
Here is what I tried
url='http://example.com'
urllib3.disable_warnings()
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0",
}
with requests.session() as s:
r=s.get(url,headers={"User-Agent":"Mozilla/5.0"},verify=False)
data=r.content
soup=BeautifulSoup(data,'html.parser')
form_data = {
"fileName":"location-downloads.xlsx",
"contentType":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}
r2=s.post('http://exampleapi.com/api/Document/Export',data=json.dumps(form_data,ensure_ascii=True).encode('utf-8'),headers=headers,verify=False)
print(r2.status_code)
Any idea how i should proceed. My status code also shows 500 for the post here
What could possibly be the reason for expressjs route to contain the following data? I am expecting it to return JSON data. I am making an ajax call to the server(expressjs) which gives me the below data with weird characters. Is this data gzipped? I have set the headers and contentType as follows:
headers: {"Access-Control-Allow-Origin":"*"}
contentType: 'application/json; charset=utf-8'
�=O�0�b��K�)�%7�܈9���G��%NOU���O'6��k�~6��S.���,��/�wأ%6�K�)��e�
The HTTP response is as follows:
General:
Request URL: http://localhost/expressRoute.js
Request Method: GET
Status Code: 200 OK
Remote Address: [::1]:80
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Accept-Ranges: bytes
Connection: Keep-Alive
Content-Length: 29396
Content-Type: application/javascript
Date: Thu, 22 Nov 2018 00:50:36 GMT
ETag: "72d4-57b124e0c372e"
Keep-Alive: timeout=5, max=100
Last-Modified: Tue, 20 Nov 2018 05:57:12 GMT
Server: Apache/2.4.34 (Win32) OpenSSL/1.1.0i PHP/7.2.10
Request Headers:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Host: localhost
Pragma: no-cache
Referer: http://localhost/index.html
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36
When using temporary sessions it works fine. Log into the auth service and calling /auth without any parameters and it shows the display name, session id, etc.
When I log in with RememberMe=true, that call returns the session information properly. But on subsequent calls to /auth without any parameters, ServiceStack returns a 401 not authenticated. The session object's IsAuthenticated property is true and actually exists. My code checks for this and if it's false, forwards the user to the login page which doesn't happen so I know the user really is authenticated.
I am not doing anything different. How can I authenticate with a permanent session and get subsequent calls to /auth to acknowledge that I am logged in?
If it helps I'm using a CustomCredentialsProvider.
Update:
AppHost code:
public override void Configure(Funq.Container container)
{
//Set JSON web services to return idiomatic JSON camelCase properties
ServiceStack.Text.JsConfig.EmitCamelCaseNames = true;
Config.RestrictAllCookiesToDomain = ConfigurationManager.AppSettings["cookieDomain"];
Plugins.Add(new AuthFeature(() => new CustomUserSession(),
new IAuthProvider[] {
new CustomCredentialsProvider()
{ SessionExpiry =
TimeSpan.FromMinutes(Convert.ToDouble(ConfigurationManager.AppSettings["SessionTimeout"]))
},
}) //end IAuthProvider
{
IncludeAssignRoleServices = false,
IncludeRegistrationService = false,
HtmlRedirect = ConfigurationManager.AppSettings["mainSiteLink"] + "Login.aspx"
} //end AuthFeature initializers
);//end plugins.add AuthFeature
Plugins.Add(new PostmanFeature() { EnableSessionExport = true });// this is only for when we want the feature and it's NOT in DebugMode
Plugins.Add(new SwaggerFeature());
Plugins.Add(new CorsFeature(allowedOrigins: "*",
allowedMethods: "GET, POST, PUT, DELETE, OPTIONS",
allowedHeaders: "Content-Type, Authorization, Accept",
allowCredentials: true));
container.Register<IRedisClientsManager>
(c => new PooledRedisClientManager(2, ConfigurationManager.AppSettings["redisIpPort"]));
container.Register<ICacheClient>(c => c.Resolve<IRedisClientsManager>().GetCacheClient());
container.Register<ISessionFactory>(c => new SessionFactory(c.Resolve<ICacheClient>()));
var userRep = new InMemoryAuthRepository();
container.Register<IUserAuthRepository>(userRep);
//Set MVC to use the same Funq IOC as ServiceStack
ControllerBuilder.Current.SetControllerFactory(new FunqControllerFactory(container));
#if DEBUG
Config.DebugMode = true;
typeof(Authenticate).AddAttributes
(
new RestrictAttribute
(RequestAttributes.HttpGet | RequestAttributes.HttpPost)
);
#else
typeof(Authenticate).AddAttributes(new RestrictAttribute(RequestAttributes.HttpPost));
#endif
RegisterTypedRequestFilter<Authenticate>((req, res, dto) =>
{
if (dto.UserName != null && dto.UserName != string.Empty
&& dto.Password != null && dto.Password != string.Empty)
if(dto.RememberMe == null)
dto.RememberMe = false;
});
RegisterTypedResponseFilter<AuthenticateResponse>((req, res, dto) =>
{
var appSettings = new ServiceStack.Configuration.AppSettings();
dto.UserId = AppHostBase.Instance.TryResolve<ICacheClient>().SessionAs<CustomUserSession>().UserId.ToString();
dto.Meta = new Dictionary<string, string>();
dto.Meta.Add("ExpiresMinutes", appSettings.Get("SessionTimeout"));
});
}
public static void Start()
{
Licensing.RegisterLicense(licenceKey);
new ServiceStackAppHost().Init();
}
Initial request headers:
https://****.com/api2/auth?username=user&password=passwordmberme=true
GET /api2/auth?username=user&password=password&rememberme=true HTTP/1.1
Accept: text/html, application/xhtml+xml, /
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: propel.zola360.com
DNT: 1
Connection: Keep-Alive
Cookie: ss-pid=P2hslABCmSs7pomRqNz5; ss-opt=perm; X-UAId=
Initial response headers:
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Powered-By: ServiceStack/4.033 Win32NT/.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, Accept
Access-Control-Allow-Credentials: true
X-AspNet-Version: 4.0.30319
Set-Cookie: ss-id=pojZkNAdMcEcACDREcRM; domain=.zola360.com; path=/; HttpOnly
Set-Cookie: ss-opt=perm; domain=.zola360.com; expires=Mon, 13-Nov-2034 16:11:09 GMT; - path=/; HttpOnly
Set-Cookie: X-UAId=; domain=.zola360.com; expires=Mon, 13-Nov-2034 16:11:09 GMT; path=/; HttpOnly
Set-Cookie: 47=0; domain=.zola360.com; path=/
Set-Cookie: UserId=47; domain=.zola360.com; path=/
X-Powered-By: ASP.NET
Date: Thu, 13 Nov 2014 16:11:09 GMT
Content-Length: 4129
Initial response body:
{"userId":"47","sessionId":"PKrITmRawxAtnaABCDgN","userName":"user","responseStatus":{},"meta":{"ExpiresMinutes":"360"}}
Subsequent call to /auth request:
GET /api2/auth HTTP/1.1
Accept: text/html, application/xhtml+xml, /
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: propel.zola360.com
DNT: 1
Connection: Keep-Alive
Cookie: ss-pid=cvgslABCmSs6pomYdLu0; ss-opt=perm; X-UAId=; ss-id=lYWZkFAdMcZcABCDcRM; 47=0; UserId=47
Subsequent call to /auth response
HTTP/1.1 401 Not Authenticated
Cache-Control: private
Content-Type: text/html
Vary: Accept
Server: Microsoft-IIS/7.5
X-Powered-By: ServiceStack/4.033 Win32NT/.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, Accept
Access-Control-Allow-Credentials: true
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 13 Nov 2014 16:11:23 GMT
Content-Length: 9731
Subsequent call to /auth body:
{"responseStatus":{"errorCode":"Not Authenticated","message":"Not Authenticated","stackTrace":"[Authenticate: 11/13/2014 3:27:49 PM]:\n[REQUEST: {}]\nServiceStack.HttpError: Not Authenticated\r\n at ServiceStack.Auth.AuthenticateService.Post(Authenticate request)\r\n at lambda_method(Closure , Object , Object )\r\n at ServiceStack.Host.ServiceRunner`1.Execute(IRequest request, Object instance, TRequest requestDto)","errors":[]}}
Update
I crafted a small Python3 script to authenticate myself and call some other web service. After authentication using RememberMe=true, the cookies come back as expected: ss-id/pid are set fine and ss-opt=perm. I figured I would print the header cookie and just paste it into a header of another request to call a different service marked with [Authenticate]. It didn't work. So I tried something silly and pasted the ss-pid cookie value into the ss-id one. It worked.
Here's the failing cookie string (session redacted :)):
cookie = "ss-id=ss-ID-session-cookie; domain=.zola360.com; path=/; HttpOnly, ss- pid=ss-PID-session-cookie; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, ss-opt=perm; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, X-UAId=; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, 47=0; domain=.zola360.com; path=/, UserId=47; domain=.zola360.com; path=/"
And simply pasting the ss-pid value into ss-id works:
cookie = "ss-id=ss-PID-session-cookie; domain=.zola360.com; path=/; HttpOnly, ss- pid=ss-PID-session-cookie; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, ss-opt=perm; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, X-UAId=; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, 47=0; domain=.zola360.com; path=/, UserId=47; domain=.zola360.com; path=/"
And the Python3 script I used:
import httplib2 as http
import json
try:
from urlparse import urlparse
except ImportError:
from urllib.parse import urlparse
headers = {
'Accept': 'application/json',
'Content-Type': 'application/json; charset=UTF-8'
}
uri = 'https://mysite.com'
path = '/api2/auth/credentials'
target = urlparse(uri+path)
method = 'POST'
body = '{"username": "username", "password": "password", "RememberMe": "true"}'.encode()
h = http.Http()
response, content = h.request(target.geturl(), method, body, headers)
#save the cookie and use it for subsequent requests
cookie = response['set-cookie']
print(cookie)
path2 = '/api2/time/start'
target2 = urlparse(uri+path2)
headers['cookie'] = cookie
response, content = h.request(target2.geturl(), 'GET', body, headers)
# assume that content is a json reply
# parse content with the json module
data = json.loads(content.decode())
print(data)
It seems that something still looks at the value of ss-id even if ss-opt=perm.
When Authenticating with GET /api2/auth?username=user&password=... it is sent with your permanent cookie ss-pid, i.e:
Cookie: ss-pid=P2hslABCmSs7pomRqNz5; ss-opt=perm; X-UAId=
The rememberme=true option tells ServiceStack to maintain the users session against the permanent ss-pid cookie. This option is maintained in the users ss-opt=perm cookie, which the HTTP Response tells the client to add with:
Set-Cookie: ss-opt=perm; domain=.zola360.com; expires=Mon, 13-Nov-2034 16:11:09 GMT; - path=/; HttpOnly
Although not important in this case, since the temporary session ss-id was missing from the Request, ServiceStack tells the client to add a new one with:
Set-Cookie: ss-id=pojZkNAdMcEcACDREcRM; domain=.zola360.com; path=/; HttpOnly
The issue is with the subsequent request to GET /api2/auth where the client is not re-sending the ss-pid cookie it originally authenticated with (i.e. P2hslABCmSs7pomRqNz5 vs cvgslABCmSs6pomYdLu0):
Cookie: ss-pid=cvgslABCmSs6pomYdLu0; ss-opt=perm; X-UAId=; ss-id=lYWZkFAdMcZcABCDcRM; 47=0; UserId=47
Which ServiceStack doesn't know about (i.e. doesn't maintain any session against) which is why it returns with a 401 Not Authenticated as expected.
HTTP Client should be configured to resend Cookies
It's not clear what HTTP Client you're using but it should be configured to re-send cookies which is normally the default behavior. Ajax will send both the permanent ss-pid cookies and the temporary ss-id only for that browser session, e.g. the temporary ss-id cookie will be discarded when the browser is closed and making a new request will receive a new ss-id cookie.
With the C# Service Clients, it only resends permanent cookies so the client needs to be authenticated with RememberMe = true, e.g:
var client = JsonServiceClient(BaseUrl);
var authResponse = client.Send(new Authenticate
{
provider = "credentials",
UserName = "user",
Password = "p#55word",
RememberMe = true,
});
authResponse.PrintDump();
Once Authenticated the same authenticated client instance can be used to access a protected record multiple times as seen in this Auth Test:
for (int i = 0; i < 500; i++)
{
var response = client.Send<SecureResponse>(new Secured { Name = "test" });
Console.WriteLine("loop : {0}", i);
}
I think this a HTTP-related problem.
I want to use my (JAX-RS) RESTeasy Service on a (JEE6) JBoss AS 7 Server from an Android Device. The RESTeasy Service is working fine. I am using on the Client-Side the Restlet-Client. This works too - without Security.
I want to use my JAAS-Formbased Security for the Pattern /rest/* in web.xml. So I have to send a HTTP-POST-Request with the Form-Data (j_username and j_password) to /foo/j_security_check.
I get the JSESSIONID from the first Response by the Server:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined; Path=/foo
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: JSF/2.0
Content-Type: text/html;charset=utf-8
Content-Length: 1028
Date: Wed, 15 Aug 2012 11:42:59 GMT
For this anonymous session I am authenticating ...
Header:
POST /foo/j_security_check HTTP/1.1
Date: Wed, 15 Aug 2012 11:42:58 GMT
Accept: text/html
Host: 172.24.47.5:8080
User-Agent: Restlet-Framework/2.0.14
Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined
Content-Length: 62
Content-Language: *
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content:
j_username=Bob&j_password=a
... and it works: JBoss-Security-TRACE:
2012-08-15 13:22:26,829 TRACE
[org.jboss.security.auth.spi.DatabaseServerLoginModule]
(http-0.0.0.0-0.0.0.0-8080-4) User 'Bob' authenticated, loginOk=true
Now the Problem: In the following request I want to GET the REST-URL (using the Cookie JSESSIONID):
GET /foo/rest/sync/products HTTP/1.1
Date: Wed, 15 Aug 2012 11:42:59 GMT
Accept: application/json
Host: 172.24.47.5:8080
User-Agent: Restlet-Framework/2.0.14
Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined
Content-Length: 0
But instead of returning the Response with JSON Content, the server is returning the JSF-Login-Page, because it want's me to authenticate again(?):
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: JSF/2.0
Content-Type: text/html;charset=utf-8
Content-Length: 936
Date: Wed, 15 Aug 2012 11:42:59 GMT
<?xml version="1.0" encoding="utf-8"?> ... ... ... </html>
If I login with the Browser and then open the REST-URL it works fine. This is the GET-Request by the Browser:
GET http://localhost:8080/foo/rest/sync/products HTTP/1.1
Host: localhost:8080
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko)
Chrome/19.0.1084.56 Safari/536.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: JSESSIONID=royq26yLd7REOz2otiZdTl6j.undefined
Anyone has an idea? I think the problem lays in the last request (GET /foo/rest/sync/products), because in the Browser it works fine.
Thanks
I have just checked the server headers for my website and this is what i got:
1 Server Response: http://www.pjnsports.co.uk
HTTP/1.1 302 Moved Temporarily
Content-Length: 0
Location: /?6690d3e0
I havent set up any 302 redirects - i assume this will be my host that is doing this. Is this normal practice?? will it have a negative affect on search results, site load speed etc? basically should i be going to them and telling them to do something about it?
Cheers
Paul
I'm not getting any 302 / 301 going to that site:
GET / HTTP/1.1
Host: www.pjnsports.co.uk
Connection: keep-alive
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.107 Safari/534.13
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: PHPSESSID=emr10g0gs9srtjccadb4k7t846; language=en; currency=GBP; __utmz=239376578.1300041169.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=239376578.1365490247.1300041169.1300041169.1300041169.1; __utmc=239376578; __utmb=239376578.1.10.1300041169
HTTP/1.1 200 OK
Date: Sun, 13 Mar 2011 18:33:41 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 6531
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8