Retrofit 2 having problem sending #FormUrlEncoded #Field no parameter is passed.
#FormUrlEncoded
#POST("GetRechargeDetail")
Call <List<IglGatewayOrderDetail>> Fetch_OrderDetail_IGL(#Field("Bpno") String Bpno);
D/OkHttp: <-- 500 https://test.com/test/test.asmx/GetRechargeDetail (188ms)
cache-control: private
D/OkHttp: content-type: text/plain; charset=utf-8
server: Microsoft-IIS/10.0
x-aspnet-version: 4.0.30319
x-powered-by: ASP.NET
x-powered-by-plesk: PleskWin
access-control-allow-origin: *
date: Sat, 23 Apr 2022 11:21:18 GMT
content-length: 396
D/OkHttp: System.InvalidOperationException: Missing parameter: Bpno.
at System.Web.Services.Protocols.ValueCollectionParameterReader.Read(NameValueCollection collection)
at System.Web.Services.Protocols.UrlParameterReader.Read(HttpRequest request)
at System.Web.Services.Protocols.HttpServerProtocol.ReadParameters()
at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
<-- END HTTP (396-byte body)
earlier everything was working fine if I remove FormUrlEncoded and change Field to Query it works but in some API i need URL encoded so i cannot remove FormUrlEncoded.
On the command line, I can do a request like: curl -I -H "Fastly-Debug: 1"
and it will return a lot of helpful information from the CDN serving that URL, in this case, Fastly:
cache-control: public, max-age=0, must-revalidate
last-modified: Tue, 20 Apr 2021 21:17:46 GMT
etag: "4c5cb3eb0ddb001584dad329b8727a9a"
content-type: text/html
server: AmazonS3
surrogate-key: /v3.6/tutorial/nav/alerts-and-monitoring/
accept-ranges: bytes
date: Fri, 30 Apr 2021 20:50:15 GMT
via: 1.1 varnish
age: 0
fastly-debug-path: (D cache-lga21923-LGA 1619815815) (F cache-lga21940-LGA 1619815815)
fastly-debug-ttl: (M cache-lga21923-LGA - - 0)
fastly-debug-digest: 04c3e527819b6a877de6577f7461e132b97665100a63ca8f667d87d049092233
x-served-by: cache-lga21923-LGA
x-cache: MISS
x-cache-hits: 0
x-timer: S1619815815.944515,VS0,VE136
vary: Accept-Encoding
content-length: 65489
How can I do this in node?
This is my attempt:
const headers = {
'Fastly-Key': environment.getFastlyToken(),
'Accept': 'application/json',
'Content-Type': 'application/json',
'Fastly-Debug': 1
};
async retrieveSurrogateKey(url) {
console.log("this is the url: ", url)
try {
request({
method: `GET`,
url: url,
headers: headers,
}, function(err, response, body) {
if (err){
console.trace(err)
}
console.log(request.headers)
})
} catch (error) {
console.log("error in retrieval: ", error)
}
}
Is there a way for me to pass in the -I and -H flags?
The -H (header) flag allows you to specify a custom header in cURL. Your code already does this - great! All that's left is emulating the -I (head) flag.
From cURL manpage for the -I option:
Fetch the headers only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document.
To use the HEAD method, you need to specify it instead of GET:
method: `HEAD`
And finally, the headers returned by the server in the response can be obtained from response.headers.
Etag is supported in SDK: https://sap.github.io/cloud-sdk/docs/java/features/odata/use-typed-odata-v4-client-in-sap-cloud-sdk-for-java/#handling-of-etags
So experimenting it by using BusinessPartner entity in S/4 EX.
But seems there's no If-Match header:
How come the header doesn't show up - any prerequisite with etag?
(entering on behalf of the implementation partner team)
I checked the VersionIdentifier of the response and it was not set to a value.
I also checked the response's JSON __metadeta and header, but there were no values that appeared to correspond to the ETag value.
[Code]
BusinessPartner bp1 = new DefaultBusinessPartnerService().getBusinessPartnerByKey(bpId).execute(dest);
log.debug("get 1: {}", bp1);
log.debug("get 1 VersionIdentifier: {}", bp1.getVersionIdentifier());
bp1.setOrganizationBPName1("SCP Update 1st:" + System.currentTimeMillis());
ODataUpdateResult result1 = new DefaultBusinessPartnerService().updateBusinessPartner(bp1).execute(dest);
log.debug("Update1 Http Status: {}", result1.getHttpStatusCode());
bp1.setOrganizationBPName1("SCP Update 2nd:" + System.currentTimeMillis());
bp1.setVersionIdentifier("dummy");
ODataUpdateResult result2 = new DefaultBusinessPartnerService().updateBusinessPartner(bp1).execute(dest);
log.debug("Update2 Http Status: {}", result2.getHttpStatusCode());
[Log]
get 1: BusinessPartner(super=VdmObject(customFields={}, changedOriginal...
get 1 VersionIdentifier: None
Update1 Http Status: 204
Update2 Http Status: 204
[GET Response JSON(__metadata) / Response Header]
(It has masked the IP address.)
"__metadata": {
"id": "https://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner('1000001')",
"uri": "https://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner('1000001')",
"type": "API_BUSINESS_PARTNER.A_BusinessPartnerType"
},
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 3152
dataserviceversion: 2.0
sap-metadata-last-modified: Thu, 14 May 2020 23:58:07 GMT
cache-control: no-store, no-cache
sap-processing-info: ODataBEP=,crp=,RAL=,st=,MedCacheHub=SHM,codeployed=X,softstate=
sap-server: true
sap-perf-fesrec: 243070.000000
I tried setting the VersionIdentifier to a meaningless value in my test code (2nd update).
The update process seems to be successful, although the request header now has "If-Match" added to it.
(I was expecting the update to fail because the values never match, so I was hoping the update would fail.)
[2nd Update(setVersionIdenfifier)]
(It has masked some of the values.)
PATCH http://xxx.xxx.xxx.xxx:xxxxxx/sap/opu/odata/sap/API_BUSINESS_PARTNER/A_BusinessPartner(BusinessPartner='1000001') HTTP/1.1
x-csrf-token: xxx
Content-Type: application/json
Accept: application/json
If-Match: dummy
Authorization: Basic xxx
SAP-Connectivity-SCC-Location_ID: xxx
Proxy-Authorization: Bearer xxx
sap-language: en
sap-client: xxx
Content-Length: 55
If ETags are not part of OData service responses, then you should approach the IT/administrators who maintain the S/4 backend. The SAP Cloud SDK is only consuming the OData service. Unfortunately it can't leverage ETag support if it's disabled.
Basically after an auth, I setting a cookie, but apparently after page refresh on the cookie that was set by cloudflare is saved
And the cookie that I transmitted with set-cookie is not used in after set-cookie requests
# Response headers
HTTP/2.0 200 OK
date: Thu, 18 Jul 2019 10:03:25 GMT
content-type: application/json; charset=utf-8
content-length: 29
set-cookie: __cfduid=d578c7a5e4378dc1b1946964a08ebc4ec1563444205; expires=Fri, 17-Jul-20 10:03:25 GMT; path=/; domain=.doc.io; HttpOnly; Secure
set-cookie: __doc=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlIjoiNzQ2NTczNzQ0MDY2NjE3Mzc0NmQ2MTY5NmMyZTZkNzgiLCJwIjoiNTUzMjQ2NzM2NDQ3NTY2YjU4MzEzODMzNTU0NjUwNTU2ZjRiMzkzMTY3NDUzNDY5NDc3MzM3MzgzOTU5MzczMDUxNjk0ZjQxNjQ0OTM5Nzg0YjZiNzU1Njc3Nzk0NDc0NjE3NDMxNTE0NzcwMzE0YjQxNmY1MjU5MzM3YTZhNDU2NDJiNmU0ZTc0NGE3NTMyNTQ1ODc2NjI1YTczNDc1MTQ1Njc0MjVhNGQ0MTNkM2QiLCJkIjoiMzEzNTM2MzMzNDM0MzQzMjMwMzUzNTM2MzMiLCJpYXQiOjE1NjM0NDQyMDV9.go1jDpc2rBe5FjK2sKX4ybW4PhCPFq1xT1WIX-mSI84; Domain=.doc.io; Path=/; Expires=Thu, 18 Jul 2019 16:03:25 GMT; HttpOnly; Secure
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 4f83a02a3dc36455-FRA
X-Firefox-Spdy: h2
reply
.code(200)
.header('Access-Control-Allow-Origin', '*')
.header('Content-Type', 'application/json; charset=utf-8')
.setCookie('__doc', token, {
domain: '.doc.io',
path: '/',
secure: true,
httpOnly: true,
expires: new Date(new Date().setHours(new Date().getHours() + 6))})
.send({ 'success': 'Sign In success' })
All my websites are https
First I do POST request for an auth on /auth, and you could see response in response headers above and after I do GET on (trying to load page) from /page and get cookies, but with reply.log.info(request.cookies) I see only cookies from cloudflare. Surely I tried to refresh and go to address in different table, there just no any cookies, but from cloudflare.
# Request headers
Host: test.doc.io
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
DNT: 1
Connection: keep-alive
Cookie: __cfduid=d578c7a5e4378dc1b1946964a08ebc4ec1563444205
Upgrade-Insecure-Requests: 1
TE: Trailers
After I tested it with an actual page with js code for a XHR request it works fine. However cookie was only displayed, but not actually saved in the storage when I was sending request directly in Firefox's inspector. Lost a day to figure out that created in inspector cookies seems preventing from installing
When using temporary sessions it works fine. Log into the auth service and calling /auth without any parameters and it shows the display name, session id, etc.
When I log in with RememberMe=true, that call returns the session information properly. But on subsequent calls to /auth without any parameters, ServiceStack returns a 401 not authenticated. The session object's IsAuthenticated property is true and actually exists. My code checks for this and if it's false, forwards the user to the login page which doesn't happen so I know the user really is authenticated.
I am not doing anything different. How can I authenticate with a permanent session and get subsequent calls to /auth to acknowledge that I am logged in?
If it helps I'm using a CustomCredentialsProvider.
Update:
AppHost code:
public override void Configure(Funq.Container container)
{
//Set JSON web services to return idiomatic JSON camelCase properties
ServiceStack.Text.JsConfig.EmitCamelCaseNames = true;
Config.RestrictAllCookiesToDomain = ConfigurationManager.AppSettings["cookieDomain"];
Plugins.Add(new AuthFeature(() => new CustomUserSession(),
new IAuthProvider[] {
new CustomCredentialsProvider()
{ SessionExpiry =
TimeSpan.FromMinutes(Convert.ToDouble(ConfigurationManager.AppSettings["SessionTimeout"]))
},
}) //end IAuthProvider
{
IncludeAssignRoleServices = false,
IncludeRegistrationService = false,
HtmlRedirect = ConfigurationManager.AppSettings["mainSiteLink"] + "Login.aspx"
} //end AuthFeature initializers
);//end plugins.add AuthFeature
Plugins.Add(new PostmanFeature() { EnableSessionExport = true });// this is only for when we want the feature and it's NOT in DebugMode
Plugins.Add(new SwaggerFeature());
Plugins.Add(new CorsFeature(allowedOrigins: "*",
allowedMethods: "GET, POST, PUT, DELETE, OPTIONS",
allowedHeaders: "Content-Type, Authorization, Accept",
allowCredentials: true));
container.Register<IRedisClientsManager>
(c => new PooledRedisClientManager(2, ConfigurationManager.AppSettings["redisIpPort"]));
container.Register<ICacheClient>(c => c.Resolve<IRedisClientsManager>().GetCacheClient());
container.Register<ISessionFactory>(c => new SessionFactory(c.Resolve<ICacheClient>()));
var userRep = new InMemoryAuthRepository();
container.Register<IUserAuthRepository>(userRep);
//Set MVC to use the same Funq IOC as ServiceStack
ControllerBuilder.Current.SetControllerFactory(new FunqControllerFactory(container));
#if DEBUG
Config.DebugMode = true;
typeof(Authenticate).AddAttributes
(
new RestrictAttribute
(RequestAttributes.HttpGet | RequestAttributes.HttpPost)
);
#else
typeof(Authenticate).AddAttributes(new RestrictAttribute(RequestAttributes.HttpPost));
#endif
RegisterTypedRequestFilter<Authenticate>((req, res, dto) =>
{
if (dto.UserName != null && dto.UserName != string.Empty
&& dto.Password != null && dto.Password != string.Empty)
if(dto.RememberMe == null)
dto.RememberMe = false;
});
RegisterTypedResponseFilter<AuthenticateResponse>((req, res, dto) =>
{
var appSettings = new ServiceStack.Configuration.AppSettings();
dto.UserId = AppHostBase.Instance.TryResolve<ICacheClient>().SessionAs<CustomUserSession>().UserId.ToString();
dto.Meta = new Dictionary<string, string>();
dto.Meta.Add("ExpiresMinutes", appSettings.Get("SessionTimeout"));
});
}
public static void Start()
{
Licensing.RegisterLicense(licenceKey);
new ServiceStackAppHost().Init();
}
Initial request headers:
https://****.com/api2/auth?username=user&password=passwordmberme=true
GET /api2/auth?username=user&password=password&rememberme=true HTTP/1.1
Accept: text/html, application/xhtml+xml, /
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: propel.zola360.com
DNT: 1
Connection: Keep-Alive
Cookie: ss-pid=P2hslABCmSs7pomRqNz5; ss-opt=perm; X-UAId=
Initial response headers:
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-Powered-By: ServiceStack/4.033 Win32NT/.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, Accept
Access-Control-Allow-Credentials: true
X-AspNet-Version: 4.0.30319
Set-Cookie: ss-id=pojZkNAdMcEcACDREcRM; domain=.zola360.com; path=/; HttpOnly
Set-Cookie: ss-opt=perm; domain=.zola360.com; expires=Mon, 13-Nov-2034 16:11:09 GMT; - path=/; HttpOnly
Set-Cookie: X-UAId=; domain=.zola360.com; expires=Mon, 13-Nov-2034 16:11:09 GMT; path=/; HttpOnly
Set-Cookie: 47=0; domain=.zola360.com; path=/
Set-Cookie: UserId=47; domain=.zola360.com; path=/
X-Powered-By: ASP.NET
Date: Thu, 13 Nov 2014 16:11:09 GMT
Content-Length: 4129
Initial response body:
{"userId":"47","sessionId":"PKrITmRawxAtnaABCDgN","userName":"user","responseStatus":{},"meta":{"ExpiresMinutes":"360"}}
Subsequent call to /auth request:
GET /api2/auth HTTP/1.1
Accept: text/html, application/xhtml+xml, /
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: propel.zola360.com
DNT: 1
Connection: Keep-Alive
Cookie: ss-pid=cvgslABCmSs6pomYdLu0; ss-opt=perm; X-UAId=; ss-id=lYWZkFAdMcZcABCDcRM; 47=0; UserId=47
Subsequent call to /auth response
HTTP/1.1 401 Not Authenticated
Cache-Control: private
Content-Type: text/html
Vary: Accept
Server: Microsoft-IIS/7.5
X-Powered-By: ServiceStack/4.033 Win32NT/.NET
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, Accept
Access-Control-Allow-Credentials: true
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 13 Nov 2014 16:11:23 GMT
Content-Length: 9731
Subsequent call to /auth body:
{"responseStatus":{"errorCode":"Not Authenticated","message":"Not Authenticated","stackTrace":"[Authenticate: 11/13/2014 3:27:49 PM]:\n[REQUEST: {}]\nServiceStack.HttpError: Not Authenticated\r\n at ServiceStack.Auth.AuthenticateService.Post(Authenticate request)\r\n at lambda_method(Closure , Object , Object )\r\n at ServiceStack.Host.ServiceRunner`1.Execute(IRequest request, Object instance, TRequest requestDto)","errors":[]}}
Update
I crafted a small Python3 script to authenticate myself and call some other web service. After authentication using RememberMe=true, the cookies come back as expected: ss-id/pid are set fine and ss-opt=perm. I figured I would print the header cookie and just paste it into a header of another request to call a different service marked with [Authenticate]. It didn't work. So I tried something silly and pasted the ss-pid cookie value into the ss-id one. It worked.
Here's the failing cookie string (session redacted :)):
cookie = "ss-id=ss-ID-session-cookie; domain=.zola360.com; path=/; HttpOnly, ss- pid=ss-PID-session-cookie; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, ss-opt=perm; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, X-UAId=; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, 47=0; domain=.zola360.com; path=/, UserId=47; domain=.zola360.com; path=/"
And simply pasting the ss-pid value into ss-id works:
cookie = "ss-id=ss-PID-session-cookie; domain=.zola360.com; path=/; HttpOnly, ss- pid=ss-PID-session-cookie; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, ss-opt=perm; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, X-UAId=; domain=.zola360.com; expires=Tue, 14-Nov-2034 01:34:25 GMT; path=/; HttpOnly, 47=0; domain=.zola360.com; path=/, UserId=47; domain=.zola360.com; path=/"
And the Python3 script I used:
import httplib2 as http
import json
try:
from urlparse import urlparse
except ImportError:
from urllib.parse import urlparse
headers = {
'Accept': 'application/json',
'Content-Type': 'application/json; charset=UTF-8'
}
uri = 'https://mysite.com'
path = '/api2/auth/credentials'
target = urlparse(uri+path)
method = 'POST'
body = '{"username": "username", "password": "password", "RememberMe": "true"}'.encode()
h = http.Http()
response, content = h.request(target.geturl(), method, body, headers)
#save the cookie and use it for subsequent requests
cookie = response['set-cookie']
print(cookie)
path2 = '/api2/time/start'
target2 = urlparse(uri+path2)
headers['cookie'] = cookie
response, content = h.request(target2.geturl(), 'GET', body, headers)
# assume that content is a json reply
# parse content with the json module
data = json.loads(content.decode())
print(data)
It seems that something still looks at the value of ss-id even if ss-opt=perm.
When Authenticating with GET /api2/auth?username=user&password=... it is sent with your permanent cookie ss-pid, i.e:
Cookie: ss-pid=P2hslABCmSs7pomRqNz5; ss-opt=perm; X-UAId=
The rememberme=true option tells ServiceStack to maintain the users session against the permanent ss-pid cookie. This option is maintained in the users ss-opt=perm cookie, which the HTTP Response tells the client to add with:
Set-Cookie: ss-opt=perm; domain=.zola360.com; expires=Mon, 13-Nov-2034 16:11:09 GMT; - path=/; HttpOnly
Although not important in this case, since the temporary session ss-id was missing from the Request, ServiceStack tells the client to add a new one with:
Set-Cookie: ss-id=pojZkNAdMcEcACDREcRM; domain=.zola360.com; path=/; HttpOnly
The issue is with the subsequent request to GET /api2/auth where the client is not re-sending the ss-pid cookie it originally authenticated with (i.e. P2hslABCmSs7pomRqNz5 vs cvgslABCmSs6pomYdLu0):
Cookie: ss-pid=cvgslABCmSs6pomYdLu0; ss-opt=perm; X-UAId=; ss-id=lYWZkFAdMcZcABCDcRM; 47=0; UserId=47
Which ServiceStack doesn't know about (i.e. doesn't maintain any session against) which is why it returns with a 401 Not Authenticated as expected.
HTTP Client should be configured to resend Cookies
It's not clear what HTTP Client you're using but it should be configured to re-send cookies which is normally the default behavior. Ajax will send both the permanent ss-pid cookies and the temporary ss-id only for that browser session, e.g. the temporary ss-id cookie will be discarded when the browser is closed and making a new request will receive a new ss-id cookie.
With the C# Service Clients, it only resends permanent cookies so the client needs to be authenticated with RememberMe = true, e.g:
var client = JsonServiceClient(BaseUrl);
var authResponse = client.Send(new Authenticate
{
provider = "credentials",
UserName = "user",
Password = "p#55word",
RememberMe = true,
});
authResponse.PrintDump();
Once Authenticated the same authenticated client instance can be used to access a protected record multiple times as seen in this Auth Test:
for (int i = 0; i < 500; i++)
{
var response = client.Send<SecureResponse>(new Secured { Name = "test" });
Console.WriteLine("loop : {0}", i);
}