Google Drive REST-API Internal error 500 - multithreading

I'm developing an application that receives several massage queue containing 20-30 gmail addresses. My app owns a single google drive folder that must be shared with all of the users. My app uses Google drive Java client and for the HTTP calls I'm using Google batching
The problem is that I'm getting internal error 500 when I'm trying to share the folder with 30 users simultaneously by multithreading. However when I use threads synchronisation everything is fine but the performance is terrible about 0.5 second per user!
Can anyone explain Why I'm receiving this error?
500 OK
{
"code" : 500,
"errors" : [ {
"domain" : "global",
"message" : "Internal Error. User message: \"An internal error has occurred which prevented the sharing of these item(s): fileame\"",
"reason" : "internalError"
} ],
"message" : "Internal Error. User message: \"An internal error has occurred which prevented the sharing of these item(s): filename\""
}
Here is the thread code:
try {
//batch start
BatchRequest batch = service.batch();
ArrayList<String> users = readUsers(this.file);
Permission[] permissions= new Permission[users.size()];
for (int i = 0 ; i < users.size(); i++){
permissions[i]= new Permission();
permissions[i].setValue(users.get(i)+"#gmail.com");
permissions[i].setType("user");
permissions[i].setRole("writer");
service.permissions().insert(fileId, permissions[i]).setSendNotificationEmails(Boolean.FALSE) .queue(batch, callback);
}
//batch execute
batch.execute();
} catch (IOException e) {
System.out.println("An error occurred and I am " + id + ": " + e);
}

Related

Getting message in ibmmq Node.js

I'm using ibmmq module https://github.com/ibm-messaging/mq-mqi-nodejs
I need to get an xml message from a queue and than make an xsl-transformation.
I put messages to the queue with JMeter and if I browse messages in rfhutil I can see them as is on the Data tab.
But when I get it in the code
function getCB(err, hObj, gmo,md,buf, hConn ) {
// If there is an error, prepare to exit by setting the ok flag to false.
if (err) {...
} else {
if (md.Format=="MQSTR") {
console.log("message <%s>", decoder.write(buf));
} else {
console.log("binary message: " + buf);
}
}
I get my message with some service information:
buf=RFH �"�MQSTR � <mcd><Msd>jms_text</Msd></mcd> X<jms><Dst>queue://MY_QM/MY_QUEUE</Dst><Tms>1657791724648</Tms><Dlv>2</Dlv></jms> ...My_message...
How can I get only My message like I do in rfhutil?
I can get it with string methods, but it looks like crutches.
That message has the headers created by a JMS application. There are various ways of dealing with it. You can
Have the sending app disable sending that structure (setting the targClient property)
Use GMO options to ignore the properties (MQGMO_NO_PROPERTIES)
Have your application deal with the RFH2 stucture. See for example the amqsget.js sample in the Node.js repo which includes this fragment:
switch (format) {
case MQC.MQFMT_RF_HEADER_2:
hdr = mq.MQRFH2.getHeader(buf);

Unable to connect to Azure Cosmos Db Account using Microsoft.EntityFrameworkCore.Cosmos - Response status code

The CosmosDb provider is sending this message:
“Response status code does not indicate success: 503 Substatus: 0 Reason: (The request failed because the client was unable to establish connections to 3 endpoints across 1 regions. Please check for client resource starvation issues and verify connectivity between client and server.”
In my tests, it works (.net core 3.1):
Task.Run(async () =>
{
var endpoint = “test”;
var masterKey = “test”;
using (var client = new DocumentClient(new Uri(endpoint), masterKey))
{
//Insert new Document
Console.WriteLine("\r\n>>>>>>>>>>>>>>>> Creating Document <<<<<<<<<<<<<<<<<<<");
dynamic candidato = new
{
Id = 1,
Nome = "Test"
};
var document1 = await client.CreateDocumentAsync(
UriFactory.CreateDocumentCollectionUri("Test", "Test"),
candidato);
Console.ReadKey();
}
}).Wait();
It does not:
Task.Run(async () =>
{
using (var context = new StudentsDbContext())
{
context.Add(new FamilyContainer(2, "Test"));
await context.SaveChangesAsync();
}
}).Wait();
public class FamilyContainer
{
public int Id { get; set; }
public string Nome { get; set; }
public FamilyContainer(int id, string nome)
{
Id = id;
Nome = nome;
}
}
public class StudentsDbContext : DbContext
{
public DbSet<FamilyContainer> FamilyContainer { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseCosmos(
"test",
"test",
"FamilyDatabase",
options =>
{ }
);
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<FamilyContainer>(x =>
{
x.ToContainer("FamilyContainer");
});
}
}
Packages
Can anyone help me? Thanks
fail: Microsoft.EntityFrameworkCore.Update[10000]
An exception occurred in the database while saving changes for context type '...'.
Microsoft.EntityFrameworkCore.Storage.RetryLimitExceededException: Maximum number of retries (6) exceeded while executing database operations with 'CosmosExecutionStrategy'. See inner exception for the most recent failure.
---> Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: 503 Substatus: 0 Reason: (Microsoft.Azure.Documents.ServiceUnavailableException: Service is currently unavailable.ActivityId: 07fbf539-0d44-4e5a-89d0-cd46838ee605, {"RequestStartTimeUtc":"2020-02-21T16:34:09.1834993Z","RequestEndTimeUtc":"2020-02-21T16:34:41.3484203Z","RequestLatency":"00:00:32.1649210","IsCpuOverloaded":false,"NumberRegionsAttempted":1,"ResponseStatisticsList":[{"ResponseTime":"2020-02-21T16:34:11.5964152Z","ResourceType":2,"OperationType":0,"StoreResult":"StorePhysicalAddress: rntbd:.../, LSN: -1, GlobalCommittedLsn: -1, PartitionKeyRangeId: , IsValid: True, StatusCode: 410, SubStatusCode: 0, RequestCharge: 0, ItemLSN: -1, SessionToken: , UsingLocalLSN: False, TransportException: A client transport error occurred: Failed to connect to the remote endpoint. (Time: 2020-02-21T16:34:11.5298608Z, activity ID: 07fbf539-0d44-4e5a-89d0-cd46838ee605, error code: ConnectFailed [0x0005], base error: socket error ConnectionRefused [0x0000274D]...
--- End of inner exception stack trace ---
I was facing same issue.
What worked for me is changing ConnectionMode to ConnectionMode.Gateway while initializing CosmosClient like :
var options = new CosmosClientOptions() { ConnectionMode = ConnectionMode.Gateway };
var client = new CosmosClient(endpoint, key, options);
For more details on refer :
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.cosmosclientoptions?view=azure-dotnet
https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.cosmos.connectionmode?view=azure-dotnet
TransportException: A client transport error occurred: Failed to connect to the remote endpoint. (Time: 2020-02-21T16:34:11.5298608Z, activity ID: 07fbf539-0d44-4e5a-89d0-cd46838ee605, error code: ConnectFailed [0x0005], base error: socket error ConnectionRefused
This means that the Connection was refused.
Either your Cosmos DB account has Firewall/VPN enabled and the application is not able to establish a connection due not not being in a whitelisted IP/Network : Try checking your account configuration.
The environment you are executing the code is restricting connections (some corporate Firewall or network might be blocking port ranges): Try running the app in a different network, or use GatewayMode. If that works, then this is related to the network.
The machine might be running low on sockets or high on CPU.
My RCA for this is: Cosmos Partitions where served by individual processes on CosmosDB, each partition serving process has it's own TCP port. When client connects to 443 (Using TCP Direct Mode), CosmosDB Proxy sends partition ports back to client so that client can talk to server-partitions in parallel. Partition ports are random (11000 upwards afaik). Normal company firewall would allow outbound 443 (connection to cosmos works) but blocks the outbound random ports. So at the end, access fails.
Workarounds:
Open firewall
Use Gateway Mode. This uses https/443 only by forwarding internally instead of redirecting to other ports.
It is because Entity framework has a default connection mode of Direct. It worked for me after overriding it to Gateway.
{
optionsBuilder.UseCosmos(
"test",
"test",
"FamilyDatabase",
options =>
{ options.ConnectionMode(ConnectionMode.Gateway); }
);
}
I just want to add this because it wasted a lot of my time. The following code would instantly die with an error message that led me to this S.O. post:
var container = _client.Client.GetContainer(_databaseName, containername);
var result = await container.CreateItemAsync(dataitem, pk);
I disbelieved the error message because everything else has worked, upsert, read, etc. After messing with it for a while, I noticed the documentation shows a template type for CreateItemAsync.
var container = _client.Client.GetContainer(_databaseName, containername);
var result = await container.CreateItemAsync<T>(dataitem, pk);
Changing the code to that fixed it (inside of a templated function).
I wanted to add: if I had been catching exceptions, I would have gotten to the meat of the problem much sooner. The library I am working with is not meant to catch exceptions, they are handled by the layer above it.

I am Face two Issues using google Drive api : socket time out exception and user limit exceed

i face issues while connecting upload Picture to Google Drive . Sockect Time Out
my Intern was Good.
This was my Code to upload receipt in Google Drive
This Sockect Time out Exeception Error
In have Increased time out session. For Connect the Google drive
private HttpRequestInitializer setHttpTimeout(final HttpRequestInitializer requestInitializer) {
return new HttpRequestInitializer() {
#Override
public void initialize(HttpRequest httpRequest) throws IOException {
requestInitializer.initialize(httpRequest);
httpRequest.setConnectTimeout(4 * 60000); // 3 minutes connect timeout
httpRequest.setReadTimeout(3 * 60000); // 3 minutes read timeout
}
};
}
Google Drive ---->Connected--> Return Code
private void handleSignInResult(Intent result) {
GoogleSignIn.getSignedInAccountFromIntent(result)
.addOnSuccessListener(googleAccount -> {
Log.d(TAG, "Signed in as " + googleAccount.getEmail());
// Use the authenticated account to sign in to the Drive service.
SharedPreferences.Editor edit = settings.edit();
edit.putString("GoogleDriveID", "" + googleAccount.getEmail());
edit.commit();
googleSignInAccount=googleAccount;
accountName=googleAccount.getEmail();
GoogleAccountCredential credential =
GoogleAccountCredential.usingOAuth2(
this, Collections.singleton(DriveScopes.DRIVE_FILE));
credential.setSelectedAccount(googleAccount.getAccount());
Drive googleDriveService = new Drive.Builder( AndroidHttp.newCompatibleTransport(),
new GsonFactory(), setHttpTimeout(credential)).setApplicationName("nnnnn")
.build();
service=googleDriveService;
2.second I was Getting "User Limit Exceed "
ERR-00006-Google Drive upload|StackTrace: GLOBAL::saveFileToDrive::IOException
com.google.api.client.googleapis.json.GoogleJsonResponseException:
403 Forbidden { "code": 403, "errors": [ { "domain": "usageLimits", "message": "User Rate
Limit Exceeded.
Rate of requests for user exceed configured project quota.
You may consider re-evaluating expected per-user traffic to the API and adjust project
quota limits accordingly.
You may monitor aggregate quota usage and adjust limits in the
API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?
project=344828444511",
"reason": "userRateLimitExceeded", "extendedHelp":
"https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?
project=************" } ],
"message": "User Rate Limit Exceeded. Rate of requests for user exceed configured project
quota.
You may consider re-evaluating expected per-user traffic to the API and adjust project
quota limits accordingly.
You may monitor aggregate quota usage and adjust limits in the API Console:
https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?
project=************" }
Increased quotes Limit in Google Developer Console . but still this Error.

MSMQ security blocking traffic from RPC traffic in Windows Server 2003

I've created a Message Queue in a server running Windows Server 2003 and created a client program that is able to send messages, but I can't pull them back. To send messages, I am using the following code (note that I am using transactional queue):
MessageQueueTransaction transaction = new MessageQueueTransaction();
transaction.Begin();
messageQueue.Send("Hello MSMQ!", "Title", transaction);
transaction.Commit();
transaction.Dispose();
To pull the messages I've tried this, without success:
MessageQueueTransaction transaction = new MessageQueueTransaction();
try
{
transaction.Begin();
MessageQueue queue = new MessageQueue(QUEUE_NAME);
Message msg = queue.Receive(new TimeSpan(0, 0, 5), transaction);
msg.Formatter = new XmlMessageFormatter(new Type[] { typeof(String) });
Console.WriteLine(msg.Label + " - " + msg.Body);
transaction.Commit();
}
catch (Exception e)
{
transaction.Abort();
Console.WriteLine(e.Message);
}
In the queue.Receive() call I am getting an exception. I've found a lot of similar questions about this, and I found this article with three possible solutions, but none of them worked for me. I disabled the firewall on the server, created the DWORD values in the registry, and one of the solutions provided seems to be specific for Windows Server 2008. Can anyone help me with remote calls from MSMQ in Windows Server 2003?

Why does IIS return a 500 status code when the server times out?

We're running IIS7 & .net 4.0. When the server times out due to a long running request, the error page is shown, but the error code is just 500 rather than 408 or possibly 503 that I'm expecting. We want to show a different error page if it's a timeout, but I can't configure this in the <customErrors> section if it's just giving a 500 error. Is this a configuration issue?
You can add code like this into your global.asax.cs
public class Global : System.Web.HttpApplication
{
protected void Application_Error(Object sender, EventArgs e)
{
Exception ex = HttpContext.Current.Server.GetLastError();
if (ex != null && ex is HttpException && ex.Message == "Request timed out.")
{
HttpContext.Current.Response.StatusCode = 503;
HttpContext.Current.Response.End();
}
}
}
I found this to not work properly and still return a 500 error without the Response.End() in there. Given your question, I'm not sure if you want to do a redirect instead to show an error page that would itself output the 503 instead of in the above.
Its really ASP.NET returning the 500 status, IIS is just passing it along.

Resources