I have implemented using semaphore and expectation , the result is same. What is the fundamental difference between both.
// Using Expectation
func testDownloadWithExpectation(){
let expect = expectation(description: "Download should exceed")
if let url = URL(string: "https://httpbin.org"){
Downloader.download(from: url, completionHandler: { (data, response, error) in
XCTAssertNil(error, "\(String(describing: error?.localizedDescription)) error occured")
XCTAssertNotNil(data,"No Payload returned")
expect.fulfill()
})
}
waitForExpectations(timeout: 10) { (error) in
XCTAssertNil(error, "Test timed Out")
}
}
// Using Semaphore
func testDownloadWIthSephaMore(){
let sepahamore = DispatchSemaphore(value: 0)
if let url = URL(string: "https://httpbin.org"){
Downloader.download(from: url, completionHandler: { (data, response, error) in
XCTAssertNil(error, "\(String(describing: error?.localizedDescription)) error occured")
XCTAssertNotNil(data,"No Payload returned")
sepahamore.signal()
})
}
let timeout = DispatchTime.now() + DispatchTimeInterval.seconds(5)
if sepahamore.wait(timeout: timeout) == DispatchTimeoutResult.timedOut {
XCTFail("Test timed out")
}
}
I believe that the expectation will wait for the expectation to be fulfilled, and will continue running other tests while waiting.
I think that the semaphore will block other tests from running while awaiting the semaphore sign
Related
I'm trying to get Solace (a queuing system) to create a session, then send a message on that session. Instead, it listens to my session creation, receives all the event handlers (I registered all of them), fails to create that session and fails to tell me why. I cannot get this to WAIT for completion. I suspect if it had a few more microseconds, the session would be completed. The promises I have are not being kept. Any awaits that I put in are dutifully ignored.
The Typescript code below is attempting to make a connection to Solace to put a message on a queue. At a high level, it works by getting an instance of the Solace module, then it creates a Session, then with that session, it sends the message. Session creation returns an actual Session and not a promise. That doesn't mean it actually works though. Instead, you have to register an event handler. Because I don't see any of the console.log()s, I believe the createSession event handlers are not being run. Despite registering an event handler for every error in the session handler, Solace neither made the connection, nor said why. As far as I can tell, there's no concept of getting the current state of the session either.
Please note, in previous attempts, I was getting a WaitingForDNS error on the send. It also runs relatively quickly, so I don't think it's doing very much. When I turned on tracing, the most I could tell is that eventually Solace decided to resolve the IP address.
Please see my wishes annotated below:
export class TopicPublisher {
public async connect() {
// Return me a Promise for the Session; either the Session is fully loaded
// loaded, or it's rejected
return new Promise<Session>((resolve, reject) => {
if (this.session !== null) {
this.log("Already connected and ready to publish");
reject();
}
try {
this.session = this.solace.SolclientFactory.createSession({
// solace.SessionProperties
url: this.hosturl,
vpnName: this.vpn,
userName: this.username,
password: this.pass,
connectRetries: 1,
});
} catch (error: any) {
this.log('Error on creating session: ' + error.toString());
reject(error);
}
//The UP_NOTICE dictates whether the session has been established
this.session.on(solace.SessionEventCode.UP_NOTICE, () => {
// *** At this point, return the session as a successfully completing promise ***
this.log("=== Successfully connected and ready to subscribe. ===");
resolve(this.session);
});
//The CONNECT_FAILED_ERROR implies a connection failure
this.session.on(solace.SessionEventCode.CONNECT_FAILED_ERROR, (sessionEvent: { infoStr: string; }) => {
this.log("Connection failed to the message router: " + sessionEvent.infoStr + " - check correct parameter values and connectivity!");
reject(`Check the settings in game-config.ts and try again!`);
});
// Register every event handler in vain attempt at getting Solace to tell me
// why it does not work
let otherErrors = [
solace.SessionEventCode.DOWN_ERROR,
solace.SessionEventCode.REJECTED_MESSAGE_ERROR,
solace.SessionEventCode.SUBSCRIPTION_ERROR,
solace.SessionEventCode.SUBSCRIPTION_OK,
solace.SessionEventCode.VIRTUALROUTER_NAME_CHANGED,
solace.SessionEventCode.REQUEST_ABORTED,
solace.SessionEventCode.REQUEST_TIMEOUT,
solace.SessionEventCode.PROPERTY_UPDATE_OK,
solace.SessionEventCode.PROPERTY_UPDATE_ERROR,
solace.SessionEventCode.CAN_ACCEPT_DATA,
solace.SessionEventCode.RECONNECTING_NOTICE,
solace.SessionEventCode.RECONNECTED_NOTICE,
solace.SessionEventCode.REPUBLISHING_UNACKED_MESSAGES,
solace.SessionEventCode.ACKNOWLEDGED_MESSAGE,
solace.SessionEventCode.UNSUBSCRIBE_TE_TOPIC_OK,
solace.SessionEventCode.UNSUBSCRIBE_TE_TOPIC_ERROR,
solace.SessionEventCode.MESSAGE,
solace.SessionEventCode.GUARANTEED_MESSAGE_PUBLISHER_DOWN
];
for (let errorCodeIndex = 0; errorCodeIndex < otherErrors.length; errorCodeIndex++) {
this.log('Registering error handler code: '+otherErrors[errorCodeIndex]);
this.session.on(otherErrors[errorCodeIndex], (sessionEvent: { infoStr: string; }) => {
this.log("Connection failed with error code : " + otherErrors[errorCodeIndex] + " " + sessionEvent.infoStr);
reject(`Check the config settings`);
});
}
//DISCONNECTED implies the client was disconnected
this.session.on(solace.SessionEventCode.DISCONNECTED, (sessionEvent: any) => {
this.log("Disconnected.");
if (this.session !== null) {
this.session.dispose();
//this.subscribed = false;
this.session = null;
}
});
try {
this.session.connect();
} catch (error: any) {
reject();
}
});
};
public async publish(topicName: string, payload: any) {
// This builds a message payload, it works fine
let solaceMessage = this.getSolaceMessage(topicName, payload);
try {
// *** It does *not* wait for the connection ***
console.log('##This point is reached');
let localSession = await this.connect();
// UP_EVENT ***SHOULD*** have happened, but it does not wait for any events
// or promises to be completed.
console.log('##This point is reached');
console.log('localSession =' + localSession);
localSession.send(solaceMessage);
} catch (error) {
}
};
}
let topicPublisher: TopicPublisher = new TopicPublisher(getInitializedSolaceModule(),
argumentParser.hosturl,
argumentParser.usernamevpn,
argumentParser.username,
argumentParser.vpn,
argumentParser.pass,
argumentParser.topicName);
topicPublisher.publish(argumentParser.topicName, readMessageFromFile(argumentParser.messageFileSpecification)).then(() => {
console.log('##This point is reached');
}, () => {
console.log('##BP10.5 Error handler on publish');
}
).catch(error => {
console.log('publish error' + error);
});
console.log('##This point is reached');
topicPublisher.disconnect();
console.log('##This point is reached');
Solace API documentation is at https://docs.solace.com/API-Developer-Online-Ref-Documentation/nodejs/index.html, but I'm not sure this is a Solace error.
I don't have great exposure to TypeScript - is it possible that the check 'this.session !== null' ends up rejecting the promise, and no session is created. An uninitialized value, if it holds undefined, a !== null check would fail. Maybe your log output sequence can shed light on this.
My apologies, this is a silly point, and not offering any direct help.
I'm using ibmmq module https://github.com/ibm-messaging/mq-mqi-nodejs
I need to get message by CorrelId and then stop listen to the queue.
async listen(queue: string, messageId?: string, waitInterval?: number) {
let mqmd = new mq.MQMD()
let gmo = new mq.MQGMO()
gmo.Options = this.MQC.MQGMO_NO_SYNCPOINT | this.MQC.MQGMO_WAIT | this.MQC.MQGMO_CONVERT | this.MQC.MQGMO_NO_PROPERTIES | this.MQC.MQGMO_FAIL_IF_QUIESCING
gmo.MatchOptions = this.MQC.MQMO_MATCH_CORREL_ID
mqmd.CorrelId = this.hexToBytes(messageId)
gmo.WaitInterval = this.MQC.MQWI_UNLIMITED
mq.Get(obj as mq.MQObject, mqmd, gmo, getCB)
}
And the getCB function:
getCB(err: mq.MQError, hObj: mq.MQObject, gmo: mq.MQGMO, mqmd: mq.MQMD, buf: Buffer, hConn: mq.MQQueueManager) {
if (err) {
...
} else {
...
console.log('GetDone:', hObj)
mq.GetDone(hObj, err => {
console.log('GetDoneError:', err)
})
}
}
I start listening to the queue. Then I put a message with the CorrelId there. The listener get it. I see 'GetDone' in the terminal.
And then I put a message with the same CorrelId. And I get that message and Error.
GetDoneError: MQError: GetDone: MQCC = MQCC_FAILED [2] MQRC = MQRC_HOBJ_ERROR [2019]
at Object.exports.GetDone (/home/apps/connector/node_modules/ibmmq/lib/mqi.js:2316:11)
at MqiConnector.getCB (/home/apps/connector/src/wmq-mqi-connector.js:206:20)
at /home/apps/connector/node_modules/ibmmq/lib/mqi.js:2263:14
at Object.<anonymous> (/home/apps/connector/node_modules/ffi-napi/lib/_foreign_function.js:115:9) {
mqcc: 2,
mqccstr: 'MQCC_FAILED',
mqrc: 2019,
mqrcstr: 'MQRC_HOBJ_ERROR',
version: '1.0.0',
verb: 'GetDone'
}
Looks like the loop with the function getCB didn't stop after GetDone.
I get messages with this CorrelId as many times as I send them. And every time I see this error. The listener is still running.
What am I doing wrong?
I suspect that you are calling GetDone twice and the second time hObj is invalid in the call to mq.GetDone.
mq.GetDone(hObj, err => {
console.log('GetDoneError:', err)
})
I think you have fallen foul of Node.js asynchronous nature and you have hit a timing issue. IE. the cleanup following GetDone has not completed around the same time as the next message is being retrieved.
The function GetDone seems to be synchronous and can be found in https://github.com/ibm-messaging/mq-mqi-nodejs/blob/3a99e0bbbeb017cc5e8498a59c32967cbd2b27fe/lib/mqi.js
The error appears to come from this snippet in GetDone -
var userContext = getUserContext(jsObject);
var err;
if (!userContext) {
err = new MQError(MQC.MQCC_FAILED,MQC.MQRC_HOBJ_ERROR,"GetDone");
} else {
deleteUserContext(jsObject);
}
First time through userContext is found and then deleted. Second time round userContext doesn't exist and the error is thrown.
The Sample in the repo - https://github.com/ibm-messaging/mq-mqi-nodejs/blob/72fba926b7010a85ce2a2c6459d2e9c58fa066d7/samples/amqsgeta.js
only calls GetDone in an error condition, ie. when there are either no messages on the queue or there has been a problem getting the next message off the queue.
function getCB(err, hObj, gmo,md,buf, hConn ) {
// If there is an error, prepare to exit by setting the ok flag to false.
if (err) {
if (err.mqrc == MQC.MQRC_NO_MSG_AVAILABLE) {
console.log("No more messages available.");
} else {
console.log(formatErr(err));
exitCode = 1;
}
ok = false;
// We don't need any more messages delivered, so cause the
// callback to be deleted after this one has completed.
mq.GetDone(hObj);
} else {
if (md.Format=="MQSTR") {
console.log("message <%s>", decoder.write(buf));
} else {
console.log("binary message: " + buf);
}
}
}
Whereas you are calling it when you have retrieved a message. You may need to create a guard that stops you calling it twice.
As for why the second message has been obtained, without an error, you might need to raise an issue on the ibmmq module.
In our program, we try to implement a task retry pattern with await.
Our main problem is our method keeps the first retry payload in subsequent ones.
Here is the retry method:
async retryTaskUntilExpectedValue({
task,
expectedValue,
messageOnError = 'Max retry number reached without expected result',
maxRetries = 10,
timeout = 10,
spinner = null
}) {
let printFn = console.log;
if (spinner !== null) {
printFn = spinner.text;
}
// Proceed retries
for (let i = 1; i <= maxRetries; i++) {
try {
let result = await task;
console.log(result); // Always display same result: {"state": "upgrading"} even if curling returns {"state": "upgraded"} after about 2 retries
result = JSON.parse(result).state;
if (result === expectedValue) {
return Promise.resolve(result);
} else if (i <= maxRetries) {
printFn(`Result "${result}" differs from expected value "${expectedValue}"`);
await wait(1000);
printFn(`Waiting ${timeout}s before retry`);
await wait(timeout * 1000);
printFn(`Retrying (${i})`);
continue;
} else {
return Promise.reject(`ERROR: ${messageOnError}`);
}
} catch (err) {
return Promise.reject(`ERROR: Unexpected error while running task`);
}
}
};
And the use in our CLI:
checkUpgrade(url) {
return retryTaskUntilExpectedValue({
task: this.makeHttpRequest('GET', url),
expectedValue: 'upgraded'
});
}
In our case, the task is an http request returning a state from our backend database.
The model is simple:
{ "state": "upgrading" } then when the backend job is done, it returns { "state": "upgraded"}.
The job takes some time to process (around 20 sec). In our tests, this behavior occured:
First call: upgrading
First retry: upgrading
After that, by curling directly the REST api by hand, I get the upgraded status
all other retries in the CLI: upgrading
So in the CLI we build, we have 10 times the result: Result "upgrading" differs from expected value "upgraded"
It seems the let response = await task; in the subsequent retries does not call the task method at each retry. Indeed if actual call was made, it would for sure retrieve the proper upgraded state since we get it through curl.
How to make the await task; to actually trigger the call task method and not to keep the result from first call?
A promise is the result for an already started operation. By passing in task as a promise inside - it will always await the same result and return the same value.
Instead, retryTaskUntilExpectedValue should take a function for a promise and await an invocation of that:
let result = await functionReturningTask();
Where functionReturningTask is whatever you used to obtain task in the first place.
I am writing a worker that uses amqplib's Channel#consume method. I want this worker to wait for jobs and process them as soon as they appear in the queue.
I wrote my own module to abstract away ampqlib, here are the relevant functions for getting a connection, setting up the queue and consuming a message:
const getConnection = function(host) {
return amqp.connect(host);
};
const createChannel = function(conn) {
connection = conn;
return conn.createConfirmChannel();
};
const assertQueue = function(channel, queue) {
return channel.assertQueue(queue);
};
const consume = Promise.method(function(channel, queue, processor) {
processor = processor || function(msg) { if (msg) Promise.resolve(msg); };
return channel.consume(queue, processor)
});
const setupQueue = Promise.method(function setupQueue(queue) {
const amqp_host = 'amqp://' + ((host || process.env.AMQP_HOST) || 'localhost');
return getConnection(amqp_host)
.then(conn => createChannel(conn)) // -> returns a `Channel` object
.tap(channel => assertQueue(channel, queue));
});
consumeJob: Promise.method(function consumeJob(queue) {
return setupQueue(queue)
.then(channel => consume(channel, queue))
});
My problem is with Channel#consume's odd signature. From http://www.squaremobius.net/amqp.node/channel_api.html#channel_consume:
#consume(queue, function(msg) {...}, [options, [function(err, ok) {...}]])
The callback is not where the magic happens, the message's processing should actually go in the second argument and that breaks the flow of promises.
This is how I planned on using it:
return queueManager.consumeJob(queue)
.then(msg => {
// do some processing
});
But it doesn't work. If there are no messages in the queue, the promise is rejected and then if a message is dropped in the queue nothing happens. If there is a message, only one message is processed and then the worker stalls because it exited the "processor" function from the Channel#consume call.
How should I go about it? I want to keep the queueManager abstraction so my code is easier to reason about but I don't know how to do it... Any pointers?
As #idbehold said, Promises can only be resolved once. If you want to process messages as they come in, there is no other way than to use this function. Channel#get will only check the queue once and then return; it wouldn't work for a scenario where you need a worker.
just as an option. You can present your application as a stream of some messages(or events). There is a library for this http://highlandjs.org/#examples
Your code should look like this(it isn`t a finished sample, but I hope it illustrates the idea):
let messageStream = _((push, next) => {
consume(queue, (msg) => {
push(null, msg)
})
)
// now you can operate with your stream in functional style
message.map((msg) => msg + 'some value').each((msg) => // do something with msg)
This approach provides you a lot of primitives for synchronization and transformation
http://highlandjs.org/#examples
Is the Callback supposed to be called on a different thread?
Using this code:
client.ExecuteAsync<List<IngredientDto>>(request, Response =>
{
textBox1.Text += Response.Data.Count;
});
I get a "InvalidOperationException":
"The calling thread cannot access this object because a different thread owns it."
Shouldn't the callback be on the UI thread, or am I wrong?
Actually, if you look at source code, you'll see
public virtual RestRequestAsyncHandle ExecuteAsync<T>(IRestRequest request, Action<IRestResponse<T>, RestRequestAsyncHandle> callback)
{
return ExecuteAsync(request, (response, asyncHandle) =>
{
IRestResponse<T> restResponse = response as RestResponse<T>;
if (response.ResponseStatus != ResponseStatus.Aborted)
{
restResponse = Deserialize<T>(request, response);
}
callback(restResponse, asyncHandle);//<--- this means that response & callback are executed at **same** thread.
});
}
That leads to:
You can't update ui objects from non-ui thread. In case of WPF you can use Dispatcher
client.ExecuteAsync<List<IngredientDto>>(request, Response =>
{
Dispatcher.Invoke((Action)() => {textBox1.Text += Response.Data.Count;});
});
In general case see syncronization context