There are many types of domain objects in my web application, such as MemberModel, PostModel, CreditsModel and so on. I find that the type of the object is needed when configuring JacksonJsonRedisSerializer, so I specified Object.class. But I got error when deserializing objects.
To work around this, I've got 2 options:
Use JdkSerializationRedisSerializer instead. But the result of the serialization is too long so it will consume lots of memory in Redis.
Configure serializer for every domian objects, which means if I have 50 domain objects then I have to configure 50 serializers. But this is obviously pretty tedious.
Is there a graceful way to solve this problem? Thanks!
There's an open PR #145 available. Untill that one is merged one can pretty much just implement a RedisSerializer the way it is done in GenericJackson2JsonRedisSerializer configuring the used ObjectMapper to inlcude type information within the json.
ObjectMapper mapper = new ObjectMapper();
mapper.enableDefaultTyping(DefaultTyping.NON_FINAL, As.PROPERTY);
byte[] bytes = mapper.writeValueAsBytes(domainObject);
// using Object.class allows the mapper fall back to the default typing.
// one could also use a concrete domain type if known to avoid the cast.
DomainObject target = (DomainObject) mapper.readValue(bytes, Object.class);
Related
I was pretty shocked to find out that "require" in node creates a singleton by default. One might assume that many people have modules which they require which have state, but are created as a singleton, so break the app as soon as there are multiple concurrent users.
We have the opposite problem, requires is creating a non-singleton, and we dont know how to fix this.
Because my brain is wired as a java developer, all our node files/modules are defined thusly:
file playerService.js
const Player = require("./player")
class PlayerService {
constructor(timeout) {
// some stuff
}
updatePlayer(player) {
// logic to lookup player in local array and change it for dev version.
// test version would lookup player in DB and update it.
}
}
module.exports = PlayerService
When we want to use it, we do this:
someHandler.js
const PlayerService = require("./playerService")
const SomeService = require("./someService")
playerService = new PlayerService(3000)
// some code which gets a player
playerService.updatePlayer(somePlayer)
Although requires() creates singletons by default, in the above case, I am guessing it is not creating a singleton as each websocket message (in our case) will instantiate a new objects in every module which is called in the stack. That is a lot of overhead - to service a single message, the service might get instantiated 5 times as there are 5 different sub services/helper classes which call each other and all do a requires(), and then multiply this by the number of concurrent users and you get a lot of unnecessary object creation.
1) How do we modify the above class to work as a singleton, as services don't have state?
2) Is there any concept of a global import or creating a global object, such that we can import (aka require) and/or instantiate an object once for a particular websocket connection and/or for all connections? We have no index.js or similar. It seems crazy to have to re-require the dependent modules/files for every js file in a stack. Note, we looked at DI options, but found them too arcane to comprehend how to use them as we are not js gurus, despite years of trying.
You can simply create an instance inside the file and export it.
let playerService = new PlayerService();
module.exports = playerService;
In this case, you may want to add setters for the member variables you would take as constructor parameters to ensure encapsulation.
Also note that, creating object instances with new in javascript is cheaper than traditional OOP language because of it's prototype model (more).
So don't hesitate when you really need new instances (as seen in your code, do you really want to share the timeout constructor parameter?), since javascript objects are pretty memory efficient with prototype methods and modern engines has excellent garbage collectors to prevent memory leak.
Description of ConcurrentBag on MSDN is not clear:
Bags are useful for storing objects when ordering doesn't matter, and unlike sets, bags support duplicates. ConcurrentBag is a thread-safe bag implementation, optimized for scenarios where the same thread will be both producing and consuming data stored in the bag.
My question is it thread safe and if this is a good practice to use ConcurrentBag in Parallel.ForEach.
For Instance:
private List<XYZ> MyMethod(List<MyData> myData)
{
var data = new ConcurrentBag<XYZ>();
Parallel.ForEach(myData, item =>
{
// Some data manipulation
data.Add(new XYZ(/* constructor parameters */);
});
return data.ToList();
}
This way I don't have to use synchronization locking in Parallel.ForEach using regular List.
Thanks a lot.
That looks fine to me. The way you're using it is thread-safe.
If you could return an IEnumerable<XYZ>, it could be made more efficient by not copying to a List<T> when you're done.
ConcurrentBag and Parallel.ForEach seems to me, no problem. If you uses this types in scenarios that has a large volume multi-user access, these classes in your implementation could rise up cpu process to levels that can crash your web-server. Furthermore, this implementation starts N tasks (threads) to execute each one of iterations, so be careful when choice this classes ans implementations. I recently spent in this situation and I had to extract memory dump to analyze whats happens into my web application core. So, be careful 'cause Concurrentbag is ThreadSafe and in web scenarios it is no better way.
I could connect to two devices from Android based cell phone simultaneously using SPP, but once I turn on the inputstream (like socket.getInputStream()), one of them will return 0 in the stream, that is, no data available on the stream.
For example, thread A(thA) and thread B(thB) connected to device A(devA) and device B(devB) respectively. So, thA uses inputstream A(inA) to receive data from devA, thB uses inputstream B(inB) to receive data from devB. As follow:
devA --->inA --->thA
devB --->inB --->thB
It works fine if I connect to each device separately. However, in the case of connecting two devices at the same time, then only inA or inB has data on it.
If it happens to you, please share your experence with me, I would be very appreciated!!
Thank you in advance.
YT
Why are you using reflection for the createRFCommSocket? device.getClass().getMethod("createRfcommSocket", new Class[] {int.class});
as opposed to
try {mBTSocket = mBTDevice.createRfcommSocketToServiceRecord(UUID_RFCOMM_GENERIC);
} catch (Exception e1) {
msg ("connect(): Failed to bind to RFCOMM by UUID. msg=" + e1.getMessage());
return false;
}
The reflection can easily be the source of problems. If there is no reason to use it then avoid it at all costs.
Furthermore, if the getClass call fails, then your "m" variable will be null, and you're not trapping for that situation. You should generalize your exception more too, instead of using specific exceptions, just use "Exception" Like in my code snippet above. It's much easier than adding a catch for every possible type of exception that might get thrown.
I'm confused about what you're doing with the handlers, it doesn't make sense to me. Can you remove the handler code to simplify things?
There's just too much complication. Remove all the reflection, extra catch's.
It's good coding practice to make your methods one page or less. When a method is more than a page it is too complicated and it makes reading it AND debugging it very difficult. Reduce the size of your methods by creating other methods to perform common tasks.
Separate your connect() logic, from your I/O logic. You should have a method for sending data, and a method for receiving data, a method for connect(). Then once you get those working, chunk up and create methods for higher level I/O for sending and receiving whole blocks of data. then perfect those methods and keep growing up and up.
in my code the read, write, connect, and ALL I/O methods are only 1-20 lines each. Keep them very simple because your I/O logic is at the core of your app and it needs to be clean clean clean.
What I need is a system I can define simple objects on (say, a "Server" than can have an "Operating System" and "Version" fields, alongside other metadata (IP, MAC address, etc)).
I'd like to be able to request objects from the system in a safe way, such that if I define a "Server", for example, can be used by 3 clients concurrently, then if 4 clients ask for a Server at the same time, one will have to wait until the server is freed.
Furthermore, I need to be able to perform requests in some sort of query-style, for example allocate(type=System, os='Linux', version=2.6).
Language doesn't matter too much, but Python is an advantage.
I've been googling for something like this for the past few days and came up with nothing, maybe there's a better name for this kind of system that I'm not aware of.
Any recommendations?
Thanks!
Resource limitation in concurrent applications - like your "up to 3 clients" example - is typically implemented by using semaphores (or more precisely, counting semaphores).
You usually initialize a semaphore with some "count" - that's the maximum number of concurrent accesses to that resource - and you decrement this counter every time a client starts using that resource and increment it when a client finishes using it. The implementation of semaphores guarantees the "increment" and "decrement" operations will be atomic.
You can read more about semaphores on Wikipedia. I'm not too familiar with Python but I think these two links can help:
Python Threading Library
Semaphore Objects in Python.
For Java there is a very good standard library that has this functionality:
http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/package-summary.html
Just create a class with Semaphore field:
class Server {
private static final MAX_AVAILABLE = 100;
private final Semaphore available = new Semaphore(MAX_AVAILABLE, true);
// ... put all other fields (OS, version) here...
private Server () {}
// add a factory method
public static Server getServer() throws InterruptedException {
available.acquire();
//... do the rest here
}
}
Edit:
If you want things to be more "configurable" look into using AOP techniques i.e. create semaphore-based synchronization aspect.
Edit:
If you want completely standalone system, I guess you can try to use any modern DB (e.g. PostgreSQL) system that support row-level locking as semaphore. For example, create 3 rows for each representing a server and select them with locking if they are free (e.g. "select * from server where is_used = 'N' for update"), mark selected server as used, unmark it in the end, commit transaction.
Slightly modified version of canonical broken double-checked locking from Wikipedia:
class Foo {
private Helper helper = null;
public Helper getHelper() {
if (helper == null) {
synchronized(this) {
if (helper == null) {
// Create new Helper instance and store reference on
// stack so other threads can't see it.
Helper myHelper = new Helper();
// Atomically publish this instance.
atomicSet(helper, myHelper);
}
}
}
return helper;
}
}
Does simply making the publishing of the newly created Helper instance atomic make this double checked locking idiom safe, assuming that the underlying atomic ops library works properly? I realize that in Java, one could just use volatile, but even though the example is in pseudo-Java, this is supposed to be a language-agnostic question.
See also:
Double checked locking Article
It entirely depends on the exact memory model of your platform/language.
My rule of thumb: just don't do it. Lock-free (or reduced lock, in this case) programming is hard and shouldn't be attempted unless you're a threading ninja. You should only even contemplate it when you've got profiling proof that you really need it, and in that case you get the absolute best and most recent book on threading for that particular platform and see if it can help you.
I don't think you can answer the question in a language-agnostic fashion without getting away from code completely. It all depends on how synchronized and atomicSet work in your pseudocode.
The answer is language dependent - it comes down to the guarantees provided by atomicSet().
If the construction of myHelper can be spread out after the atomicSet() then it doesn't matter how the variable is assigned to the shared state.
i.e.
// Create new Helper instance and store reference on
// stack so other threads can't see it.
Helper myHelper = new Helper(); // ALLOCATE MEMORY HERE BUT DON'T INITIALISE
// Atomically publish this instance.
atomicSet(helper, myHelper); // ATOMICALLY POINT UNINITIALISED MEMORY from helper
// other thread gets run at this time and tries to use helper object
// AT THE PROGRAMS LEISURE INITIALISE Helper object.
If this is allowed by the language then the double checking will not work.
Using volatile would not prevent a multiple instantiations - however using the synchronize will prevent multiple instances being created. However with your code it is possible that helper is returned before it has been setup (thread 'A' instantiates it, but before it is setup thread 'B' comes along, helper is non-null and so returns it straight away. To fix that problem, remove the first if (helper == null).
Most likely it is broken, because the problem of a partially constructed object is not addressed.
To all the people worried about a partially constructed object:
As far as I understand, the problem of partially constructed objects is only a problem within constructors. In other words, within a constructor, if an object references itself (including it's subclass) or it's members, then there are possible issues with partial construction. Otherwise, when a constructor returns, the class is fully constructed.
I think you are confusing partial construction with the different problem of how the compiler optimizes the writes. The compiler can choose to A) allocate the memory for the new Helper object, B) write the address to myHelper (the local stack variable), and then C) invoke any constructor initialization. Anytime after point B and before point C, accessing myHelper would be a problem.
It is this compiler optimization of the writes, not partial construction that the cited papers are concerned with. In the original single-check lock solution, optimized writes can allow multiple threads to see the member variable between points B and C. This implementation avoids the write optimization issue by using a local stack variable.
The main scope of the cited papers is to describe the various problems with the double-check lock solution. However, unless the atomicSet method is also synchronizing against the Foo class, this solution is not a double-check lock solution. It is using multiple locks.
I would say this all comes down to the implementation of the atomic assignment function. The function needs to be truly atomic, it needs to guarantee that processor local memory caches are synchronized, and it needs to do all this at a lower cost than simply always synchronizing the getHelper method.
Based on the cited paper, in Java, it is unlikely to meet all these requirements. Also, something that should be very clear from the paper is that Java's memory model changes frequently. It adapts as better understanding of caching, garbage collection, etc. evolve, as well as adapting to changes in the underlying real processor architecture that the VM runs on.
As a rule of thumb, if you optimize your Java code in a way that depends on the underlying implementation, as opposed to the API, you run the risk of having broken code in the next release of the JVM. (Although, sometimes you will have no choice.)
dsimcha:
If your atomicSet method is real, then I would try sending your question to Doug Lea (along with your atomicSet implementation). I have a feeling he's the kind of guy that would answer. I'm guessing that for Java he will tell you that it's cheaper to always synchronize and to look to optimize somewhere else.