Here is the code:
var timePointer *[]time.Time
func UpdateHolidayList() error {
//updating logic: pulling list of holidays from API
holidaySlice := make([]time.Time, 0)
//appending some holidays of type time.Time to holidaySlice
//now holidaySlice contains a few time.Time values
timePointer = &holidaySlice
return nil
}
func main() {
//run UpdateHoliday every 7 days
go func() {
for {
UpdateHoliday()
time.Sleep(7 * 3600 * time.Hour)
}
}()
}
I have 4 questions:
holidaySlice is a local variable, is it safe to point a (global) pointer to it?
Is this whole code multi-thread safe?
After pointing timePointer to holidaySlice, can I access the values via timePointer
(If answer to 3. is "yes") The holiday list is constantly changing, so holidaySlice will be different each update. Will the values accessed via timePointer change accordingly then?
holidaySlice is a local variable allocated on the heap. Any variable pointing to the same heap location can access the data structure at that location. Whether or not it is safe depends on how you access it. Even if holidaySlice was not explicitly allocated on the heap, once you make a global variable point to it, Go compiler would detect that it "escapes", so it would allocate that on the heap.
The code is not thread-safe. You are modifying a shared variable (the global variable) without any explicit synchronization, so there is no guarantee on when or if other goroutines will see the updates to that variable.
If you update the contents of the timePointer array without explicit synchronization, there is no guarantee on when or if other goroutines will see those updates. You have to use synchronization primitives like sync/Mutex to delimit read/write access to data structures that can be updated/read by multiple goroutines.
As long as your main does not end timePointer will have access to the pointer that holidaySlice creates - since its heap allocated, compiler will detect its escape and not free the memory.
Nope, absolutely not. Look at the sync package
Yes you can. Just remember to iterate using *timePointer instead of timePointer
It will change - but not accordingly. Since you have not done any synchronization - you have no defined way of knowing what data is stored at slice pointed to by timePointer when it is read.
Related
Consider the following scenarios.
let mut map = HashMap::new();
map.insert(2,5);
thread::scope(|s|{
s.spawn(|_|{
map.insert(1,5);
});
s.spawn(|_|{
let d = map.get(&2).unwrap();
});
}).unwrap();
This code cannot be compiled because we borrow the variable map mutably in h1 and borrow again in h2. The classical solution is wrapping map by Arc<Mutex<...>>. But in the above code, we don't need to lock whole hashmap. Because, although two threads concurrently access to same hashmap, they access completely different region of it.
So I want to share map through thread without using lock, but how can I acquire it? I'm also open to use unsafe rust...
in the above code, we don't need to lock whole hashmap
Actually, we do.
Every insert into the HashMap may possibly trigger its reallocation, if the map is at that point on its capacity. Now, imagine the following sequence of events:
Second thread calls get and retrieves reference to the value (at runtime it'll be just an address).
First thread calls insert.
Map gets reallocated, the old chunk of memory is now invalid.
Second thread dereferences the previously-retrieved reference - boom, we get UB!
So, if you need to insert something in the map concurrently, you have to synchronize that somehow.
For the standard HashMap, the only way to do this is to lock the whole map, since the reallocation invalidates every element. If you used something like DashMap, which synchronizes access internally and therefore allows inserting through shared reference, this would require no locking from your side - but can be more cumbersome in other parts of API (e.g. you can't return a reference to the value inside the map - get method returns RAII wrapper, which is used for synchronization), and you can run into unexpected deadlocks.
As I understand it, there are three ways for a WebAssembly instance to return values to the external caller:
returning a value directly as the result of a method call (is this only a single value of the basic datatypes {i32, i64, f32, f64}?)
calling an imported function with arguments, this function can store the arguments or access the memory while it is called and save the contents somewhere else
writing to a memory and have the external environment access it after execution
Are there more? What are the details?
Your assessment is roughly correct. Functions can either return a WebAssembly value (i32, i64, f32, f64) directly or return data indirectly via modifying elements in the global environment such as Memory, Table and Globals. If you are talking about returns data to a caller outside of the WebAssembly module than any such elemennts would themselves need to to be exported from he module. i.e. you can't use the WebAssembly memory to indirectly return data unless he memory itself is also exports.
With the multi-value proposal you will also be able to directly return more than one value: https://github.com/WebAssembly/multi-value. Likewise the GC proposal will allow you to directly return complex structures. But neither of those is available today.
Your list is almost complete- the only other mechanism I can think of is global variables. These can also be exported to allow access by the host environment so could conceivably be used as a mechanism for returning results.
Yes, you are correct in your statement that you can only return a single numeric value from exported functions
Is there a way to zero buffers containing e. g. private keys after
using them and make sure that compilers don't delete the zeroing code as
unused? Something tells me that a simple:
copy(privateKey, make([]byte, keySize))
Is not guaranteed to stay there.
Sounds like you want to prevent sensitive data remaining in memory. But have you considered the data might have been replicated, or swapped to disk?
For these reasons I use the https://github.com/awnumar/memguard package.
It provides features to destroy the data when no longer required, while keeping it safe in the mean time.
You can read about its background here; https://spacetime.dev/memory-security-go
How about checking (some of) the content of the buffer after zeroing it and passing it to another function? For example:
copy(privateKey, make([]byte, keySize))
if privateKey[0] != 0 {
// If you pass the buffer to another function,
// this check and above copy() can't be optimized away:
fmt.Println("Zeroing failed", privateKey[0])
}
To be absolutely safe, you could XOR the passed buffer content with random bytes, but if / since the zeroing is not optimized away, the if body is never reached.
You might think a very intelligent compiler might deduce the above copy() zeros privateKey[0] and thus determine the condition is always false and still optimize it away (although this is very unlikely). The solution to this is not to use make([]byte, keySize) but e.g. a slice coming from a global variable or a function argument (whose value can only be determined at runtime) so the compiler can't be smart enough to deduce the condition is going to be always false at compile time.
As I've read all objects in D are fully location independent. How this requirement is achieved?
One thing that comes to my mind, is that all references are not pointers to the objects, but to some proxy, so when you move object (in memory) you just update that proxy, not all references used in program.
But this is just my guess. How it is done in D for real?
edit: bottom line up front, no proxy object, objects are referenced directly through regular pointers. /edit
structs aren't allowed to keep a pointer to themselves, so if they get copied, they should continue to just work. This isn't strictly enforced by the language though:
struct S {
S* lol;
void beBad() {
lol = &this; // this compiler will allow this....
}
}
S pain() {
S s;
s.beBad();
return s;
}
void main() {
S s;
s = pain();
assert(s.lol !is &s); // but it will also move the object without notice!
}
(EDIT: actually, I guess you could use a postblit to update internal pointers, so it isn't quite without notice. If you're careful enough, you could make it work, but then again, if you're careful enough, you can shoot between your toes without hitting your foot too. EDIT2: Actually no, the compiler/runtime is still allowed to move it without even calling the postblit. One example of where this happens is if it copies a stack frame to the heap to make a closure. The struct data is moved to a new address without being informed. So yeah. /edit)
And actually, that assert isn't guaranteed to pass, the compiler might choose to call pain straight on the local object declared in main, so the pointer would work (though I'm not able to force this optimization here for a demo, generally, when you return a struct from a function, it is actually done via a hidden pointer the caller passes - the caller says "put the return value right here" thus avoiding a copy/move in some cases).
But anyway, the point just is that the compiler is free to copy or not to copy a struct at its leisure, so if you do keep the address of this around in it, it may become invalid without notice; keeping that pointer is not a compile error, but it is undefined behavior.
The situation is different with classes. Classes are allowed to keep references to this internally since a class is (in theory, realized by the garbage collector implementation)) an independent object with an infinite lifetime. While it may be moved (such as be a moving GC (not implemented in D today)), if it is moved, all references to it, internal and external, would also be required to be updated.
So classes can't have the memory pulled out from under them like structs can (unless you the programmer take matters into your own hands and bypass the GC...)
The location independent thing I'm pretty sure is referring only to structs and only to the rule that they can't have pointers to themselves. There's no magic done with references or pointers - they indeed work with memory addresses, no proxy objects.
I have a method as below
SomeStruct* abc;
void NullABC()
{
abc = NULL;
}
This is just example and not very interesting.
Many thread could call this method at the same time.
Do I need to lock "abc = NULL" line?
I think it is just pointer so it could be done in one shot and there isn't really need for it but just wanted to make sure.
Thanks
It depends on the platform on which you are running. On many platforms, as long as abc is correctly aligned, the write will be atomic.
However, if your platform does not have such a guarantee, you need to synchronize access to the variable, using a lock, an atomic variable, or an interlocked operation.
No you do not need a lock, at least not on x86. A memory barrier is required in may real world situations though, and locking is one way to get this (the other would be an explicit barrier). You may also consider using an interlocked operation, like VisualC's InterlockedExchangePointer if you need access to the original pointer. There are equivalent intrinsics supported by most compilers.
If no other threads are ever using abc for any other purpose, then the code as shown is fine... but of course it's a bit silly to have a pointer that never gets used except to set it to NULL.
If there is some other code somewhere that does something like this, OTOH:
if (abc != NULL)
{
abc->DoSomething();
}
Then in this case both the code that uses the abc pointer (above) and the code that changes it (that you posted) needed to lock a mutex before accessing (abc). Otherwise the code above risks crashing if the value of abc gets set to NULL after the if statement but before the DoSomething() call.
A borderline case would be if the other code does this:
SomeStruct * my_abc = abc;
if (my_abc != NULL)
{
my_abc->DoSomething();
}
That will probably work, because at the time the abc pointer's value is copied over to my_abc, the value of abc is either NULL or it isn't... and my_abc is a local variable, so other thread's won't be able to change it before DoSomething() is called. The above could theoretically break on some platforms where copying of pointers isn't atomic though (in which case my_abc might end up being an invalid pointer, with have of abc's bits and half NULL bits)... but common/PC hardware will copy pointers atomically, so it shouldn't be an issue there. It might be worthwhile to use a Mutex anyway just to for paranoia's sake though.