.foreach (runtime {!da 00000086c74c3b70 })
{
.foreach(obj {!dumpobj poi(poi(${runtime})+0x8)})
{
.if(0 == $sicmp("${obj}", "abcdxxxxxxx"))
{
.echo *****
!dumpobj poi(poi(${runtime})+0x8)
!dumpobj poi(${runtime})
!dumpobj poi(${runtime}+0x8)
.echo *****
}
}
}
I run the code above by windbg, and the windbg process consumed almost all of my PC memory(6G), and wouldn't released it even the above script ran done. BTW,the results of !da 00000086c74c3b70 is about 8000 rows
My questions is how to release the memory or prevent the memory leak?
lets go step by step
the result of !da is as follows
how can you do .foreach on this without filtering
.foreach will be passing Name: , System.Int32 , MethodTable , 621ff680 ....
[0] ,,, [n] , [actual address] in your script and no wonder windbg
tries like hell to read crap from Name , System.int32 strings and trying to dump them as an object
i would say even if you have 60 gb it wouldnt be sufficient if you do .foreach like this take some time to read the docs
0:004> !da 016e1da4
Name: System.Int32[]
MethodTable: 621ff680
EEClass: 61e3fd78
Size: 300(0x12c) bytes
Array: Rank 1, Number of elements 72, Type Int32
Element Methodtable: 621ff6bc
[0] 016e1dac
[1] 016e1db0
[2] 016e1db4
[3] 016e1db8
you probably want to do .foreach on 16e1dac,db0,db4,db8 etc ??
if so your first line should be mimicking this ie
instead of dd place l1
you should be using your {? poi(${place}) } whatever
also make sure every dereference here can actually be dereferenced
0:004> .foreach /pS 16 /ps 1 (place { !da 016e1da4 } ) {dd place l1 }
016e1dac 00000003
016e1db0 00000007
016e1db4 0000000b
016e1db8 00000011
016e1dbc 00000017
016e1dc0 0000001d
016e1dc4 00000025
in this example i cant dereference anything > poi(${place}) because it is an Int Array
0:004> .foreach /pS 16 /ps 1 (place { !da -length 5 016e1da4 } ) {? poi(${place}) }
Evaluate expression: 3 = 00000003
Evaluate expression: 7 = 00000007
Evaluate expression: 11 = 0000000b
Evaluate expression: 17 = 00000011
Evaluate expression: 23 = 00000017
you can't do arbitrary things and expect sane results
Related
I'm pretty new to SPIN and Promela and I came across this error when I'm trying to verify the liveness property in my models.
Error code:
unreached in proctype P
(0 of 29 states)
unreached in proctype monitor
mutex_assert.pml:39, state 1, "assert(!((mutex>1)))"
mutex_assert.pml:42, state 2, "-end-"
(2 of 2 states)
unreached in init
(0 of 3 states)
unreached in claim ltl_0
_spin_nvr.tmp:10, state 13, "-end-"
(1 of 13 states)
pan: elapsed time 0 seconds
The code is basically an implementation of Peterson's algorithm and I checked for safety and it seems to be valid. But whenever I try to validate the liveness property using the ltl {[]((wait -> <> (cs)))}, it comes up with the above errors. I'm not sure what they mean so I don't know how to proceed...
My code is as follows:
#define N 3
#define wait (P[1]#WAIT)
#define cs (P[1]#CRITICAL)
int pos[N];
int step[N];
int enter;
byte mutex;
ltl {[]((wait -> <> (cs)))}
proctype P(int i) {
int t;
int k;
WAIT:
for (t : 1 .. (N-1)){
pos[i] = t
step[t] = i
k = 0;
do
:: atomic {(k != i && k < N && (pos[k] < t|| step[t] != i)) -> k++}
:: atomic {k == i -> k++}
:: atomic {k == N -> break}
od;
}
CRITICAL:
atomic {mutex++;
printf("MSC: P(%d) HAS ENTERED THE CRITICAL SECTION.\n", i);
mutex--;}
pos[i] = 0;
}
init {
atomic { run P(0); }
}
General Answer
This is a warning telling you that some states are unreachable due to transitions that are never taken.
In general, this is not an error, but it is a good practice to take a close look to the unreachable states for every routine that you modelled, and check that you expect none of them to be reachable. i.e. in the case the model is not correct wrt. the intended behaviour.
Note. You can use the label end: in front of a particular line of code to mark valid terminating states, so to get rid of those warnings, e.g. when your procedure does not terminate. More info here.
Specific Answer
I can not reproduce your output. In particular, by running
~$ spin -a file.pml
~$ gcc pan.c
~$ ./a.out -a
I get the following output, which is different from yours:
(Spin Version 6.4.3 -- 16 December 2014)
+ Partial Order Reduction
Full statespace search for:
never claim + (ltl_0)
assertion violations + (if within scope of claim)
acceptance cycles + (fairness disabled)
invalid end states - (disabled by never claim)
State-vector 64 byte, depth reached 47, errors: 0
41 states, stored (58 visited)
18 states, matched
76 transitions (= visited+matched)
0 atomic steps
hash conflicts: 0 (resolved)
Stats on memory usage (in Megabytes):
0.004 equivalent memory usage for states (stored*(State-vector + overhead))
0.288 actual memory usage for states
128.000 memory used for hash table (-w24)
0.534 memory used for DFS stack (-m10000)
128.730 total actual memory usage
unreached in proctype P
(0 of 29 states)
unreached in init
(0 of 3 states)
unreached in claim ltl_0
_spin_nvr.tmp:10, state 13, "-end-"
(1 of 13 states)
pan: elapsed time 0 seconds
In particular, I lack the warnings about the unreached states in the monitor process. As far as I am concerned, juding from the source code, none of the warnings I obtained is problematic.
Either you are using a different version of Spin than me, or you did not include the full source code in your question. In the latter case, could you edit your question and add the code? I'll update my answer afterwards.
EDIT: in the comments, you ask what does the following message mean: "unreached in claim ltl_0 _spin_nvr.tmp:10, state 13, "-end-"".
If you open the file _spin_nvr.tmp, you can see the following piece of Promela code, which corresponds to a Büchi automaton that accepts all and only the execution which violate your ltl property []((wait -> <> (cs))).
never ltl_0 { /* !([] ((! ((P[1]#WAIT))) || (<> ((P[1]#CRITICAL))))) */
T0_init:
do
:: (! ((! ((P[1]#WAIT)))) && ! (((P[1]#CRITICAL)))) -> goto accept_S4
:: (1) -> goto T0_init
od;
accept_S4:
do
:: (! (((P[1]#CRITICAL)))) -> goto accept_S4
od;
}
The message simply warns you that the execution of this code will never reach the last closing bracket } (state "-end-"), meaning that the procedure does never terminate.
I trying to use this package in Github for string matching. My dictionary is 4 MB. When creating the Trie, I got fatal error: runtime: out of memory. I am using Ubuntu 14.04 with 8 GB of RAM and Golang version 1.4.2.
It seems the error come from the line 99 (now) here : m.trie = make([]node, max)
The program stops at this line.
This is the error:
fatal error: runtime: out of memory
runtime stack:
runtime.SysMap(0xc209cd0000, 0x3b1bc0000, 0x570a00, 0x5783f8)
/usr/local/go/src/runtime/mem_linux.c:149 +0x98
runtime.MHeap_SysAlloc(0x57dae0, 0x3b1bc0000, 0x4296f2)
/usr/local/go/src/runtime/malloc.c:284 +0x124
runtime.MHeap_Alloc(0x57dae0, 0x1d8dda, 0x10100000000, 0x8)
/usr/local/go/src/runtime/mheap.c:240 +0x66
goroutine 1 [running]:
runtime.switchtoM()
/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc208518a60 sp=0xc208518a58
runtime.mallocgc(0x3b1bb25f0, 0x4d7fc0, 0x0, 0xc20803c0d0)
/usr/local/go/src/runtime/malloc.go:199 +0x9f3 fp=0xc208518b10 sp=0xc208518a60
runtime.newarray(0x4d7fc0, 0x3a164e, 0x1)
/usr/local/go/src/runtime/malloc.go:365 +0xc1 fp=0xc208518b48 sp=0xc208518b10
runtime.makeslice(0x4a52a0, 0x3a164e, 0x3a164e, 0x0, 0x0, 0x0)
/usr/local/go/src/runtime/slice.go:32 +0x15c fp=0xc208518b90 sp=0xc208518b48
github.com/mf/ahocorasick.(*Matcher).buildTrie(0xc2083c7e60, 0xc209860000, 0x26afb, 0x2f555)
/home/go/ahocorasick/ahocorasick.go:104 +0x28b fp=0xc208518d90 sp=0xc208518b90
github.com/mf/ahocorasick.NewStringMatcher(0xc208bd0000, 0x26afb, 0x2d600, 0x8)
/home/go/ahocorasick/ahocorasick.go:222 +0x34b fp=0xc208518ec0 sp=0xc208518d90
main.main()
/home/go/seme/substrings.go:66 +0x257 fp=0xc208518f98 sp=0xc208518ec0
runtime.main()
/usr/local/go/src/runtime/proc.go:63 +0xf3 fp=0xc208518fe0 sp=0xc208518f98
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208518fe8 sp=0xc208518fe0
exit status 2
This is the content of the main function (taken from the same repo: test file)
var dictionary = InitDictionary()
var bytes = []byte(""Partial invoice (€100,000, so roughly 40%) for the consignment C27655 we shipped on 15th August to London from the Make Believe Town depot. INV2345 is for the balance.. Customer contact (Sigourney) says they will pay this on the usual credit terms (30 days).")
var precomputed = ahocorasick.NewStringMatcher(dictionary)// line 66 here
fmt.Println(precomputed.Match(bytes))
Your structure is awfully inefficient in terms of memory, let's look at the internals. But before that, a quick reminder of the space required for some go types:
bool: 1 byte
int: 4 bytes
uintptr: 4 bytes
[N]type: N*sizeof(type)
[]type: 12 + len(slice)*sizeof(type)
Now, let's have a look at your structure:
type node struct {
root bool // 1 byte
b []byte // 12 + len(slice)*1
output bool // 1 byte
index int // 4 bytes
counter int // 4 bytes
child [256]*node // 256*4 = 1024 bytes
fails [256]*node // 256*4 = 1024 bytes
suffix *node // 4 bytes
fail *node // 4 bytes
}
Ok, you should have a guess of what happens here: each node weighs more than 2KB, this is huge ! Finally, we'll look at the code that you use to initialize your trie:
func (m *Matcher) buildTrie(dictionary [][]byte) {
max := 1
for _, blice := range dictionary {
max += len(blice)
}
m.trie = make([]node, max)
// ...
}
You said your dictionary is 4 MB. If it is 4MB in total, then it means that at the end of the for loop, max = 4MB. It it holds 4 MB different words, then max = 4MB*avg(word_length).
We'll take the first scenario, the nicest one. You are initializing a slice of 4M of nodes, each of which uses 2KB. Yup, that makes a nice 8GB necessary.
You should review how you build your trie. From the wikipedia page related to the Aho-Corasick algorithm, each node contains one character, so there is at most 256 characters that go from the root, not 4MB.
Some material to make it right: https://web.archive.org/web/20160315124629/http://www.cs.uku.fi/~kilpelai/BSA05/lectures/slides04.pdf
The node type has a memory size of 2084 bytes.
I wrote a litte program to demonstrate the memory usage: https://play.golang.org/p/szm7AirsDB
As you can see, the three strings (11(+1) bytes in size) dictionary := []string{"fizz", "buzz", "123"} require 24 MB of memory.
If your dictionary has a length of 4 MB you would need about 4000 * 2084 = 8.1 GB of memory.
So you should try to decrease the size of your dictionary.
Set resource limit to unlimited worked for me
if ulimit -a return 0 run ulimit -c unlimited
Maybe set a real size limit to be more secure
I am currently working on some scientific calculations for which my base calculation loops are executed over and over again with recursive calls as long as at least one parameter is false.
Currently my nodejs server stops at around 905 - 915th recursive function call.
The weird thing is, that it doesn't crash, nor output any error. It just stops doing anything -> no more logs etc.
Is this some protection behaviour from node to avoid overflow?
I am struggling with this for a few weeks now whilst trying to limit the "loops" with an as intelligent as possible software.
Thank you for your help & advice.
Greetings Noa.
As requested I provide some abstraction of my actual code
I hope this helps. I cannot put my original code on here because it consists of more than 1.5k lines - theres a lot to check.
But the below example covers the base logic behind the recursive call.
// Contains objects which contain an array
// which represents the amount of the ex_obj terms
var amount = {
a:[10,10],
b:[7.5,7.5],
c:[2.5,2.5,2.5,2.5]
}
// Contains objects, which contain an array of other objects
// that represent some selection
// Each object in an array consists of different values per attribute per 1 amount
var ex_obj = {
a: [
{aa: 1.41, ab: 0.143, ac: 0.5},
{aa: 1.30, ab: 1.43, ac: 0.42}
],
b: [
{aa: 0.32, ab: 5.54, ac: 1.3},
{aa: 0.33, ab: 4.49, ac: 2.5}
],
c: [
{aa: 0.54, ab: 1.4, ac: 3.5},
{aa: 0.39, ab: 1.434, ac: 3.91},
{aa: 0.231, ab: 1.44324, ac: 2.91},
{aa: 0.659, ab: 1.554, ac: 3.9124},
]
}
// Here we have an object that represents
// the "to be" state which should be achieved
var should_be ={
aa: 14.534,
ab: 3.43,
ac: 5.534
}
function calculate(){
// Now we want to mulitply the amount object values with
// the ex_obj values
for(let prop in ex_obj){
for(let i = 0, i < ex_obj[prop].length, i++){
for(let propa in ex_obj[prop][i]){
// here every aa,ab,ac gets mulitplied with the
// contains of the amount obj for the matching
// propertyname
}
}
}
// the next step is to check if the sum of all ex_obj property
// child values per aa, ab and ac match the should_be propertie values
// if everything is above the should_be and not too high then the
// programm can stop here and print out the amount obj.
// if the sum of the ex_obj properties is too little
// compared to the should_be obj
// we need to check which property is too little
// e.g. aa is too little
// then we check where aa in the ex_obj per 1 value is
// the highest
// then we increment the matching amount property child
// and start calculate() again
// same procedure for - if something is too much
}
Since your code isn't complete it's hard to say exactly what's going wrong. If you exceed node's call stack limit you will get an exception, although 1000 recursions isn't normally a problem.
It might be that you're choking the Node.js event-loop. Instead of directly calling the recursive function, you can try to call
process.nextTick(function(){calculate()})
It's not suprised that direct recursive call will result in stack overflow, however, it seems that frequent function calls will get node hang up(BTW, I don't know why this happens :( ).
For example, the following script get frozen at around 4k ~ 5k loops on my machine with 1.6 GHz CPU and 3.3 GiB memory, and node keeps swallowing my available memory after that.
var i = 0;
function cal() {
console.log(++i);
process.nextTick(cal);
}
cal();
But if I changed process.nextTick(cal); to setTimeout(cal, 0);, everything works fine.
So as for your question, I think you can use something like setTimeout(calculate, 100) in your script to prevent recursive call and defer it slightly.
I have a server that has a function to return the registration date of a user. Any user can see any non-hidden user (in this example only user2 is hidden). The server gets a dateRequest from a client and uses id from that to find the corresponding user's date in the user file:
package main
import (
"bytes"
"fmt"
)
const datelen = 10
//format: `name 0xfe date age 0xfd password`; each user is seperated by 0xff
var userfile = []byte("admin\xfe2014-01-0140\xfdadminpassword\xffuser1\xfe2014-03-0423\xfduser1password\xffuser2\xfe2014-09-2736\xfduser2password")
func main() {
c := clientInfo{0, 0, 0}
fmt.Println(string(getDate(dateRequest{c, user{0, "admin"}})))
fmt.Println(string(getDate(dateRequest{c, user{0, "admin______________"}})))
fmt.Println(string(getDate(dateRequest{c, user{1, "user1"}})))
//fmt.Println(string(getDate(dateRequest{c,user{2,"user2"}}))) // panic
fmt.Println(string(getDate(dateRequest{c, user{1, "user1_________________________________________________"}})))
}
func getDate(r dateRequest) []byte {
if r.id == 2 {
panic("hidden user")
}
user := bytes.Split(userfile, []byte{0xff})[r.id]
publicSection := bytes.Split(user, []byte{0xfd})[0]
return publicSection[len(r.username)+1 : len(r.username)+1+datelen]
}
type dateRequest struct {
clientInfo
user
}
type clientInfo struct {
reqTime uint64
ip uint32
ver uint32
}
type user struct {
id int
username string
}
$ go run a.go
2014-01-01
dminpasswo
2014-03-04
r2password
As you can see, it works correctly to receive the dates of the users, but if the request has extra bytes on the username, part of the user's password gets returned instead. Not only that, but if you keep adding more bytes, it returns data from user2, which is supposed to be hidden. Why?
When executing main's third line of code, user and publicSection in getData are "admin�2014-01-0140�adminpassword" and "admin�2014-01-0140". But then it returns "dminpasswo". How is slicing publicSection ("admin�2014-01-0140") returning "dminpasswo"? It looks like a buffer overrun problem, but that shouldn't happen because Go is memory-safe. I even tried reading past the buffer of publicSection by printing publicSection[len(publicSection)], but it panics as expected.
I also tried replacing all the []byte with string, and that fixes the issue for some reason.
Slice expressions check the bounds of the upper index against the slice capacity, not just the length of the slice. In effect, you can slice past the length of the slice. You cannot however slice outside the bounds of the underlying array to access uninitialized memory.
http://play.golang.org/p/oIxXLG-YEV
s := make([]int, 5, 10)
copy(s, []int{1, 2, 3, 4, 5})
fmt.Printf("len:%d cap:%d\n", len(s), cap(s))
// > len:5 cap:10
fmt.Printf("raw slice: %+v\n", s)
// > raw slice: [1 2 3 4 5]
fmt.Printf("sliced past length: %+v\n", s[:10])
// > sliced past length: [1 2 3 4 5 0 0 0 0 0]
// panics
_ = s[:11]
// > panic: runtime error: slice bounds out of range
If you really want to prevent slicing past the length of an array, in go1.3 or later you can set the capacity as the third argument when slicing.
// set the capacity to 5
s := s[:5:5]
// now this will panic
_ = s[:6]
I'm trying to reproduce ALGOL 60 code written by Dijkstra in the paper titled "Cooperating sequential processes", the code is the first attempt to solve the mutex problem, here is the syntax:
begin integer turn; turn:= 1;
parbegin
process 1: begin Ll: if turn = 2 then goto Ll;
critical section 1;
turn:= 2;
remainder of cycle 1; goto L1
end;
process 2: begin L2: if turn = 1 then goto L2;
critical section 2;
turn:= 1;
remainder of cycle 2; goto L2
end
parend
end
So I tried to reproduce the above code in Promela and here is my code:
#define true 1
#define Aturn true
#define Bturn false
bool turn, status;
active proctype A()
{
L1: (turn == 1);
status = Aturn;
goto L1;
/* critical section */
turn = 1;
}
active proctype B()
{
L2: (turn == 2);
status = Bturn;
goto L2;
/* critical section */
turn = 2;
}
never{ /* ![]p */
if
:: (!status) -> skip
fi;
}
init
{ turn = 1;
run A(); run B();
}
What I'm trying to do is, verify that the fairness property will never hold because the label L1 is running infinitely.
The issue here is that my never claim block is not producing any error, the output I get simply says that my statement was never reached..
here is the actual output from iSpin
spin -a dekker.pml
gcc -DMEMLIM=1024 -O2 -DXUSAFE -DSAFETY -DNOCLAIM -w -o pan pan.c
./pan -m10000
Pid: 46025
(Spin Version 6.2.3 -- 24 October 2012)
+ Partial Order Reduction
Full statespace search for:
never claim - (not selected)
assertion violations +
cycle checks - (disabled by -DSAFETY)
invalid end states +
State-vector 44 byte, depth reached 8, errors: 0
11 states, stored
9 states, matched
20 transitions (= stored+matched)
0 atomic steps
hash conflicts: 0 (resolved)
Stats on memory usage (in Megabytes):
0.001 equivalent memory usage for states (stored*(State-vector + overhead))
0.291 actual memory usage for states
128.000 memory used for hash table (-w24)
0.534 memory used for DFS stack (-m10000)
128.730 total actual memory usage
unreached in proctype A
dekker.pml:13, state 4, "turn = 1"
dekker.pml:15, state 5, "-end-"
(2 of 5 states)
unreached in proctype B
dekker.pml:20, state 2, "status = 0"
dekker.pml:23, state 4, "turn = 2"
dekker.pml:24, state 5, "-end-"
(3 of 5 states)
unreached in claim never_0
dekker.pml:30, state 5, "-end-"
(1 of 5 states)
unreached in init
(0 of 4 states)
pan: elapsed time 0 seconds
No errors found -- did you verify all claims?
I've read all the documentation of spin on the never{..} block but couldn't find my answer (here is the link), also I've tried using ltl{..} blocks as well (link) but that just gave me syntax error, even though its explicitly mentioned in the documentation that it can be outside the init and proctypes, can someone help me correct this code please?
Thank you
You've redefined 'true' which can't possibly be good. I axed that redefinition and the never claim fails. But, the failure is immaterial to your goal - that initial state of 'status' is 'false' and thus the never claim exits, which is a failure.
Also, it is slightly bad form to assign 1 or 0 to a bool; assign true or false instead - or use bit. Why not follow the Dijkstra code more closely - use an 'int' or 'byte'. It is not as if performance will be an issue in this problem.
You don't need 'active' if you are going to call 'run' - just one or the other.
My translation of 'process 1' would be:
proctype A ()
{
L1: turn !=2 ->
/* critical section */
status = Aturn;
turn = 2
/* remainder of cycle 1 */
goto L1;
}
but I could be wrong on that.