Go binary data being truncated on string conversion - string
I am using GORM. My data field is varbinary(2000) but why is the data being inserted (new) being truncated to around 20 bytes?
var charData CharacterData
db.Where(CharacterData{CharId: charId, Segment: 0}).FirstOrInit(&charData)
charData.Data = data[16:]
db.Save(&charData)
len(data[16:]) is 2000.
type CharacterData struct {
CharId uint32
Data []byte
Segment uint8
SegmentId uint16
CreatedAt time.Time
UpdatedAt time.Time
DeletedAt time.Time
}
I also tried:
fmt.Println(string(charData0))
fmt.Println(len(charData0))
fmt.Println(len(string(charData0)))
0uyuyuyyuyuyuyy���������������������������� // only this gets inserted to the db which is short of the actual 1587
1587
1587
And it is also truncating the data. So I guess the problem lies with the to string casting.
The insert log shows:
INSERT INTO character_data (char_id, data, segment) VALUES ('2', '0tututututt����������������������������', '0')
data[16:] contains:
[0 0 48 6 1 0 1 0 104 101 104 101 104 101 104 101 104 101 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 255 255 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
update
Actual data inserted to database:
0tututututtÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
Instead of what normally should be something similar to:
pdukelÐÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿHHllbÍfÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿúúúúúúúÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿþ ú
úúþ
þþþþÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ'ÿÿ
using raw queries
var charData0 []byte = data[16:]
hexData := fmt.Sprintf("%x", charData0)
fmt.Println(hexData)
db.Exec("INSERT INTO character_data (char_id, data, segment) VALUES (?, UNHEX(?), ?)", charId, hexData, 0)
=> 000030060100010079697969796979697979697900000000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
But the database only got:
0yiyiyiyiyyiyÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
update
Actually tried running the query on sqlpro:
INSERT INTO character_data (char_id, data, segment) VALUES ('4', UNHEX('000030060100010074757475747574757475740000000000050000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff0000000000000000000000000000ffff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000'), '0')
And I still got truncated data. So does the problem lie with MYSQL?
EDIT:
You mentioned in the comment that you use MariaDB (not MySQL).
See MariaDB Tutorial: DATA TYPES:
VARBINARY(size) - Maximum size of 255 characters. - Where size is the number of characters to store. Variable-length string.
varbinary length is limited to 255 characters... Also it is in section STRING DATATYPES. This means MariaDB handles varbinary as a string...
This may very well explain the value you see: after about 255 bytes it contains just 0 values and nothing else, most likely because only 255 bytes come from the database and the rest is just filled with the zero-value of byte (which is 0).
A string in Go is stored as a byte array and interpreted using UTF-8 encoding.
A string may contain any byte sequence, but not all byte sequences represent valid UTF-8 encoded text.
So if you use Data to store arbitrary seuqence of bytes, you can't always display or treat it as a text. Just because printing the result of string(Data) conversion displays only 16 characters it doesn't mean Data contains only 16 bytes (or the bytes of 16 characters).
You can check it by printing its length:
fmt.Println(len(charData.Data))
Read this blog post to learn more about the topic:
Strings, bytes, runes and characters in Go
Edit:
You mentioned you use MySQL. What version?
MySQL version 5.0.2 and below only allows 255 bytes in varbinary, version 5.0.3 and above allows 64 KB. If the data inserted does not fit into the max (or defined) length, it is truncated.
Also please see MySQL 5.6 Reference Manual :: Section 11 Data Types :: 11.7 Data Type Storage Requirements:
The internal representation of a table has a maximum row size of 65,535 bytes, even if the storage engine is capable of supporting larger rows.
So if you (obviously) have other values in the row, they also have to fit into the row limit (which is 64 KB). Note that this is MySQL 5.6.
Related
edit sample ID in vcf file using sed command
I have a vcf file in which ID is represented like this: GTEX-1117F_GTEX-1117F I want to edit the ID to look like: only GTEX-1117F and exclude other part of the ID for all GTEX values. Id GTEX-1117F_GTEX-1117F GTEX-111CU_GTEX-111CU GTEX-111FC_GTEX-111FC GTEX-111VG_GTEX-111VG GTEX-111YS_GTEX-111YS GTEX-1122O_GTEX-1122O GTEX-1128S_GTEX-1128S GTEX-113IC_GTEX-113IC I tried to use this command: sed -e '$s/\[[[:digit:]]\+\]//g; s/_GTEX[[:digit:]]\+//g' genotype_chr22_filtered_dosage2.txt > genotype_chr22_filtered_dosage3.txt This is how my file looks like: Id GTEX-1117F_GTEX-1117F GTEX-111CU_GTEX-111CU GTEX-111FC_GTEX-111FC GTEX-111VG_GTEX-111VG GTEX-111YS_GTEX-111YS GTEX-1122O_GTEX-1122O GTEX- 1128S_GTEX-1128S GTEX-113IC_GTEX-113IC GTEX-113JC_GTEX-113JC GTEX-117XS_GTEX-117XS GTEX-117YW_GTEX-117YW GTEX-117YX_GTEX-117YX GTEX-1192W_GTEX-11 92W GTEX-1192X_GTEX-1192X GTEX-11DXW_GTEX-11DXW GTEX-11DXX_GTEX-11DXX GTEX-11DXY_GTEX-11DXY GTEX-11DXZ_GTEX-11DXZ GTEX-11DYG_GTEX-11DYG GTEX-11 DZ1_GTEX-11DZ1 GTEX-11EI6_GTEX-11EI6 GTEX-11EM3_GTEX-11EM3 GTEX-11EMC_GTEX-11EMC GTEX-11EQ8_GTEX-11EQ8 GTEX-11EQ9_GTEX-11EQ9 GTEX-ZVZP_GTEX-ZVZP GTEX-ZVZQ_GTEX-ZVZQ GTEX-ZWKS_GTEX-ZWKS GTEX-ZXES_GTEX-ZXES GTEX-ZXG5_GTEX-ZXG5 GTEX-ZY6K_GTEX-ZY6K GTEX-ZYFC_GTEX-ZYFC GTEX-ZYFD_GTEX-ZYFD GTEX-ZYFG_GTEX-ZYFG GTEX-ZYT6_GTEX-ZYT6 GTEX-ZYVF_GTEX-ZYVF GTEX-ZYW4_GTEX-ZYW4 GTEX-ZYWO_GTEX-ZYWO GTEX-ZYY3_GTEX-ZYY3 GTEX-ZZ64_GTEX-ZZ64 GTEX-ZZPT_GTEX-ZZPT GTEX-ZZPU_GTEX-ZZPU 22_20000146_G_A_b37 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
difference between reading the /proc/stat files via syscall fucntion read () or via glibc function fread()
Program reading /proc/stat files via read(like cat) or via fread (like nl) return different values. read() /proc/stat return right result, while fread() return wrong result.
I had write simple C programs that just read or fread /proc/stat. read() return right values, like cat /proc/stat. fread() return wrong values,like nl /proc/stat.
I execut "top" command in my Centos7 system and then press "1" to show every cpu's detail information, it only show cpu0's information, while my system have 8 cpu cores. further more, the cpu0's information is wrong.
So I read the source code of top command, found out that it user fread() to get cpu info from /proc/stat.I change the file reading interface from glibc fread() function to read(), and it works well.
I use fread() to retrieve other file in /proc,like /proc/meminfo, it works well too.
fread() to get info from /proc/stat（command 'nl' use fread() to read content in a file）
[root#pro_app_server workspace]# nl /proc/stat
1 cpu 0 0 3375 498153 97 0 10 0 0 0
2 cpu0 0 0 1699 249246 71 0 6 0 0 0
3 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
4 ctxt 543672
5 btime 1544675938
6 processes 2774
7 procs_running 4
8 procs_blocked 0
9 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
10 cpu0 0 0 1699 249246 71 0 6 0 0 0
11 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12 ctxt 543672
13 btime 1544675938
14 processes 2774
15 procs_running 4
16 procs_blocked 0
17 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
18 cpu0 0 0 1699 249246 71 0 6 0 0 0
19 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
20 ctxt 543672
21 btime 1544675938
22 processes 2774
23 procs_running 4
24 procs_blocked 0
25 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
26 cpu0 0 0 1699 249246 71 0 6 0 0 0
27 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
28 ctxt 543672
29 btime 1544675938
30 processes 2774
31 procs_running 4
32 procs_blocked 0
33 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
34 cpu0 0 0 1699 249246 71 0 6 0 0 0
35 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
36 ctxt 543672
37 btime 1544675938
38 processes 2774
39 procs_running 4
40 procs_blocked 0
41 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
42 cpu0 0 0 1699 249246 71 0 6 0 0 0
43 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
44 ctxt 543672
45 btime 1544675938
46 processes 2774
47 procs_running 4
48 procs_blocked 0
49 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
50 cpu0 0 0 1699 249246 71 0 6 0 0 0
51 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
52 ctxt 543672
53 btime 1544675938
54 processes 2774
55 procs_running 4
56 procs_blocked 0
57 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
58 cpu0 0 0 1699 249246 71 0 6 0 0 0
59 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
60 ctxt 543672
61 btime 1544675938
62 processes 2774
63 procs_running 4
64 procs_blocked 0
65 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
66 cpu0 0 0 1699 249246 71 0 6 0 0 0
67 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
68 ctxt 543672
69 btime 1544675938
70 processes 2774
71 procs_running 4
72 procs_blocked 0
73 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
74 cpu0 0 0 1699 249246 71 0 6 0 0 0
75 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
76 ctxt 543672
77 btime 1544675938
78 processes 2774
79 procs_running 4
80 procs_blocked 0
81 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
82 cpu0 0 0 1699 249246 71 0 6 0 0 0
83 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
84 ctxt 543672
85 btime 1544675938
86 processes 2774
87 procs_running 4
88 procs_blocked 0
89 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
90 cpu0 0 0 1699 249246 71 0 6 0 0 0
91 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
92 ctxt 543672
93 btime 1544675938
94 processes 2774
95 procs_running 4
96 procs_blocked 0
97 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
98 cpu0 0 0 1699 249246 71 0 6 0 0 0
99 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
100 ctxt 543672
101 btime 1544675938
102 processes 2774
103 procs_running 4
104 procs_blocked 0
105 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
106 cpu0 0 0 1699 249246 71 0 6 0 0 0
107 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
108 ctxt 543672
109 btime 1544675938
110 processes 2774
111 procs_running 4
112 procs_blocked 0
113 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
114 cpu0 0 0 1699 249246 71 0 6 0 0 0
115 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
116 ctxt 543672
117 btime 1544675938
118 processes 2774
119 procs_running 4
120 procs_blocked 0
121 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337cpu 0 0 3375 498153 97 0 10 0 0 0
122 cpu0 0 0 1699 249246 71 0 6 0 0 0
123 intr 0 28 0 0 0 378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 0 4129 0 124 1 0 1251 0 3 0 6879 6680 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
124 ctxt 543672
125 btime 1544675938
126 processes 2774
127 procs_running 4
128 procs_blocked 0
129 softirq 188202 0 129718 1 457 0 0 1 35688 0 22337
[root#pro_app_server workspace]#
read() to get info from /proc/stat（command 'cat' use read() to read content in a file）
[root#pro_app_server workspace]# cat -n /proc/stat
1 cpu 2058017 66 399693 188962556 16705 0 18516 5046 0 0
2 cpu0 192984 3 47418 23679955 1327 0 3599 658 0 0
3 cpu1 360654 16 59590 23478307 3861 0 2961 850 0 0
4 cpu2 322760 24 53901 23533358 3234 0 2684 736 0 0
5 cpu3 271322 4 52696 23600674 2217 0 2168 660 0 0
6 cpu4 246241 13 48638 23643590 1678 0 1918 557 0 0
7 cpu5 236649 2 46872 23660881 1362 0 1772 553 0 0
8 cpu6 208005 2 45099 23691054 1219 0 1718 484 0 0
9 cpu7 219399 1 45476 23674734 1804 0 1694 545 0 0
10 intr 731124153 637 15753 0 0 0 0 3 0 0 0 0 34 44262 0 0 0 0 0 0 0 0 0 0 0 0 74547 0 0 358100 16030715 243 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
11 ctxt 698139400
12 btime 1559549547
13 processes 1436019
14 procs_running 2
15 procs_blocked 0
16 softirq 240304636 1 96899354 179 55128202 0 0 6370 22706849 0 65563681
[root#pro_app_server workspace]#
user read() and fread() to retrieve other file under /proc, it seems normal
My C program with read()
[root#pro_app_server workspace]# cat readstat.c -n
1 #include <stdio.h>
2 #include <stdlib.h>
3 #include <sys/types.h>
4 #include <sys/stat.h>
5 #include <fcntl.h>
6
7 #define BUF_INC_SIZE 1024
8
9 int main()
10 {
11 int fd = 0;
12 FILE *file = NULL;
13 char *buf =NULL;
14 size_t buf_size = 0, tot_read = 0, sgl_read = 0;
15
16 fd = open("/proc/stat", O_RDONLY);
17 if(fd < 0)
18 perror("read /proc/stat fail!");
19
20 buf = malloc(BUF_INC_SIZE);
21 if(!buf)
22 perror("malloc fail!");
23 buf_size += BUF_INC_SIZE;
24
25 while(-1 != (sgl_read = read(fd, buf + tot_read, buf_size - tot_read)))
26 {
27 tot_read += sgl_read;
28 if(tot_read < buf_size)
29 break;
30
31 buf = realloc(buf, (buf_size += BUF_INC_SIZE));
32 }
33
34 free(buf);
35 close(fd);
36 return 0;
37 }
[root#pro_app_server workspace]#
My C program with fread()
[root#pro_app_server workspace]# cat freadstat.c -n
1 #include <stdio.h>
2 #include <stdlib.h>
3
4 #define BUF_INC_SIZE 1024
5
6 int main()
7 {
8 FILE *file = NULL;
9 char *buf =NULL;
10 size_t buf_size = 0, tot_read = 0, sgl_read = 0;
11
12 file = fopen("/proc/stat", "r+");
13 if(!file)
14 perror("read /proc/stat fail!");
15
16 buf = malloc(BUF_INC_SIZE);
17 if(!buf)
18 perror("malloc fail!");
19
20 buf_size += BUF_INC_SIZE;
21
22 while(0 < (sgl_read = fread(buf + tot_read, 1, buf_size - tot_read, file)))
23 {
24 tot_read += sgl_read;
25 if(tot_read < buf_size)
26 break;
27
28 buf = realloc(buf, (buf_size += BUF_INC_SIZE));
29 }
30
31 free(buf);
32 fclose(file);
33 return 0;
34 }
[root#pro_app_server workspace]#
As the adviser of aschepler, I strace my C program. :
6、strace info of read() /proc/stat (part of result):
[root#pro_app_server workspace]# strace ./readstat
execve("./readstat", ["./readstat"], [/* 23 vars */]) = 0
brk(0) = 0xcfb000
.
.
.
.
open("/proc/stat", O_RDONLY) = 3
read(3, "cpu 2275345 66 527271 203616681"..., 1024) = 1024
read(3, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 "..., 1024) = 602
close(3) = 0
exit_group(0) = ?
+++ exited with 0 +++
[root#pro_app_server workspace]#
7、strace info of fread() /proc/stat (part of result)
[root#pro_app_server workspace]# strace ./freadstat
execve("./freadstat", ["./freadstat"], [/* 23 vars */]) = 0
brk(0) = 0xdeb000
.
.
.
open("/tmp/tmpfeQoAKy", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
unlink("/tmp/tmpfeQoAKy") = 0
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|O_LARGEFILE)
open("/proc/stat", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7692850000
read(4, "cpu 2277177 66 528841 203745262"..., 1024) = 1024
fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f769284f000
write(3, "cpu 0 0 3375 498153 97 0 10 0 0"..., 4096) = 4096
write(3, " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"..., 4096) = 4096
read(4, "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 "..., 1024) = 602
write(3, " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"..., 4096) = 4096
write(3, " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"..., 4096) = 4096
read(4, "", 1024) = 0
close(4) = 0
munmap(0x7f7692850000, 4096) = 0
write(3, "5938\nprocesses 2774\nprocs_runnin"..., 1200) = 1200
lseek(3, 0, SEEK_SET) = 0
read(3, "cpu 0 0 3375 498153 97 0 10 0 0"..., 4096) = 4096
read(3, " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"..., 4096) = 4096
read(3, " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"..., 4096) = 4096
read(3, " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"..., 4096) = 4096
read(3, "5938\nprocesses 2774\nprocs_runnin"..., 4096) = 1200
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7f769284f000, 4096) = 0
exit_group(0) = ?
+++ exited with 0 +++
[root#pro_app_server workspace]#
Python3.x: subtracting y values at the same x
I have two data sets that have slightly different x values. I want to subtract the y values of the two data sets at the same x values. Th following is how the data look like:
First data:
data1_x data1_y1 data1_y2 data1_y3
-566.692382 0 0 0
-456.971091 0 0 0
-347.2498 0 0 0
-237.528509 0 0 0
-127.807218 0 0 0
-18.085927 0 0 0
91.635364 0 0 0
201.356655 0 0 0
311.077946 0 0 0
420.799237 0 0 0
530.520528 0 0 0
640.241819 0 0 0
749.96311 0 0 0
859.684401 0 0 0
969.405692 0 0 0
1079.126983 0 0 0
1188.848274 0 0 0
1298.569565 0 0 0
1408.290856 0 0 0
1518.012147 0 0 0
1627.733439 0 0 0
1737.45473 0 0 0
1847.176021 0 0 0
1956.897312 0 0 0
2066.618603 0 0 0
2176.339894 0 0 0
2286.061185 0 0 0
2395.782476 0 0 0
2505.503767 0 0 0
2615.225058 0 0 0
2724.946349 0 0 0
2834.66764 0 0 0
2944.388931 0 0 0
3054.110222 0 0 0
3163.831513 0.000756 0 0.000225
3273.552804 0.007979 0 0.005741
3383.274095 0.155249 0 0.141201
3492.995386 1 0 0.987494
3602.716677 0.966901 0 1
3712.437968 0.155149 0 0.160298
3822.159259 0.013988 0 0.013816
3931.88055 0.004039 0 0.002835
4041.601841 0.002268 0.029346 0.002405
4151.323132 0.000577 0.028708 0.000225
4261.044424 0.000298 0.336204 0.000256
4370.765715 0 0.889952 0.000307
4480.487006 0 1 0.000133
4590.208297 0 0.73429 0.000194
4699.929588 0 0.540032 0
4809.650879 0 0.481659 0
4919.37217 0 0.227113 0
5029.093461 0 0.155662 0
5138.814752 0 0.051675 0
5248.536043 0 0.023285 0
5358.257334 0.000318 0.035726 0
5467.978625 0 0 0
5577.699916 0 0.016268 0
5687.421207 0 0 0.000133
5797.142498 0 0 0
5906.863789 0 0.026794 0
6016.58508 0 0.004466 0
6126.306371 0 0.004466 0
6236.027662 0 0 0
6345.748953 0 0 0
6455.470244 0 0 0
6565.191535 0 0 0
6674.912826 0 0 0
6784.634117 0 0 0
6894.355408 0 0 0
7004.0767 0 0 0
7113.797991 0 0 0
7223.519282 0 0 0
7333.240573 0 0 0
7442.961864 0 0 0
7552.683155 0 0 0
7662.404446 0 0 0
7772.125737 0 0 0
7881.847028 0 0 0
7991.568319 0 0 0
8101.28961 0 0 0
8211.010901 0 0 0
8320.732192 0 0 0
8430.453483 0 0 0
8540.174774 0 0 0
8649.896065 0 0 0
8759.617356 0 0 0
8869.338647 0 0 0
8979.059938 0 0 0
9088.781229 0 0 0
9198.50252 0 0 0
9308.223811 0 0 0
9417.945102 0 0 0
9527.666393 0 0 0
9637.387685 0 0 0
9747.108976 0 0 0
9856.830267 0 0 0
9966.551558 0 0 0
10076.27285 0 0 0
10185.99414 0 0 0
10295.71543 0 0 0
10405.43672 0 0 0
10515.15801 0 0 0
10624.8793 0 0 0
10734.6006 0 0 0
10844.32189 0 0 0
10954.04318 0 0 0
11063.76447 0 0 0
11173.48576 0 0 0
11283.20705 0 0 0
11392.92834 0 0 0
11502.64963 0 0 0
11612.37092 0 0 0
11722.09221 0 0 0
11831.81351 0 0 0
11941.5348 0 0 0
12051.25609 0 0 0
12160.97738 0 0 0
12270.69867 0 0 0
12380.41996 0 0 0
12490.14125 0 0 0
12599.86254 0 0 0
12709.58383 0 0 0
12819.30513 0 0 0
12929.02642 0 0 0
13038.74771 0 0 0
13148.469 0 0 0
13258.19029 0 0 0
13367.91158 0 0 0
13477.63287 0 0 0
13587.35416 0 0 0
13697.07545 0 0 0
13806.79674 0 0 0
13916.51804 0 0 0
14026.23933 0 0 0
14135.96062 0 0 0
14245.68191 0 0 0
14355.4032 0 0 0
14465.12449 0 0 0
14574.84578 0 0 0
14684.56707 0 0 0
14794.28836 0 0 0
14904.00965 0 0 0
15013.73095 0 0 0
15123.45224 0 0 0
15233.17353 0 0 0
15342.89482 0 0 0
15452.61611 0 0 0
15562.3374 0 0 0
15672.05869 0 0 0
15781.77998 0 0 0
15891.50127 0 0 0
16001.22257 0 0 0
16110.94386 0 0 0
16220.66515 0 0 0
16330.38644 0 0 0
16440.10773 0 0 0
16549.82902 0 0 0
16659.55031 0 0 0
16769.2716 0 0 0
16878.99289 0 0 0
16988.71418 0 0 0
17098.43548 0 0 0
17208.15677 0 0 0
17317.87806 0 0 0
17427.59935 0 0 0
17537.32064 0 0 0
17647.04193 0 0 0
17756.76322 0 0 0
17866.48451 0 0 0
17976.2058 0 0 0
18085.9271 0 0 0
18195.64839 0 0 0
18305.36968 0 0 0
18415.09097 0 0 0
18524.81226 0 0 0
18634.53355 0 0 0
18744.25484 0 0 0
18853.97613 0 0 0
18963.69742 0 0 0
19073.41871 0 0 0
19183.14001 0 0 0.000297
19292.8613 0 0 0
19402.58259 0 0 0
19512.30388 0 0 0
19622.02517 0 0 0
19731.74646 0 0 0
19841.46775 0 0 0
19951.18904 0 0 0
20060.91033 0 0 0
20170.63162 0 0 0
20280.35292 0 0 0
20390.07421 0 0 0
20499.7955 0 0 0
20609.51679 0 0 0
20719.23808 0 0 0
20828.95937 0 0 0
20938.68066 0 0 0
21048.40195 0 0 0
21158.12324 0 0 0
21267.84454 0 0 0
The second data look like
data2_x data2_y1 data2_y2 data2_y3
-613.863532 0 0 0
-504.331019 0 0 0
-394.798507 0 0 0
-285.265994 0 0 0
-175.733482 0 0 0
-66.200969 0 0 0
43.331543 0 0 0
152.864056 0 0 0
262.396569 0 0 0
371.929081 0 0 0
481.461594 0 0 0
590.994106 0 0 0
700.526619 0 0 0
810.059131 0 0 0
919.591644 0 0 0
1029.124156 0 0 0
1138.656669 0 0 0
1248.189181 0 0 0
1357.721694 0 0 0
1467.254207 0 0 0
1576.786719 0 0 0
1686.319232 0 0 0
1795.851744 0 0 0
1905.384257 0 0 0
2014.916769 0 0 0
2124.449282 0 0 0
2233.981794 0 0 0
2343.514307 0 0 0
2453.04682 0 0 0
2562.579332 0 0 0
2672.111845 0 0 0
2781.644357 0 0 0
2891.17687 0 0 0
3000.709382 0 0.00421 0
3110.241895 0 0 0
3219.774407 0 0.000472 0.000258
3329.30692 0.005998 0.005817 0.046254
3438.839433 0.126953 0.025986 0.447706
3548.371945 0.595733 0.007664 1
3657.904458 0.977669 0.010667 0.981762
3767.43697 0.846061 0.000766 0.481531
3876.969483 0.787706 0 0.182611
3986.501995 1 0 0.102414
4096.034508 0.964644 0.03512 0.051297
4205.56702 0.54548 0.161153 0.017741
4315.099533 0.178064 0.558853 0.003784
4424.632045 0.037034 0.789553 0.001666
4534.164558 0.007972 0.96508 0.000394
4643.697071 0.003013 1 0
4753.229583 0.002961 0.903092 0.000163
4862.762096 0.002831 0.669578 0.000293
4972.294608 0.001025 0.375463 0.000108
5081.827121 0.000761 0.251614 0.000161
5191.359633 0 0.139304 0
5300.892146 0 0.044527 0
5410.424658 0.000413 0.008105 0
5519.957171 0 0.003013 6.50E-05
5629.489684 0 0 0
5739.022196 0.000211 0.003381 0
5848.554709 0 0.003633 0
5958.087221 0 0.004651 0.000121
6067.619734 0 0.017901 0
6177.152246 0 0.001134 0
6286.684759 0 0.002394 0
6396.217271 0 0.000462 0
6505.749784 0 0 0
6615.282297 0 0 0
6724.814809 0 0 0
6834.347322 0 0.000504 0
6943.879834 0 0.002016 0
7053.412347 0 0.004326 0
7162.944859 0.000185 0.00546 6.00E-05
7272.477372 0 0 0
7382.009884 0 0 0
7491.542397 0 0 0
7601.074909 0 0 0.000131
7710.607422 0 0 0
7820.139935 0 0 0
7929.672447 0 0 0
8039.20496 0 0 0
8148.737472 0 0 0
8258.269985 0 0 0
8367.802497 0.000273 0 0
8477.33501 0 0 0
8586.867522 0 0.001522 0
8696.400035 0 0 0
8805.932548 0 0 0.000133
8915.46506 0 0 0
9024.997573 0 0 0
9134.530085 0 0 7.20E-05
9244.062598 0 0 0
9353.59511 0 0 0
9463.127623 0 0 0
9572.660135 0 0 0
9682.192648 8.50E-05 0 0
9791.725161 0 0 0
9901.257673 0 0 0
10010.79019 0 0 0
10120.3227 0 0 6.60E-05
10229.85521 0 0 0.000128
10339.38772 0 0 0
10448.92024 0 0 0
10558.45275 0 0 0
10667.98526 0 0 0
10777.51777 0 0 6.60E-05
10887.05029 0 0 6.70E-05
10996.5828 0 0 3.00E-05
11106.11531 0 0 0
11215.64782 0 0 0
11325.18034 0 0 0
11434.71285 0 0 0
11544.24536 0 0 0
11653.77787 0 0 0
11763.31039 0 0 0
11872.8429 0 0 0
11982.37541 0 0 0
12091.90792 0 0 0
12201.44044 0 0 0
12310.97295 0 0 0
12420.50546 0 0 0
12530.03797 0 0 0
12639.57049 0 0 0
12749.103 0 0 0
12858.63551 0 0 0
12968.16803 0 0 0
13077.70054 0 0 0
13187.23305 0 0 0
13296.76556 0 0 0
13406.29808 0 0 0
13515.83059 0 0 0
13625.3631 0 0 0.00011
13734.89561 0 0 0
13844.42813 0 0 0
13953.96064 0 0 0
14063.49315 0 0 0
14173.02566 0 0 0
14282.55818 0 0 0
14392.09069 0 0 0
14501.6232 0 0 0
14611.15571 0 0 0
14720.68823 0 0 0
14830.22074 0 0 0
14939.75325 0.000177 0 0
15049.28576 0 0 0
15158.81828 0 0 0
15268.35079 0 0 0
15377.8833 0 0 0
15487.41581 0.00017 0 0
15596.94833 0 0 0
15706.48084 0 0 0
15816.01335 0 0 4.80E-05
15925.54586 0 0 0
16035.07838 0 0 0
16144.61089 0 0 0
16254.1434 0.000216 0 0
16363.67591 0 0 0
16473.20843 0 0 0
16582.74094 0 0 0
16692.27345 0.000239 0 0
16801.80596 0 0 0
16911.33848 0 0 0
17020.87099 0 0 0
17130.4035 0 0 0
17239.93601 0 0 0
17349.46853 0 0 0
17459.00104 0 0 0
17568.53355 0 0 6.10E-05
17678.06606 0 0 4.50E-05
17787.59858 0 0 0
17897.13109 0 0 0
18006.6636 0 0 0
18116.19611 0.000125 0 0
18225.72863 0 0 0
18335.26114 0 0 0
18444.79365 0 0 0
18554.32617 0 0 0
18663.85868 0 0 0
18773.39119 0 0 0
18882.9237 0 0 0
18992.45622 0 0 0
19101.98873 0 0 0
19211.52124 0 0 0
19321.05375 0 0 0
19430.58627 0 0 0
19540.11878 0 0 0
19649.65129 0 0 0
19759.1838 0 0 0
19868.71632 0 0 0
19978.24883 0 0 0
20087.78134 0 0 0
20197.31385 0 0 0
20306.84637 0 0 0
20416.37888 0 0 0
20525.91139 0 0 0
20635.4439 0 0 0
20744.97642 0 0 0
20854.50893 5.20E-05 0 0
20964.04144 0 0 0
21073.57395 0 0 0
21183.10647 0 0 0
As you can see, the x values are not exactly the same. Given that, data1-data2 would not yield the results I desire. I can infer y values at the same x values using interpolation. For instance, I can do the interpolation using the following code:
Interpolation = interp1d(data1_y, data1_x, assume_sorted = False)
I still haven't come up with how I can subtract the y values of the two data at the same x using the interpolation... Any ideas?
Edit 1):
I meant that data1-data2 to be the following.
data1=pd.read_csv('Data1.csv')
data2=pd.read_csv('Data2.csv')
data1.iloc[:,1:]-data2.iloc[:,1:] #this line is what I meant by data1-data2
Sorry for my insufficient description. Nonethless,
data1.iloc[:,1:]-data2.iloc[:,1:]
will not work, because it will subtract the values from the same index number. However, y values in the same index number has different corresponding x values for each data set.
how can i visualize data using t-SNE?
I'm working with K-means in python3 for educational purpose. I have a data set of clusters and centroids with multi-dimension. I need to visualize those data in 2D using t-SNE. Can anyone help me to do that. (little explanation about the code will be very helpful for me.) Data set is given below: Centroids: [0.0, 0.0, 1.125, 0.5, 0.25, 0.375, 0.125, 0.0, 0.75, 0.0, 0.0, 0.0, 0.0, 1.5, 0.5, 0.125, 0.0, 0.75, 0.25, 0.0, 1.75, 0.0, 0.0, 1.125, 0.125, 0.625, 0.25, 0.0, 0.25, 0.0, 0.625, 0.75, 0.0, 0.0, 0.625, 0.0, 0.75, 0.0, 0.625, 0.0, 0.0, 1.375, 0.625, 0.0, 1.0, 1.25, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.25, 0.625, 0.875, 0.0, 0.75, 1.25, 1.5, 0.0, 0.0] [1.6666666666666667, 1.5833333333333333, 0.4166666666666667, 0.16666666666666666, 0.16666666666666666, 0.08333333333333333, 0.08333333333333333, 0.0, 0.08333333333333333, 0.16666666666666666, 0.0, 0.0, 0.0, 1.3333333333333333, 1.0, 0.0, 0.0, 0.0, 0.3333333333333333, 0.0, 0.3333333333333333, 0.0, 0.08333333333333333, 0.08333333333333333, 0.16666666666666666, 0.0, 0.0, 0.25, 0.0, 0.0, 0.16666666666666666, 0.0, 0.0, 0.0, 1.1666666666666667, 1.1666666666666667, 0.0, 0.0, 0.0, 0.16666666666666666, 0.0, 0.0, 0.9166666666666666, 0.0, 0.0, 0.16666666666666666, 0.0, 0.16666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.16666666666666666, 0.25, 0.0, 0.4166666666666667, 0.75, 0.4166666666666667, 0.0, 0.0] [0.0, 0.0, 0.16666666666666666, 0.6666666666666666, 0.16666666666666666, 1.6666666666666667, 1.6111111111111112, 0.2222222222222222, 0.5, 0.0, 0.05555555555555555, 1.2222222222222223, 0.5555555555555556, 0.0, 0.0, 0.1111111111111111, 0.05555555555555555, 0.0, 0.6666666666666666, 0.0, 0.6111111111111112, 0.0, 0.4444444444444444, 0.3333333333333333, 0.5, 0.0, 0.0, 0.05555555555555555, 0.0, 0.0, 0.05555555555555555, 0.16666666666666666, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.05555555555555555, 0.7222222222222222, 0.0, 0.0, 1.2777777777777777, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.9444444444444444, 0.05555555555555555, 0.3888888888888889, 0.0, 0.3888888888888889, 0.5, 0.4444444444444444, 0.3333333333333333, 0.0] [0.06976744186046512, 0.13953488372093023, 0.13953488372093023, 0.627906976744186, 0.11627906976744186, 0.37209302325581395, 0.18604651162790697, 0.023255813953488372, 0.0, 0.0, 0.0, 0.27906976744186046, 0.13953488372093023, 0.20930232558139536, 0.11627906976744186, 0.13953488372093023, 0.06976744186046512, 0.09302325581395349, 0.18604651162790697, 0.023255813953488372, 0.8372093023255814, 0.13953488372093023, 0.11627906976744186, 0.27906976744186046, 0.06976744186046512, 0.023255813953488372, 0.18604651162790697, 0.046511627906976744, 0.0, 0.0, 0.046511627906976744, 0.11627906976744186, 0.06976744186046512, 0.18604651162790697, 0.046511627906976744, 0.023255813953488372, 0.023255813953488372, 0.06976744186046512, 0.09302325581395349, 0.06976744186046512, 0.13953488372093023, 0.023255813953488372, 0.2558139534883721, 0.023255813953488372, 0.18604651162790697, 0.2558139534883721, 0.0, 0.23255813953488372, 0.18604651162790697, 0.023255813953488372, 0.06976744186046512, 0.11627906976744186, 0.0, 0.06976744186046512, 0.046511627906976744, 0.3023255813953488, 0.046511627906976744, 0.23255813953488372, 0.2558139534883721, 0.3023255813953488, 0.0, 0.09302325581395349] [0.0, 0.0, 0.5, 0.0, 1.5, 0.25, 0.0, 2.5, 1.75, 0.75, 1.0, 0.0, 0.0, 1.0, 0.0, 0.25, 1.5, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0, 1.25, 0.5, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.75, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.25, 0.0, 0.5, 0.25, 0.0, 0.0, 0.0, 1.5, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.5, 1.0, 0.0, 0.0, 0.0] Clusters: cluster_01 :> [0 0 1 0 1 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 2 0 0 2 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 2 0 0 2 1 0 0 0 0 0 2 0 0 0 0 0 1 2 0 0 0] [0 0 1 0 1 0 0 0 2 0 0 0 0 2 0 0 0 0 0 0 2 0 0 2 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 2 0 0 2 1 0 0 0 0 0 2 0 0 0 0 0 1 2 0 0 0] [0 0 0 2 0 1 0 0 1 0 0 0 0 1 2 0 0 0 0 0 0 0 0 1 0 3 1 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 2 3 0 0 0 2 0 0] [0 0 0 2 0 1 0 0 1 0 0 0 0 1 2 0 0 0 0 0 0 0 0 1 1 2 1 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 2 2 0 0 0 2 0 0] [0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 0 0 2 0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 2 0 0 0 0 2 1 0 2 0 0 0 0 0 0 1 0 1 0 0 0 1 2 3 0 0] [0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 0 0 2 0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 2 0 0 0 0 2 1 0 2 0 0 0 0 0 0 2 0 1 0 0 0 1 2 2 0 0] [0 0 1 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 2 0 3 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 2 0 0 0 0 0 1 0 0 1 0 0 1 2 1 0 0] [0 0 2 0 0 1 1 0 0 0 0 0 0 0 0 1 0 2 0 0 3 0 0 1 0 0 0 0 0 0 1 2 0 0 0 0 2 0 1 0 0 2 0 0 0 2 0 0 0 0 0 0 0 0 0 2 0 1 0 2 0 0] cluster_02 :> [1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [2 2 0 0 1 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 2 0 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0] [1 2 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 0 1 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0] [2 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2 2 0 0 0] [2 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2 2 0 0 0] [0 3 2 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 2 0 0] [3 2 2 0 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 2 0 0 1 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 2 2 0 0] [3 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 2 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [2 1 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [2 1 0 1 0 0 0 0 0 0 0 0 0 2 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [2 1 0 1 0 0 0 0 0 0 0 0 0 2 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] cluster_03 :> [0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 2 0 2 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2 0] [0 0 0 0 0 2 0 2 1 0 0 2 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2 0] [0 0 0 2 0 1 2 0 1 0 1 0 0 0 0 1 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 1 0 2 0 1 0 0 0 0] [0 0 0 2 0 1 2 0 1 0 0 0 1 0 0 1 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 1 0 2 0 1 0 0 0 0] [0 0 0 0 0 2 1 1 0 0 0 2 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 0 1 0] [0 0 0 0 0 2 1 1 0 0 0 2 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 0 1 0] [0 0 1 0 0 2 1 0 1 0 0 2 1 0 0 0 0 0 0 0 1 0 2 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 1 0 1 0 0 1 2 0 0] [0 0 1 0 0 1 1 0 1 0 0 2 0 0 0 0 0 0 0 0 1 0 2 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1 0 1 0 0 1 2 0 0] [0 0 0 2 1 2 3 0 1 0 0 1 0 0 0 0 0 0 3 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 2 1 2 2 0 1 0 0 1 0 0 0 0 0 0 2 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 2 1 0 0 0 0 2 0 0 0 0 0 0 0 0 3 0 2 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 2 0 0] [0 0 0 0 1 1 3 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 1 0 0 0 2 1 0 0 0] [0 0 1 0 0 2 1 0 1 0 0 3 2 0 0 0 0 0 1 0 1 0 2 1 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 1 1 0 0 2 2 2 0 0] [0 0 0 0 0 2 2 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 2 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0] [0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0] cluster_04 :> [0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [1 0 0 3 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 1 1 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 3 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 2 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 1 0 0 0 0] [0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2 2 0 0 0 0 0 0 1 0 0 1 2 0 0] [0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 2 2 0 0 0 0 0 0 1 0 0 1 1 0 0] [0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 1 2 0 1 1 0 0 0] [0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 1 2 0 1 1 0 0 0] [0 0 0 2 0 0 3 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0] [0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 2 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0] [0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 2 0 0 1 0 0 0 2 0 0 0 0 2 2 0 0 0 0 0 1 0 0 1 0 0 0 0 2 0 0 1 0 0 0 0 2 0 1 1 1 0 2] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 2 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 2 0 0 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 2 0 0 0 0 0 0 0 0 2 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 2 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 2 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0 0 2 0 0 0 0 0 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0] [0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 2 0 0 0 0 2 2 0 0 0 0 0 0 0 0 1 0 2 0 0] [0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 2 0 0] [0 1 0 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 1 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0] [0 0 0 0 1 2 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 1 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0] [0 0 0 0 1 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0] [0 0 0 0 1 2 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] cluster_05 :> [0 0 0 0 1 0 0 2 2 1 0 0 0 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2 1 0 0 0 1 0 0 0 0 0 0 0 2 0 0 0 0] [0 0 0 0 0 0 0 2 2 0 2 0 0 1 0 0 2 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0] [0 0 2 0 3 0 0 3 2 0 2 0 0 1 0 0 2 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 1 0 0 0] [0 0 0 0 2 1 0 3 1 2 0 0 0 2 0 1 2 0 0 0 0 0 0 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 0]
You can use sklearn's TSNE.fit_transform() on all the data points and receive them in new reduced dimensions.
from sklearn.manifold import TSNE
all_nodes = clus1 + clus2 + clus3 + clus4 + clus5 + centroids
result = TSNE(n_components=2, learning_rate=100, early_exaggeration=50).fit_transform(all_nodes)
To get the same plot:
import seaborn as sns
import pandas as pd
X = {
'x': result[:,0],
'y': result[:,1],
'col' : ['clus1'] * len(clus1) + ['clus2'] * len(clus2) + ['clus3'] * len(clus3) + ['clus4'] * len(clus4) + ['clus5'] * len(clus5) + ['centroids'] * len(centroids),
}
data = pd.DataFrame(data=X)
sns.set(style="white", color_codes=True)
sns.lmplot( x="x", y="y", data=data, fit_reg=False, hue='col',
markers=['o', 'o', 'o', 'o', 'o', 'x'])
You can adjust the parameters to the TSNE yourself. You can find all parameters here.
UPDATE
If you want to plot the graph in matplotlib I've quickly put together this code:
group_sizes = [len(arr) for arr in [clus1, clus2, clus3, clus4, clus5]]
colors = ['red', 'blue', 'green', 'purple', 'orange']
start_pos = 0
for idx, (pos, col) in enumerate(zip(group_sizes, colors)):
plt.scatter(X['x'][start_pos:start_pos+pos], X['y'][start_pos:start_pos+pos], c=col, marker='o', label='Cluster {}'.format(idx+1))
start_pos += pos
plt.scatter(X['x'][-5:], X['y'][-5:], c='black', marker='x', label='Centroids')
_ = plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)
which outputs
Linux string functions
How is this so complex or am I missing something. I simply want to get the integer value of a substring within an existing string and place it in a variable - same as php strpos.
The closest I have found is:
echo $haystack | awk '{print index($0,"<tagtosearch>")}';
Tried
myvar=$($haystack | awk '{print index($0,"<tagtosearch>")}');
but says command not found
The application is to automate include custom bash scripts on a given linux box, but not overwrite existing. Therefore I decided to insert custom start and end tags to denote the custom section. So I simply expected to get the start and end positions, delete this part of the file and pull in a latest version.
So this is as far as I have that function:
function install-env(){
mkdir -p /etc/datadimension/tmp;
cd /etc/datadimension/tmp;
cp /etc/bash.bashrc tempbash.bashrc;
newbash=$(cat tempbash.bashrc);
echo "$newbash" > newbash.bashrc;
insertstart=$(echo "$newbash" | awk '{print index($0,"<starttag>")}');
echo $insertstart;
}
Output
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Do I need to apt install something that can handle strings - this seems really basic requirement that is lacking.
With $($haystack), you invoke the value assigned to $haystack as a command.
To pipe $haystack value to awk try this:
myvar=$(echo "$haystack" | awk '{print index($0,"<tagtosearch>")}');
Or as #anubhava suggests in comment, using a here string:
myvar=$(awk '{print index($0, "<tagtosearch>")}' <<< "$haystack")
Could not find anything acceptable aside from php handling
If you want to print position of a search word in file then you can use this awk command:
awk -v kw='<starttag>' 'p=index($0, kw){p+=b; exit} {b+=length($0)+1} END{print p}' file
We keep adding each line's length into a running variable b until index retuns greater than 0. At that time we add current line's index into b and exit.
This one seems to work like php strpos() function
strpos() {
local string=$1
local findme=$2
awk -v x=$string -v y=$findme '{print index($0, y)}' <<< $string
}
Some tests
strpos azerty a
1
strpos azerty t
5
strpos azerty er
3
strpos azerty/uio /
7
strpos "azerty uio" " "
7
EDIT:
If you want exactly the php behaviour you can do:
php_strpos() {
local haystack="$1"
local needle="$2"
local offset=${3:-0}
php -r "echo strpos('$haystack', '$needle', $offset);"
}