Getting hardware info in nim-lang - nim-lang

What is the most elegant way to get hardware info, such us GPU name, CPU name, RAM in nim? I haven't found a library and not really sure how to implement it from pure nim

When you want to search for Nim packages, you should look first at:
awesome-nim: A curated list of packages
the nimble search functionality. Ex: nimble search hardware cpu
I have found the pkg/sysinfo package and pkg/psutil-nim which does probably what you want.
Shell scripts are usually faster and more adapted for this kind of application. There are many CLI applications that you can adapt or leverage directly.
On Linux, you can get hostname and model information with hostnamectl.
There are other utilities that aggregate hardware information, the most well-known is probably neofetch. It has many variants, see: https://alternativeto.net/software/neofetch/
Based on neofetch, I have written a small script that works on Linux only:
import std/[strutils, strformat, osproc]
type
MyComputerInfo = object
os: string
host: string
kernel: string
cpu: string
gpu: string
proc cacheUnameInfo(): tuple[os: string, host: string, kernel: string] =
## Only defined for posix systems
when defined(posix):
let (output, errorCode) = execCmdEx("uname -srm")
if errorCode != 0:
echo fmt"`uname -srm` command failed, with Error code: {errorCode}"
quit(2)
let
outSeq: seq[string] = output.split(' ')
os = outSeq[0]
host = outSeq[1]
kernel = outSeq[2]
return (os, host, kernel)
proc getGpuInfo(): string =
when defined(posix):
var (gpu_driver_name, exitCode) = execCmdEx("""lspci -nnk | awk -F ': ' \ '/Display|3D|VGA/{nr[NR+2]}; NR in nr {printf $2 ", "; exit}'""")
if exitCode == 0:
gpu_driver_name = gpu_driver_name.split(",")[0]
if gpu_driver_name == "nvidia":
var (gpu_name, exitCodeSmi) = execCmdEx("nvidia-smi --query-gpu=name --format=csv,noheader")
if exitCodeSmi != 0:
echo "nvidia-smi command failed with exit code: {exitCodeSmi}"
quit(2)
return gpu_name
return gpu_driver_name.toUpperAscii()
else:
echo fmt"GpuInfo error code: {exitCode}"
quit(2)
proc getCpuInfo(): string =
when defined(posix):
let filename = "/proc/cpuinfo"
var (cpu_name, exitCode) = execCmdEx("""cat /proc/cpuinfo | awk -F '\\s*: | #' '/model name|Hardware|Processor|^cpu model|chip type|^cpu type/ { cpu=$2; if ($1 == "Hardware") exit } END { print cpu }' "$cpu_file" """)
return cpu_name
proc `$`(mci: MyComputerInfo): string =
## Nice Output of a `MyComputerInfo` object
for name, val in fieldPairs(mci):
if $val != "":
result.add name.capitalizeAscii()
result.add ": "
result.add ($val).strip(leading=false, chars = {'\n'})
result.add "\n"
result.strip(leading=false, chars = {'\n'})
when isMainModule:
var mci: MyComputerInfo
(mci.os, mci.host, mci.kernel) = cacheUnameInfo()
mci.gpu = getGpuInfo()
mci.cpu = getcpuInfo()
echo mci
On my computer, I get:
Os: Linux
Host: 6.0.2-zen1-1-zen
Kernel: x86_64
Cpu: 11th Gen Intel(R) Core(TM) i5-11400F
Gpu: NVIDIA GeForce RTX 3070
We could replace the awk's calls in execCmdEx by Nim parsing procedures for more Nim purity. We could add a configurator (with a CLI parser, or a DSL) so that the user can select the options he wants to print out. I leave the rest to you.

Related

How do I replace a character in the string with another charater in fortran

I have a string "f://asd/aww/asdf.txt" and I need to change it to "f:\asd\aww\asdf.txt". is it feasible?
I have done this in the past, so here is a recursive way of replacing parts of an input string that match the pattern of arbitrary length given by search input argument with a substitute string of also arbitrary size given by the input argument substitute.
module String_mod
implicit none
public
!***********************************************************************************************************************************
!***********************************************************************************************************************************
contains
!***********************************************************************************************************************************
!***********************************************************************************************************************************
pure recursive function replaceStr(string,search,substitute) result(modifiedString)
implicit none
character(len=*), intent(in) :: string, search, substitute
character(len=:), allocatable :: modifiedString
integer :: i, stringLen, searchLen
stringLen = len(string)
searchLen = len(search)
if (stringLen==0 .or. searchLen==0) then
modifiedString = ""
return
elseif (stringLen<searchLen) then
modifiedString = string
return
end if
i = 1
do
if (string(i:i+searchLen-1)==search) then
modifiedString = string(1:i-1) // substitute // replaceStr(string(i+searchLen:stringLen),search,substitute)
exit
end if
if (i+searchLen>stringLen) then
modifiedString = string
exit
end if
i = i + 1
cycle
end do
end function replaceStr
!***********************************************************************************************************************************
!***********************************************************************************************************************************
end module String_mod
program testReplaceStr
use, intrinsic :: iso_fortran_env, only: output_unit
use String_mod
implicit none
write( output_unit, "(*(g0,:,' '))" ) replaceStr( string = "f://asd/aww/asdf.txt" &
, search = "/" &
, substitute = "\" &
)
end program testReplaceStr
D:\>ifort /debug:full /Zi /CB /Od /Qinit:snan /warn:all /gen-interfaces /traceback /check:all /check:bounds /fpe-all:0 /Qdiag-error-limit:10 /Qtrapuv main.f90 -o main.exe
Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.0.4.245 Build 20190417
Copyright (C) 1985-2019 Intel Corporation. All rights reserved.
Microsoft (R) Incremental Linker Version 14.22.27905.0
Copyright (C) Microsoft Corporation. All rights reserved.
-out:main.exe
-debug
-pdb:main.pdb
-subsystem:console
-incremental:no
main.obj
D:\>main.exe
f:\\asd\aww\asdf.txt
Keep in mind that this approach is likely not the best for performance considerations as it involves multiple recursive memory allocations for the final string construction, depending on how many times the search pattern appears in the input string.

Why physical address of Aarch64 kernel image is nonnegative?

I'm recently learning about booting system of Linux kernel. (v4.6, with ARM64 arch.)
In the source code arch/arm64/kernel/head.S, definition of __PHYS_OFFSET is:
#define __PHYS_OFFSET (KERNEL_START - TEXT_OFFSET)
where KERNEL_START is simply defined to be _text section.
And if I'm right, TEXT_OFFSET is a random number determined during kernel compile, as /arch/arm64/Makefile says:
TEXT_OFFSET := $(shell awk 'BEGIN {srand(); printf "0x%03x000\n", int(512 * rand())}')
so that the kernel image file has random location, as the linker script /arch/arm64/kernel/vmlinux.lds.S includes:
. = KIMAGE_VADDR + TEXT_OFFSET;
.head.text : {
_text = .;
HEAD_TEXT
}
Here, KIMAGE_VADDR is a virtual address 0xFFFF000000000000 + 128M. Since TEXT_OFFSET is added, section _text will be randomly located.
Rest parts of head.S map KIMAGE_VADDR to __PHYS_OFFSET to enable MMU.
My question is this: is __PHYS_OFFSET = _text - TEXT_OFFSET always nonnegative?
I don't know where would be exact physical location of _text, but I think 512 * rand() might be as big as 512 * 32767 ~ 10MB.
Do I make sense? Is there any reason makes these codes safe?
vmlinux.lds.S does:
. = KIMAGE_VADDR + TEXT_OFFSET;
followed by
_text = .;
So _text = KIMAGE_VADDR + TEXT_OFFSET. When you then subtract TEXT_OFFSET, __PHYS_OFFSET will be the same as KIMAGE_VADDR.
Thus, if KIMAGE_VADDR is non-negative, so is __PHYS_OFFSET.

Handling long probe paths in systemtap userspace scripts (variables)?

I am trying to instrument a user-space program using systemtap, Linux 4.2.0-42-generic #49~14.04.1-Ubuntu SMP; stap --version says: "Systemtap translator/driver (version 2.3/0.158, Debian version 2.3-1ubuntu1.4 (trusty))"
So I first tried getting a list of all callable functions, with:
stap -L 'process("/home/username/pathone/subpathtwo/subpaththree/subpathfour/subpathFive/subpathsix/subpathSeven/subpatheight/myprogramab").function("*").call' 2>&1 | tee /tmp/stap
This worked - however, note that the absolute path to my program is massive. So from the /tmp/stap file, I read some probes of interest, however, they are even longer, such as:
process("/home/username/pathone/subpathtwo/subpaththree/subpathfour/subpathFive/subpathsix/subpathSeven/subpatheight/myprogramab").function("DoSomething#/home/username/pathone/subpathtwo/subpaththree/subpathfour/subpathFive/src/BasicTestCodeFileInterface.cpp:201").call
... and as this is very difficult to read/unreadable for me, I wanted to do something to split this line into more readable pieces. The first thing I thought of was using variables, so I tried this test.stp script:
#!/usr/bin/env stap
global exepath = "/home/username/pathone/subpathtwo/subpaththree/subpathfour/subpathFive/subpathsix/subpathSeven/subpatheight/myprogramab"
global srcpath = "/home/username/pathone/subpathtwo/subpaththree/subpathfour/subpathFive/src"
probe begin {
printf("%s\n", exepath)
exit() # must have; else probe end runs only upon Ctrl-C
}
probe end {
newstr = "DoSomething#" . srcpath # concatenate strings
printf("%s\n", newstr)
}
This works, I can run sudo stap /path/to/test.stp, and I get the two strings printed.
However, when I try to use these strings in probes, they fail:
Writing a probe like this:
probe process(exepath).function("DoSomething#".srcpath."/BasicTestCodeFileInterface.cpp:201").call { ...
... fails with: parse error: expected literal string or number; saw: identifier 'exepath' ....
Trying to put the "process" part in a variable:
global tproc = process("/home/username/pathone/subpathtwo/subpaththree/subpathfour/subpathFive/subpathsix/subpathSeven/subpatheight/myprogramab")
... fails with parse error: expected literal string or number; saw: identifier 'process' ....
So, what options do I have, to somehow shorten the probe lines in a script?
Your best bet is probably to use macros:
#define part1 %( "/path/part/1" %)
#define part2 %( "/part/2" %)
probe process(#part1 #part2).function("...") { }
Note no explicit concatenation operator between the (macros that expand to) string literals. The parser will auto-concatenate them, just as in C.

SystemTap script to analyze the cache behavior of functions

I would like to profile the cache behavior of a kernel module with SystemTap (#cache references, #cache misses, etc). There is an example script online which shows how SystemTap can be used to read the perf events and counters, including cache-related ones:
https://sourceware.org/systemtap/examples/profiling/perf.stp
This sample script works by default for a process:
probe perf.hw.cache_references.process("/usr/bin/find").counter("find_insns") {}
I replaced the process keyword with module and the path to the executable with the name of my kernel module:
probe perf.hw.cache_references.module(MODULE_NAME).counter("find_insns") {}
I'm pretty sure that my module has the debug info, but running the script I get:
semantic error: while resolving probe point: identifier 'perf' at perf.stp:14:7
source: probe perf.hw.instructions.module(MODULE_NAME).counter("find_insns") {}
Any ideas what might be wrong?
Edit:
Okay, I realized that the perf counters could be bound to processes only not to modules (Explained here: https://sourceware.org/systemtap/man/stapprobes.3stap.html). Therefore I changed it back to:
probe perf.hw.cache_references.process(PATH_TO_BINARY).counter("find_insns") {}
Now, as the sample script suggests, I have:
probe module(MODULE_NAME).function(FUNC_NAME) {
#save counter values on entrance
...
}
But now running it, I get:
semantic error: perf counter 'find_insns' not defined semantic error:
while resolving probe point: identifier 'module' at perf.stp:26:7
source: probe module(MODULE_NAME).function(FUNC_NAME)
Edit2:
So here is my complete script:
#! /usr/bin/env stap
# Usage: stap perf.stp <path-to-binary> <module-name> <function-name>
global cycles_per_insn
global branch_per_insn
global cacheref_per_insn
global insns
global cycles
global branches
global cacherefs
global insn
global cachemisses
global miss_per_insn
probe perf.hw.instructions.process(#1).counter("find_insns") {}
probe perf.hw.cpu_cycles.process(#1).counter("find_cycles") {}
probe perf.hw.branch_instructions.process(#1).counter("find_branches") {}
probe perf.hw.cache_references.process(#1).counter("find_cache_refs") {}
probe perf.hw.cache_misses.process(#1).counter("find_cache_misses") {}
probe module(#2).function(#3)
{
insn["find_insns"] = #perf("find_insns")
insns <<< (insn["find_insns"])
insn["find_cycles"] = #perf("find_cycles")
cycles <<< insn["find_cycles"]
insn["find_branches"] = #perf("find_branches")
branches <<< insn["find_branches"]
insn["find_cache_refs"] = #perf("find_cache_refs")
cacherefs <<< insn["find_cache_refs"]
insn["find_cache_misses"] = #perf("find_cache_misses")
cachemisses <<< insn["find_cache_misses"]
}
probe module(#2).function(#3).return
{
dividend = (#perf("find_cycles") - insn["find_cycles"])
divisor = (#perf("find_insns") - insn["find_insns"])
q = dividend / divisor
if (q > 0)
cycles_per_insn <<< q
dividend = (#perf("find_branches") - insn["find_branches"])
q = dividend / divisor
if (q > 0)
branch_per_insn <<< q
dividend = (#perf("find_cycles") - insn["find_cycles"])
q = dividend / divisor
if (q > 0)
cacheref_per_insn <<< q
dividend = (#perf("find_cache_misses") - insn["find_cache_misses"])
q = dividend / divisor
if (q > 0)
miss_per_insn <<< q
}
probe end
{
if (#count(cycles_per_insn)) {
printf ("Cycles per Insn\n\n")
print (#hist_log(cycles_per_insn))
}
if (#count(branch_per_insn)) {
printf ("\nBranches per Insn\n\n")
print (#hist_log(branch_per_insn))
}
if (#count(cacheref_per_insn)) {
printf ("Cache Refs per Insn\n\n")
print (#hist_log(cacheref_per_insn))
}
if (#count(miss_per_insn)) {
printf ("Cache Misses per Insn\n\n")
print (#hist_log(miss_per_insn))
}
}
Systemtap can't read hardware perfctr values for kernel probes, because linux doesn't provide a suitable (e.g., atomic) internal API for safely reading those values from all contexts. The perf...process probes work only because that context is not atomic: the systemtap probe handler can block safely.
I cannot answer your detailed question about the two (?) scripts you last experimented with, because they're not complete.

Perl script execution keeps getting killed - running out of memory

I am trying to execute a perl script that processes a small 12 x 2 text file (approx. 260 bytes) and a large .bedgraph file (at least 1.3 MB in size). From these two files, the script outputs a new bedgraph file.
I have ran this script on 3 other .bedgraph files but I try to run it on the rest of them the process keeps getting Killed.
It should take about 20 minutes on average for the perl script to run on each of the .bedgraph files.
I'm running the perl script on my local machine (not from a server). I'm using a Linux OS Ubuntu 12.04 system 64-bit 4GB RAM.
Why does my perl script execution keeps getting killed and how can I fix this?
Here's the script:
# input file handle
open(my $sizes_fh, '<', 'S_lycopersicum_chromosomes.size') or die $!;
# output file handles
open(my $output, '+>', 'tendaysafterbreaker_output.bedgraph') or die $!;
my #array;
while(<$sizes_fh>){
chomp;
my ($chrom1, $size) = split(/\t/, $_);
#array = (0) x $size;
open(my $bedgraph_fh, '<', 'Solanum_lycopersicum_tendaysafterbreaker.bedgraph') or die $!;
while(<$bedgraph_fh>){
chomp;
my ($chrom2, $start, $end, $FPKM) = split(/\t/, $_);
if ($chrom1 eq $chrom2){
for(my $i = $start; $i < $end; $i++){
$array[$i] += $FPKM;
}
}
}
close $bedgraph_fh or warn $!;
my ($last_start, $last_end) = 0;
my $last_value = $array[0];
for (my $i = 1; $i < $#array; $i++){
my $curr_val = $array[$i];
my $curr_pos = $i;
# if the current value is not equal to the last value
if ($curr_val != $last_value){
my $last_value = $curr_val;
print $output "$chrom1\t$last_start\t$last_end\t$last_value\n";
$last_start = $last_end = $curr_pos;
} else {
$last_end = $i;
}
}
}
close $sizes_fh or warn $!;
You are trying to allocate an array of 90,000,000 elements. Perl, due to its flexible typing and other advanced variable features, uses a lot more memory for this than you would expect.
On my (Windows 7) machine, a program that just allocates such an array and does nothing else eats up 3.5 GB of RAM.
There are various ways to avoid this huge memory usage. Here are a couple:
The PDL module for scientific data processing, which is designed to efficiently store huge numeric arrays in memory. This will change the syntax for allocating and using the array, though (and it messes around with Perl's syntax in various other ways).
DBM::Deep is a module that allocates a database in a file--and then lets you access that database through a normal array or hash:
use DBM::Deep;
my #array;
my $db = tie #array, "DBM::Deep", "array.db";
#Now you can use #array like a normal array, but it will be stored in a database.
If you know a bit of C, it is quite simple to offload the array manipulation into low-level code. Using a C array takes less space, and is a lot faster. However, you loose nice stuff like bounds checking. Here is an implementation with Inline::C:
use Inline 'C';
...;
__END__
__C__
// note: I don't know if your data contains only ints or doubles. Adjust types as needed
int array_len = -1; // last index
int *array = NULL;
void make_array(int size) {
free(array);
// if this fails, start checking return value of malloc for != NULL
array = (int*) malloc(sizeof(int) * size);
array_len = size - 1;
}
// returns false on bounds error
int array_increment(int start, int end, int fpkm) {
if ((end - 1) > array_len) return 0;
int i;
for (i = start; i < end; i++) {
array[i] += fpkm;
}
return 1;
}
// please check if this is actually equivalent to your code.
// I removed some unneccessary-looking variables.
void loop_over_array(char* chrom1) {
int
i,
last_start = 0,
last_end = 0,
last_value = array[0];
for(i = 1; i < array_len; i++) { // are you sure not `i <= array_len`?
if (array[i] != last_value) {
last_value = array[i];
// I don't know how to use Perl filehandles from C,
// so just redirect the output on the command line
printf("%s\t%d\t%d\t%d\n", chrom1, last_start, last_end, last_value);
last_start = i;
}
last_end = i;
}
}
void free_array {
free(array);
}
Minimal testing code:
use Test::More;
make_array(15);
ok !array_increment(0, 16, 2);
make_array(95_000_000);
ok array_increment(0, 3, 1);
ok array_increment(2, 95_000_000, 1);
loop_over_array("chrom");
free_array();
done_testing;
The output of this test case is
chrom 0 1 2
chrom 2 2 1
(with testing output removed). It may take a second to compile, but after that it should be quite fast.
In the records read from $bedgraph_fh, what's a typical value for $start? Although hashes have more overhead per entry than arrays, you may be able to save some memory if #array starts with a lot of unused entries. e.g., If you have an #array of 90 million elements, but the first 80 million are never used, then there's a good chance you'll be better off with a hash.
Other than that, I don't see any obvious cases of this code holding on to data that's not needed by the algorithm it implements, although, depending on your actual objective, it is possible that there may be an alternative algorithm which doesn't require as much data to be held in memory.
If you really need to be dealing with a set of 90 million active data elements, though, then your primary options are going to be either buy a lot of RAM or use some form of database. In the latter case, I'd opt for SQLite (via DBD::SQLite) for simplicity and light weight, but YMMV.

Resources