write error: No space left on device in embedded linux - linux

all
I have a embedded board, run linux OS. and I use yaffs2 as rootfs.
I run a program on it, but after some times, it got a error "error No space left on device.". but I checked the flash, there still have a lot free space.
I just write some config file. the config file is rarely update. the program will write some log to flash. log size is limited to 2M.
I don't know why, and how to solve.
Help me please!(my first language is not English,sorry. hope you understand what I say)
some debug info:
# ./write_test
version 1.0
close file :: No space left on device
return errno 28
# cat /proc/yaffs
YAFFS built:Nov 23 2015 16:57:34
Device 0 "rootfs"
start_block........... 0
end_block............. 511
total_bytes_per_chunk. 2048
use_nand_ecc.......... 1
no_tags_ecc........... 1
is_yaffs2............. 1
inband_tags........... 0
empty_lost_n_found.... 0
disable_lazy_load..... 0
refresh_period........ 500
n_caches.............. 10
n_reserved_blocks..... 5
always_check_erased... 0
data_bytes_per_chunk.. 2048
chunk_grp_bits........ 0
chunk_grp_size........ 1
n_erased_blocks....... 366
blocks_in_checkpt..... 0
n_tnodes.............. 749
n_obj................. 477
n_free_chunks......... 23579
n_page_writes......... 6092
n_page_reads.......... 11524
n_erasures............ 96
n_gc_copies........... 5490
all_gcs............... 1136
passive_gc_count...... 1136
oldest_dirty_gc_count. 95
n_gc_blocks........... 96
bg_gcs................ 96
n_retired_writes...... 0
n_retired_blocks...... 0
n_ecc_fixed........... 0
n_ecc_unfixed......... 0
n_tags_ecc_fixed...... 0
n_tags_ecc_unfixed.... 0
cache_hits............ 0
n_deleted_files....... 0
n_unlinked_files...... 289
refresh_count......... 1
n_bg_deletions........ 0
Device 2 "data"
start_block........... 0
end_block............. 927
total_bytes_per_chunk. 2048
use_nand_ecc.......... 1
no_tags_ecc........... 1
is_yaffs2............. 1
inband_tags........... 0
empty_lost_n_found.... 0
disable_lazy_load..... 0
refresh_period........ 500
n_caches.............. 10
n_reserved_blocks..... 5
always_check_erased... 0
data_bytes_per_chunk.. 2048
chunk_grp_bits........ 0
chunk_grp_size........ 1
n_erased_blocks....... 10
blocks_in_checkpt..... 0
n_tnodes.............. 4211
n_obj................. 24
n_free_chunks......... 658
n_page_writes......... 430
n_page_reads.......... 467
n_erasures............ 7
n_gc_copies........... 421
all_gcs............... 20
passive_gc_count...... 13
oldest_dirty_gc_count. 3
n_gc_blocks........... 6
bg_gcs................ 4
n_retired_writes...... 0
n_retired_blocks...... 0
n_ecc_fixed........... 0
n_ecc_unfixed......... 0
n_tags_ecc_fixed...... 0
n_tags_ecc_unfixed.... 0
cache_hits............ 0
n_deleted_files....... 0
n_unlinked_files...... 2
refresh_count......... 1
n_bg_deletions........ 0
#
log and config file stored in "data".
thanks!!

In General this could be your disk space (here Flash), first of all check your flash space with with df -h (or other commands you have.. df is present in BusyBox). But if your flash space (specially on your program partition) is ok, this could be your "inode" (directory) space problem, you could see your inode usage with df -i command. (a good link for this: https://wiki.gentoo.org/wiki/Knowledge_Base:No_space_left_on_device_while_there_is_plenty_of_space_available)
If non of these is the problem cause, I think you have to have a deeper look at your code, specially if you deal with disk I/O!
Also good to mention that be aware of memory & heap space & free all allocated spaces in you functions.

Related

Tasklet counts in /proc/softirqs increase very rapidly on USB operation in Linux

I have a legacy device with following configuration:
Chipset Architecture : Intel NM10 express
CPU : Atom D2250 Dual Core
Volatile Memory : 1GB
CPU core : 4
USB Host controller driver : ehci-pci
When I perform any USB operation I observe increasing tasklet count linearly and if USB operation continues for a long time(approx half an hour) then tasklet count crosses a million and this seems to be very strange to me.
I read about interrupt handling mechanism used by ehci-pci which is not latest(i.e. PCI pin-based) but still the tasklet counts are so high in numbers.
I use /proc/softirqs to read tasklet count.
Any lead on this?
root#panther1:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
23: 0 0 0 1411443 IO-APIC 23-fasteoi ehci_hcd:usb1, uhci_hcd:usb2
root#panther1:~# cat /proc/softirqs
CPU0 CPU1 CPU2 CPU3
HI: 7 3 13 1125529
TIMER: 2352846 2325384 2533628 2675821
NET_TX: 0 0 2 1703
NET_RX: 1161 1193 2730 51184
BLOCK: 0 0 0 0
IRQ_POLL: 0 0 0 0
TASKLET: 256 164 90 1162220
SCHED: 1078965 1015261 1155661 1207484
HRTIMER: 0 0 0 0
RCU: 1370647 1367098 1485356 1503762

Is it possible to get current CPU utilisation from a specific core via /sys in Linux?

I would like to write a shellscript that reads the current CPU utilisation on a per-core basis. Is it possible to read this from the /sys directory in Linux (CentOS 8)? I have found /sys/bus/cpu/drivers/processor/cpu0 which does give me a fair bit of information (like current frequency), but I've yet to figure out how to read CPU utilisation.
In other words: Is there a file that gives me current utilisation of a specific CPU core in Linux, specifically CentOS 8?
I believe that you should be able to extract information from /proc/stat - the lines that start with cpu$N, where $N is 0, 1, 2, ...... For example:
Strongly suggesting reading articles referenced on other answer.
cpu0 101840 1 92875 80508446 4038 0 4562 0 0 0
cpu1 81264 0 68829 80842548 4424 0 2902 0 0 0
Repeated call will show larger values:
cpu 183357 1 162020 161382289 8463 0 7470 0 0 0
cpu0 102003 1 93061 80523961 4038 0 4565 0 0 0
cpu1 81354 0 68958 80858328 4424 0 2905 0 0 0
Notice CPU0 5th column (idle count) moving from 80508446 to 80523961
Format of each line in
cpuN user-time nice-time system-time idle-time io-wait ireq softirq
steal guest guest_nice
So a basic solution:
while true ;
for each cpu
read current counters, at least user-time system-time and idle
usage = current(user-time + system-time) - prev(user-time+system-time)
idle = current(idle) - prev(idle)
utilization = usage/(usage+idle)
// print or whatever.
set prev=current
done

Mainframe ICETOOL joins two files

I have two VB files.
File1 is 406 bytes long, and the key is in position 401 to 406.
File2 is 565 bytes long, and the key is in position 87 to 92.
I would like the result file to have all the 406 bytes from file1 plus 8 bytes from file2 starts from column 42.
I have the following codes:
//COPY01 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN01 DD DSN=CCSY.CCSYAC.BNS.C2,DISP=SHR
//IN02 DD DSN=CCSY.CCSYAC.UID.XBAS,DISP=SHR
//OUT01 DD DSN=CCSY.CCSYAC.BNS.C2.ID,DISP=(,CATLG,DELETE),
// DCB=(RECFM=FB,LRECL=410,BLKSIZE=41000),
// SPACE=(CYL,(200,100),RLSE)
//TOOLIN DD *
COPY JKFROM TO(OUT01) VSAMTYPE(V) USING(BNSC)
/*
//BNSCCNTL DD *
JOINKEYS F1=IN01,FIELDS=(401,6,A)
JOINKEYS F2=IN02,FIELDS=(87,6,A)
REFORMAT FIELDS=(F1:5,402,F2:42,8)
/*
I got S013. Anyone would like to help? I'll appreciate.
JESMSGLG:
13.02.33 JOB38364 ---- WEDNESDAY, 10 JAN 2018 ----
13.02.33 JOB38364 IRR010I USERID CCSYAC IS ASSIGNED TO THIS JOB.
13.02.34 JOB38364 ICH70001I CCSYAC LAST ACCESS AT 12:56:19 ON WEDNESDAY, JANUARY 10, 2018
13.02.34 JOB38364 $HASP373 CCSYAC02 STARTED - INIT 68 - CLASS J - SYS SY07
13.02.34 JOB38364 IEF403I CCSYAC02 - STARTED - TIME=13.02.34
13.02.35 JOB38364 IEC141I 013-68,IFG0196L,CCSYAC02,COPY01,OUT01,B275,STRP49, 005
005 CCSY.CCSYAC.BNS.C2.ID
13.02.35 JOB38364 IEA995I SYMPTOM DUMP OUTPUT 006
006 SYSTEM COMPLETION CODE=013 REASON CODE=00000068
006 TIME=13.02.35 SEQ=61720 CPU=0000 ASID=0058
006 PSW AT TIME OF ERROR 075C1000 80E70476 ILC 2 INTC 0D
006 NO ACTIVE MODULE FOUND
006 NAME=UNKNOWN
006 DATA AT PSW 00E70470 - 4100302C 0A0D010D A7E5014B
006 AR/GR 0: 008FE990/00E70780 1: 00000000/A4013000
006 2: 00000000/0000DCC8 3: 00000000/00E70754
006 4: 00000000/008C4388 5: 00000000/008C471C
006 6: 00000000/008C46C4 7: 00000000/008C471C
006 8: 00000000/008C46E4 9: 00000000/008D1C50
006 A: 00000000/80E72A62 B: 00000000/00E73622
006 C: 00000000/80E7387E D: 00000000/7F59FCE8
006 E: 00000000/80E6FCD4 F: 00000000/00000068
006 END OF SYMPTOM DUMP
13.02.47 JOB38364 IEF450I CCSYAC02 COPY01 - ABEND=S013 U0000 REASON=00000068 011
011 TIME=13.02.47
13.02.47 JOB38364 IEF404I CCSYAC02 - ENDED - TIME=13.02.47
13.02.47 JOB38364 $HASP395 CCSYAC02 ENDED - ABEND=S013
------ JES2 JOB STATISTICS ------
10 JAN 2018 JOB EXECUTION DATE
26 CARDS READ
563 SYSOUT PRINT RECORDS
0 SYSOUT PUNCH RECORDS
54 SYSOUT SPOOL KBYTES
0.22 MINUTES EXECUTION TIME
TOOLMSG:
ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED
ICE650I 0 VISIT http://www.ibm.com/storage/dfsort FOR ICETOOL PAPERS,
EXAMPLES AND MORE
ICE632I 0 SOURCE FOR ICETOOL STATEMENTS: TOOLIN
ICE630I 0 MODE IN EFFECT: STOP
COPY JKFROM TO(OUT01) VSAMTYPE(V) USING(BNSC)
DFSMSG:
ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
ICE411I 0 THIS IS THE JOINKEYS MAIN TASK FOR JOINING F1 AND F2
ICE416I 0 JOINKEYS IS USING THE F1 SUBTASK FOR IN01 - SEE JNF1JMSG MESSAGES
ICE416I 1 JOINKEYS IS USING THE F2 SUBTASK FOR IN02 - SEE JNF2JMSG MESSAGES
ICE419I 0 JOINED RECORDS: TYPE=F, LENGTH=410
ICE201I A RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-I40658 C6-I35397 C7-I35397 C8-I40658 E9-I40658 C9-I35397 E5-
I38877 E7-I40658
ICE143I 0 BLOCKSET COPY TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2 - 13:02 ON WED JAN 10, 2018 -
JOINKEYS F1=IN01,FIELDS=(401,6,A)
JOINKEYS F2=IN02,FIELDS=(87,6,A)
REFORMAT FIELDS=(F1:5,402,F2:42,8)
ICE146I 0 END OF STATEMENTS FROM BNSCCNTL - PARAMETER LIST STATEMENTS FOLLOW
DEBUG NOABEND,ESTAE
OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=BNSC,SORTOUT=OUT01*
,DYNALLOC
SORT FIELDS=COPY
RECORD TYPE=V
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE089I 0 CCSYAC02.COPY01 . , INPUT LRECL = 410, TYPE = F
ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1024000,MINLIM=122800,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=DFSMSG
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N ,ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=MAX ,EXPOLD=50% ,EXPRES=10%
ICE084I 0 EXCP ACCESS METHOD USED FOR OUT01
ICE751I 2 EF-I35397 DA-I40658
ICE805I 0 JOBNAME: CCSYAC02 , STEPNAME: COPY01
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE906I 0 ST=ABOVE,SR=6291456,RC=0
ICE907I 0 ST=ABOVE,SA=6291440,NF=1,LF=6291440,SF=6291440
ICE906I 0 ST=BELOW,SR=1036288,RC=0
ICE907I 0 ST=BELOW,SA=1036272,NF=1,LF=1036272,SF=1036272
ICE889I 0 CT=MAX , SB=8, L=0, D=0000, CCW=1MAM
ICE902I 0 O RP10 I
ICE906I 1 ST=ABOVE,SR=6234096,RC=0
ICE907I 1 ST=ABOVE,SA=6234080,NF=1,LF=6234080,SF=6234080
ICE906I 1 ST=BELOW,SR=94680,RC=0
ICE907I 1 ST=BELOW,SA=49608,NF=1,LF=49608,SF=49608
ICE185A 0 AN S013 ABEND WAS ISSUED BY DFSORT, ANOTHER PROGRAM OR AN EXIT (PHASE C 3)
JNF1JMSG:
ICE201I A RECORD TYPE IS V - DATA STARTS IN POSITION 5
ICE751I 0 C5-I40658 C6-I35397 C7-I35397 C8-I40658 E4-I40658 C9-I35397 E5-
I38877 E6-I31999 C4-I31999 E7-I40658
ICE417I 0 THIS IS THE JOINKEYS F1 SUBTASK FOR IN01
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2 - 13:02 ON WED JAN 10, 2018 -
SORT FORMAT=BI,FIELDS=(401,6,A)
RECORD TYPE=F
DEBUG NOABEND,ESTAE
OPTION EQUALS,MSGPRT=ALL,LIST,NOCHECK,RESINV=0,DYNALLOC,SORTDD=JNF1,MSG*
DDN=JNF1JMSG,SORTIN=IN01
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE088I 0 CCSYAC02.COPY01 . , INPUT LRECL = 406, BLKSIZE = 27998, TYPE = VB
ICE093I 0 MAIN STORAGE = (MAX,9251025,9251025)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (9193665,9193665)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=9251025,MAXLIM=1024000,MINLIM=122800,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=JNF1JMSG
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT==N,DYNALOC=(DISK ,004),ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=128
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=MAX ,EXPOLD=50% ,EXPRES=10%
ICE084I 0 EXCP ACCESS METHOD USED FOR IN01
ICE750I 0 DC 589973550 TC 0 CS DSVXX KSZ 10 VSZ 10
ICE752I 0 FSZ=589973550 BC IGN=0 E AVG=203 0 WSP=766274 C DYN=520 53216
ICE751I 1 D8-I35397 D4-I35397 EA-I35933 F1-I35933 E8-I40658
ICE091I 0 OUTPUT LRECL = 406, TYPE = V
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 1433042, OUT: 0
ICE134I 0 NUMBER OF BYTES SORTED: 581815052
ICE253I 0 RECORDS SORTED - PROCESSED: 1433042, EXPECTED: 2906273
ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 406, EXPECTED: 203
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 540 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES
ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 563M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT
JNF2JMSG:
ICE201I A RECORD TYPE IS V - DATA STARTS IN POSITION 5
ICE751I 0 C5-I40658 C6-I35397 C7-I35397 C8-I40658 E4-I40658 C9-I35397 E5-
I38877 E6-I31999 E7-I40658
ICE417I 0 THIS IS THE JOINKEYS F2 SUBTASK FOR IN02
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 0 - CONTROL STATEMENTS FOR 5650-ZOS, Z/OS DFSORT V2R2 - 13:02 ON WED JAN 10, 2018 -
SORT FORMAT=BI,FIELDS=(87,6,A)
RECORD TYPE=F
DEBUG NOABEND,ESTAE
OPTION EQUALS,MSGPRT=ALL,LIST,NOCHECK,RESINV=0,DYNALLOC,SORTDD=JNF2,MSG*
DDN=JNF2JMSG,SORTIN=IN02
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE088I 0 CCSYAC02.COPY01 . , INPUT LRECL = 565, BLKSIZE = 27998, TYPE = VB
ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1024000,MINLIM=122800,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=JNF2JMSG
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(DISK ,004),ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y,TUNE=STOR,EXPMAX=MAX ,EXPOLD=50% ,EXPRES=10%
ICE084I 0 EXCP ACCESS METHOD USED FOR IN02
ICE750I 0 DC 1063772 TC 0 CS DSVXX KSZ 10 VSZ 10
ICE752I 0 FSZ=1063772 BC IGN=0 E AVG=282 0 WSP=1381 C DYN=0 0
ICE751I 1 D8-I35397 D4-I35397 D1-I35397 E8-I40658
ICE091I 0 OUTPUT LRECL = 565, TYPE = V
ICE080I 0 IN MAIN STORAGE SORT
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 3603, OUT: 0
ICE134I 0 NUMBER OF BYTES SORTED: 1034061
ICE253I 0 RECORDS SORTED - PROCESSED: 3603, EXPECTED: 3772
ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 287, EXPECTED: 282
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES
ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 0M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT
Here is the description for the S013 error you encountered:
The SORTIN, SORTOUT, or OUTFIL BLKSIZE parameter:
Was greater than 32760 for a disk data set, or was greater than the maximum block size
supported by the access method for a tape data set.
Was not an integer multiple of the LRECL parameter for fixed-length records, or was not
at least four bytes longer than the LRECL for variable-length
records.
and if you look at the JCL you provided:
//COPY01 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN01 DD DSN=CCSY.CCSYAC.BNS.C2,DISP=SHR
//IN02 DD DSN=CCSY.CCSYAC.UID.XBAS,DISP=SHR
//OUT01 DD DSN=CCSY.CCSYAC.BNS.C2.ID,DISP=(,CATLG,DELETE),
// DCB=(RECFM=FB,LRECL=410,BLKSIZE=41000),
// SPACE=(CYL,(200,100),RLSE)
//TOOLIN DD *
COPY JKFROM TO(OUT01) VSAMTYPE(V) USING(BNSC)
/*
//BNSCCNTL DD *
JOINKEYS F1=IN01,FIELDS=(401,6,A)
JOINKEYS F2=IN02,FIELDS=(87,6,A)
REFORMAT FIELDS=(F1:5,402,F2:42,8)
/*
You can see that your block size of 41000 is greater than that 32760 maximum. Try not specifying the block size. If you do not specify the block size in the JCL, the system will calculate an efficient one for you.

Node.js - spawn is cutting off the results

I'm creating a node program to return the output of linux top command, is working fine the only issue is that the name of command is cutted, instead the full command name like /usr/local/libexec/netdata/plugins.d/apps.plugin 1 returns /usr/local+
My code
const topparser=require("topparser")
const spawn = require('child_process').spawn
let proc=null
let startTime=0
exports.start=function(pid_limit,callback){
startTime=new Date().getTime()
proc = spawn('top', ['-c','-b',"-d","3"])
console.log("started process, pid: "+proc.pid)
let top_data=""
proc.stdout.on('data', function (data) {
console.log('stdout: ' + data);
})
proc.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}//start
exports.stop=function(){
console.log("stoped process...")
if(proc){proc.kill('SIGINT')}// SIGHUP -linux ,SIGINT -windows
}//stop
The results
14861 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/1+
14864 root 20 0 0 0 0 S 0.0 0.0 0:00.02 [kworker/0+
15120 root 39 19 102488 3344 2656 S 0.0 0.1 0:00.09 /usr/bin/m+
16904 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/0+
19031 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/u+
21500 root 20 0 0 0 0 Z 0.0 0.0 0:00.00 [dsc] <def+
22571 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kworker/0+
Any way to fix it?
Best regards
From a top manpage:
In Batch mode, when used without an argument top will format output using the COLUMNS= and LINES=
environment variables, if set. Otherwise, width will be fixed at the maximum 512 columns. With an
argument, output width can be decreased or increased (up to 512) but the number of rows is consid‐
ered unlimited.
Add '-w', '512' to the arguments.
Since you work with node, you can query netdata running on localhost for this.
Example:
http://london.my-netdata.io/api/v1/data?chart=apps.cpu&after=-1&options=ms
For localhost netdata:
http://localhost:19999/api/v1/data?chart=apps.cpu&after=-1&options=ms
You can also get systemd services:
http://london.my-netdata.io/api/v1/data?chart=services.cpu&after=-1&options=ms
If you are not planning to update the screen per second, you can instruct netdata to return the average of a longer duration:
http://london.my-netdata.io/api/v1/data?chart=apps.cpu&after=-5&points=1&group=average&options=ms
The above returns the average of the last 5 seconds.
Finally, you get the latest values all the metrics netdata monitors, with this:
http://london.my-netdata.io/api/v1/allmetrics?format=json
For completeness, netdata can export all the metrics in BASH format for shell scripts. Check this: https://github.com/firehol/netdata/wiki/receiving-netdata-metrics-from-shell-scripts

SGE faild to submit job, attribute is not a memory value

I failed to submit a job with mem attribute. Since I am novice and after google two days, I look for help here. Any suggestion would be grateful!
Following is what had I done:
\1. Submit my script:
qsub -S /bin/bash -A assembly -pe threads 16 -l mem=2GB -cwd -N "pBcR_correct_asm" -j y -o /dev/null runCorrection.sh
Unable to run job: unknown resource "mem".
Exiting.
\2. Considering that I had replace "h" to "host", which solve my problem according to SGE unknown resource "nodes", I replace "m" to "mem”, and it didn't work.
\3. After google, I know "h" is the shortcut that was defined in "/opt/gridengine/util/resources/centry/
hostname", and can be confirmed with "qconf -sc":
qconf -sc
#name shortcut type relop requestable consumable default urgency
#----------------------------------------------------------------------------------------
arch a RESTRING == YES NO NONE 0
calendar c RESTRING == YES NO NONE 0
cpu cpu DOUBLE >= YES NO 0 0
display_win_gui dwg BOOL == YES NO 0 0
h_core h_core MEMORY <= YES NO 0 0
h_cpu h_cpu TIME <= YES NO 0:0:0 0
h_data h_data MEMORY <= YES NO 0 0
h_fsize h_fsize MEMORY <= YES NO 0 0
h_rss h_rss MEMORY <= YES NO 0 0
h_rt h_rt TIME <= YES NO 0:0:0 0
h_stack h_stack MEMORY <= YES NO 0 0
h_vmem h_vmem MEMORY <= YES NO 0 0
hostname h HOST == YES NO NONE 0
load_avg la DOUBLE >= NO NO 0 0
load_long ll DOUBLE >= NO NO 0 0
load_medium lm DOUBLE >= NO NO 0 0
load_short ls DOUBLE >= NO NO 0 0
m_core core INT <= YES NO 0 0
m_socket socket INT <= YES NO 0 0
m_topology topo RESTRING == YES NO NONE 0
m_topology_inuse utopo RESTRING == YES NO NONE 0
mem_free mf MEMORY <= YES NO 0 0
mem_total mt MEMORY <= YES NO 0 0
mem_used mu MEMORY >= YES NO 0 0
\4. I thus replaced "mt" to "mem", but it complained attribute problem. According to above output, it seemed the mem_total is almost same as "hostname" that worked previously. Then, I think jsv may be a problem after going through the SGE guide, but I can't find any scripts contain "Unable to run job: attribute......", which under the director of "/opt/gridengine/util/resources/jsv". I think I have to configure some files, but what these files, and what should I do?
qsub -S /bin/bash -A assembly -pe threads 16 -l mt=2GB -cwd -N "pBcR_correct_asm" -j y -o test.out runCorrection.sh
Unable to run job: attribute "mem_total" is not a memory value.
Exiting.
#Vince!
Thanks for your reply very much.
Finally I solve my problem, by using "h_vmem=2g" ("2GB" would give error), but I don't know where to find how to design the value of the complex (MEMORY).
Following information is un-necessary now.
I had read the website you gave, and configured the attribute of h_vmem and s_vmeme in complex to "consumable", but it didn't work. I guess I have to configure "complex_value" of queue which is "NONE" at the moment. However, I can't open the web http://gridscheduler.sourceforge.net/htmlman/htmlman5/sge_types.html?pathrev=V62u5_TAG that may tell me how to configure. Am I right to configure to configure queue? Do I have to configure host too?
Any suggestions would be grateful!
Fowllowing is what had I done:
\1. Change the attribute of consumable to "YES" for h_vmem and s_vmem:
qconf -sc
#name shortcut type relop requestable consumable default urgency
#----------------------------------------------------------------------------------------
arch a RESTRING == YES NO NONE 0
calendar c RESTRING == YES NO NONE 0
cpu cpu DOUBLE >= YES NO 0 0
display_win_gui dwg BOOL == YES NO 0 0
h_core h_core MEMORY <= YES NO 0 0
h_cpu h_cpu TIME <= YES NO 0:0:0 0
h_data h_data MEMORY <= YES NO 0 0
h_fsize h_fsize MEMORY <= YES NO 0 0
h_rss h_rss MEMORY <= YES NO 0 0
h_rt h_rt TIME <= YES NO 0:0:0 0
h_stack h_stack MEMORY <= YES NO 0 0
h_vmem h_vmem MEMORY <= YES YES 0 0
hostname h HOST == YES NO NONE 0
load_avg la DOUBLE >= NO NO 0 0
load_long ll DOUBLE >= NO NO 0 0
load_medium lm DOUBLE >= NO NO 0 0
load_short ls DOUBLE >= NO NO 0 0
m_core core INT <= YES NO 0 0
m_socket socket INT <= YES NO 0 0
m_topology topo RESTRING == YES NO NONE 0
m_topology_inuse utopo RESTRING == YES NO NONE 0
mem_free mf MEMORY <= YES NO 0 0
mem_total mt MEMORY <= YES NO 0 0
mem_used mu MEMORY >= YES NO 0 0
min_cpu_interval mci TIME <= NO NO 0:0:0 0
np_load_avg nla DOUBLE >= NO NO 0 0
np_load_long nll DOUBLE >= NO NO 0 0
np_load_medium nlm DOUBLE >= NO NO 0 0
np_load_short nls DOUBLE >= NO NO 0 0
num_proc p INT == YES NO 0 0
qname q RESTRING == YES NO NONE 0
rerun re BOOL == NO NO 0 0
s_core s_core MEMORY <= YES NO 0 0
s_cpu s_cpu TIME <= YES NO 0:0:0 0
s_data s_data MEMORY <= YES NO 0 0
s_fsize s_fsize MEMORY <= YES NO 0 0
s_rss s_rss MEMORY <= YES NO 0 0
s_rt s_rt TIME <= YES NO 0:0:0 0
s_stack s_stack MEMORY <= YES NO 0 0
s_vmem s_vmem MEMORY <= YES YES 0 0
seq_no seq INT == NO NO 0 0
slots s INT <= YES YES 1 1000
swap_free sf MEMORY <= YES NO 0 0
swap_rate sr MEMORY >= YES NO 0 0
swap_rsvd srsv MEMORY >= YES NO 0 0
swap_total st MEMORY <= YES NO 0 0
swap_used su MEMORY >= YES NO 0 0
tmpdir tmp RESTRING == NO NO NONE 0
virtual_free vf MEMORY <= YES NO 0 0
virtual_total vt MEMORY <= YES NO 0 0
virtual_used vu MEMORY >= YES NO 0 0
# >#< starts a comment but comments are not saved across edits --------
\2. Submit my job to the queue of smp.q, and it complained the same problem:
qsub -S /bin/bash -A assembly -q smp.q -pe newPe 16 -l h_vmem=2GB -cwd -N "pBcR_correct_asm" -j y -o runCorrection.sh
Unable to run job: attribute "h_vmem" is not a memory value.
Exiting.
\3. The information of smp.q. I think "complex_values" should be changed and "h_vmem" can keep unchanged:
qconf -sq smp.q
qname smp.q
hostlist #smp.q
seq_no 0
load_thresholds np_load_avg=1.75
suspend_thresholds NONE
nsuspend 1
suspend_interval 00:05:00
priority 0
min_cpu_interval 00:05:00
processors UNDEFINED
qtype BATCH INTERACTIVE
ckpt_list NONE
pe_list make newPe
rerun FALSE
slots 160
tmpdir /tmp
shell /bin/csh
prolog NONE
epilog NONE
shell_start_mode posix_compliant
starter_method NONE
suspend_method NONE
resume_method NONE
terminate_method NONE
notify 00:00:60
owner_list NONE
user_lists NONE
xuser_lists NONE
subordinate_list NONE
complex_values NONE
projects NONE
xprojects NONE
calendar NONE
initial_state default
s_rt INFINITY
h_rt INFINITY
s_cpu INFINITY
h_cpu INFINITY
s_fsize INFINITY
h_fsize INFINITY
s_data INFINITY
h_data INFINITY
s_stack INFINITY
h_stack INFINITY
s_core INFINITY
h_core INFINITY
s_rss INFINITY
h_rss INFINITY
s_vmem INFINITY
h_vmem INFINITY
\4. The information of hosts in #smp.q:
qconf -sconf smp03.local
#smp03.local:
mailer /bin/mail
xterm /usr/bin/X11/xterm
execd_spool_dir /opt/gridengine/default/spool
\5. The global information. Have I add h_vmem and s_vmem here?
qconf -sconf
#global:
execd_spool_dir /opt/gridengine/default/spool
mailer /bin/mail
xterm /usr/bin/X11/xterm
load_sensor none
prolog none
epilog none
shell_start_mode posix_compliant
login_shells sh,ksh,csh,tcsh
min_uid 0
min_gid 0
user_lists none
xuser_lists none
projects none
xprojects none
enforce_project false
enforce_user auto
load_report_time 00:00:40
max_unheard 00:05:00
reschedule_unknown 00:00:00
loglevel log_warning
administrator_mail none
set_token_cmd none
pag_cmd none
token_extend_time none
shepherd_cmd none
qmaster_params none
execd_params ENABLE_ADDGRP_KILL=TRUE H_MEMORYLOCKED=infinity
reporting_params accounting=true reporting=true \
flush_time=00:00:15 joblog=true sharelog=00:00:00
finished_jobs 100
gid_range 20000-20100
qlogin_command builtin
qlogin_daemon builtin
rlogin_command builtin
rlogin_daemon builtin
rsh_command builtin
rsh_daemon builtin
max_aj_instances 2000
max_aj_tasks 75000
max_u_jobs 0
max_jobs 0
max_advance_reservations 0
auto_user_oticket 0
auto_user_fshare 0
auto_user_default_project none
auto_user_delete_time 86400
delegated_file_staging false
reprioritize 0
jsv_url none
jsv_allowed_mod ac,h,i,e,o,j,M,N,p,w
What you likely want is h_vmem. At least that is the attribute I always use to specify the memory I want request for a job.
See:
http://gridscheduler.sourceforge.net/htmlman/htmlman5/queue_conf.html?pathrev=V62u5_TAG
Specifically,
The resource limit parameters s_vmem and h_vmem are imple-
mented by Sun Grid Engine as a job limit. They impose a
limit on the amount of combined virtual memory consumed by
all the processes in the job. If h_vmem is exceeded by a job
running in the queue, it is aborted via a SIGKILL signal
(see kill(1)). If s_vmem is exceeded, the job is sent a
SIGXCPU signal which can be caught by the job. If you wish
to allow a job to be "warned" so it can exit gracefully
before it is killed then you should set the s_vmem limit to
a lower value than h_vmem. For parallel processes, the
limit is applied per slot which means that the limit is mul-
tiplied by the number of slots being used by the job before
being applied.
Also, you may need to set this as consumable using qconf.

Resources