I have a node script I pass a large json stream into as command line arg. It's worth noting I am running Mac OSX High Sierra
The script is unable to run because of this error:
-bash: /usr/local/bin/node: Argument list too long
Upon reading, this is caused my a limit in the ARG_MAX or MAX_ARG_STRLEN
according to https://www.in-ulm.de/~mascheck/various/argmax/
I attempted to raise these limits in the config file here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/syslimits.h
Here is a section of the file where I have raised the ARG_MAX size from (256 * 1024) and I have added the MAX_ARG_STRLEN config.
#define ARG_MAX (512 * 1024) /* max bytes for an exec function */
#define MAX_ARG_STRLEN (PAGE_SIZE * 64)
#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE)
#define CHILD_MAX 266 /* max simultaneous processes */
#define GID_MAX 2147483647U /* max value for a gid_t (2^31-2) */
#endif /* (_POSIX_C_SOURCE && !_DARWIN_C_SOURCE) */
#define LINK_MAX 32767 /* max file link count */
#define MAX_CANON 1024 /* max bytes in term canon input line */
#define MAX_INPUT 1024 /* max bytes in terminal input */
#define NAME_MAX 255 /* max bytes in a file name */
#define NGROUPS_MAX 16 /* max supplemental group id's */
#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE)
#define UID_MAX 2147483647U /* max value for a uid_t (2^31-2) */
#define OPEN_MAX 10240 /* max open files per process - todo, make a config option? */
#endif /* (_POSIX_C_SOURCE && !_DARWIN_C_SOURCE) */
#define PATH_MAX 1024 /* max bytes in pathname */
#define PIPE_BUF 512 /* max bytes for atomic pipe writes */
#define BC_BASE_MAX 99 /* max ibase/obase values in bc(1) */
#define BC_DIM_MAX 2048 /* max array elements in bc(1) */
#define BC_SCALE_MAX 99 /* max scale value in bc(1) */
#define BC_STRING_MAX 1000 /* max const string length in bc(1) */
#define CHARCLASS_NAME_MAX 14 /* max character class name size */
#define COLL_WEIGHTS_MAX 2 /* max weights for order keyword */
#define EQUIV_CLASS_MAX 2
#define EXPR_NEST_MAX 32 /* max expressions nested in expr(1) */
#define LINE_MAX 2048 /* max bytes in an input line */
#define RE_DUP_MAX 255 /* max RE's in interval notation */
#if __DARWIN_UNIX03
#define NZERO 20 /* default priority [XSI] */
/* = ((PRIO_MAX - PRIO_MIN) / 2) + 1 */
/* range: 0 - 39 [(2 * NZERO) - 1] */
/* 0 is not actually used */
#else /* !__DARWIN_UNIX03 */
#define NZERO 0 /* default priority */
/* range: -20 - 20 */
/* (PRIO_MIN - PRIO_MAX) */
#endif /* __DARWIN_UNIX03 */
#endif /* !_ANSI_SOURCE */
#endif /* !_SYS_SYSLIMITS_H_ */
After these changes, my file which contains 330,000 characters is still failing. The file is 300kb upon running the node script.
Your expertise is appreciated! I am unsure if this is caused by an overflow in the string size or the arg size (what is the difference?) Either way I have not found a way to make it work
These values cannot be changed in Mac OS X kernel.
It was strange to me when I saw this:
#define HI3518_IOCH1_PHYS 0x10000000 /* 0x1000_0000 ~ 0x1020_0000 */
#define HI3518_IOCH2_PHYS 0x20000000 /* 0x2000_0000 ~ 0x2020_0000 */
#define HI3518_IOCH1_SIZE 0x200000
#define HI3518_IOCH2_SIZE 0x700000
#define HI3518_IOCH1_VIRT 0xFE000000
#define HI3518_IOCH2_VIRT (HI3518_IOCH1_VIRT + HI3518_IOCH1_SIZE)
This is located at arch/arm/mach-hi3518/include/mach/io.h
The last 2 line puzzle me, is that mean user can determine the virtual address themselves, don't need to use APIs?
Thanks in advance!
For Unix Sockets , there is the struct sockaddress_un which has two members : sun_family which refers to the communication domain in which the socket is to be created and sun_path which refers to the path name
Can someone explain the meaning of the words in bold ??
Why is the rationale for naming the two members : sun_family & sun_path
What does the un in sockaddress_un mean ??
For the communication domain you might want to look at where the macro for AF_UNIX (which is mentioned in /usr/include/un.h) is set, using grep -FR AF_UNIX /usr/include/. On my system/usr/include/x86_64-linux-gnu/bits/socket.h` gives:
/* Protocol families. */
#define PF_UNSPEC 0 /* Unspecified. */
#define PF_LOCAL 1 /* Local to host (pipes and file-domain). */
#define PF_UNIX PF_LOCAL /* POSIX name for PF_LOCAL. */
#define PF_FILE PF_LOCAL /* Another non-standard name for PF_LOCAL. */
#define PF_INET 2 /* IP protocol family. */
#define PF_AX25 3 /* Amateur Radio AX.25. */
#define PF_IPX 4 /* Novell Internet Protocol. */
#define PF_APPLETALK 5 /* Appletalk DDP. */
#define PF_NETROM 6 /* Amateur radio NetROM. */
#define PF_BRIDGE 7 /* Multiprotocol bridge. */
#define PF_ATMPVC 8 /* ATM PVCs. */
#define PF_X25 9 /* Reserved for X.25 project. */
#define PF_INET6 10 /* IP version 6. */
#define PF_ROSE 11 /* Amateur Radio X.25 PLP. */
...
Which indicates that the domain is the type of communication that the socket is handling.
The un stands for unified as this combines the different additional informations (port number, path, pipe, etc) these different domains use, the specifics of which are passed in the sun_path part of the struct sockaddr_un.
I have following code to get width and height of screen in Linux.
#ifdef TIOCGSIZE
struct ttysize ts;
ioctl(STDIN_FILENO, TIOCGSIZE, &ts);
cols = ts.ts_cols;
lines = ts.ts_lines;
#elif defined(TIOCGWINSZ)
struct winsize ts;
ioctl(STDIN_FILENO, TIOCGWINSZ, &ts);
cols = ts.ws_col;
lines = ts.ws_row;
#endif /* TIOCGSIZE */
What is difference between ttysize and winsize?
The ttysize was the original implementation for SunOS 3.0 (February 1986), and soon after made obsolete by winsize, which adds the size of the window in pixels. Here's
what the definitions look like in <sys/ttycom.h> from SunOS 4:
/*
* Window/terminal size structure.
* This information is stored by the kernel
* in order to provide a consistent interface,
* but is not used by the kernel.
*
* Type must be "unsigned short" so that types.h not required.
*/
struct winsize {
unsigned short ws_row; /* rows, in characters */
unsigned short ws_col; /* columns, in characters */
unsigned short ws_xpixel; /* horizontal size, pixels - not used */
unsigned short ws_ypixel; /* vertical size, pixels - not used */
};
#define TIOCGWINSZ _IOR(t, 104, struct winsize) /* get window size */
#define TIOCSWINSZ _IOW(t, 103, struct winsize) /* set window size */
/*
* Sun version of same.
*/
struct ttysize {
int ts_lines; /* number of lines on terminal */
int ts_cols; /* number of columns on terminal */
};
#define TIOCSSIZE _IOW(t,37,struct ttysize)/* set tty size */
#define TIOCGSIZE _IOR(t,38,struct ttysize)/* get tty size */
The data types are different (an integer would waste memory), and the fields have different names.
The ttysize structure has long been obsolete: if either is provided by the system, winsize is supported. That wasn't true when porting ncurses to SCO OpenServer in 1997, as noted in this chunk from lib_setup.c:
/*
* SCO defines TIOCGSIZE and the corresponding struct. Other systems (SunOS,
* Solaris, IRIX) define TIOCGWINSZ and struct winsize.
*/
#ifdef TIOCGSIZE
# define IOCTL_WINSIZE TIOCGSIZE
# define STRUCT_WINSIZE struct ttysize
# define WINSIZE_ROWS(n) (int)n.ts_lines
# define WINSIZE_COLS(n) (int)n.ts_cols
#else
# ifdef TIOCGWINSZ
# define IOCTL_WINSIZE TIOCGWINSZ
# define STRUCT_WINSIZE struct winsize
# define WINSIZE_ROWS(n) (int)n.ws_row
# define WINSIZE_COLS(n) (int)n.ws_col
# endif
#endif
You might notice that Linux is not mentioned in the comment. According to comments in asm-sparc64/ioctls.h, the ioctl for ttysize was unsupported as of 2.6.16:
/* Note that all the ioctls that are not available in Linux have a
* double underscore on the front to: a) avoid some programs to
* think we support some ioctls under Linux (autoconfiguration stuff)
*/
...
#define TIOCCONS _IO('t', 36)
#define __TIOCSSIZE _IOW('t', 37, struct sunos_ttysize) /* SunOS Specific */
#define __TIOCGSIZE _IOR('t', 38, struct sunos_ttysize) /* SunOS Specific */
#define TIOCGSOFTCAR _IOR('t', 100, int)
#define TIOCSSOFTCAR _IOW('t', 101, int)
#define __TIOCUCNTL _IOW('t', 102, int) /* SunOS Specific */
#define TIOCSWINSZ _IOW('t', 103, struct winsize)
#define TIOCGWINSZ _IOR('t', 104, struct winsize)
A much earlier comment in 1995 added the definitions (without the double-underscore). Possibly a few programs used that with Linux, although winsize was well established on most platforms before Linux was begun. A little more digging finds that the double-underscore was introduced in 1996 (patch-2.1.9 linux/include/asm-sparc/ioctls.h). Given that, very few programs would have used it with Linux.
Further reading:
Garbage-collect struct ttysize (OpenBSD mailing list)
/dev/console MKS manual page
They have the same function (at least in this case - maybe one structure has an additional field not used here).
Historically some (mainly older) Linux versions have different definitions of the IOCTL codes. Therefore some Linux versions have only TIOCGSIZE defined (using a ttysize structure) and some Linux versions have only TIOCGWINSZ defined.
Using the "#ifdef" construction the program can be compiled for both Linux versions.
Newer Linux versions should have both IOCTLs defined.
I used readelf on several binaries on my linux box and saw something that surprised me in the program headers. This eample is from the 'ld' utility, but it also occurs with anything I compile with gcc.
PHDR 0x000034 0x08048034 0x08048034 0x00120 0x00120 R E 0x4
This segment spans the entirety of the program headers. Why is is marked as executable? It doesn't contain machine code. But also, why is even this present in the headers? I don't really want it in my program image.
The PHDR pointing to the PHDRs tells the loader that the PHDRs themselves should be mapped to the process address space, in order to make them accessible to the program itself.
This is useful mainly for dynamic linking.
The reason the memory is marked as executable is because the PHDRs are smaller than one page, and live right next to the start of the executable code. If the permissions for the PHDRs were different from those of the program text, the linker would have to insert padding between them.
The main File ELF headers are there to easily find the offset in the file where other sections are stored. Then each subheader describes the data in it's section.
Main ELF header looks like this:
/* ELF File Header */
typedef struct
{
unsigned char e_ident[EI_NIDENT]; /* Magic number and other info */
Elf32_Half e_type; /* Object file type */
Elf32_Half e_machine; /* Architecture */
Elf32_Word e_version; /* Object file version */
Elf32_Addr e_entry; /* Entry point virtual address */
Elf32_Off e_phoff; /* Program header table file offset */
Elf32_Off e_shoff; /* Section header table file offset */
Elf32_Word e_flags; /* Processor-specific flags */
Elf32_Half e_ehsize; /* ELF header size in bytes */
Elf32_Half e_phentsize; /* Program header table entry size */
Elf32_Half e_phnum; /* Program header table entry count */
Elf32_Half e_shentsize; /* Section header table entry size */
Elf32_Half e_shnum; /* Section header table entry count */
Elf32_Half e_shstrndx; /* Section header string table index */
} Elf32_Ehdr;
The program header(s) are there because they describe the executable parts of the ELF executable.
The next portion of the program are
the ELF program headers. These
describe the sections of the program
that contain executable program code
to get mapped into the program address
space as it loads.
/* Program segment header. */
typedef struct
{
Elf32_Word p_type; /* Segment type */
Elf32_Off p_offset; /* Segment file offset */
Elf32_Addr p_vaddr; /* Segment virtual address */
Elf32_Addr p_paddr; /* Segment physical address */
Elf32_Word p_filesz; /* Segment size in file */
Elf32_Word p_memsz; /* Segment size in memory */
Elf32_Word p_flags; /* Segment flags */
Elf32_Word p_align; /* Segment alignment */
} Elf32_Phdr;
This is taken from here