Input line is too long - Spark - apache-spark

I am getting following error while executing sparkling-shell2.cmd bat file. I walked through and I am getting this error while executing spark-shell.cmd with following paramters
cd %TOPDIR%
%SPARK_HOME%/bin/spark-shell.cmd --jars %TOPDIR%/assembly/build/libs/%FAT_JAR% --driver-memory %DRIVER_MEMORY% --conf spark.driver.extraJavaOptions="-XX:MaxPermSize=384m" %*
Error: The input line is too long.
How do I solve this issue?
Thanks

In Windows the maximum length of command line is 260 characters and you are hitting the limit.
Your options are to change multiple of %DEF% operators into one single operator and reduce the overall length with some experimentation.

Related

Error about TMP_DIR space using parallel program

I am using parallel for freebayes to call variants using,
TMPDIR=/path/tmp freebayes-parallel <(fasta_generate_regions.py genome.fasta.fai 100000) 36 \
-f genome.fasta -L $bam_list.txt > freebayes.vcf
But it gives error:
parallel: Error: Output is incomplete. Cannot append to buffer file tmp. Is the disk full?
parallel: Error: Change $TMPDIR with --tmpdir or use --compress.
I found some other posts where they suggested to change the tmp where space is available. I am not sure how to do that. Thank you for your help!

Running trinity for de novo assembly

I have a problem in running rnaseqtrinity, my command line is here:
/home/marziyeh/Software/trinityrnaseq-Trinity-v2.4.0/
Trinity --seqType fq --max_memory 4G --left /home/marziyeh/Data/RNA-seq\ data/Main\ data/paired_32_L3_1_trimmed.fq.gz paired_33_L3_1_
trimmed.fq.gz --right paired_32_L3_2_trimmed.fq.gz paired_33_L3_2_trimmed.fq.gz --CPU 2 --output trinity_out_dir_32_32.fq.gz
but the following error occours:
Error, cannot locate file: paired_33_L3_1_trimmed.fq.gz at /home/marziyeh/Software/trinityrnaseq-Trinity-v2.4.0/Trinity line
2429. main::create_full_path(ARRAY(0x555a78a86df0), 1) called at /home/marziyeh/Software/trinityrnaseq-Trinity-v2.4.0/Trinity line
1159
Thanks so much for your suggestions.
Look, your pramater of --left have a fatal error .
You command line indicates that your reads is paired end. But your path of left read have some problems.
1. Your should use \ to instead of /.
2. Your command shouldn't have the spaceļ¼Œ and the space in linux means a delimiter.

yaml syntax error for ansible base64 multi line variable

Ansible Version: 2.1.2.0
So I have a yaml file with a multi line variable that's from a binary file converted to base 64.
My variable file: self-cert.yml
selfcert: |
MIIKCAIBAzCCCcIGCSqGSIb3DQEHAaCCCbMEggmvMIIJqzCCBWgGCSqGSIb3DQEHAaCCBVkEggVV
MIIFUTCCBU0GCyqGSIb3DQEMCgECoIIE+jCCBPYwKAYKKoZIhvcNAQwBAzAaBBQFa98IY7UgblDK
qGwMjTIQCK+3DwICBAAEggTIvA/VFm3j3oSN6cknp5qFyUxXAI5TxURnyx8UVRm8UfMcA0LHlh+z
06ztcwApIrxMSV26ezu0p1FrHInpbABNuO0rlk4XlQwTkLynUyg58iBwK7IyV5SqT2UC8djaOiMN
b9ViC3yn7SrRdS3MmCQznu6dScRIHbhG46yZNJrzrJh038X2KAPpS/LfC9DJBjaEzkZY8BwyARYe
When I try to run my playbook that includes this variable, I get:
ERROR! Syntax Error while loading YAML.
The error appears to have been in '/home/ansible/projects/install-cert/self-cert.yml': line 3, column 1, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
MIIKCAIBAzCCCcIGCSqGSIb3DQEHAaCCCbMEggmvMIIJqzCCBWgGCSqGSIb3DQEHAaCCBVkEggVV
MIIFUTCCBU0GCyqGSIb3DQEMCgECoIIE+jCCBPYwKAYKKoZIhvcNAQwBAzAaBBQFa98IY7UgblDK
^ here
Any idea whats wrong? I've tried changing | to > , didn't work, and i've also tried indenting the whole base64 output too.
So it turns out you do need to make an indentation to the the multiline variable. My original indentation was an actual tab instead of spaces (Stupid Sublime) and so the indentation failed the syntax check, but using actual spaces made everything work.

I am not able to execute the following aws cli command

I am trying to execute the below command but I am getting an error can anyone please help
aws --profile awsprofile logs put-log-events --log-group-name mylogs --log-stream-name 20150601 --log-events file:///var/log/kafka/server.log.2016-09-25-02
Error:-
Expecting ',' delimiter or ']': line 1 column 6 (char 5)
So I determine the problem is with file:///
I want to specify /var/log/kafka/server.log.2016-09-25-02
as when I specify var/log/kafka/server.log.2016-09-25-02 it is saying file not found so I want to specify /var/log/kafka/server.log.2016-09-25-02. But when I specify in that way it is showing error as
Expecting ',' delimiter or ']': line 1 column 6 (char 5)
So please help me. Help will be appreciated.
The problem is not with the CLI command line, but in your log_events file /var/log/kafka/server.log.2016-09-25-02. It is not a valid JSON file.
Take a look at: put-log-event example
Put " or ' around file:///var/log/kafka/server.log.2016-09-25-02

Why is vbc mangling the command line?

Following the technique here, I got the vbc command line used to compile my project. I've used this to compile the project from the command line and it's worked fine the several times I've tried it. The command line is about 25,000 characters long, if that means anything.
But today when I ran the command, I got a bunch of errors that look like this:
vbc : Command line error BC2001 : file 'admin\TestShare.aspxvb' could not be found
However, when I look at the actual command line for the reference to that file, it's correct:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Vbc.exe ... admin\TestShare.aspx.vb ...
All of the other "file could not be found" errors are similar: it's dropping characters from the filename (not just dots but text characters as well), and then it says it can't find that file.
What's going on here?
Update: That first bad file is right near character 8192 in the command line, which suggests that the problem is related to the path being too long for Windows. But if this is the problem, how can I build my app from the command line?
If you just need to build from the command line, why not use msbuild instead? All that is necessary is a msbuild YourProject.proj to get things rolling, and much shorter than the verbose way ;-) Run from VS command prompt.

Resources