Julia 1.3.1 works fine in my ubuntu 18.04 platform:
Linux KenServ 5.3.0-45-generic #37~18.04.1-Ubuntu SMP Fri Mar 27 15:58:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
But when I download and try to run julia 1.4.1 it fails and show me this messages:
Invalid instruction at 0x7faf524e39f8: 0x66, 0x0f, 0x3a, 0x0f, 0xc0, 0x08, 0x0f, 0x83, 0x61, 0x05, 0x00, 0x00, 0x48, 0xc1, 0xe0
signal (4): Instrucción ilegal
in expression starting at none:0
_ZN4llvm13LexicalScopes23assignInstructionRangesERNS_15SmallVectorImplISt4pairIPKNS_12MachineInstrES5_EEERNS_8DenseMapIS5_PNS_12LexicalScopeENS_12DenseMapInfoIS5_EENS_6detail12DenseMapPairIS5_SB_EEEE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
_ZN4llvm13LexicalScopes10initializeERKNS_15MachineFunctionE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
_ZN4llvm18LiveDebugVariables20runOnMachineFunctionERNS_15MachineFunctionE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /home/kenneth/Descargas/julia/julia-1.4.1/bin/../lib/julia/libLLVM-8jl.so (unknown line)
operator() at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:356
addModule at /buildworker/worker/package_linux64/build/usr/include/llvm/ExecutionEngine/Orc/IRCompileLayer.h:84 [inlined]
addModule at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:485
jl_add_to_ee at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:747 [inlined]
jl_finalize_function at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:755
getAddressForFunction at /buildworker/worker/package_linux64/build/src/codegen.cpp:1414
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/codegen.cpp:1508
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1912
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2153 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
#764 at ./client.jl:370
jfptr_#764_8156 at /home/kenneth/Descargas/julia/julia-1.4.1/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1700 [inlined]
do_apply at /buildworker/worker/package_linux64/build/src/builtins.c:643
jl_f__apply_latest at /buildworker/worker/package_linux64/build/src/builtins.c:693
#invokelatest#1 at ./essentials.jl:712 [inlined]
invokelatest at ./essentials.jl:711 [inlined]
run_main_repl at ./client.jl:367
exec_options at ./client.jl:305
_start at ./client.jl:484
jfptr__start_2075 at /home/kenneth/Descargas/julia/julia-1.4.1/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux64/build/ui/../src/julia.h:1700 [inlined]
true_main at /buildworker/worker/package_linux64/build/ui/repl.c:96
main at /buildworker/worker/package_linux64/build/ui/repl.c:217
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_start at ./julia (unknown line)
Allocations: 2505 (Pool: 2496; Big: 9); GC: 0
Instrucción ilegal (`core' generado)
I try to compile from source. The same thing happens.
Invalid instruction at 0x7fad3a976ad0: 0x66, 0x0f, 0x3a, 0x0f, 0xc0, 0x08, 0x0f, 0x83, 0xae, 0x04, 0x00, 0x00, 0x48, 0xc1, 0xe0
signal (4): Instrucción ilegal
Allocations: 1 (Pool: 1; Big: 0); GC: 0
Illegal instruction (core dumped)
*** This error might be fixed by running `make clean`. If the error persists, try `make cleanall`. ***
sysimage.mk:67: fallo en las instrucciones para el objetivo '/home/kenneth/Descargas/julia/julia-1.4.1/usr/lib/julia/sys.ji'
make[1]: *** [/home/kenneth/Descargas/julia/julia-1.4.1/usr/lib/julia/sys.ji] Error 1
Makefile:81: fallo en las instrucciones para el objetivo 'julia-sysimg-ji'
make: *** [julia-sysimg-ji] Error 2
Please help me, What am I doing wrong?
Below is the logs i am getting in my console.
.spark.executor.Executor : Finished task 185.0 in stage 189.0 (TID 4477). 11508 bytes result sent to driver
2017-05-06 10:00:18.767 INFO 3336 --- [er-event-loop-2] o.apache.spark.scheduler.TaskSetManager : Starting task 188.0 in stage 189.0 (TID 4480, localhost, executor driver, partition 188, ANY, 6317 bytes)
2017-05-06 10:00:18.769 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.769 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.769 INFO 3336 --- [result-getter-1] o.apache.spark.scheduler.TaskSetManager : Finished task 185.0 in stage 189.0 (TID 4477) in 75 ms on localhost (executor driver) (185/200)
2017-05-06 10:00:18.769 INFO 3336 --- [launch worker-5] org.apache.spark.executor.Executor : Running task 188.0 in stage 189.0 (TID 4480)
2017-05-06 10:00:18.770 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.770 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.771 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Getting 2 non-empty blocks out of 201 blocks
2017-05-06 10:00:18.771 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.773 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Getting 3 non-empty blocks out of 401 blocks
2017-05-06 10:00:18.773 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.773 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.773 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.773 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.774 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 1 ms
2017-05-06 10:00:18.775 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 2 non-empty blocks out of 201 blocks
2017-05-06 10:00:18.775 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.777 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 3 non-empty blocks out of 401 blocks
2017-05-06 10:00:18.777 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.786 INFO 3336 --- [launch worker-6] org.apache.spark.executor.Executor : Finished task 182.0 in stage 189.0 (TID 4474). 11508 bytes result sent to driver
2017-05-06 10:00:18.786 INFO 3336 --- [er-event-loop-1] o.apache.spark.scheduler.TaskSetManager : Starting task 189.0 in stage 189.0 (TID 4481, localhost, executor driver, partition 189, ANY, 6317 bytes)
2017-05-06 10:00:18.787 INFO 3336 --- [result-getter-2] o.apache.spark.scheduler.TaskSetManager : Finished task 182.0 in stage 189.0 (TID 4474) in 132 ms on localhost (executor driver) (186/200)
2017-05-06 10:00:18.787 INFO 3336 --- [launch worker-6] org.apache.spark.executor.Executor : Running task 189.0 in stage 189.0 (TID 4481)
2017-05-06 10:00:18.790 INFO 3336 --- [launch worker-5] org.apache.spark.executor.Executor : Finished task 188.0 in stage 189.0 (TID 4480). 11356 bytes result sent to driver
2017-05-06 10:00:18.790 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.790 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.791 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.791 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.792 INFO 3336 --- [er-event-loop-2] o.apache.spark.scheduler.TaskSetManager : Starting task 190.0 in stage 189.0 (TID 4482, localhost, executor driver, partition 190, ANY, 6317 bytes)
2017-05-06 10:00:18.792 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Getting 2 non-empty blocks out of 201 blocks
2017-05-06 10:00:18.792 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.794 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Getting 2 non-empty blocks out of 401 blocks
2017-05-06 10:00:18.794 INFO 3336 --- [launch worker-6] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.796 INFO 3336 --- [launch worker-4] org.apache.spark.executor.Executor : Finished task 187.0 in stage 189.0 (TID 4479). 11356 bytes result sent to driver
2017-05-06 10:00:18.798 INFO 3336 --- [er-event-loop-0] o.apache.spark.scheduler.TaskSetManager : Starting task 191.0 in stage 189.0 (TID 4483, localhost, executor driver, partition 191, ANY, 6317 bytes)
2017-05-06 10:00:18.798 INFO 3336 --- [launch worker-5] org.apache.spark.executor.Executor : Running task 190.0 in stage 189.0 (TID 4482)
2017-05-06 10:00:18.798 INFO 3336 --- [result-getter-3] o.apache.spark.scheduler.TaskSetManager : Finished task 188.0 in stage 189.0 (TID 4480) in 31 ms on localhost (executor driver) (187/200)
2017-05-06 10:00:18.798 INFO 3336 --- [result-getter-3] o.apache.spark.scheduler.TaskSetManager : Finished task 187.0 in stage 189.0 (TID 4479) in 35 ms on localhost (executor driver) (188/200)
2017-05-06 10:00:18.800 INFO 3336 --- [launch worker-4] org.apache.spark.executor.Executor : Running task 191.0 in stage 189.0 (TID 4483)
2017-05-06 10:00:18.801 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.801 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.802 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.802 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.803 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.803 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.803 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 201 blocks
2017-05-06 10:00:18.803 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.804 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 1 blocks
2017-05-06 10:00:18.804 INFO 3336 --- [launch worker-4] o.a.s.s.ShuffleBlockFetcherIterator : Started 0 remote fetches in 0 ms
2017-05-06 10:00:18.804 INFO 3336 --- [launch worker-5] o.a.s.s.ShuffleBlockFetcherIterator : Getting 1 non-empty blocks out of 401 blocks
Below is my POM file.
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web-services</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
<groupId>info.debatty</groupId>
<artifactId>java-string-similarity</artifactId>
<version>RELEASE</version>
</dependency>
<dependency>
<groupId>com.univocity</groupId>
<artifactId>univocity-parsers</artifactId>
<version>2.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>commons-compiler</artifactId>
<version>2.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>11.2.0.3</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-network-common_2.10</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>commons-compiler</artifactId>
<version>2.7.5</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
I think you can change log level as
sparkContext.setLogLevel("WARN")
You can choose the log level among
ALL, DEBUG, ERROR, FATAL, INFO, OFF, TRACE, WARN
If the log printed is in the spark-shell then you can change the log level from config file located at conf/log4j.properties (change the name from conf/log4j.properties.template)
then change the log level as you want
log4j.rootCategory=INFO, console
and reopen the shell you will see less output.
Set this setting spark.history.fs.cleaner.enabled to true in spark-default-conf. This will help in purging the logs from the hdfs after 7 days by default, which can be altered by setting spark.history.fs.cleaner.maxAge
I use below code in scala :
Logger.getLogger("org").setLevel(Level.OFF)
Logger.getLogger("akka").setLevel(Level.WARN)
you can try something like above in Java.
I'm trying to build apktool (tool for decompiling android apk files).
The instructions how to build are at http://ibotpeaches.github.io/Apktool/build/
The problem is with the command ./gradlew build fatJar
The errors that I receive are:
brut.androlib.SharedLibraryTest > isSharedResourceDecodingAndRebuildingWorking FAILED
brut.androlib.AndrolibException: brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 21, --version-code, 21, --version-name, 5.0-eng.ibotpeaches.20141225.072308, -F, /tmp/APKTOOL134713816630716118.tmp, -0, arsc, -I, /tmp/BRUT4423679278565619004.tmp/1.apk, -I, /tmp/BRUT4423679278565619004.tmp/2-shared.apk, -S, /tmp/BRUT4423679278565619004.tmp/client.apk.out/res, -M, /tmp/BRUT4423679278565619004.tmp/client.apk.out/AndroidManifest.xml]
at brut.androlib.Androlib.buildResourcesFull(Androlib.java:459)
at brut.androlib.Androlib.buildResources(Androlib.java:397)
at brut.androlib.Androlib.build(Androlib.java:285)
at brut.androlib.SharedLibraryTest.isSharedResourceDecodingAndRebuildingWorking(SharedLibraryTest.java:102)
Caused by:
brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 21, --version-code, 21, --version-name, 5.0-eng.ibotpeaches.20141225.072308, -F, /tmp/APKTOOL134713816630716118.tmp, -0, arsc, -I, /tmp/BRUT4423679278565619004.tmp/1.apk, -I, /tmp/BRUT4423679278565619004.tmp/2-shared.apk, -S, /tmp/BRUT4423679278565619004.tmp/client.apk.out/res, -M, /tmp/BRUT4423679278565619004.tmp/client.apk.out/AndroidManifest.xml]
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:419)
at brut.androlib.Androlib.buildResourcesFull(Androlib.java:445)
... 3 more
Caused by:
brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 21, --version-code, 21, --version-name, 5.0-eng.ibotpeaches.20141225.072308, -F, /tmp/APKTOOL134713816630716118.tmp, -0, arsc, -I, /tmp/BRUT4423679278565619004.tmp/1.apk, -I, /tmp/BRUT4423679278565619004.tmp/2-shared.apk, -S, /tmp/BRUT4423679278565619004.tmp/client.apk.out/res, -M, /tmp/BRUT4423679278565619004.tmp/client.apk.out/AndroidManifest.xml]
at brut.util.OS.exec(OS.java:93)
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:413)
... 4 more
Caused by:
java.io.IOException: Cannot run program "/tmp/brut_util_Jar_9054478823788249311.tmp": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
at brut.util.OS.exec(OS.java:84)
... 5 more
Caused by:
java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
... 8 more
brut.androlib.ProviderAttributeTest > isProviderStringReplacementWorking FAILED
brut.androlib.AndrolibException: brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 22, --version-code, 1, --version-name, 1.0, -F, /tmp/APKTOOL5763864884088802643.tmp, -0, arsc, -I, /home/ran/apktool/framework/1.apk, -S, /tmp/BRUT4214352291584370372.tmp/issue636.apk.out/res, -M, /tmp/BRUT4214352291584370372.tmp/issue636.apk.out/AndroidManifest.xml]
at brut.androlib.Androlib.buildResourcesFull(Androlib.java:459)
at brut.androlib.Androlib.buildResources(Androlib.java:397)
at brut.androlib.Androlib.build(Androlib.java:285)
at brut.androlib.ProviderAttributeTest.isProviderStringReplacementWorking(ProviderAttributeTest.java:59)
Caused by:
brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 22, --version-code, 1, --version-name, 1.0, -F, /tmp/APKTOOL5763864884088802643.tmp, -0, arsc, -I, /home/ran/apktool/framework/1.apk, -S, /tmp/BRUT4214352291584370372.tmp/issue636.apk.out/res, -M, /tmp/BRUT4214352291584370372.tmp/issue636.apk.out/AndroidManifest.xml]
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:419)
at brut.androlib.Androlib.buildResourcesFull(Androlib.java:445)
... 3 more
Caused by:
brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --min-sdk-version, 21, --target-sdk-version, 22, --version-code, 1, --version-name, 1.0, -F, /tmp/APKTOOL5763864884088802643.tmp, -0, arsc, -I, /home/ran/apktool/framework/1.apk, -S, /tmp/BRUT4214352291584370372.tmp/issue636.apk.out/res, -M, /tmp/BRUT4214352291584370372.tmp/issue636.apk.out/AndroidManifest.xml]
at brut.util.OS.exec(OS.java:93)
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:413)
... 4 more
Caused by:
java.io.IOException: Cannot run program "/tmp/brut_util_Jar_9054478823788249311.tmp": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
at brut.util.OS.exec(OS.java:84)
... 5 more
Caused by:
java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
... 8 more
brut.androlib.BuildAndDecodeJarTest > classMethod FAILED
brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, -F, /tmp/BRUT3478421848789513632.tmp/testjar.jar, -0, arsc, /tmp/BRUT3478421848789513632.tmp/testjar-orig/build/apk]
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:419)
at brut.androlib.Androlib.buildApk(Androlib.java:661)
at brut.androlib.Androlib.build(Androlib.java:290)
at brut.androlib.BuildAndDecodeJarTest.beforeClass(BuildAndDecodeJarTest.java:45)
Caused by:
brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, -F, /tmp/BRUT3478421848789513632.tmp/testjar.jar, -0, arsc, /tmp/BRUT3478421848789513632.tmp/testjar-orig/build/apk]
at brut.util.OS.exec(OS.java:93)
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:413)
... 3 more
Caused by:
java.io.IOException: Cannot run program "/tmp/brut_util_Jar_9054478823788249311.tmp": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
at brut.util.OS.exec(OS.java:84)
... 4 more
Caused by:
java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
... 7 more
brut.androlib.BuildAndDecodeTest > classMethod FAILED
brut.androlib.AndrolibException: brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --version-code, 1, --version-name, 1.0, -F, /tmp/APKTOOL4080305473457824664.tmp, -0, arsc, -I, /home/ran/apktool/framework/1.apk, -S, /tmp/BRUT8741597800933526375.tmp/testapp-orig/res, -M, /tmp/BRUT8741597800933526375.tmp/testapp-orig/AndroidManifest.xml]
at brut.androlib.Androlib.buildResourcesFull(Androlib.java:459)
at brut.androlib.Androlib.buildResources(Androlib.java:397)
at brut.androlib.Androlib.build(Androlib.java:285)
at brut.androlib.BuildAndDecodeTest.beforeClass(BuildAndDecodeTest.java:47)
Caused by:
brut.androlib.AndrolibException: brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --version-code, 1, --version-name, 1.0, -F, /tmp/APKTOOL4080305473457824664.tmp, -0, arsc, -I, /home/ran/apktool/framework/1.apk, -S, /tmp/BRUT8741597800933526375.tmp/testapp-orig/res, -M, /tmp/BRUT8741597800933526375.tmp/testapp-orig/AndroidManifest.xml]
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:419)
at brut.androlib.Androlib.buildResourcesFull(Androlib.java:445)
... 3 more
Caused by:
brut.common.BrutException: could not exec command: [/tmp/brut_util_Jar_9054478823788249311.tmp, p, --forced-package-id, 127, --version-code, 1, --version-name, 1.0, -F, /tmp/APKTOOL4080305473457824664.tmp, -0, arsc, -I, /home/ran/apktool/framework/1.apk, -S, /tmp/BRUT8741597800933526375.tmp/testapp-orig/res, -M, /tmp/BRUT8741597800933526375.tmp/testapp-orig/AndroidManifest.xml]
at brut.util.OS.exec(OS.java:93)
at brut.androlib.res.AndrolibResources.aaptPackage(AndrolibResources.java:413)
... 4 more
Caused by:
java.io.IOException: Cannot run program "/tmp/brut_util_Jar_9054478823788249311.tmp": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:485)
at brut.util.OS.exec(OS.java:84)
... 5 more
Caused by:
java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
... 8 more
6 tests completed, 4 failed
:brut.apktool:apktool-lib:test FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':brut.apktool:apktool-lib:test'.
> There were failing tests. See the report at: file:///home/ran/Apktool/brut.apktool/apktool-lib/build/reports/tests/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
I tried it several times, but received the same errors.
What can I do to fix it?
(Somehow in Ubuntu 12.04 it works, but not on Ubuntu 14.04).
EDIT
The jar file ./brut.apktool/apktool-cli/build/libs/apktool-cli.jar was created, but the errors remain.
I found the problem: aapt was missing...,
To add aapt, I had to install Android SDK, then run Android SDK Manager (./android in android-sdk-linux/tools), and install Android SDK Platform-tools.
aapt is in android-sdk-linux/built-tools/22.0.1 directory.
I also had to install lib32z1 (with sudo apt-get install) and add the aapt directory to PATH env variable in order to make aapt working.
Let's say that I have a number of URL strings and I want to extract 'meaningful' information from. Namely, I want to know what is the page which that url specifies. So if the site is subaru.com, did this come from the about page, special deals page, etc.
[1] "http://www.subaru.com/vehicles/impreza/index.html"
[2] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=214495e6-dbe0-6668-9222-00003d7cd876&prid=87&k_affcode=76602"
[3] "http://www.subaru.com/index.html?s_kwcid=subaru models&k_clickid=3ec14630-aa7f-b968-c389-00003e9a93f9&prid=87&k_affcode=77236"
[4] "http://www.subaru.com/customer-support.html"
[5] "http://www.subaru.com/"
[6] "http://www.subaru.com/vehicles/forester/index.html"
[7] "http://www.subaru.com/auto-show/detroit-2014.html"
[8] "http://www.subaruofchampaigncounty.com/index.htm"
[9] "http://www.subaru.com/build-your-own/impreza.html?zip=92106"
[10] "http://www.subaru.com/mobile/index.html"
[11] "http://www.subaru.com/"
[12] "http://www.subaru.com/"
[13] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=2361a001-195a-29c8-7323-00003c593714&prid=87&k_affcode=76602"
[14] "http://www.subaru.ru/index"
[15] "http://www.subarugeorgetown.com/certified/subaru/2013-subaru-outback-georgetown-tx-1b523a570a0a00de63937097e2f3723d.htm"
[16] "http://www.subaru.com/"
[17] "http://www.subaru.com/?s_kwcid=suburau&k_clickid=41a2c6dc-c9fa-6ac8-9bf0-000044fe28d7&prid=87&k_affcode=2966&gclid=cprrlygp-rscfugs7aodbkiaaw"
[18] "http://www.subaru.com/mobile/index.html"
[19] "http://www.subaru.com/mobile/index.html"
[20] "http://www.subaru.com/enthusiasts/index.html"
[21] "http://www.subaru.ru/index"
[22] "http://www.subaru.ru/index"
[23] "http://www.subaru.com/mobile/index.html"
[24] "http://www.subaru.com/"
[25] "http://www.subaru.com/"
[26] "http://www.subaru.com/"
[27] "http://www.subaru.com/enthusiasts/index.html"
[28] "http://www.subaruofdayton.com/tcd/home/?tcdkwid=22194961&tcdcmpid=19148&tcdadid=6852747105&locale=en_us"
[29] "http://www.subaru.com/build-your-own/outback.html?sc_brochure=subaru.outback.2014-specifications"
[30] "http://www.subaruofatlanta.com/featured-vehicles/used.htm?reset=inventorylisting"
[31] "http://www.subaru.com/customer-support.html"
[32] "http://www.subarupacific.com/index.htm?cikw=+subaru&cimt=b&cipl=&cinetwork=search&ciagaid=49620691888&gclid=clhf0uoq-rscffpm7aodtv0aiw"
[33] "http://www.subaru.ru/index"
[34] "http://www.subaru.ru/lineup/forester/spec/spec"
[35] "http://www.subaru.com/build-your-own/forester.html?zip=37211"
[36] "http://www.subaru.com/mobile/index.html"
[37] "http://www.subaruelcajon.com/index.htm"
[38] "http://www.subaru.com/customer-support.html"
[39] "http://www.subaru.com/vehicles/brz/index.html?s_kwcid=brz&k_clickid=1ec224f1-18c6-a228-5afb-000047ecef67&prid=87&k_affcode=197257&gclid=cpik35-r-rscfrsffgodhk4ajg"
[40] "http://www.subaru.com/mobile/index.html"
[41] "http://www.subaru.com/mobile/index.html"
[42] "http://www.subaru.ru/index"
[43] "http://www.subaru.com/"
[44] "http://www.subaru.com/vehicles/xv-crosstrek/index.html"
[45] "http://www.subaru.com/customer-support.html"
[46] "http://www.subaru.com/mobile/index.html"
[47] "http://www.subaru.ru/index"
[48] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=293d9ff9-a1ad-8489-82d3-00001e3a514f&prid=87&k_affcode=76602"
[49] "http://www.subaruofkingsautomall.com/index.htm"
[50] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=5ed77da1-f786-55e9-02d1-000055d135fc&prid=87&k_affcode=76602"
[51] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=1645e9d9-05b5-1fe8-d2b1-00002a3ce9e8&prid=87&k_affcode=76602"
[52] "https://www.subaru.com/my-subaru/account.html"
[53] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=0c0e3142-706d-4cc8-830f-00001ba63c96&prid=87&k_affcode=76602"
[54] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=3a594c6a-4485-d2c9-aabf-000051bdfc1d&prid=87&k_affcode=76602"
[55] "http://www.subaru.com/"
[56] "http://www.subaru.com/customer-support.html"
[57] "http://www.subaru.com/build-your-own/index.html"
[58] "http://www.subaru.com/"
[59] "http://www.subaru.com/mobile/index.html"
[60] "http://www.subaru.com/vehicles/brz/photos-videos.html?site=370595&placement=96106620&ad=7514606&creative=0"
[61] "http://www.subaru.com/customer-support.html"
[62] "http://www.subaru.com/"
[63] "http://www.subaru.com/"
[64] "http://www.subaru.com/customer-support.html"
[65] "http://www.subaru.com/mobile/index.html"
[66] "http://www.subaru.com/mobile/index.html"
[67] "http://www.subaru.com/"
[68] "http://www.subaru.com/mobile/index.html"
[69] "http://www.subaru.com/build-your-own/impreza.html?zip=01504"
[70] "http://www.subaru.com/enthusiasts/badge-of-ownership/index.html"
[71] "http://www.subaru.com/"
[72] "http://www.subaru.com/mobile/index.html"
[73] "http://www.subaruofcolumbia.com/used-inventory/index.htm"
[74] "http://www.subaru.com/customer-support.html"
[75] "http://www.subaru.com/"
[76] "http://www.subaruofpuyallup.com/tcd/home/?tcdkwid=22163386&tcdcmpid=13971&tcdadid=35753423988&locale=en_us"
[77] "http://www.subaru.com/mobile/vehicles/forester/index.html"
[78] "http://www.subaru.com/mobile/index.html"
[79] "http://www.subaru.com/"
[80] "http://www.subaru.com/"
As you can see, there's no specific rule I can use to extract just one thing from the URL string because each is different. Furthermore, notice that some have an extension of .ru and not .com. Right now, I've put together the following code but I still want to extract the page (xv-crosstrek, customer-support, etc)
mydat$URL_One <- gsub(".*www\\.([[:alpha:]]+\\.com).*","\\1", mydat$URL)
mydat$URL_Two <- gsub(".*\\.com","", mydat$URL)
Can anyone help with this task?
I'm thinking that I may want to remove all incidents of /index in each URL string.
So for some examples.
before:
"http://www.subaru.com/vehicles/forester/index.html"
after:
forester
before:
http://www.subaruofcolumbia.com/used-inventory/index.htm
after:
used-inventory
before:
http://www.subaru.com/build-your-own/forester.html?zip=37211
after:
build-your-own
There's the httr package, which has the function parse_url. For example, you can do
<r> parse_url("http://www.subaru.com/vehicles/forester/index.html")
$scheme
[1] "http"
$hostname
[1] "www.subaru.com"
$port
NULL
$path
[1] "vehicles/forester/index.html"
$query
NULL
$params
NULL
$fragment
NULL
$username
NULL
$password
NULL
attr(,"class")
[1] "url"
Which perhaps gets you part of the way there.
R has two handy functions for this. basename returns the basename of a url, while dirname returns the directory name (or path). Taking urls to be your first ten urls, I think we can achieve the results you're looking for with the following.
> urls
# [1] "http://www.subaru.com/vehicles/impreza/index.html"
# [2] "http://www.subaru.com/index.html?s_kwcid=subaru&k_clickid=214495e6-dbe0-6668-9222-00003d7cd876&prid=87&k_affcode=76602"
# [3] "http://www.subaru.com/index.html?s_kwcid=subaru models&k_clickid=3ec14630-aa7f-b968-c389-00003e9a93f9&prid=87&k_affcode=77236"
# [4] "http://www.subaru.com/customer-support.html"
# [5] "http://www.subaru.com/"
# [6] "http://www.subaru.com/vehicles/forester/index.html"
# [7] "http://www.subaru.com/auto-show/detroit-2014.html"
# [8] "http://www.subaruofchampaigncounty.com/index.htm"
# [9] "http://www.subaru.com/build-your-own/impreza.html?zip=92106"
# [10] "http://www.subaru.com/mobile/index.html"
> ifelse(grepl('index|zip', basename(urls)),
gsub('^.*/', '', dirname(urls)),
gsub('\\.html', '', basename(urls)))
# [1] "impreza" "www.subaru.com"
# [3] "www.subaru.com" "customer-support"
# [5] "www.subaru.com" "forester"
# [7] "detroit-2014" "www.subaruofchampaigncounty.com"
# [9] "build-your-own" "mobile"