I am currently using GraphFrames to retrieve connected components from a graph.
My code is very simple as follows:
v = sqlContext.createDataFrame(node,["id","name"])
print v.take(15)
e = sqlContext.createDataFrame(edge,["src","dst"])
print e.take(15)
g = GraphFrame(v,e)
# NullPointerException comes from connectedComponents function
res = g.connectedComponents()
Below is the output of the code snippet, which seems alright to me too.
Print Vertices:
[Row(id=6, name=u'6'), Row(id=12, name=u'12'), Row(id=1, name=u'1'), Row(id=3, name=u'3'), Row(id=9, name=u'9'), Row(id=2, name=u'2'), Row(id=11, name=u'11'), Row(id=10, name=u'10'), Row(id=5, name=u'5'), Row(id=4, name=u'4')]
Print Edges:
[Row(src=2, dst=9), Row(src=2, dst=5), Row(src=2, dst=6), Row(src=9, dst=10), Row(src=11, dst=12), Row(src=4, dst=10), Row(src=1, dst=2), Row(src=1, dst=3), Row(src=1, dst=12)]
However, when g.connectedComponents() is executed, the program starts to give the following NullPointerException.
Would appreciate any suggestions on what's going wrong here!
ERROR LiveListenerBus: Listener JobProgressListener threw an exception
java.lang.NullPointerException at
org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onTaskEnd$1.apply(JobProgressListener.scala:361)
at
org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onTaskEnd$1.apply(JobProgressListener.scala:360)
at scala.collection.immutable.List.foreach(List.scala:318) at
scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
at
org.apache.spark.ui.jobs.JobProgressListener.onTaskEnd(JobProgressListener.scala:360)
at
org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:42)
at
org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
at
org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31)
at
org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:55)
at
org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:37)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(AsynchronousListenerBus.scala:80)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(AsynchronousListenerBus.scala:65)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) at
org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:64)
at
org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1183)
at
org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:63)
Related
I am trying to connect ADS1015 ADC with my Pico Pro Maker. Unfortunately, I got folowing error:
E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.cililing.harvbox.thingsapp, PID: 1855
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:448)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
Caused by: java.lang.reflect.InvocationTargetException
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
Caused by: com.google.android.things.pio.PioException: android.os.ServiceSpecificException: I2C2 is already in use (code 16)
at com.google.android.things.pio.I2cDeviceImpl.<init>(I2cDeviceImpl.java:43)
at com.google.android.things.pio.PeripheralManager.openI2cDevice(PeripheralManager.java:246)
at com.google.android.things.contrib.driver.adc.ads1xxx.Ads1xxx.<init>(Ads1xxx.java:200)
at com.google.android.things.contrib.driver.adc.ads1xxx.Ads1xxx.<init>(Ads1xxx.java:187)
at com.cililing.harvbox.thingsapp.thingscontroller.controllers.ADS1015ControllerImpl.<init>(ADS1015Controller.kt:12)
at com.cililing.harvbox.thingsapp.ui.MainActivity$i2CController$2.invoke(MainActivity.kt:40)
at com.cililing.harvbox.thingsapp.ui.MainActivity$i2CController$2.invoke(MainActivity.kt:33)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at com.cililing.harvbox.thingsapp.ui.MainActivity.getI2CController(Unknown Source:25)
at com.cililing.harvbox.thingsapp.ui.MainActivity.access$getI2CController$p(MainActivity.kt:33)
at com.cililing.harvbox.thingsapp.ui.MainActivity$i2cLogger$1.run(MainActivity.kt:47)
at android.os.Handler.handleCallback(Handler.java:790)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6494)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
Caused by: android.os.ServiceSpecificException: I2C2 is already in use (code 16)
at android.os.Parcel.readException(Parcel.java:2027)
at android.os.Parcel.readException(Parcel.java:1959)
at com.google.android.things.pio.IPeripheralManagerClient$Stub$Proxy.OpenI2cDevice(IPeripheralManagerClient.java:1243)
at com.google.android.things.pio.I2cDeviceImpl.<init>(I2cDeviceImpl.java:41)
at com.google.android.things.pio.PeripheralManager.openI2cDevice(PeripheralManager.java:246)
at com.google.android.things.contrib.driver.adc.ads1xxx.Ads1xxx.<init>(Ads1xxx.java:200)
at com.google.android.things.contrib.driver.adc.ads1xxx.Ads1xxx.<init>(Ads1xxx.java:187)
at com.cililing.harvbox.thingsapp.thingscontroller.controllers.ADS1015ControllerImpl.<init>(ADS1015Controller.kt:12)
at com.cililing.harvbox.thingsapp.ui.MainActivity$i2CController$2.invoke(MainActivity.kt:40)
at com.cililing.harvbox.thingsapp.ui.MainActivity$i2CController$2.invoke(MainActivity.kt:33)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at com.cililing.harvbox.thingsapp.ui.MainActivity.getI2CController(Unknown Source:25)
at com.cililing.harvbox.thingsapp.ui.MainActivity.access$getI2CController$p(MainActivity.kt:33)
at com.cililing.harvbox.thingsapp.ui.MainActivity$i2cLogger$1.run(MainActivity.kt:47)
at android.os.Handler.handleCallback(Handler.java:790)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6494)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
I am using following driver:
https://github.com/androidthings/contrib-drivers/tree/master/adc
What I've already done:
uninstall all apps
reinstall whole AndroidThings
I tired both, I2C1 and I2C2
try to read value on new app (only example from library).
I have no idea what can I do more. I am pretty sure that device is connected properly (scl-scl, sda-sda, addr-gnd, a0-other_analog_output) to main board and error is actually kind of strange, as I am sure that none other app uses those pins.
code responsible for connecting with peripheral:
class ADS1015ControllerImpl(val i2cName: String,
val range: Int) : ADS1015Controller {
private val instance: Ads1xxx = Ads1xxx(i2cName, Ads1xxx.Configuration.ADS1015)
init {
instance.inputRange = range
}
override fun read(channel: Int, logger: (Int) -> Unit) {
logger.invoke(
instance.readSingleEndedInput(channel)
)
}
override fun release() {
instance.close()
}
}
The error message I2C2 is already in use (code 16) means you are trying to open more than one connection to the same I2C bus without closing the previous instance. The code to open the I2C connection lives within the Ads1xxx constructor. You must either manage a single instance of this class in your code, or be certain that you call close() on the current Ads1xxx before initializing another one.
Based on the stack trace, it also looks like this device is being initialized by a lazy load when the value it first accessed, so it's possible that this constructor is not getting invoked at the point you expected, causing things to come out of order.
In fact, the problem was a badly soldered plug. :)
I am using spark-sql-2.4.1 version with Java 8.
I have raw_df with column "eventTs" which is timestamp in Long data type.
I am trying to convert it back to Timestamp as below :
Dataset<Row> modified_df = raw_df
.withColumn("eventTimeStamp", ( col("eventTs").divide(1000) ).cast(DataTypes.TimestampType))
When use ( col("eventTs").divide(1000) ).cast(DataTypes.TimestampType)
or ( col("eventTs").divide(lit(1000)) ).cast(DataTypes.TimestampType)
It is working fine but it is throwing below AssertionError
java.lang.AssertionError: assertion failed: unsafe symbol Unstable (child of package InterfaceStability) in runtime reflection universe
at scala.reflect.internal.Symbols$Symbol.<init>(Symbols.scala:184)
at scala.reflect.internal.Symbols$TypeSymbol.<init>(Symbols.scala:3009)
at scala.reflect.internal.Symbols$ClassSymbol.<init>(Symbols.scala:3201)
at scala.reflect.internal.Symbols$StubClassSymbol.<init>(Symbols.scala:3496)
at scala.reflect.internal.Symbols$Symbol.newStubSymbol(Symbols.scala:498)
at scala.reflect.internal.pickling.UnPickler$Scan.readExtSymbol$1(UnPickler.scala:258)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbol(UnPickler.scala:284)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolRef(UnPickler.scala:649)
at scala.reflect.internal.pickling.UnPickler$Scan.readType(UnPickler.scala:417)
at scala.reflect.internal.pickling.UnPickler$Scan$$anonfun$readTypeRef$1.apply(UnPickler.scala:658)
at scala.reflect.internal.pickling.UnPickler$Scan$$anonfun$readTypeRef$1.apply(UnPickler.scala:658)
at scala.reflect.internal.pickling.UnPickler$Scan.at(UnPickler.scala:179)
at scala.reflect.internal.pickling.UnPickler$Scan.readTypeRef(UnPickler.scala:658)
at scala.reflect.internal.pickling.UnPickler$Scan.readAnnotationInfo(UnPickler.scala:492)
at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolAnnotation(UnPickler.scala:515)
at scala.reflect.internal.pickling.UnPickler$Scan.run(UnPickler.scala:97)
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:38)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply$mcV$sp(SymbolLoaders.scala:28)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:189)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:189)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.info(SynchronizedSymbols.scala:189)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:286)
at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:341)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply$mcV$sp(SymbolLoaders.scala:74)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$LazyPackageType.complete(SymbolLoaders.scala:71)
at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:174)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:174)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.info(SynchronizedSymbols.scala:174)
at scala.reflect.internal.Types$TypeRef.thisInfo(Types.scala:2194)
at scala.reflect.internal.Types$TypeRef.baseClasses(Types.scala:2199)
at scala.reflect.internal.tpe.FindMembers$FindMemberBase.<init>(FindMembers.scala:17)
at scala.reflect.internal.tpe.FindMembers$FindMember.<init>(FindMembers.scala:219)
at scala.reflect.internal.Types$Type.scala$reflect$internal$Types$Type$$findMemberInternal$1(Types.scala:1014)
at scala.reflect.internal.Types$Type.findMember(Types.scala:1016)
at scala.reflect.internal.Types$Type.memberBasedOnName(Types.scala:631)
at scala.reflect.internal.Types$Type.member(Types.scala:600)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:45)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:77)
at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:161)
at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:22)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1$$typecreator48$1.apply(ScalaReflection.scala:735)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:232)
at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:232)
at org.apache.spark.sql.catalyst.ScalaReflection$class.localTypeOf(ScalaReflection.scala:921)
at org.apache.spark.sql.catalyst.ScalaReflection$.localTypeOf(ScalaReflection.scala:46)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:735)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:724)
at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:56)
at org.apache.spark.sql.catalyst.ScalaReflection$class.cleanUpReflectionObjects(ScalaReflection.scala:906)
at org.apache.spark.sql.catalyst.ScalaReflection$.cleanUpReflectionObjects(ScalaReflection.scala:46)
at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:723)
at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:720)
at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$1.apply(literals.scala:160)
at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$1.apply(literals.scala:159)
at scala.util.Try$.apply(Try.scala:192)
at org.apache.spark.sql.catalyst.expressions.Literal$.create(literals.scala:159)
at org.apache.spark.sql.functions$.typedLit(functions.scala:127)
at org.apache.spark.sql.functions$.lit(functions.scala:110)
at org.apache.spark.sql.Column.$div(Column.scala:747)
at org.apache.spark.sql.Column.divide(Column.scala:762)
at
What is wrong here ?
how to fix this?
Maybe you're missing spark-tags_2.11.jar from your classpath? This JAR defines InterfaceStability package.
I am facing an issue with the execution of following Groovy Script snippet.
GroovyShell sh = new GroovyShell();
sh.evaluate("\"abcd\".length() >= .34");
I am getting the following exceptions. The entire stack trace is mentioned below.
Exception in thread "main" org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 1: unexpected token: >= # line 1, column 17.
"abcd".length() >= .34d
If I change .34 to 0.34, it works. However, because of some limitation, I won't be able to change the script content.
Any help to overcome will be appreciated.
I am getting the following exceptions
Exception in thread "main" org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 1: unexpected token: >= # line 1, column 17.
"abcd".length() >= .34d
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:350)
at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:144)
at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:110)
at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:234)
at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:168)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:943)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:584)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:623)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:594)
at groovytest.Testtest.main(Testtest.java:18)
Your Groovy snippet is incorrect - Groovy does not support notation without leading zero in case of decimal numbers smaller than 1.0. If you try to compile following expression directly using groovyc:
"abcd".length() >= .34
compilation will fail with error like:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
test.groovy: 2: Unexpected input: '.' # line 2, column 20.
"abcd".length() >= .34
^
1 error
Java supports such notation, however Groovy from 2.x up to 3.0.0-alpha-3 version does not support it.
Solid solution
Fix the input Groovy code snippet to contain only a valid and compile-ready code. Any invalid Groovy statements or expressions will lead to failures and compilation errors.
Workaround: add leading zeros with replaceAll() method
The only way to compile such incorrect snippet is to replace all .\d+ (dots followed by at least one space and ended with a number) with 0.$1. Consider following example:
def snippet = "\"abcd\".length() >= .34; \"efgh\".length() >= .22; \"xyz\".length() >= 0.11;"
println snippet.replaceAll(' \\.(\\d+)', ' 0.$1')
It adds 0 to all decimal numbers where leading zero is missing. Running this example prints following output to the console:
"abcd".length() >= 0.34; "efgh".length() >= 0.22; "xyz".length() >= 0.11;
If you pass such modified snippet to GroovyShell.evaluate() method it will run with no errors.
Of course this is not a rock-solid solution and it is just a way to automatically fix some of the syntax errors introduced in the code snippet. There are some corner cases where this workaround may cause some side effects, you have to be aware of it.
I'm implementing some RL in PyTorch and had to write my own mse_loss function (which I found on Stackoverflow ;) ).
The loss function is:
def mse_loss(input_, target_):
return torch.sum(
(input_ - target_) * (input_ - target_)) / input_.data.nelement()
Now, in my training loop, the first input is something like:
tensor([-1.7610e+10]), tensor([-6.5097e+10])
With this input I'll get the error:
Unable to get repr for <class 'torch.Tensor'>
Computing a = (input_ - target_) works fine, while b = a * a respectively b = torch.pow(a, 2) will fail with the error metioned above.
Does anyone know a fix for this?
Thanks a lot!
Update:
I just tried using torch.nn.functional.mse_loss which will result in the same error..
I had the same error,when I use the below code
criterion = torch.nn.CrossEntropyLoss().cuda()
output=output.cuda()
target=target.cuda()
loss=criterion(output, target)
but I finally found my wrong：output is like tensor([[0.5746,0.4254]]) and target is like tensor([2]),the number 2 is out of indice of output
when I not use GPU,this error message is:
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed. at /opt/conda/conda-bld/pytorch-nightly_1547458468907/work/aten/src/THNN/generic/ClassNLLCriterion.c:93
Are you using a GPU ?
I had simillar problem (but I was using gather operations), and when I moved my tensors to CPU I could get a proper error message. I fixed the error, switched back to GPU and it was alright.
Maybe pytorch has trouble outputing the correct error when it comes from inside the GPU.
Clojure 1.5.1 and Core.async library version "0.1.267.0-0d7780-alpha" is used to run CPU intensive calculations. A bunch of functions wrapped by time out function thunk-timeout are sent to a channel as shown by below mock code.
(defn toSendToGo [args timeoutUnits]
(let [result (atom [false])
timeout? (atom false)]
(try
( thunk-timeout
(fn [] (reset! result (myFunction args))) timeoutUnits)
(catch java.util.concurrent.TimeoutException e (do (prn "!Time out after " timeoutUnits " seconds!!") (reset! timeout? true)) ))
(if #timeout? (do sth))
#result))
(let [c ( chan)]
(go (>! c (toSendToGo args timeoutUnits))))
On Linux server with large memory, the code runs fine without issue. On windows server with smaller memory, if a few cases in a row experienced time out, there'd be this strange exception that I don't quite understand. Why is this related to time out?
Exception in thread "my-async-dispatch-4" java.lang.IllegalStateException: Pop w
ithout matching push
at clojure.lang.Var.popThreadBindings(Var.java:364)
at clojure.core$pop_thread_bindings.invoke(core.clj:1737)
at regtest$fn__40$processRegtestFiles__41$fn__96$state_machine__3962__auto____97$fn__99.invoke(regtest.clj:158)
at regtest$fn__40$processRegtestFiles__41$fn__96$state_machine__3962__auto____97.invoke(regtest.clj:158)
at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:945)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:949)
at regtest$fn__40$processRegtestFiles__41$fn__96.invoke(regtest.clj:158)
at clojure.lang.AFn.run(AFn.java:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Exception in thread "my-async-dispatch-5" java.lang.IllegalStateException: Pop w
ithout matching push
at clojure.lang.Var.popThreadBindings(Var.java:364)
at clojure.core$pop_thread_bindings.invoke(core.clj:1737)
at regtest$fn__40$processRegtestFiles__41$fn__96$state_machine__3962__au
to____97$fn__99.invoke(regtest.clj:158)
at regtest$fn__40$processRegtestFiles__41$fn__96$state_machine__3962__au
to____97.invoke(regtest.clj:158)
at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macro
s.clj:945)
at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(i
oc_macros.clj:949)
at regtest$fn__40$processRegtestFiles__41$fn__96.invoke(regtest.clj:158)
at clojure.lang.AFn.run(AFn.java:24)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
[UPDATE]
The issue is gone after adding a catch expression after the catch expression for TimeoutException:
(try...
(catch java.util.concurrent.TimeoutException e (do (prn "!Time out after " timeoutUnits " seconds!!") (reset! timeout? true)) )
(catch Exception e (prn "Unexpected exception " e) ))
With the caught exception being
Unexpected exception #<ExecutionException java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: slingshot/Stone
I'm not sure this is your problem but (toSendToGo) returns nil via the result atom if an exception is caught.
That nil is then put to chan c. Core.async channels shouldn't (and normally can't) have nil as a value. Do you still get the error when resetting result in the catch to anything but 'nil' (try 'false' perhaps?)?
As an aside, clojure, being a lisp, tends to prefer lower-case-with-hyphens for names of most things. toSendToGo should probably be to-send-to-go instead. I don't know of any official style guide but here's a link to the most thorough one I know of: https://github.com/bbatsov/clojure-style-guide
(P.S. This should probably have been a comment but I don't have the reputation to make them yet.)