I'm having trouble implementing a struct to load data from a table. I understand that the error is telling me that the trait bounds are not satisfied to load the data but it wont tell specifically what field in the struct is problematic. I already check the docs and the types seems correct. What process would I take to identify the problematic field since the compiler wont be specific?
I have this table (generated by diesel):
diesel::table! {
product (id) {
id -> Varchar,
title -> Varchar,
visible -> Tinyint,
description -> Varchar,
section -> Varchar,
price -> Decimal,
discount -> Decimal,
isFeatured -> Tinyint,
availableQuantity -> Nullable<Integer>,
createdAt -> Datetime,
updatedAt -> Datetime,
category_id -> Varchar,
}
}
This is the struct to load data:
#[derive(Queryable, Clone, Identifiable, Associations, Debug)]
#[diesel(belongs_to(ProductCategory, foreign_key = category_id))]
#[diesel(table_name = product)]
pub struct Product {
pub id: String, //diesel::sql_types::Text / id -> Varchar
pub title: String, //diesel::sql_types::Text / title -> Varchar
pub visible: i8, //TinyInt / visible -> Tinyint
pub description: String, //diesel::sql_types::Text / description -> Varchar
pub section: String, //diesel::sql_types::Text / section -> Varchar
pub price: BigDecimal, //diesel::sql_types::Numeric / price -> Decimal
pub discount: BigDecimal, //diesel::sql_types::Numeric / discount -> Decimal
pub is_featured: i8, //TinyInt / isFeatured -> Tinyint
pub available_quantity: Option<i32>, //diesel::sql_types::Nullable<diesel::sql_types::Integer> / availableQuantity -> Nullable<Integer>
pub created_at: NaiveDateTime, //diesel::sql_types::Datetime / createdAt -> Datetime
pub updated_at: NaiveDateTime, //diesel::sql_types::Datetime / updatedAt -> Datetime
pub category_id: String, //diesel::sql_types::Text / categoryId -> Varchar
}
impl Product {
pub fn get_all(conn: &mut MysqlConnection) -> Result<Vec<Product>, DatabaseError> {
let products = dsl::product.load::<Product>(conn)?;
Ok(products)
}
}
Getting the following error:
error[E0277]: the trait bound `(diesel::sql_types::Text, diesel::sql_types::Text, TinyInt, diesel::sql_types::Text, diesel::sql_types::Text, diesel::sql_types::Numeric, diesel::sql_types::Numeric, TinyInt, diesel::sql_types::Nullable<diesel::sql_types::Integer>, diesel::sql_types::Datetime, diesel::sql_types::Datetime, diesel::sql_types::Text): load_dsl::private::CompatibleType<models::product::Product, _>` is not satisfied
--> src/models/product.rs:34:37
| 34 | let products = dsl::product.load::<Product>(conn)?;
| ^^^^ the trait `load_dsl::private::CompatibleType<models::product::Product, _>` is not implemented for `(diesel::sql_types::Text, diesel::sql_types::Text, TinyInt, diesel::sql_types::Text, diesel::sql_types::Text, diesel::sql_types::Numeric, diesel::sql_types::Numeric, TinyInt, diesel::sql_types::Nullable<diesel::sql_types::Integer>, diesel::sql_types::Datetime, diesel::sql_types::Datetime, diesel::sql_types::Text)`
|
= help: the following other types implement trait `load_dsl::private::CompatibleType<U, DB>`:
(ST0, ST1)
(ST0, ST1, ST2)
(ST0, ST1, ST2, ST3)
(ST0, ST1, ST2, ST3, ST4)
(ST0, ST1, ST2, ST3, ST4, ST5)
(ST0, ST1, ST2, ST3, ST4, ST5, ST6)
(ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7)
(ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7, ST8)
and 56 others
= note: required because of the requirements on the impl of `LoadQuery<'_, _, models::product::Product>` for `schema::product::table` note: required by a bound in `diesel::RunQueryDsl::load`
--> /home/david/.cargo/registry/src/github.com-1ecc6299db9ec823/diesel-2.0.2/src/query_dsl/mod.rs:1499:15
| 1499 | Self: LoadQuery<'query, Conn, U>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `diesel::RunQueryDsl::load`
error[E0277]: the trait bound `models::product::Product: FromSqlRow<_,
_>` is not satisfied
--> src/models/product.rs:34:37
| 34 | let products = dsl::product.load::<Product>(conn)?;
| ^^^^ the trait `FromSqlRow<_, _>` is not implemented for `models::product::Product`
|
= help: the following other types implement trait `FromSqlRow<ST, DB>`:
<(T1, T0) as FromSqlRow<(ST1, Untyped), __DB>>
<(T1, T2, T0) as FromSqlRow<(ST1, ST2, Untyped), __DB>>
<(T1, T2, T3, T0) as FromSqlRow<(ST1, ST2, ST3, Untyped), __DB>>
<(T1, T2, T3, T4, T0) as FromSqlRow<(ST1, ST2, ST3, ST4, Untyped), __DB>>
<(T1, T2, T3, T4, T5, T0) as FromSqlRow<(ST1, ST2, ST3, ST4, ST5, Untyped), __DB>>
<(T1, T2, T3, T4, T5, T6, T0) as FromSqlRow<(ST1, ST2, ST3, ST4, ST5, ST6, Untyped), __DB>>
<(T1, T2, T3, T4, T5, T6, T7, T0) as FromSqlRow<(ST1, ST2, ST3, ST4, ST5, ST6, ST7, Untyped), __DB>>
<(T1, T2, T3, T4, T5, T6, T7, T8, T0) as FromSqlRow<(ST1, ST2, ST3, ST4, ST5, ST6, ST7, ST8, Untyped), __DB>>
and 55 others
= note: required because of the requirements on the impl of `LoadQuery<'_, _, models::product::Product>` for `schema::product::table` note: required by a bound in `diesel::RunQueryDsl::load`
--> /home/david/.cargo/registry/src/github.com-1ecc6299db9ec823/diesel-2.0.2/src/query_dsl/mod.rs:1499:15
| 1499 | Self: LoadQuery<'query, Conn, U>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `diesel::RunQueryDsl::load`
For more information about this error, try `rustc --explain E0277`.
Related
I'm trying to make a function to do the following:
accept a closure f of the form fn(T) -> T
return a closure of the form fn(T, bool) -> T, that conditionally performs f depending on the bool parameter.
I come from a haskell-ish background, and in Haskell this would be something like this:
conditionally :: (a -> a) -> a -> Bool -> a
conditionally f x True = f x
conditionally f x False = x
Translating that to something more rust-like:
conditionally :: ((t) -> t) -> ((t, Bool) -> t)
conditionally f = \(x, b) -> if b then (f x) else (x)
I tried the following in rust:
fn conditionally<T>(f: &'static (dyn Fn(T) -> T + 'static)) -> Box<dyn Fn(T, bool) -> T> {
Box::new(&|x, b| if b { f(x) } else { x } )
}
and was helpfully told to use the move keyword to ensure the closure takes ownership of f. However, the following still doesn't work:
fn conditional<T>(f: &'static (dyn Fn(T) -> T + 'static)) -> Box<dyn Fn(T, bool) -> T> {
Box::new(&move|x, b| if b { f(x) } else { x } )
}
I'm getting the following error (this also appeared prior to adding move):
error[E0515]: cannot return value referencing temporary value
--> src/main.rs:216:5
|
216 | Box::new(&move|x, b| if b { f(x) } else { x } )
| ^^^^^^^^^^-----------------------------------^^
| | |
| | temporary value created here
| returns a value referencing data owned by the current function
I'm thinking that the 'data owned by the current function' is either the closure I've defined, or the f that I've moved, but I can't get my head around how it all fits together.
As a smoke check, I made sure I'm able to box simpler closures that I define in the function body, and the following compiles:
fn conditional_increment() -> Box<dyn Fn(i32, bool) -> i32> {
Box::new(&|x, b| if b { x + 1 } else { x } )
}
What am I missing here? Is this possible in rust?
I'm also wondering if there's a more specific name for what I'm trying to do than simply higher-order functions, as I'm struggling to find resources on this kind of problem.
Update: I've realised that "currying in rust" would have been a good term to search for. While this isn't an example of currying, it would use the same language features, and would have led me to the answer given by vallentin.
You're attempting to return a boxed reference to a closure defined within the conditional function. Which you can't do, given that the closure only lives for the duration of the call. Instead you can return the closure itself, and in short just have to remove the &, i.e. turn &move |x, b| ... into move |x, b| ....
fn conditional<T>(f: &'static (dyn Fn(T) -> T + 'static)) -> Box<dyn Fn(T, bool) -> T> {
Box::new(move |x, b| if b { f(x) } else { x })
}
However, the more idiomatic way to write what you're attempting, which is to use generics and a type parameter for the closure. Check out:
Rust By Example - (Closures) As input parameters
Rust By Example - (Closures) As output parameters.
In short, you could rewrite it into this:
fn conditional<F, T>(f: F) -> Box<dyn Fn(T, bool) -> T>
where
F: Fn(T) -> T + 'static,
T: 'static,
{
Box::new(move |x, b| if b { f(x) } else { x })
}
You can actually also do without the box, and use the impl Trait syntax.
fn conditional<F, T>(f: F) -> impl Fn(T, bool) -> T
where
F: Fn(T) -> T,
{
move |x, b| if b { f(x) } else { x }
}
You can also use the impl Trait syntax for parameters, which the links shows, but personally I find it to noisy when dealing with closuring.
Using it the boils down to something as simple as this:
let f = conditional(move |x| x * 2);
println!("{}", f(2, false)); // Prints `2`
println!("{}", f(2, true)); // Prints `4`
Recently, I tried to write a piece of code similar to the following:
pub struct Foo<'a, F> /* where F: Fn(&u32) -> bool */ {
u: &'a u32,
f: F
}
impl<'a, F> Foo<'a, F>
where F: Fn(&u32) -> bool
{
pub fn new_foo<G: 'static>(&self, g: G) -> Foo<impl Fn(&u32) -> bool + '_>
where G: Fn(&u32) -> bool
{
Foo { u: self.u, f: move |x| (self.f)(x) && g(x) }
}
}
Here, an instance of Foo represents a condition on a piece of data (the u32), where a more restrictive Foo can be built from a less restrictive one via new_foo, without consuming the old. However, the above code does not compile as written, but gives the rather cryptic error message:
error[E0308]: mismatched types
--> src/lib.rs:9:52
|
9 | pub fn new_foo<G: 'static>(&self, g: G) -> Foo<impl Fn(&u32) -> bool + '_>
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected type `std::ops::FnOnce<(&u32,)>`
found type `std::ops::FnOnce<(&u32,)>`
error: higher-ranked subtype error
--> src/lib.rs:9:5
|
9 | / pub fn new_foo<G: 'static>(&self, g: G) -> Foo<impl Fn(&u32) -> bool + '_>
10 | | where G: Fn(&u32) -> bool
11 | | {
12 | | Foo { u: self.u, f: move |x| (self.f)(x) && g(x) }
13 | | }
| |_____^
error: aborting due to 2 previous errors
After much experimentation, I did find a way to make the code compile, and I believe it then functions as intended. I am used to the convention of placing bounds on impls rather than declarations when the declaration can be written without relying on those bounds, but for some reason uncommenting the where clause above, that is, copying the bound F: Fn(&u32) -> bool from the impl to the declaration of Foo itself resolved the problem. However, I don't have a clue why this makes a difference (nor do I really understand the error message in the first place). Does anyone have an explanation of what's going on here?
The only subtypes that exist in Rust are lifetimes, so your errors (cryptically) hint that there's some sort of lifetime problem at play. Furthermore, the error clearly points at the signature of your closure, which involves two lifetimes:
the lifetime of the closure itself, which you have explicitly stated outlives the anonymous lifetime '_; and
the lifetime of its argument &u32, which you have not explicitly stated, so a higher-ranked lifetime is inferred as if you had stated the following:
pub fn new_foo<G: 'static>(&self, g: G) -> Foo<impl for<'b> Fn(&'b u32) -> bool + '_>
where G: Fn(&u32) -> bool
Using the more explicit signature above gives a (very) slightly more helpful error:
error[E0308]: mismatched types
--> src/lib.rs:9:52
|
9 | pub fn new_foo<G: 'static>(&self, g: G) -> Foo<impl for<'b> Fn(&'b u32) -> bool + '_>
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ one type is more general than the other
|
= note: expected type `std::ops::FnOnce<(&'b u32,)>`
found type `std::ops::FnOnce<(&u32,)>`
At least we can now see that "one type is more general than the other": we expected a closure that can take an argument with any lifetime but for some reason Rust thinks that what we have instead is a closure that takes an argument that may have some more restricted range of lifetimes.
What's happened? Well, the function's return value is the following expression:
Foo { u: self.u, f: move |x| (self.f)(x) && g(x) }
This is of course an instance of struct Foo<'a, F>, where this F bears no relation to that declared on the impl block (with its trait bound). Indeed, since there's no explicit bound on F in the struct definition, the compiler must fully infer this type F from the expression itself. By giving the struct definition a trait bound, you are telling the compiler that instances of Foo, including the above expression, have an F that implements for<'b> Fn(&'b u32) -> bool: i.e. the range of lifetimes for the &u32 argument are unbounded.
Okay, so the compiler needs to infer F instead, and indeed it does infer that it implements Fn(&u32) -> bool. However, it's just not quite smart enough to determine to what range of lifetimes that &u32 argument might be restricted. Adding an explicit type annotation, as suggested in #rodrigo's comment above, states that the argument can indeed have any lifetime.
If there are in fact some restrictions on the possible lifetimes of the closure's argument, you would need to indicate that more explicitly by changing the definition of 'b from a higher-ranked trait bound (i.e. for<'b> in the return type above) to whatever is appropriate for your situation.
Hopefully once chalk is fully integrated into the compiler it will be able to perform this inference in both the unrestricted and restricted cases. In the meantime, the compiler is erring on the side of caution and not making potentially erroneous assumptions. The errors could definitely have been a bit more helpful, though!
My program has a bunch of functions that operate on generic integer. They are usually of the following form:
use num::{FromPrimitive, Integer, ToPrimitive};
use std::cmp::Ord;
use std::ops::{Add, Mul};
fn function<'a, I>(n: &'a I) -> I
where
I: Integer + Clone + FromPrimitive + ToPrimitive,
for<'b> &'b I: Mul<Output = I> + Add<Output = I> + Ord,
{
}
I want to alias the generic type requirements:
I: Integer + Clone + FromPrimitive + ToPrimitive,
for<'b> &'b I: Mul<Output = I> + Add<Output = I> + Ord,
so that I won't need to rewrite them every time. Initially, I thought macros would help but it looks like they don't work like in C so I looked for another way.
I found a way to do it for the first requirement. One has to apply the default implementation on the defined trait over any type T.
trait GInteger: Integer + Clone + FromPrimitive + ToPrimitive {}
impl<T: Integer + Clone + FromPrimitive + ToPrimitive> GInteger for T {}
Then I can simply write:
I: GInteger
instead of
I: Integer + Clone + FromPrimitive + ToPrimitive,
How can I alias the second requirement? Is it possible?
for<'b> &'b I: Mul<Output = I> + Add<Output = I> + Ord,
No it's not possible to use a new trait for this.
While it's possible to include the second requirement into the trait definition...
trait GInteger: Integer + Clone + FromPrimitive + ToPrimitive
where
for<'b> &'b Self: Mul<Output = Self> + Add<Output = Self> + Ord,
{
}
rustc will not elaborate the where clause for you, so in the declaration of function() you still need to write the where for<'b> &'b I: ... bound. It is a known bug.
fn function<I: GInteger>(n: &I) -> I
where
for<'b> &'b I: Mul<Output = I> + Add<Output = I> + Ord, // meh
{
n * n
}
If you're using nightly Rust, you could use trait alias (RFC 1733) instead, which exactly solves this problem.
#![feature(trait_alias)]
use num::{FromPrimitive, Integer, ToPrimitive};
use std::cmp::Ord;
use std::ops::{Add, Mul};
// Define a trait alias
trait GInteger = Integer + Clone + FromPrimitive + ToPrimitive
where
for<'b> &'b Self: Mul<Output = Self> + Add<Output = Self> + Ord;
// Just use it
fn function<I: GInteger>(n: &I) -> I {
n * n
}
I can't implement this LISP construction
(defun foo (n)
(lambda (i) (incf n i)))
in Rust.
I have tried this:
use std::ops::Add;
fn f<T: Add>(n: T) -> Box<Fn(T) -> T> {
Box::new(move |i: T| n + i)
}
fn main() {
let adder = f(2);
assert_eq!(4, adder(2));
}
But it causes an error:
error: mismatched types:
expected `T`,
found `<T as core::ops::Add>::Output`
(expected type parameter,
found associated type) [E0308]
Box::new(move |i: T| n + i)
^~~~~
Seems like the trait Add defined for the outer function was not transferred into the inner closure.
Is it possible to implement such a construction?
It is possible to implement this function with a concrete type instead of a generic:
fn f(n: i32) -> Box<Fn(i32) -> i32> {
Box::new(move |i| n + i)
}
There are several problems with the generic version.
First, the error you have provided occurs because just T: Add is not enough to specify output type: you need to put a constraint onto the associated type <T as Add>::Output as well (see Add docs):
fn f<T: Add<Output=T>>(n: T) -> Box<Fn(T) -> T> {
Box::new(move |i: T| n + i)
}
Alternatively, you can make the closure return the output type of <T as Add>:
fn f<T: Add>(n: T) -> Box<Fn(T) -> T::Output> {
Box::new(move |i: T| n + i)
}
However, now you'll get the following error:
<anon>:4:10: 4:37 error: the parameter type `T` may not live long enough [E0310]
<anon>:4 Box::new(move |i: T| n + i)
^~~~~~~~~~~~~~~~~~~~~~~~~~~
<anon>:4:10: 4:37 help: see the detailed explanation for E0310
<anon>:4:10: 4:37 help: consider adding an explicit lifetime bound `T: 'static`...
<anon>:4:10: 4:37 note: ...so that the type `[closure#<anon>:4:19: 4:36 n:T]` will meet its required lifetime bounds
<anon>:4 Box::new(move |i: T| n + i)
^~~~~~~~~~~~~~~~~~~~~~~~~~~
The problem here is that T may contain references inside it (afterwards, it is a generic type - it can contain anything); however, Box<Fn(T) -> T> implicitly means that anything inside this trait object must be 'static, i.e. the compiler automatically adds 'static constraint: Box<Fn(T) -> T + 'static>. However, your closure captures T which can contain any references, not only 'static.
The most general way to fix it would be to add an explicit lifetime constraint of T and Box<Fn(T) -> T>:
fn f<'a, T: Add<Output=T> + 'a>(n: T) -> Box<Fn(T) -> T + 'a> {
Box::new(move |i: T| n + i)
}
Alternatively, you can specify that T is 'static, though this unnecessarily limits the genericity of your code:
fn f<T: Add<Output=T> + 'static>(n: T) -> Box<Fn(T) -> T> {
Box::new(move |i: T| n + i)
}
However, this still does not compile:
<anon>:4:31: 4:32 error: cannot move out of captured outer variable in an `Fn` closure
<anon>:4 Box::new(move |i: T| n + i)
^
This error happens because addition in Rust (i.e. Add trait) works by value - it consumes both arguments. For Copy types, like numbers, it is fine - they are always copied. However, the compiler can't assume that generic type parameters also designate Copy types because there is no respective bound, therefore it assumes that values of type T can only be moved around. However, you're specifying that the returned closure is Fn, so it takes its environment by reference. You can't move out of a reference, and this is what this error is about.
There are several ways to fix this error, the simplest one being to add Copy bound:
fn f<'a, T: Add<Output=T> + Copy + 'a>(n: T) -> Box<Fn(T) -> T + 'a> {
Box::new(move |i: T| n + i)
}
And now it compiles.
One possible alternative would be to return FnOnce closure which takes its environment by value:
fn f<'a, T: Add<Output=T> + 'a>(n: T) -> Box<FnOnce(T) -> T + 'a> {
Box::new(move |i: T| n + i)
}
There are two problems with it, however. First, as its name implies, FnOnce can only be called once, because upon its first invocation its environment is consumed, and there is nothing to call it on the next time. This may be overly limiting. Second, unfortunately, Rust cannot call Box<FnOnce()> closures at all. This is an implementation problem and it should be solved in the future; for now there is an unstable FnBox trait to work around this.
Even another alternative would be to use references instead of values:
fn f<'a, T: 'a>(n: T) -> Box<Fn(T) -> T + 'a> where for<'b> &'b T: Add<T, Output=T> {
Box::new(move |i: T| &n + i)
}
Now we specify that instead of T, &'b T for any lifetime 'b must be summable with T. Here we use the fact that Add trait is overloaded for references to primitive types as well. This is probably the most generic version of this function.
If you are OK with moving away from function calling, you could wrap the x in your own type:
use std::ops::Add;
struct Adder<X> {
x: X
}
impl<X: Copy> Adder<X> {
fn add<Y: Add<X>>(&self, y: Y) -> <Y as Add<X>>::Output {
y + self.x
}
}
fn main() {
let x = Adder { x: 1usize };
x.add(2); // as opposed to x(2)
}
This means you can get rid of Box and don't need to allocate anything. Implementing Fn(..) is not possible in stable Rust, and the unstable version may break in future Rust versions. Look at std::ops::Fn for further information.
I'm trying to use the zipPartitions function defined in Spark's RDD class (url to Spark Scala Docs here: http://spark.apache.org/docs/0.9.1/api/core/index.html#org.apache.spark.rdd.RDD).
The function is overloaded, and contains several implementations.
def zipPartitions[B, C, D, V](rdd2: RDD[B], rdd3: RDD[C], rdd4: RDD[D])(f: (Iterator[T], Iterator[B], Iterator[C], Iterator[D]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[D], arg3: ClassTag[V]): RDD[V]
def zipPartitions[B, C, D, V](rdd2: RDD[B], rdd3: RDD[C], rdd4: RDD[D], preservesPartitioning: Boolean)(f: (Iterator[T], Iterator[B], Iterator[C], Iterator[D]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[D], arg3: ClassTag[V]): RDD[V]
def zipPartitions[B, C, V](rdd2: RDD[B], rdd3: RDD[C])(f: (Iterator[T], Iterator[B], Iterator[C]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[V]): RDD[V]
def zipPartitions[B, C, V](rdd2: RDD[B], rdd3: RDD[C], preservesPartitioning: Boolean)(f: (Iterator[T], Iterator[B], Iterator[C]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[V]): RDD[V]
def zipPartitions[B, V](rdd2: RDD[B])(f: (Iterator[T], Iterator[B]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[V]): RDD[V]
def zipPartitions[B, V](rdd2: RDD[B], preservesPartitioning: Boolean)(f: (Iterator[T], Iterator[B]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[V]): RDD[V]
I defined a function, merge, with type signature:
merge(iter1: Iterator[(Int,Int)], iter2: Iterator[(Int,Int)]): Iterator[(Int,Int)]
and have two RDD's of type [Int].
However, when I do Rdd1.zipPartitions(Rdd2,merge), the spark shell throws an error and says:
error: missing arguments for method merge;
follow this method with `_' if you want to treat it as a partially applied function
This is strange, because elsewhere, I am able to pass a function as an argument into another method fine. However, if I add two _'s to merge, and try
Rdd1.zipPartitions(Rdd2,merge(_:Iterator[(Int,Int)], _: Iterator[(Int,Int)]), then I get a different error:
error: overloaded method value zipPartitions with alternatives:
[B, C, D, V](rdd2: org.apache.spark.rdd.RDD[B], rdd3: org.apache.spark.rdd.RDD[C], rdd4: org.apache.spark.rdd.RDD[D])(f: (Iterator[(Int, Int)], Iterator[B], Iterator[C], Iterator[D]) => Iterator[V])(implicit evidence$34: scala.reflect.ClassTag[B], implicit evidence$35: scala.reflect.ClassTag[C], implicit evidence$36: scala.reflect.ClassTag[D], implicit evidence$37: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V] <and>
[B, C, D, V](rdd2: org.apache.spark.rdd.RDD[B], rdd3: org.apache.spark.rdd.RDD[C], rdd4: org.apache.spark.rdd.RDD[D], preservesPartitioning: Boolean)(f: (Iterator[(Int, Int)], Iterator[B], Iterator[C], Iterator[D]) => Iterator[V])(implicit evidence$30: scala.reflect.ClassTag[B], implicit evidence$31: scala.reflect.ClassTag[C], implicit evidence$32: scala.reflect.ClassTag[D], implicit evidence$33: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V] <and>
[B, C, V](rdd2: org.apache.spark.rdd.RDD[B], rdd3: org.apache.spark.rdd.RDD[C])(f: (Iterator[(Int, Int)], Iterator[B], Iterator[C]) => Iterator[V])(implicit evidence$27: scala.reflect.ClassTag[B], implicit evidence$28: scala.reflect.ClassTag[C], implicit evidence$29: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V] <and>
[B, C, V](rdd2: org.apache.spark.rdd.RDD[B], rdd3: org.apache.spark.rdd.RDD[C], preservesPartitioning: Boolean)(f: (Iterator[(Int, Int)], Iterator[B], Iterator[C]) => Iterator[V])(implicit evidence$24: scala.reflect.ClassTag[B], implicit evidence$25: scala.reflect.ClassTag[C], implicit evidence$26: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V] <and>
[B, V](rdd2: org.apache.spark.rdd.RDD[B])(f: (Iterator[(Int, Int)], Iterator[B]) => Iterator[V])(implicit evidence$22: scala.reflect.ClassTag[B], implicit evidence$23: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V] <and>
[B, V](rdd2: org.apache.spark.rdd.RDD[B], preservesPartitioning: Boolean)(f: (Iterator[(Int, Int)], Iterator[B]) => Iterator[V])(implicit evidence$20: scala.reflect.ClassTag[B], implicit evidence$21: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V]
cannot be applied to (org.apache.spark.rdd.RDD[(Int, Int)], (Iterator[(Int, Int)], Iterator[(Int, Int)]) => Iterator[(Int, Int)])
val RDD_combined = RDD1.zipPartitions(RDD1:org.apache.spark.rdd.RDD[(Int, Int)],merge(_:Iterator[(Int,Int)],_:Iterator[(Int,Int)]))
I suspect the error lies in this bottom line:
The function definition that I'm trying to match with this call is:
[B, V](rdd2: org.apache.spark.rdd.RDD[B])(f: (Iterator[(Int, Int)], Iterator[B]) => Iterator[V])(implicit evidence$22: scala.reflect.ClassTag[B], implicit evidence$23: scala.reflect.ClassTag[V])org.apache.spark.rdd.RDD[V]
however, what scala sees is
val RDD_combined = RDD1.zipPartitions(RDD1:org.apache.spark.rdd.RDD[(Int, Int)],merge(_:Iterator[(Int,Int)],_:Iterator[(Int,Int)]))
where the [B] type parameter has already been converted to [(Int,Int)].
Any insights into how to get this to work would be very appreciated!
If you look at the signature, you'll see that this is actually a function with multiple parameter lists, not one list with multiple parameters. The invocation you need is more like:
RDD1.zipPartitions(RDD1)(merge)
(Not sure about the type references you added in your original?)
There may still be some other adjustments you need to make this work, but that is the essence of fixing the error you currently see.