Spark: load csv file with a different schema - apache-spark

Question

I have a csv file like this :
product price,product origin,phone number
20,US,200200
I would like to load the csv file using a new schema so that my dataset should look like this:
|price | origin | number |
|20 | US | 200200 |
I tried to create a schema using structfield :
sparkSession.read().format("csv")
.option("header", "false")
.option("delimiter", ",")
.schema(myScheme).load(csv)
but what I got is like this:
|price | origin | number |
|200200 | US | 20 |
What is the correct way to load the csv with a new scheme with correct column orders ?

Answer 1

Using a csv file with the exact contents that you posted in your question:
product price,product origin,phone number
20,US,200200
You should be able to create a schema by using types from org.apache.spark.sql.types._. You could do something like this:
import org.apache.spark.sql.types._
val mySchema = new StructType()
.add("product price", IntegerType)
.add("product origin", StringType)
.add("phone number", StringType)
val df = spark
.read
.option("header", "true")
.schema(mySchema)
.csv("./simpleCSV.csv")
df.show
+-------------+--------------+------------+
|product price|product origin|phone number|
+-------------+--------------+------------+
| 20| US| 200200|
+-------------+--------------+------------+
df.printSchema
root
|-- product price: integer (nullable = true)
|-- product origin: string (nullable = true)
|-- phone number: string (nullable = true)
Hope this helps!