I receive the data from sensors and save it in an .csv file. but the formatt that I have is like 0.234.234.234.123.5434
What I Need is the Format like this 0,23452342234234234
private fun formatRowData(x: Float,
y: Float,
z: Float,
length: Double,
angle: Double): String {
return String.format(
"${SimpleDateFormat("H:m:s:SSS")
.format(today.time)};$x;${y};${z};${length};${angle}", Locale.GERMANY
)
}
Try this :
var value:String ="0.234.234.234.123.5434"
value = value.replaceFirst(".", ",")
value = value.replace(".","")
println(" : "+value)
Output :
: 0,2342342341235434
Related
I would like to write a yeardiff function that works similarly to datediff. yeardiff should take two Column arguments and return a Column with the number of years between those parameter Columns.
Let's use the following sample data:
val testDf = Seq(
("2016-09-10", "2001-08-10"),
("2016-04-18", "2010-05-18"),
("2016-01-10", "2013-08-10")
)
.toDF("first_datetime", "second_datetime")
.withColumn("first_datetime", $"first_datetime".cast("timestamp"))
.withColumn("second_datetime", $"second_datetime".cast("timestamp"))
We can run this to get the date difference:
testDf.withColumn("num_days", datediff(col("first_datetime"), col("second_datetime")))
I want to be able to run this:
testDf.withColumn("num_years", yeardiff(col("first_datetime"), col("second_datetime")))
I tried to define a yeardiff function with the necessary method signature and didn't get anywhere:
def yeardiff(end: Column, start: Column): Column = {
// what do I do here
}
Here is a hacked transformation solution that I came up with and don't like:
def yearDiff(end: String, start: String)(df: DataFrame): DataFrame = {
val c = s"${end}_${start}_datediff"
df
.withColumn(c, datediff(col(end), col(start)))
.withColumn("yeardiff", col(c) / 365)
}
EDIT
I started digging into the Spark source code to see how datediff works. Here is the datediff function definition:
def datediff(end: Column, start: Column): Column = withExpr { DateDiff(end.expr, start.expr) }
Here is the DateDiff case class:
case class DateDiff(endDate: Expression, startDate: Expression)
extends BinaryExpression with ImplicitCastInputTypes {
override def left: Expression = endDate
override def right: Expression = startDate
override def inputTypes: Seq[AbstractDataType] = Seq(DateType, DateType)
override def dataType: DataType = IntegerType
override def nullSafeEval(end: Any, start: Any): Any = {
end.asInstanceOf[Int] - start.asInstanceOf[Int]
}
override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
defineCodeGen(ctx, ev, (end, start) => s"$end - $start")
}
}
This may solve your problem:
def yearDiff(end: Column, start: Column): Column = {
datediff(end, start)/365
}
We can use the built in year function and a udf to adjust for cases where the month has not past.
def yeardiff(end: Column, start: Column): Column = {
def getAdjustment(monthStart : Int, monthEnd : Int, dayStart : Int, dayEnd : Int) : Int = {
if (monthEnd>monthStart) return -1
if (monthStart==monthEnd && dayEnd > dayStart) return -1
else return 0
}
val udfGetAdjustment = udf[Int,Int,Int,Int,Int](getAdjustment)
val adj = udfGetAdjustment(month(start),month(end),dayofmonth(start),dayofmonth(end))
year(end) - year(start) + adj
}
I have streaming data coming as follows
id, date, value
i1, 12-01-2016, 10
i2, 12-02-2016, 20
i1, 12-01-2016, 30
i2, 12-05-2016, 40
Want to reduce by id to get aggregate value info by date like
output required from rdd is for a given id and list(days 365)
I have to put the value in the list position based on day of year like 12-01-2016 is 336 and as there are two instances for device i1 with same date they should be aggregated
id, List [0|1|2|3|... |336| 337| |340| |365]
i1, |10+30| - this goes to 336 position
i2, 20 40 -- this goes to 337 and 340 position
Please guide the reduce or group by transformation to do this.
I'll provide you the basic code snippet with few assumptions as you haven't specified about the language, data source or data format.
JavaDStream<String> lineStream = //Your data source for stream
JavaPairDStream<String, Long> firstReduce = lineStream.mapToPair(line -> {
String[] fields = line.split(",");
String idDate = fields[0] + fields[1];
Long value = Long.valueOf(fields[2]);
return new Tuple2<String, Long>(idDate, value);
}).reduceByKey((v1, v2) -> {
return (v1+v2);
});
firstReduce.map(idDateValueTuple -> {
String idDate = idDateValueTuple._1();
Long valueSum = idDateValueTuple._2();
String id = idDate.split(",")[0];
String date = idDate.split(",")[];
//TODO parse date and put the sumValue in array as you wish
}
Can only reach this far. Am not sure how to add each element of an array in the final step. Hope this helps!!!If you get the last step or any alternate way,appreciate if you post it here!!
def getDateDifference(dateStr:String):Int = {
val startDate = "01-01-2016"
val formatter = DateTimeFormatter.ofPattern("MM-dd-yyyy")
val oldDate = LocalDate.parse(startDate, formatter)
val currentDate = dateStr
val newDate = LocalDate.parse(currentDate, formatter)
return newDate.toEpochDay().toInt - oldDate.toEpochDay().toInt
}
def getArray(numberofDays:Int,data:Int):Iterable[Int] = {
val daysArray = new Array[Int](366)
daysArray(numberofDays) = data
return daysArray
}
val idRDD = <read from stream>
val idRDDMap = idRDD.map { rec => ((rec.split(",")(0),rec.split(",")(1)),
(getDateDifference(rec.split(",")(1)),rec.split(",")(2).toInt))}
val idRDDconsiceMap = idRDDMap.map { rec => (rec._1._1,getArray(rec._2._1, rec._2._2)) }
val finalRDD = idRDDconsiceMap.reduceByKey((acc,value)=>(???add each element of the arrays????))
I'm having trouble converting a String to Int in my Swift OS X Xcode project. I have some data saved in a text file in a comma delimited format. The contents of the text file is below:
1,Cessna 172,3,54.4,124,38.6112
(and a line break at the end)
I read the text file and seperate it, first by \n to get each line by itself, and then by , to get each element by itself. The code to do this is below:
if let dir : NSString = NSSearchPathForDirectoriesInDomains(NSSearchPathDirectory.DocumentDirectory, NSSearchPathDomainMask.AllDomainsMask, true).first {
let path = dir.stringByAppendingPathComponent("FSPassengers/aircraft.txt")
do {
let content = try NSString(contentsOfFile: path, encoding: NSUTF8StringEncoding)
if content != "" {
let astrContent:[String] = content.componentsSeparatedByString("\n")
for aeroplane in astrContent {
let aSeperated:[String] = aeroplane.componentsSeparatedByString(",")
print(aSeperated[0])
print(Int(aSeperated[0]))
//self.aAircraft.append(Aircraft(id: aSeperated[0], type: aSeperated[1], passengerCapacity: Int(aSeperated[2])!, cargoCapacityKg: Double(aSeperated[3])!, cruiseSpeed: Int(aSeperated[4])!, fuelLitresPerHour: Double(aSeperated[5])!))
}
}
}
catch {
print("Error")
}
}
The end result here will be to assign each record (each line of the text file) into the array aAircraft. This array is made up of a custom object called Aircraft. The custom class is below:
class Aircraft: NSObject {
var id:Int = Int()
var type:String = String()
var passengerCapacity:Int = Int()
var cargoCapacityKg:Double = Double()
var cruiseSpeed:Int = Int()
var fuelLitresPerHour:Double = Double()
override init() {}
init(id:Int, type:String, passengerCapacity:Int, cargoCapacityKg:Double, cruiseSpeed:Int, fuelLitresPerHour:Double) {
self.id = id
self.type = type
self.passengerCapacity = passengerCapacity
self.cargoCapacityKg = cargoCapacityKg
self.cruiseSpeed = cruiseSpeed
self.fuelLitresPerHour = fuelLitresPerHour
}
}
In the first code extract above, where I split the text file contents and attempt to assign them into the array, you will see that I have commented out the append line. I have done this to get the application to compile, at the moment it is throwing me errors.
The error revolves around the conversion of the String values to Int and Double values as required. For example, Aircraft.id, or aSeperated[0] needs to be an Int. You can see that I use the line Int(aSeperated[0]) to convert the String to Int in order to assign it into the custom object. However, this line of code is failing.
The two print statements in the first code extract output the following values:
1
Optional(1)
If I add a ! to the end of the second print statement to make them:
print(aSeperated[0])
print(Int(aSeperated[0])!)
I get the following output:
I understand what the error means, that it tried to unwrap an optional value because I force unwrapped it, and it couldn't find an Int value within the string I passed to it, but I don't understand why I am getting the error. The string value is 1, which is very clearly an integer. What am I doing wrong?
Because Casena 172 is not convertible to an Int. You also have other decimal numbers which you will lose precision when casting them to Int. Use NSScanner to create an initializer from a CSV string:
init(csvString: String) {
let scanner = NSScanner(string: csvString)
var type: NSString?
scanner.scanInteger(&self.id)
scanner.scanLocation += 1
scanner.scanUpToString(",", intoString: &type)
self.type = type as! String
scanner.scanLocation += 1
scanner.scanInteger(&self.passengerCapacity)
scanner.scanLocation += 1
scanner.scanDouble(&self.cargoCapacityKg)
scanner.scanLocation += 1
scanner.scanInteger(&self.cruiseSpeed)
scanner.scanLocation += 1
scanner.scanDouble(&self.fuelLitresPerHour)
}
Usage:
let aircraft = Aircraft(csvString: "1,Cessna 172,3,54.4,124,38.6112")
As #mrkxbt mentioned, the issue was related to the blank line after the data in the text file. The string was being split at the \n which was assigning two values into the array. The first value was a string containing the data and the second was an empty string, so obviously the second set of splitting (by ,) was failing. Amended and working code is below:
if let dir : NSString = NSSearchPathForDirectoriesInDomains(NSSearchPathDirectory.DocumentDirectory, NSSearchPathDomainMask.AllDomainsMask, true).first {
let path = dir.stringByAppendingPathComponent("FSPassengers/aircraft.txt")
do {
let content = try NSString(contentsOfFile: path, encoding: NSUTF8StringEncoding)
if content != "" {
let astrContent:[String] = content.componentsSeparatedByString("\n")
for aeroplane in astrContent {
if aeroplane != "" {
let aSeperated:[String] = aeroplane.componentsSeparatedByString(",")
print(aSeperated[0])
print(Int(aSeperated[0])!)
self.aAircraft.append(Aircraft(id: Int(aSeperated[0])!, type: aSeperated[1], passengerCapacity: Int(aSeperated[2])!, cargoCapacityKg: Double(aSeperated[3])!, cruiseSpeed: Int(aSeperated[4])!, fuelLitresPerHour: Double(aSeperated[5])!))
}
}
}
}
catch {
print("Error")
}
}
A function in swift takes any numeric type in Swift (Int, Double, Float, UInt, etc).
the function converts the number to a string
the function signature is as follows :
func swiftNumbers <T : NumericType> (number : T) -> String {
//body
}
NumericType is a custom protocol that has been added to numeric types in Swift.
inside the body of the function, the number should be converted to a string:
I use the following
var stringFromNumber = "\(number)"
which is not so elegant, PLUS : if the absolute value of the number is strictly inferior to 0.0001 it gives this:
"\(0.000099)" //"9.9e-05"
or if the number is a big number :
"\(999999999999999999.9999)" //"1e+18"
is there a way to work around this string interpolation limitation? (without using Objective-C)
P.S :
NumberFormater doesn't work either
import Foundation
let number : NSNumber = 9_999_999_999_999_997
let formatter = NumberFormatter()
formatter.minimumFractionDigits = 20
formatter.minimumIntegerDigits = 20
formatter.minimumSignificantDigits = 40
formatter.string(from: number) // "9999999999999996.000000000000000000000000"
let stringFromNumber = String(format: "%20.20f", number) // "0.00000000000000000000"
Swift String Interpolation
1) Adding different types to a string
2) Means the string is created from a mix of constants, variables, literals or expressions.
Example:
let length:Float = 3.14
var breadth = 10
var myString = "Area of a rectangle is length*breadth"
myString = "\(myString) i.e. = \(length)*\(breadth)"
Output:
3.14
10
Area of a rectangle is length*breadth
Area of a rectangle is length*breadth i.e. = 3.14*10
Use the Swift String initializer: String(format: <#String#>, arguments: <#[CVarArgType]#>)
For example:
let stringFromNumber = String(format: "%.2f", number)
String and Characters conforms to StringInterpolationProtocol protocol which provide more power to the strings.
StringInterpolationProtocol - "Represents the contents of a string literal with interpolations while it’s being built up."
String interpolation has been around since the earliest days of Swift, but in Swift 5.0 it’s getting a massive overhaul to make it faster and more powerful.
let name = "Ashwinee Dhakde"
print("Hello, I'm \(name)")
Using the new string interpolation system in Swift 5.0 we can extend String.StringInterpolation to add our own custom interpolations, like this:
extension String.StringInterpolation {
mutating func appendInterpolation(_ value: Date) {
let formatter = DateFormatter()
formatter.dateStyle = .full
let dateString = formatter.string(from: value)
appendLiteral(dateString)
}
}
Usage: print("Today's date is \(Date()).")
We can even provide user-defined names to use String-Interpolation, let's understand with an example.
extension String.StringInterpolation {
mutating func appendInterpolation(JSON JSONData: Data) {
guard
let JSONObject = try? JSONSerialization.jsonObject(with: JSONData, options: []),
let jsonData = try? JSONSerialization.data(withJSONObject: JSONObject, options: .prettyPrinted) else {
appendInterpolation("Invalid JSON data")
return
}
appendInterpolation("\n\(String(decoding: jsonData, as: UTF8.self))")
}
}
print("The JSON is \(JSON: jsonData)")
Whenever we want to provide "JSON" in the string interpolation statement, it will print the .prettyPrinted
Isn't it cool!!
I'm trying to organize data I am given from a text file, there are for 4 pieces of info on each line (City, country, population, and date). I wanted to have an array for each so I first put it all into one big String array and started to separate them into 4 arrays but I needed to change the Population info to an int array but it says *
"Type mismatch: cannot convert from element type int to String"
//Separate the information by commas
while(sc.hasNextLine()){
String line = sc.nextLine();
input = line.split(",");
//Organize the data into 4 seperate arrays
for(int x=0; x<input.length;x++){
if(x%4==0){
cities[x] = input[x];
}
if(x%4==1){
countries[x] = input[x];
}
if(x%4==2){
population[x] = Integer.parseInt(input[x]);
}
if(x%4==3){
dates[x] = input[x];
}
}
}
And when I print out the arrays they have a bunch of nulls in between each data. I'm planning to create objects that have the 4 pieces of data so that I can then sort them by population, dates etc... I'm pretty new to working with objects so if anyone has a better way of getting the 4 pieces of data into an object cause I haven't figured a way yet :/ My end goal was to have an array of these objects that I can u different sorting methods on them
I would recommend doing something like this:
public class MyData {
private String city;
private String country;
private Integer population;
private String date;
public MyData(String city, String, country, Integer population, String date) {
this.city = city;
this.country = country;
this.population = population;
this.date = date;
}
// Add getters and setters here
}
And then in the file you're posting about:
...
ArrayList<MyData> allData = new ArrayList<MyData>();
while(sc.hasNextLine()) {
String[] values = sc.nextLine().split(",");
allData.add(new MyData(values[0], values[1], Integer.parseInt(values[2]), values[3]));
}
...
You need an object to store the data in so that you keep the relationship between the values in each column.
Also, I'm just assuming you're using Java here. Which language we're talking about is something you should include in your question or as a tag.
The problem is with your x index. If you look carefully at your "for" you will see that it will insert a value at every 3 positions.
try
int index = 0;
while(sc.hasNextLine()){
String line = sc.nextLine();
input = line.split(",");
//Organize the data into 4 seperate arrays
for(int x=0; x<input.length;x++){
if(x%4==0){
cities[index] = input[x];
}
if(x%4==1){
countries[index] = input[x];
}
if(x%4==2){
population[index] = Integer.parseInt(input[x]);
}
if(x%4==3){
dates[index] = input[x];
}
}
++index;
}