I am trying to add changes data in a csv file:
This is the sample data:
DATE status code value value2
"2016-01-26","Subscription All","119432660","1315529431362550","0.0080099833517888"
"2016-01-26","Subscription All","119432664","5836995058433524","0.033825584764444"
"2016-01-26","Subscription All","119432664","8287300074499777","0.076913377834744"
"2016-01-26","Subscription All","119432664","14870697739968326","0.0074188355187426"
My code used to format the data:
CSVReader reader = new CSVReader(new FileReader(new File(fileToChange)), CSVParser.DEFAULT_SEPARATOR, CSVParser.NULL_CHARACTER, CSVParser.NULL_CHARACTER, 1)
info "Read all rows at once"
List<String[]> allRows = reader.readAll();
CSVWriter writer = new CSVWriter(new FileWriter(fileToChange), CSVWriter.DEFAULT_SEPARATOR, CSVWriter.NO_QUOTE_CHARACTER)
writer.writeAll(allRows)
writer.close()
The output i get is this, with extra quote added instead of removing it.
""2016-01-26"",""Subscription All"",""119432660"",""1315529431362550"",""0.0080099833517888""
""2016-01-26"",""Subscription All"",""119432664"",""5836995058433524"",""0.033825584764444""
""2016-01-26"",""Subscription All"",""119432664"",""8287300074499777"",""0.076913377834744""
""2016-01-26"",""Subscription All"",""119432664"",""14870697739968326"",""0.0074188355187426""
I want to remove the quotes.
Please can someone help.
Also, is it possible to change the date format to yyyymmdd instead of yyyy-mm-dd?
allRows.each { String[] theLine ->
String newDate = theLine[0].replaceAll('-', '')
String newline = theLine.eachWithIndex { String s, int i -> return i > 0 ? s : newDate}
writer.writeLine(newline)
}
Thanks
When you instantiated your CSVReader you told it to treat no characters as quotes, therefore it read the existing quotes as data and did not remove them.
When you told CSVWriter not to add any quotes it honored your request. However, the input data contained quote characters, and the convention for including quotes inside a string in CSV is to double the quotes. Thus the
string value
ABC"DEF
gets coded in CSV as
"ABC""DEF"
So the result you see is the combination of not removing the quotes on input (you told it not to) and then doubling the quotes on output.
To solve this change the input option from NULL_CHARACTER to DEFAULT_QUOTE_CHARACTER. However be aware that if any of your data actually contains embedded quotes or commas the resulting output will not be valid CSV.
Also I think this might be a valid bug report against OpenCSV. I believe that OpenCSV needs to inform you if it is about to generate invalid CSV when you told it to omit quotes, probably via a runtime exception. Although I suppose they might argue that you chose to work without a net and should accept whatever you get. Personally I go for the "principle of least surprise", which IMHO would be not to double quotes when the output is unquoted.
Because quotation in your CSVReader is set to CSVParser.NULL_CHARACTER " is treated as normal character which is part of read token. This causes your array to contain data in form:
["2016-01-26", "Subscription All", "119432660", "1315529431362550", "0.0080099833517888"]
rather than:
[2016-01-26, Subscription All, 119432660, 1315529431362550, 0.0080099833517888]
So try changing option from CSVParser.NULL_CHARACTER to either
'"'
CSVParser.DEFAULT_QUOTE_CHARACTER (it also stores '"').
CsvToBean csvToBean = new CsvToBeanBuilder(new StringReader(csv))
.withMappingStrategy(strategy)
.withIgnoreLeadingWhiteSpace(true)
.withSeparator(',')
.withIgnoreEmptyLine(true)
.withQuoteChar('\'')
.withQuoteChar('"')
.build();
Related
I want to update a string in quotes in yaml file in Jenkins Job. While updating the file, single quotes around the string are replaced by triple quotes. Following is the method that I have written:
{
def fileName = 'config.yml'
datas = readYaml file: fileName
var = "'" + params.ReleaseBranchName + "'"
println var // this shows output as expected, string in single quotes -> 'rel-21.9'
datas.branchName = var
println datas // this prints the yaml with single quotes -> productiveBranch='rel-21.9',
writeYaml file: fileName, data: datas, overwrite: true //this show value in triple quotes -> productiveBranch: '''rel-21.9'''
}
Could someone suggest how can I save string with single quotes in yaml file? Thanks!
The value of var, as written, is 'rel-21.9', i.e. the single quotes are part of the value.
In YAML input, when 'rel-21.9' is encountered, the single quotes are not part of the value; they are part of the syntax and enclose the value, so the value is rel-21.9.
Therefore, if you want your value to be rel-21.9, which is most probably what you want, do not put single quotes in the value; just do var = params.ReleaseBranchName.
Your code does not do anything with var; I assume what you're trying is to put it into datas. This would result in YAML writing out "'rel-21.9'" (not triple single quotes, that can't happen since it would be invalid YAML). By surrounding the value with double quotes, the single quotes become part of the value just like your code requested.
When you do not put single quotes into the data, YAML will probably serialise it without any quotes. This is expected since rel-21.9 does not contain any special characters that would require quoting. There are ways to force a YAML processor to quote a value, but they are complex and I am unsure whether the API is exposed to Groovy. For references, this is how you would do it in Java.
Since you are editing a YAML file, you might want to read this question which details how and why updating YAML files in code can lead to style changes.
I have a query that looks like this:
INSERT INTO table VALUES ('47677;2019;2019;10T-1001-10010AS;A05;International;TieLineKoman-KosovoB;L_KOM-KOSB;2018;NULL;NULL;;NULL;Tieline;NULL;10XAL-KESH-----J;0;3')
that is produced by parsing a csv file.
The query is not in a valid form, I have to replace all semicolons with the string ',' (comma inside single quotes). What I want to get is:
('47677','2019','2019','10T-1001-10010AS','A05','International','TieLineKoman-KosovoB','L_KOM-KOSB','2018','NULL','NULL','','NULL','Tieline','NULL','10XAL-KESH-----J','0','3')
I have tried to do this in many different ways, but I end up with backshlashes added in my string. This is what I get:
"INSERT INTO AllocatedEICDetail VALUES ('47677\\',\\'2019\\',\\'2019\\',\\'10T-1001-10010AS\\',\\'A05\\',\\'International\\',\\'TieLineKoman-KosovoB\\',\\'L_KOM-KOSB\\',\\'2018\\',\\'NULL\\',\\'NULL\\',\\'\\',\\'NULL\\',\\'Tieline\\',\\'NULL\\',\\'10XAL-KESH-----J\\',\\'0\\',\\'3')"
Any ideas how to do this properly without having the backslashes added?
Thank you!
//the string you have
const string = '47677;2019;2019;10T-1001-10010AS;A05;International;TieLineKoman-KosovoB;L_KOM-KOSB;2018;NULL;NULL;;NULL;Tieline;NULL;10XAL-KESH-----J;0;3';
//the string you need:
const targetString = string.replace(/\;/g,',');
You specify a small regex between the forward slashes in replace which is a simple ';', give it a 'g' flag for global which will replace all instances, and in the second argument supply what you need it replaced with.
I see a very weird problem when json when used in nodejs, it is skipping single quote from revision key . I want to pass this json as input to node request module and since single quote is missing from 'revision' key so it is not taking as valid json input. Could someone help how to retain it so that I can use it. I have tried multiple attempts but not able to get it correct.
What did I try ?
console.log(jsondata)
jsondata = {
'splits': {
'os-name': 'ubuntu',
'platform-version': 'os',
'traffic-percent': 100,
'revision': 'master'
}
}
Expected :-
{ splits:
{ 'os-name': 'ubuntu',
'platform-version': 'os',
'traffic-percent': 100,
'revision': 'master'
}
}
But in actual output single quote is missing from revision key :-
{ splits:
{ 'os-name': 'ubuntu',
'platform-version': 'os',
'traffic-percent': 100,
revision: 'master'
}
}
Run 2 :- Tried below code this also produce same thing.
data = JSON.stringify(jsondata)
result = JSON.parse(data)
console.log(result)
Run 3:- Used another way to achieve it
jsondata = {}
temp = {}
splits = []
temp['revision'] = 'master',
temp['os-name'] = 'ubuntu'
temp['platform-version'] = 'os'
temp['traffic-percent'] = 100
splits.push(temp)
jsondata['splits'] = splits
console.log(jsondata)
Run 4: tries replacing single quotes to double quotes
Run 5 : Change the order of revision line
This is what is supposed to happen. The quotes are kept only if the object key it’s not a valid JavaScript identifier. In your example, the 'splits' & 'revision' don't have a dash in their name, so they are the only ones with the quotes removed.
You shouldn't receive any error using this object - if you do, update this post mentioning the scenario and the error.
You should note that JSON and JavaScript are not the same things.
JSON is a format where all keys and values are surrounded by double quotes ("key" and "value"). A JSON string is produced by JSON.stringify, and is required by JSON.parse.
A JavaScript object has very similar syntax to the JSON file format, but is more flexible - the values can be surrounded by double quotes or single quotes, and the keys can have no quotes at all as long as they are valid JavaScript identifiers. If the keys have spaces, dashes, or other non-valid characters, then they need to be surrounded by single quotes or double quotes.
If you need your string to be valid JSON, generate it with JSON.stringify. If it's OK for it to be just valid JavaScript, then it's already fine - it does not matter whether the quotes are there or not.
If, for some reason, you need some imaginary third option (perhaps you are interacting with an API where someone has written their own custom string parser, and they are demanding that all keys are surrounded by single quotes?) you will probably need to write your own little string generator.
I try to import a CSV file in Excel, using ; as delimiters, but some columns contains
; and/or quotes.
My problem is : I can use double quotes to ignore the delimiters for a specific string, but if there is a double quote inside the string, it ignores delimiters until the first double quote, but not after.
I don't know if it's clear, it's not that easy to explain.
I will try to explain with a example :
Suppose I have this string this is a;test : I use double quotes around the string, to ignore the delimiter => It works.
Now if this string contains delimiters AND double quotes : my trick doesn't work anymore. For example if I have the string this; is" a;test : My added double quotes around the string ignore delimiters for the first part (the delimiter in the part this; is is correctly ignored, but since there is a double quote after, Excel doesn't ignore the next delimiter in the a;test part.
I tried my best to be as clear as possible, I hope you'll understand what is the problem.
When reading in a quoted string in a csv file, Excel will interpret all pairs of double-quotes ("") with single double-quotes(").
so "this; is"" a;test" will be converted to one cell containing this; is" a;test
So replace all double-quotes in your strings with pairs of double quotes.
Excel will reverse this process when exporting as CSV.
Here is some CSV
a,b,c,d,e
"""test1""",""",te"st2,"test,3",test"4,test5
And this is how it looks after importing into Excel:
Import your Excel file in openOffice and export as CSV (column escaped with " unlike Excel csv, utf8, comma against ";").
I have something like:
def newProps = new Properties()
def fileWriter = new OutputStreamWriter(new FileOutputStream(propsFile,true), 'UTF-8')
def lineSeparator = System.getProperty("line.separator")
newProps.setProperty('SFTP_USER_HASH', userSftpHome.toString())
newProps.setProperty('GD_SFTP_URI', sftpHost.toString())
fileWriter.write(lineSeparator)
newProps.store(fileWriter, null)
fileWriter.close()
The problem is that store() method escapes ":" or "=" characters with backslash (). I don't want that because I store there some passwords and tokens and need to copy those values strictly in the key=value format.
Also, when I use the configSlurper, it stores the values with single quotes, like:
key='value'
Is there any solution for that? Saving in unescaped key=value format to properties file in Groovy?
You could do this:
def newProps = new Properties()
newProps.setProperty('SFTP_USER_HASH', 'woo')
newProps.setProperty('GD_SFTP_URI', 'ftp://woo.com')
propsFile.withWriterAppend( 'UTF-8' ) { fileWriter ->
fileWriter.writeLine ''
newProps.each { key, value ->
fileWriter.writeLine "$key=$value"
}
}
BUT, so long as you are reading the properties in with load, there should be no need for this as it should de-escape any escaped characters
The JDK's built in Properties class does that escaping by design. According to the Docs:
Then every entry in this Properties table is written out, one per
line. For each entry the key string is written, then an ASCII =, then
the associated element string. For the key, all space characters are
written with a preceding \ character. For the element, leading space
characters, but not embedded or trailing space characters, are written
with a preceding \ character. The key and element characters #, !, =,
and : are written with a preceding backslash to ensure that they are
properly loaded.
You can however, override this behavior by sub-classing the Properties class yourself. You'd need to override the load and store methods yourself and read/write yourself. It would be pretty straight forward; pretty good examples found here: Link