Removing accents and diacritics in kotlin - string

Question

Is there any way to convert string like 'Dziękuję' to 'Dziekuje' or 'šećer' to 'secer' in kotlin. I have tried using java.text.Normalizer but it doesn't seem to work the desired way.

Answer 1

Normalizer only does half the work. Here's how you could use it:
private val REGEX_UNACCENT = "\\p{InCombiningDiacriticalMarks}+".toRegex()
fun CharSequence.unaccent(): String {
val temp = Normalizer.normalize(this, Normalizer.Form.NFD)
return REGEX_UNACCENT.replace(temp, "")
}
assert("áéíóů".unaccent() == "aeiou")
And here's how it works:
We are calling the normalize(). If we pass à, the method returns a + ` . Then using a regular expression, we clean up the string to keep only valid US-ASCII characters.
Source: http://www.rgagnon.com/javadetails/java-0456.html
Note that Normalizer is a Java class; this is not pure Kotlin and it will only work on JVM.

Answer 2

TL;DR:
Use Normalizer to canonically decomposed the Unicode thext.
Remove non-spacing combining characters (\p{Mn}).
fun String.removeNonSpacingMarks() =
Normalizer.normalize(this, Normalizer.Form.NFD)
.replace("\\p{Mn}+".toRegex(), "")
Long answer:
Using Normalizer you can transform the original text into an equivalent composed or decomposed form.
NFD: Canonical decomposition.
NFC: Canonical decomposition, followed by canonical composition.
.
(more info about normalization can be found in the Unicode® Standard Annex #15)
In our case, we are interested in NFD normalization form because it allows us to separate all the combined characters from the base character.
After decomposing the text, we have to run a regex to remove all the new characters resulting from the decomposition that correspond to combined characters.
Combined characters are special characters intended to be positioned relative to an associated base character. The Unicode Standard distinguishes two types of combining characters: spacing and nonspacing.
We are only interested in non-spacing combining characters. Diacritics are the principal class (but not the only one) of this group used with Latin, Greek, and Cyrillic scripts and their relatives.
To remove non-spacing characters with a regex we have to use \p{Mn}. This group includes all the 1,826 non-spacing characters.
Other answers uses \p{InCombiningDiacriticalMarks}, this block only includes combining diacritical marks. It is a subset of \p{Mn} that includes only 112 characters.

Answer 3

This is an extension function you can use and extend further:
fun String.normalize(): String {
val original = arrayOf("ę", "š")
val normalized = arrayOf("e", "s")
return this.map { it ->
val index = original.indexOf(it.toString())
if (index >= 0) normalized[index] else it
}.joinToString("")
}
Use it like this:
val originalText = "aębšc"
val normalizedText = originalText.normalize()
println(normalizedText)
will print
aebsc
Extend the arrays original and normalized with as many elements as you need.

Answer 4

In case anyone is strugling to do this in kotlin, this code works like a charm.
To avoid inconsistencies I also use .toUpperCase and Trim(). then i cast this function:
fun stripAccents(s: String):String{
if (s == null) {
return "";
}
val chars: CharArray = s.toCharArray()
var sb = StringBuilder(s)
var cont: Int = 0
while (chars.size > cont) {
var c: kotlin.Char
c = chars[cont]
var c2:String = c.toString()
//these are my needs, in case you need to convert other accents just Add new entries aqui
c2 = c2.replace("Ã", "A")
c2 = c2.replace("Õ", "O")
c2 = c2.replace("Ç", "C")
c2 = c2.replace("Á", "A")
c2 = c2.replace("Ó", "O")
c2 = c2.replace("Ê", "E")
c2 = c2.replace("É", "E")
c2 = c2.replace("Ú", "U")
c = c2.single()
sb.setCharAt(cont, c)
cont++
}
return sb.toString()
}
to use these fun cast the code like this:
var str: String
str = editText.text.toString() //get the text from EditText
str = str.toUpperCase().trim()
str = stripAccents(str) //call the function