#Test
public void test() throws InvalidProtocolBufferException {
byte[] testString = StringValue.newBuilder()
.setValue("testString")
.build()
.getValueBytes()
.toByteArray();
StringValue stringValue = StringValue.parseFrom(testString);
System.out.println(stringValue);
}
Created a byte array from the Protobuf StringValue, but when converted it back, I got:
com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
Any idea why would this happen?
Related
Dear Members,
As we know that Text data type of Hadoop uses UTF-8 encoding. That is if the character can be stored in a byte it will store it in a byte. If a character cannot be stored in a byte then it will store it in 2 bytes. The same way, as a performance boost, for word count program in Hadoop Map phase, for mapper value data type if ByteWritable can be used then the amount of mapper output data can be reduced. IntWritable requires 4 bytes whereas ByteWritable needs 1 byte. If I use VIntWritable instead of IntWritable then if an integer can be stored in a byte it will store it in a byte otherwise it will store it in 4 bytes reduce memory footprint.
How the following Java program can be modified so that mapper's key type is Text and value is ByteWritable and reducer's key type is Text and value is VIntWritable.
// Now we have non-empty lines, lets split them into words
JavaRDD<String> words = nonEmptyLines.flatMap(new FlatMapFunction<String, String>() {
#Override
public Iterable<String> call(String s) throws Exception {
return Arrays.asList(s.split(" "));
}
});
// Convert words to Pairs, remember the TextPair class in Hadoop world
JavaPairRDD<String, Integer> wordPairs = words.mapToPair(new PairFunction<String, String, Integer>() {
public Tuple2<String, Integer> call(String s) {
return new Tuple2<String, Integer>(s, 1);
}
});
JavaPairRDD<String, Integer> wordCount = wordPairs.reduceByKey(new Function2<Integer, Integer, Integer>() {
#Override
public Integer call(Integer integer, Integer integer2) throws Exception {
return integer + integer2;
}
});
// Just for debugging, NOT FOR PRODUCTION
wordCount.foreach(new VoidFunction<Tuple2<String, Integer>>() {
#Override
public void call(Tuple2<String, Integer> stringIntegerTuple2) throws Exception {
System.out.println(String.format("%s - %d", stringIntegerTuple2._1(), stringIntegerTuple2._2()));
}
});
I have written a method (mentioned below) which returns a Map.
Class MyTest
static Map<String, String> getApplicationPreferences() {
String host = System.getProperty("property1")
String namespace = System.getProperty("property2")
String username = System.getProperty("property3")
String password = System.getProperty("property4")
String url = "http://" + host + ":10000/v3/namespaces/" + namespace + "/apps/MyApplication/preferences"
String token = RestUtil.getToken(host, username, password)
HttpURLConnection connection = RestUtil.sendRequest(url, token, true, RestUtil.REQUEST_METHOD.GET)
String response = connection.getInputStream().getText()
Map<String, String> preferences = new ObjectMapper().readValue(response, Map.class)
return preferences
}
I am calling the above method as:
static Map<String, String> envPreferences = new HashMap<String, String>();
envPreferences = MyTest.getApplicationPreferences()
and printing the Values from the Map by following way:
println("\nAll preferences="+envPreferences.toString()+"\n")
Now I want to assign the values to String variables which I have to pass to another function.
static String a = envPreferences.get('messaging.kafkaBrokers')
println("Kafka preferences="+a)
static String b = envPreferences.get("zookeeper.server").toString()
println("Zookeeper preferences="+b)
The Output that I am getting for the above print statements are:
All preferences=[zookeeper.server:localhost:2181, messaging.kafkaBrokers:localhost:9092]
Kafka preferences=null
Zookeeper preferences="null"
How can I get the values instead of null?
Thanks in advance.
In the code below, sBar is an arraylist. I am trying to convert it to String and then write it to file. However, I don't know what I did wrong here as I keep getting error message saying:
- NullPointerException
- Exception in thread "Thread-1" java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.String
try{
FileWriter writer = new FileWriter("stime.txt");
for (Iterator it = sBar.iterator(); it.hasNext();) {
String str = (String) it.next();
writer.write(str);
}
}
} catch (IOException e) {
}
You can't convert a long to a string by casting. Change
String str = (String) it.next();
to
String str = it.next().toString();
Another way to write it would be to use a for each loop:
for (Long val : sBar)
writer.write(String.valueOf(val));
As the error clearly states, you cannot cast a Long to a String.
Java casts can only be used to convert an object to a type that it actually is; you cannot use a cast to convert an object to a different type.
You may want to call .toString().
This is my transformation function call:
<p><%# MyFunctions.getDocumentCategory(Eval("DocumentID"))%></p>
This is the function:
public static string getDocumentCategory(int documentID)
{
string category;
StringBuilder sb = new StringBuilder();
// Get document categories
var ds = CategoryInfoProvider.GetDocumentCategories(documentID, "CategoryEnabled = 1", null);
// Check whether exists at least one category
if (!DataHelper.DataSourceIsEmpty(ds))
{
// Loop thru all categories
foreach (DataRow dr in ds.Tables[0].Rows)
{
sb.Append(Convert.ToString(dr["CategoryDisplayName"]) + ",");
}
}
string content = sb.ToString();
category = content.Split(',')[0];
return category;
}
}
This is the error:
MyFunctions.getDocumentCategory(int) has some invalid arguments.
I've tried an alternate form of the function that accepts strings rather than ints but it throws the same error. I've verified that the Eval("DocumentID") works correctly when placed by itself. Any ideas?
Eval returns an object. You either need to convert it to an int, or change the function to accept an object, and convert that object to an int.
<p><%# MyFunctions.getDocumentCategory( Convert.ToInt32( Eval("DocumentID") ) )%></p>
OR
public static string getDocumentCategory(object document)
{
int documentID = Convert.ToInt32( document );
etc...
}
Thanks to Doozer for the nice explanation and example.
The second approach - to accept the object and make the conversion inside your custom function - may be better to keep the transformation code cleaner. The result is equal.
Just to add a little bit - you can use Kentico's ValidationHelper for conversions, for example:
transformation:
<%# MyFunctions.getDocumentCategory(Eval("DocumentID"))%>
code:
public static string getDocumentCategory(object docID)
{
int documentID = ValidationHelper.GetInteger(docID, 0); //0 is the default value
...
Here is my idea
I am reading a string from my .resx file
And here is a sample of such string :"I am writing this from {}"
I wrote a function to pass values to those arguments. I don't know the number of arguments expected by the string
public string MyFormattedString (string resourceName, object param=null)
{
string fStr= Resources.ResourceManager.GetString(resourceName);
fStr= string.Format(fStr, param);
return fStr;
}
If I call my function with MyFormattedString ("resourceName", "noWhere"), I do not get what I am expecting
What's wrong?
I found a solution to my issue with using params object[] I just discovered
public string MyFormattedString (string resourceName, params object[] param)
{
string fStr= Resources.ResourceManager.GetString(resourceName);
fStr= string.Format(fStr, param);
return fStr;
}
The resource string should be "I am writing this from {0}" with a numeric position in it.