Can someone help me understand how to make the following using object-oriented programming?
It is a short program that compares how quickly two different sort algorithms run on arrays of various sizes. When trying to rewrite it using object orientation, I start by making an object called "OO" at the top of Main, then prefixing all the method calls with "OO.", then remvoing all the static modifiers, but it still throws errors. I'm sure there is an easy fix, I'm just very new to this. Thanks. (below is the version that runs, with no attempt yet at OOP)
import java.util.Arrays;
public class SortingExercise {
//Create constants to give array size
final static int CONST100 = 100000; // The length of arrays that will be sorted.
final static int CONST10 = 10000; // The length of arrays that will be sorted.
final static int CONST1 = 1000; // The length of arrays that will be sorted.
final static int CONSTMIL = 1000000; //Constant for use in making/sorting a million-element array
/*
* Creates an array populated with random integers varying widely in size
* #param count : the length of the array that is created.
*/
private static int[] randomInts(int count) {
int[] numbers = new int[count];
for (int i = 0; i < count; i++)
numbers[i] = (int)(Integer.MAX_VALUE * Math.random());
return numbers;
}
/*
* Sorts an array of integers using the selection sort algorithm.
*/
private static void selectionSort(int[] numbers) {
for (int end = numbers.length-1; end > 0; end-- ) {
int maxloc = 0;
for (int i = 1; i <= end; i++) {
if (numbers[i] > numbers[maxloc])
maxloc = i;
}
int temp = numbers[end];
numbers[end] = numbers[maxloc];
numbers[maxloc] = temp;
}
}
public static void main(String[] args) {
double startTime;
double runTime;
int[] Array1;
int[] Array2;
//FIRST RUN - 1,000 ELEMENTS
//Create Arrays
Array1 = randomInts(CONST1);
Array2 = Arrays.copyOf(Array1, CONST1);
//Sort and print comparative run times of selectionSort and the Array.sort method on identical 1,000-element arrays
startTime = System.currentTimeMillis();
selectionSort(Array1);
runTime = System.currentTimeMillis() - startTime;
System.out.println(runTime + " milliseconds for Array1 with 1,000 elements using selectionSort");
startTime = System.nanoTime();
Arrays.sort(Array2);
runTime = System.nanoTime() - startTime;
System.out.println(runTime/1000000 + " milliseconds for Array2 with 1,000 elements using built-in Array.sort method");
System.out.println();
//SECOND RUN - 10,000 ELEMENTS
//Create Arrays
Array1 = randomInts(CONST10);
Array2 = Arrays.copyOf(Array1, CONST10);
//Sort and print comparative run times of selectionSort and the Array.sort method on identical 10,000-element arrays
startTime = System.currentTimeMillis();
selectionSort(Array1);
runTime = System.currentTimeMillis() - startTime;
System.out.println(runTime + " milliseconds for Array1 with 10,000 elements using selectionSort");
startTime = System.currentTimeMillis();
Arrays.sort(Array2);
runTime = System.currentTimeMillis() - startTime;
System.out.println(runTime + " milliseconds for Array2 with 10,000 elements using built-in Array.sort method");
System.out.println();
//THIRD RUN - 100,000 ELEMENTS
//Create Arrays
Array1 = randomInts(CONST100);
Array2 = Arrays.copyOf(Array1, CONST100);
//Sort and print comparative run times of selectionSort and the Array.sort method on identical 100,000-element arrays
startTime = System.currentTimeMillis();
selectionSort(Array1);
runTime = System.currentTimeMillis() - startTime;
System.out.println(runTime + " milliseconds for Array1 with 100,000 elements using selectionSort");
startTime = System.currentTimeMillis();
Arrays.sort(Array2);
runTime = System.currentTimeMillis() - startTime;
System.out.println(runTime + " milliseconds for Array2 with 100,000 elements using built-in Array.sort method");
System.out.println();
//OPTIONAL FINAL RUN ONLY USING .sort() -> 1 MILLION ELEMENTS
//Create Array
Array1 = randomInts(CONSTMIL);
//Sort and print runtime for a million-element array
startTime = System.currentTimeMillis();
Arrays.sort(Array1);
runTime = System.currentTimeMillis() - startTime;
System.out.println(runTime + " milliseconds for Array1 with 1 million elements using built-in Array.sort method");
//END PROGRAM
}
}
I want to calculate the median value of a PairDStream for the values of each key.
I already tried the following, which is very unefficient:
JavaPairDStream<String, Iterable<Float>> groupedByKey = pairDstream.groupByKey();
JavaPairDStream<String, Float> medianPerPlug1h = groupedByKey.transformToPair(new Function<JavaPairRDD<String,Iterable<Float>>, JavaPairRDD<String,Float>>() {
public JavaPairRDD<String, Float> call(JavaPairRDD<String, Iterable<Float>> v1) throws Exception {
return v1.mapValues(new Function<Iterable<Float>, Float>() {
public Float call(Iterable<Float> v1) throws Exception {
List<Float> buffer = new ArrayList<Float>();
long count = 0L;
Iterator<Float> iterator = v1.iterator();
while(iterator.hasNext()) {
buffer.add(iterator.next());
count++;
}
float[] values = new float[(int)count];
for(int i = 0; i < buffer.size(); i++) {
values[i] = buffer.get(i);
}
Arrays.sort(values);
float median;
int startIndex;
if(count % 2 == 0) {
startIndex = (int)(count / 2 - 1);
float a = values[startIndex];
float b = values[startIndex + 1];
median = (a + b) / 2.0f;
} else {
startIndex = (int)(count/2);
median = values[startIndex];
}
return median;
}
});
}
});
medianPerPlug1h.print();
Can somebody help me with a more efficient transaction? I have about 1950 different keys, each can get to 3600 (1 data point per second, window of 1 hour) values, where to find the median of.
Thank you!
First thing is that I don't know why are you using Spark for that kind of task. It doesn't seem to be related to big data considering you got just few thousand of values. It may make things more complicated. But let's assume you're planning to scale up to bigger datasets.
I would try to use some more optimized algorithm for finding median than just sorting values. Sorting an array of values runs in O(n * log n) time.
You could think about using some linear-time median algorithm like Median of medians
1) avoid using groupbykey; reducebykey is more efficient than groupbykey.
2) reduceByKeyAndWindow(Function2,windowduration,slideDuration) can serve you better.
example:
JavaPairDStream merged=yourRDD.reduceByKeyAndWindow(new Function2() {
public String call(String arg0, String arg1) throws Exception {
return arg0+","+arg1;
}
}, Durations.seconds(windowDur), Durations.seconds(slideDur));
Assume output from this RDD will be like this :
(key,1,2,3,4,5,6,7)
(key,1,2,3,4,5,6,7).
now for each key , you can parse this , you will have the count of values,
so : 1+2+3+4+5+6+7/count
Note: i used string just to concatenate.
I hope it helps :)
Can somebody verify this method. I need a long type number inside a range of two longs. I use the .NET Random.Next(min, max) function which return int's. Is my reasoning correct if I simply divide the long by 2, generate the random number and finally multiply it by 2 again? Or am I too enthusiastic...
I understand that my random resolution will decrease but are there any other mistakes which will lead to no such a random number.
long min = st.MinimumTime.Ticks; //long is Signed 64-bit integer
long max = st.MaximumTime.Ticks;
int minInt = (int) (min / 2); //int is Signed 64-bit integer
int maxInt = (int) (max / 2); //int is Signed 64-bit integer
Random random = new Random();
int randomInt = random.Next(minInt, maxInt);
long randomLong = (randomInt * 2);
Why don't you just generate two random Int32 values and make one Int64 out of them?
long LongRandom(long min, long max, Random rand) {
long result = rand.Next((Int32)(min >> 32), (Int32)(max >> 32));
result = (result << 32);
result = result | (long)rand.Next((Int32)min, (Int32)max);
return result;
}
Sorry, I forgot to add boundaries the first time. Added min and max params. You can test it like that:
long r = LongRandom(100000000000000000, 100000000000000050, new Random());
Values of r will lie in the desired range.
EDIT: the implementation above is flawed. It's probably worth it to generate 4 16-bit integers rather than 2 32-bit ones to avoid signed-unsigned problems. But at this point the solution loses its elegancy, so I think it's best to stick with Random.NextBytes version:
long LongRandom(long min, long max, Random rand) {
byte[] buf = new byte[8];
rand.NextBytes(buf);
long longRand = BitConverter.ToInt64(buf, 0);
return (Math.Abs(longRand % (max - min)) + min);
}
It looks pretty well in terms of value distribution (judging by very simple tests I ran).
Some other answers here have two issues: having a modulo bias, and failing to correctly handle values of max = long.MaxValue. (Martin's answer has neither problem, but his code is unreasonably slow with large ranges.)
The following code will fix all of those issues:
//Working with ulong so that modulo works correctly with values > long.MaxValue
ulong uRange = (ulong)(max - min);
//Prevent a modolo bias; see https://stackoverflow.com/a/10984975/238419
//for more information.
//In the worst case, the expected number of calls is 2 (though usually it's
//much closer to 1) so this loop doesn't really hurt performance at all.
ulong ulongRand;
do
{
byte[] buf = new byte[8];
random.NextBytes(buf);
ulongRand = (ulong)BitConverter.ToInt64(buf, 0);
} while (ulongRand > ulong.MaxValue - ((ulong.MaxValue % uRange) + 1) % uRange);
return (long)(ulongRand % uRange) + min;
The following fully-documented class can be dropped into your codebase to implement the above solution easily and brain-free. Like all code on Stackoverflow, it's licensed under CC-attribution, so you can feel free to use to use it for basically whatever you want.
using System;
namespace MyNamespace
{
public static class RandomExtensionMethods
{
/// <summary>
/// Returns a random long from min (inclusive) to max (exclusive)
/// </summary>
/// <param name="random">The given random instance</param>
/// <param name="min">The inclusive minimum bound</param>
/// <param name="max">The exclusive maximum bound. Must be greater than min</param>
public static long NextLong(this Random random, long min, long max)
{
if (max <= min)
throw new ArgumentOutOfRangeException("max", "max must be > min!");
//Working with ulong so that modulo works correctly with values > long.MaxValue
ulong uRange = (ulong)(max - min);
//Prevent a modolo bias; see https://stackoverflow.com/a/10984975/238419
//for more information.
//In the worst case, the expected number of calls is 2 (though usually it's
//much closer to 1) so this loop doesn't really hurt performance at all.
ulong ulongRand;
do
{
byte[] buf = new byte[8];
random.NextBytes(buf);
ulongRand = (ulong)BitConverter.ToInt64(buf, 0);
} while (ulongRand > ulong.MaxValue - ((ulong.MaxValue % uRange) + 1) % uRange);
return (long)(ulongRand % uRange) + min;
}
/// <summary>
/// Returns a random long from 0 (inclusive) to max (exclusive)
/// </summary>
/// <param name="random">The given random instance</param>
/// <param name="max">The exclusive maximum bound. Must be greater than 0</param>
public static long NextLong(this Random random, long max)
{
return random.NextLong(0, max);
}
/// <summary>
/// Returns a random long over all possible values of long (except long.MaxValue, similar to
/// random.Next())
/// </summary>
/// <param name="random">The given random instance</param>
public static long NextLong(this Random random)
{
return random.NextLong(long.MinValue, long.MaxValue);
}
}
}
Usage:
Random random = new Random();
long foobar = random.NextLong(0, 1234567890L);
This creates a random Int64 by using random bytes, avoiding modulo bias by retrying if the number is outside the safe range.
static class RandomExtensions
{
public static long RandomLong(this Random rnd)
{
byte[] buffer = new byte[8];
rnd.NextBytes (buffer);
return BitConverter.ToInt64(buffer, 0);
}
public static long RandomLong(this Random rnd, long min, long max)
{
EnsureMinLEQMax(ref min, ref max);
long numbersInRange = unchecked(max - min + 1);
if (numbersInRange < 0)
throw new ArgumentException("Size of range between min and max must be less than or equal to Int64.MaxValue");
long randomOffset = RandomLong(rnd);
if (IsModuloBiased(randomOffset, numbersInRange))
return RandomLong(rnd, min, max); // Try again
else
return min + PositiveModuloOrZero(randomOffset, numbersInRange);
}
static bool IsModuloBiased(long randomOffset, long numbersInRange)
{
long greatestCompleteRange = numbersInRange * (long.MaxValue / numbersInRange);
return randomOffset > greatestCompleteRange;
}
static long PositiveModuloOrZero(long dividend, long divisor)
{
long mod;
Math.DivRem(dividend, divisor, out mod);
if(mod < 0)
mod += divisor;
return mod;
}
static void EnsureMinLEQMax(ref long min, ref long max)
{
if(min <= max)
return;
long temp = min;
min = max;
max = temp;
}
}
Here is a solution that leverages from the other answers using Random.NextBytes, but also pays careful attention to boundary cases. I've structured it as a set of extension methods. Also, I've accounted for modulo bias, by sampling another random number it falls out of range.
One of my gripes (at least for the situation I was trying to use it) is that the maximum is usually exclusive so if you want to roll a die, you do something like Random.Next(0,7). However, this means you can never get this overload to return the .MaxValue for the datatype (int, long, ulong, what-have-you). Therefore, I've added an inclusiveUpperBound flag to toggle this behavior.
public static class Extensions
{
//returns a uniformly random ulong between ulong.Min inclusive and ulong.Max inclusive
public static ulong NextULong(this Random rng)
{
byte[] buf = new byte[8];
rng.NextBytes(buf);
return BitConverter.ToUInt64(buf, 0);
}
//returns a uniformly random ulong between ulong.Min and Max without modulo bias
public static ulong NextULong(this Random rng, ulong max, bool inclusiveUpperBound = false)
{
return rng.NextULong(ulong.MinValue, max, inclusiveUpperBound);
}
//returns a uniformly random ulong between Min and Max without modulo bias
public static ulong NextULong(this Random rng, ulong min, ulong max, bool inclusiveUpperBound = false)
{
ulong range = max - min;
if (inclusiveUpperBound)
{
if (range == ulong.MaxValue)
{
return rng.NextULong();
}
range++;
}
if (range <= 0)
{
throw new ArgumentOutOfRangeException("Max must be greater than min when inclusiveUpperBound is false, and greater than or equal to when true", "max");
}
ulong limit = ulong.MaxValue - ulong.MaxValue % range;
ulong r;
do
{
r = rng.NextULong();
} while(r > limit);
return r % range + min;
}
//returns a uniformly random long between long.Min inclusive and long.Max inclusive
public static long NextLong(this Random rng)
{
byte[] buf = new byte[8];
rng.NextBytes(buf);
return BitConverter.ToInt64(buf, 0);
}
//returns a uniformly random long between long.Min and Max without modulo bias
public static long NextLong(this Random rng, long max, bool inclusiveUpperBound = false)
{
return rng.NextLong(long.MinValue, max, inclusiveUpperBound);
}
//returns a uniformly random long between Min and Max without modulo bias
public static long NextLong(this Random rng, long min, long max, bool inclusiveUpperBound = false)
{
ulong range = (ulong)(max - min);
if (inclusiveUpperBound)
{
if (range == ulong.MaxValue)
{
return rng.NextLong();
}
range++;
}
if (range <= 0)
{
throw new ArgumentOutOfRangeException("Max must be greater than min when inclusiveUpperBound is false, and greater than or equal to when true", "max");
}
ulong limit = ulong.MaxValue - ulong.MaxValue % range;
ulong r;
do
{
r = rng.NextULong();
} while(r > limit);
return (long)(r % range + (ulong)min);
}
}
private long randomLong()
{
Random random = new Random();
byte[] bytes = new byte[8];
random.NextBytes(bytes);
return BitConverter.ToInt64(bytes, 0);
}
This will get you a secure random long:
using (RNGCryptoServiceProvider rg = new RNGCryptoServiceProvider())
{
byte[] rno = new byte[9];
rg.GetBytes(rno);
long randomvalue = BitConverter.ToInt64(rno, 0);
}
Start at the minimum, add a random percentage of the difference between the min and the max. Problem with this is that NextDouble returns a number x such that 0 <= x < 1, so there's a chance you'll never hit the max number.
long randomLong = min + (long)(random.NextDouble() * (max - min));
Your randomLong will always be even and you will have eliminated even more values because you are very far away from the maximum for long, The maximum for long is 2^32 * max for int. You should use Random.NextBytes.
You can try CryptoRandom of the Inferno library:
public class CryptoRandom : Random
// implements all Random methods, as well as:
public byte[] NextBytes(int count)
public long NextLong()
public long NextLong(long maxValue)
public long NextLong(long minValue, long maxValue)
I wrote some Test Methods and check my own method and many of the answers from this and the same questions. Generation of redundant values is a big problem. I found #BlueRaja - Danny Pflughoeft answer at this address Is good enough and did not generate redundant values at least for first 10,000,000s. This is a Test Method:
[TestMethod]
public void TestRand64WithExtensions()
{
Int64 rnum = 0;
HashSet<Int64> hs = new HashSet<long>();
Random randAgent = new Random((int)DateTime.Now.Ticks);
for (int i = 0; i < 10000000; i++)
{
rnum = randAgent.NextLong(100000000000000, 999999999999999);
//Test returned value is greater than zero
Assert.AreNotEqual(0, rnum);
//Test Length of returned value
Assert.AreEqual(15, rnum.ToString().Length);
//Test redundancy
if (!hs.Contains(rnum)) { hs.Add(rnum); }
else
{
//log redundant value and current length we received
Console.Write(rnum + " | " + hs.Count.ToString());
Assert.Fail();
}
}
}
I didn't want to post this as an answer but I can't stuff this in the comment section and I didn't want to add as an edit to answer without author consent. So pardon me as this is not an independent answer and maybe just a prove to one of the answers.
I wrote a benchmarking C# console app that tests 5 different methods for generating unsigned 64-bit integers. Some of those methods are mentioned above. Method #5 appeared to consistently be the quickest. I claim to be no coding genius, but if this helps you, you're welcome to it. If you have better ideas, please submit. - Dave (sbda26#gmail.com)
enter code here
static private Random _clsRandom = new Random();
private const int _ciIterations = 100000;
static void Main(string[] args)
{
RunMethod(Method1);
RunMethod(Method2);
RunMethod(Method3);
RunMethod(Method4);
RunMethod(Method5);
Console.ReadLine();
}
static void RunMethod(Func<ulong> MethodX)
{
ulong ulResult;
DateTime dtStart;
TimeSpan ts;
Console.WriteLine("--------------------------------------------");
Console.WriteLine(MethodX.Method.Name);
dtStart = DateTime.Now;
for (int x = 1; x <= _ciIterations; x++)
ulResult = MethodX.Invoke();
ts = DateTime.Now - dtStart;
Console.WriteLine(string.Format("Elapsed time: {0} milliseconds", ts.TotalMilliseconds));
}
static ulong Method1()
{
int x1 = _clsRandom.Next(int.MinValue, int.MaxValue);
int x2 = _clsRandom.Next(int.MinValue, int.MaxValue);
ulong y;
// lines must be separated or result won't go past 2^32
y = (uint)x1;
y = y << 32;
y = y | (uint)x2;
return y;
}
static ulong Method2()
{
ulong ulResult = 0;
for(int iPower = 0; iPower < 64; iPower++)
{
double dRandom = _clsRandom.NextDouble();
if(dRandom > 0.5)
{
double dValue = Math.Pow(2, iPower);
ulong ulValue = Convert.ToUInt64(dValue);
ulResult = ulResult | ulValue;
}
}
return ulResult;
}
static ulong Method3() // only difference between #3 and #2 is that this one (#3) uses .Next() instead of .NextDouble()
{
ulong ulResult = 0;
for (int iPower = 0; iPower < 64; iPower++)
if (_clsRandom.Next(0, 1) == 1)
ulResult = ulResult | Convert.ToUInt64(Math.Pow(2, iPower));
return ulResult;
}
static ulong Method4()
{
byte[] arr_bt = new byte[8];
ulong ulResult;
_clsRandom.NextBytes(arr_bt);
ulResult = BitConverter.ToUInt64(arr_bt, 0);
return ulResult;
}
// Next method courtesy of https://stackoverflow.com/questions/14708778/how-to-convert-unsigned-integer-to-signed-integer-without-overflowexception/39107847
[System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Explicit)]
struct EvilUnion
{
[System.Runtime.InteropServices.FieldOffset(0)] public int Int32;
[System.Runtime.InteropServices.FieldOffset(0)] public uint UInt32;
}
static ulong Method5()
{
var evil = new EvilUnion();
ulong ulResult = 0;
evil.Int32 = _clsRandom.Next(int.MinValue, int.MaxValue);
ulResult = evil.UInt32;
ulResult = ulResult << 32;
evil.Int32 = _clsRandom.Next(int.MinValue, int.MaxValue);
ulResult = ulResult | evil.UInt32;
return ulResult;
}
}
I'll add my solution for generating random unsigned long integer (random ulong) below max value.
public static ulong GetRandomUlong(ulong maxValue)
{
Random rnd = new Random();
//This algorithm works with inclusive upper bound, but random generators traditionally have exclusive upper bound, so we adjust.
//Zero is allowed, function will return zero, as well as for 1. Same behavior as System.Random.Next().
if (maxValue > 0) maxValue--;
byte[] maxValueBytes = BitConverter.GetBytes(maxValue);
byte[] result = new byte[8];
int i;
for(i = 7; i >= 0; i--)
{
//senior bytes are either zero (then Random will write in zero without our help), or equal or below that of maxValue
result[i] = (byte)rnd.Next( maxValueBytes[i] + 1 );
//If, going high bytes to low bytes, we got ourselves a byte, that is lower than that of MaxValue, then lower bytes may be of any value.
if ((uint)result[i] < maxValueBytes[i]) break;
}
for(i--; i >= 0; i--) // I like this row
{
result[i] = (byte)rnd.Next(256);
}
return BitConverter.ToUInt64(result, 0);
}
C#10 now has long randoms built in.
Use NextInt64 if you can.
You're better off taking the difference between minimum and maximum (if it fits in an int), getting a random between 0 and that, and adding it to the minimum.
Is there anything wrong with using this simple approach?
long min = 10000000000001;
long max = 99999999999999;
Random random = new Random();
long randomNumber = min + random.Next() % (max - min);
d
My worked solution. Tested for 1000+ times:
public static long RandomLong(long min, long max)
{
return min + (long)RandomULong(0, (ulong)Math.Abs(max - min));
}
public static ulong RandomULong(ulong min, ulong max)
{
var hight = Rand.Next((int)(min >> 32), (int)(max >> 32));
var minLow = Math.Min((int)min, (int)max);
var maxLow = Math.Max((int)min, (int)max);
var low = (uint)Rand.Next(minLow, maxLow);
ulong result = (ulong)hight;
result <<= 32;
result |= (ulong)low;
return result;
}
How about generating bytes and converting to int64?
/* generate a byte array, then convert to unint64 */
var r = new Random(); // DONT do this for each call - use a static Random somewhere
var barray = new byte[64/8];
r.NextBytes(barray);
var rint64 = BitConverter.ToUInt64(barray, 0);
Sees to work for me (:
What's wrong with generating a double to be intended as a factor to be used to calculate the actual long value starting from the max value a long can be?!
long result = (long)Math.Round( random.NextDouble() * maxLongValue );
NextDouble generates a random number between [0.0, 0.99999999999999978] (msdn doc)
You multiply this random number by your maxLongValue.
You Math.Round that result so you can get the chance to get maxLongValue anyway (eg: simulate you got 1.0 from the NextDouble).
You cast back to long.