I'm trying to solve a simple TSP with Jsprit. I only have 1 vehicle and a total of about 200 pickups.
This is my vehicle routing algorithm code:
val vrpBuilder = VehicleRoutingProblem.Builder.newInstance()
vrpBuilder.setFleetSize(VehicleRoutingProblem.FleetSize.FINITE)
vrpBuilder.setRoutingCost(costMatrix)
vrpBuilder.addAllVehicles(vehicles)
vrpBuilder.addAllJobs(pickups)
val vrp = vrpBuilder.build()
val builder = Jsprit.Builder.newInstance(vrp)
val stateManager = StateManager(vrp)
val constraintManager = ConstraintManager(vrp, stateManager)
constraintManager.addConstraint(object : HardActivityConstraint {
override fun fulfilled(
context: JobInsertionContext?,
prevActivity: TourActivity?,
newActivity: TourActivity?,
nextActivity: TourActivity?,
previousActivityDepartureTime: Double
): HardActivityConstraint.ConstraintsStatus {
if (prevActivity !== null && newActivity !== null && nextActivity !== null && context !== null) {
if (prevActivity.location.id === nextActivity.location.id) {
return HardActivityConstraint.ConstraintsStatus.FULFILLED
}
val distanceBetweenPrevAndNew = costMatrix.getDistance(
prevActivity.location,
newActivity.location,
prevActivity.endTime,
context.newVehicle
)
val distanceBetweenPrevAndNext = costMatrix.getDistance(
prevActivity.location,
nextActivity.location,
prevActivity.endTime,
context.newVehicle
)
if (distanceBetweenPrevAndNext > distanceBetweenPrevAndNew) {
return HardActivityConstraint.ConstraintsStatus.FULFILLED
} else {
return HardActivityConstraint.ConstraintsStatus.NOT_FULFILLED
}
}
return HardActivityConstraint.ConstraintsStatus.FULFILLED
}
}, ConstraintManager.Priority.CRITICAL)
builder.setProperty(Jsprit.Parameter.FAST_REGRET, true.toString())
builder.setProperty(Jsprit.Parameter.CONSTRUCTION, Construction.REGRET_INSERTION.toString())
builder.setProperty(Jsprit.Parameter.THREADS, threads.toString())
builder.setStateAndConstraintManager(stateManager, constraintManager)
val vra = builder.buildAlgorithm()
As routing cost i'm using a distance only FastVehicleRoutingTransportCostsMatrix built with the Graphhopper Matrix Api.
The vehicle should avoid passing in front of an uncollected pickup, I tried to set up an hard activity constraint that checks if the new inserted activity is further away than the next one. If it's not further away, the constraint is not fulfilled. However the constraint does not work quite well.
Here's an example of an optimized route with the constraint:
example
The correct order for my case should be: 46 43 45 44 , however Jsprit orders them in that way because after 44 the vehicle has to do an uturn and run through the street again to reach 47,48,49...
I'm not sure if setting up a constraint is the right way to solve this, do you have any advices?
Thanks
Related
I am trying to find a way to iterate all objects from a large collection of data in Firebase Database.
My best attempt follows but I found it odd for several reasons:
startAt() values are always inclusive. So after fetching 100 elements, I had to use my last fetched key as an argument to startAt which results in the last item being fetched again
DataSnapshot's forEach method doesn't allow a callback with an index count as you would think it would based on JS's standards so I had to create a manual index - not sure it will work in every case as i'm not sure if forEach works perfectly synchronously
Here is my code, given the assumption my collection is located at users.
const mapAllTripsPaginated = function (database, childSnapshotCallback, start = '', limit = 100, totalNb = 0) {
return database.ref('/users').orderByKey().startAt(start).limitToFirst(limit).once('value').then((snapshot) => {
let childrenPromises = []
let lastChildKey = null
let idx = 0
snapshot.forEach((childSnapshot) => {
lastChildKey = childSnapshot.key
if (start !== '' && idx === 0) {
// console.log(`Skipping ${childSnapshot.key} as 1st element of page`)
} else {
childrenPromises.push(childSnapshotCallback(childSnapshot))
}
idx = idx + 1
})
return Promise.all(childrenPromises)
.then((result) => {
let newTotal = totalNb + result.length
if (snapshot.numChildren() === limit) {
console.log(`Paginating from ${lastChildKey}`)
return mapAllTripsPaginated(database, childSnapshotCallback, start = lastChildKey, limit = limit, totalNb = newTotal)
} else {
// Done paginating
return newTotal
}
})
})
}
Any idea on how I could make this method more elegant?
Firebase queries are inclusive both for their start and end conditions. You will indeed have to deduplicate the overlapping item on the client.
Firebase's Snapshot.forEach() is a synchronous operation.
I'd normally deduplicate based on already having the key of the item. That will also remove the need for the idx counter.
snapshot.forEach((childSnapshot) => {
if (lastChildKey !== childSnapshot.key) {
childrenPromises.push(childSnapshotCallback(childSnapshot))
}
lastChildKey = childSnapshot.key
})
I have a customzied filter function writtern as:
def filterSpiderBots(inputDF: DataFrame, whitelistMatcher: Matcher) {
val filterFunc = udf(
(ua:String) => ua == null || whitelistMatcher.matches(ua)
)
inputDF.filter(filterFunc($"ua"))
}
The matcher reads from
FileInputStream(SparkFiles.get("iab-whitelist.txt"))
I made sure the Match is not null, and I have a check for null ua in place.
but when I start the program, I still get NullPointerException. I don't know what is the cause of it..
Update:
after not use matcher, i still get the same error
def filterSpiderBots(inputDF: DataFramer) {
val filterFunc = udf(
(ua:String) => ua == null
)
inputDF.filter(filterFunc($"ua"))
}
filter() preserves records that evaluate to true .. and in your case, all the null records evaluate to true. So, change your filterFunc to "ua != null" ?
I am using spark to read and analyse a data file, file contains data like following:
1,unit1,category1_1,100
2,unit1,category1_2,150
3,unit2,category2_1,200
4,unit3,category3_1,200
5,unit3,category3_2,300
The file contains around 20 million records. If user input unit or category, spark need filter the data by inputUnit or inputCategory.
Solution 1:
sc.textFile(file).map(line => {
val Array(id,unit,category,amount) = line.split(",")
if ( (StringUtils.isNotBlank(inputUnit) && unit != inputUnit ) ||
(StringUtils.isNotBlank(inputCategory) && category != inputCategory)){
null
} else {
val obj = new MyObj(id,unit,category,amount)
(id,obj)
}
}).filter(_!=null).collectAsMap()
Solution 2:
var rdd = sc.textFile(file).map(line => {
val (id,unit,category,amount) = line.split(",")
(id,unit,category,amount)
})
if (StringUtils.isNotBlank(inputUnit)) {
rdd = rdd.filter(_._2 == inputUnit)
}
if (StringUtils.isNotBlank(inputCategory)) {
rdd = rdd.filter(_._3 == inputCategory)
}
rdd.map(e => {
val obj = new MyObject(e._1, e._2, e._3, e._4)
(e._1, obj)
}).collectAsMap
I want to understand, which solution is better, or both of them are poor? If both are poor, how to make a good one? Personally, I think second one is better, but I am not quite sure whether it is nice to declare a rdd as var... (I am new to Spark, and I am using Spark 1.5.0 and Scala 2.10.4 to write the code, this is my first time asking a question in StackOverFlow, feel free to edit if it is not well formatted) Thanks.
I would like to write a method for querying table with one method by null cheking parameters using SqlExpressionVisitor of Ormlite
Here is my method :
public static List<UserChatsDTO> GetUserChats(int startRow, int rowCount, DateTime? startDate, DateTime? endDate, string operatorName, short? rating, string visitorName)
{
using (IDbConnection db = DbFactory.OpenDbConnection())
{
SqlExpressionVisitor<UserChatsDTO> ev = OrmLiteConfig.DialectProvider.ExpressionVisitor<UserChatsDTO>();
ev.Where(q =>
(startDate.HasValue && q.Dated >= startDate) &&
(endDate.HasValue && q.Dated <= endDate) &&
(!string.IsNullOrEmpty(operatorName) && q.TakenByUser.Contains(operatorName)) &&
(rating.HasValue && q.Rating == (short)rating) &&
(!string.IsNullOrEmpty(visitorName) && q.VisitorName.Contains(visitorName)));
//ev.OrderBy();
ev.Limit(startRow, rowCount);
return db.Select<UserChatsDTO>(ev);
}
}
But Object reference not set to an instance of an object. NullReferenceException is thrown when i call ev.Where part.
Is there a bug here or i am missing something ?
Thank you.
You can actually build up the ExpressionVisitor inside the Select method like so:
var chats = db.Select<UserChatsDTO>(q => q
.Where(x => startDate.HasValue && x.Date >= startDate)
.Where(x => endDate.HasValue && x.Date <= endDate)
.Where(x => string.IsNullOrEmpty(operatorName) || x.TakeByUser.Contains(operatorName))
.Where(x => rating.HasValue && x.Rating == (short)rating)
.Where(x => string.IsNullOrEmpty(visitorName) || x.VisitorName.Contains(visitorName)
.Limit(startRow, rowCount));
I know this question is 7 months old but I had a similar issue & this was the first question that came up when I searched. I wanted to add my working solution since Master Morality's didn't work for me.
Originally, I tried syntax roughly similar to mustafasturan's first attempt. I got the same NullReferenceException he did. Master Morality's answer did not help either...
I'm trying to build a search function that performs LIKE searches rather than exact match. There are multiple criteria on the request object which may or may not be null (for simplicity's sake we'll use an example with 2 criteria). Following Master Morality's suggestion, I tried this:
var searchResults = db.Select<Item>(q => q
.Where(x => string.IsNullOrWhiteSpace(request.Criteria1) || x.Criteria1.Contains(request.Criteria1))
.Where(x => string.IsNullOrWhiteSpace(request.Criteria2) || x.Criteria2.Contains(request.Criteria2))
);
I still got a NullReferenceException. Seems like the && and || operators are not using short-circuit evaluation inside the lambda expression.
What I finally had to go with is this:
SqlExpressionVisitor<Item> ev = new ServiceStack.OrmLite.MySql.MySqlExpressionVisitor<Item>();
if (!String.IsNullOrWhiteSpace(request.Criteria1)) {
ev.Where(q => q.Criteria1.Contains(request.Criteria1));
}
if (!String.IsNullOrWhiteSpace(request.Criteria2)) {
ev.Where(q => q.Criteria2.Contains(request.Criteria2));
}
searchResults = db.Select<Item>(ev);
It doesn't feel very elegant, but it's the only thing I could find that works.
Following a couple different tutorials, I've been trying to build a "Silverlight Business Application" against a database I've created. I've found I have two problems. The one I'm asking about here is how to filter the query.
The query that is built in the DomainService is when using the VS2010 template is:
[EnableClientAccess]
public class ChargesService : LinqToEntitiesDomainService<ChargesEntities>
{
public IQueryable<tblChargeCode> GetCharges()
{
return ObjectContext.tblChargeCodes.OrderBy(e => e.Chrgs_Code_01).Take(10);
}
}
I'm trying to create another query against the same ObjectContext.tblChargeCodes. Pulling the entire table (30 columns by ~7k rows) creates a timeout error.
I can't figure out how to do a select. I want to select Charge_Codes_01 and Bill_Description with a "starts with" type functionality (dynamic drop down search functionality). I've tried different variations of this without success. Something just isn't clicking in my brain.
public IQueryable<tblChargeCode> SearchCharges(string num)
{
var min = System.Convert.ToInt32(num.PadRight(7, '0'));
var max = System.Convert.ToInt32(num.PadRight(7, '9'));
return ObjectContext.tblChargeCodes
.Select(e => e.Chrgs_Code_01, e.Chrgs_Code_01_Desc)
.Where(e => e.Chrgs_Code_01 >= min && e.Chrgs_Code_01 <= max)
.OrderBy(e => e.Chrgs_Code_01)
.Take(10);
}
(sorry for my bad english)
The problem with you code is the "Select" - your method signature says that it must return a IQueryable of tblChargeCode, so you cannot return a projection. Here is two ways you can write your query:
In the server:
public IQueryable<tblChargeCode> SearchCharges(int min, int max, string description)
{
return ObjectContext.tblChargeCodes
.Where(e => e.Chrgs_Code_01 >= min && e.Chrgs_Code_01 <= max)
.Where(e => e.Bill_Description.StartsWith(description))
.OrderBy(e => e.Chrgs_Code_01)
.Take(10);
}
And call it on the client:
context.Load(context.SearchChargesQuery(0, 9999999, "Bill"), (op) =>
{
//op.Entities has your entities loaded
}, null);
Or you can just leave the query on the server:
public IQueryable<tblChargeCode> GetCharges()
{
return ObjectContext.tblChargeCodes.OrderBy(e => e.Chrgs_Code_01);
}
And call it from the client (it will filter on the server)
context.Load(context.GetChargesQuery().Where(e => e.Chrgs_Code_01 >= 0 && e.Chrgs_Code_01 <= 9999999)
.Where(e => e.Bill_Description.StartsWith("Bill"))
.OrderBy(e => e.Chrgs_Code_01)
.Take(10), (op) =>
{
//op.Entities has your entities loaded
}, null);
You also can use "Contains" instead of "StartsWith" in your query.