I have a node.js chat app that uses socket.io and other dependencies but I have no db currently.
I would like some advice on how to approach this and how to implement it, as I'm working on an app where when 2 users at some point have a matching message, they don’t have to have the matching message at the same time.
User1 might of said the word "apple" at the start of the chat and user2 might of said apple half through the chat. When this match occurs they both get points. I've been looking around for a good base chat app that might facilitate this but nothing I've found has met that.
Thanks.
This is how i could implement the whole game. Its clear that you will get the answers from both players. I guess that in your chat programm you'd have something of an identifier to check for the users ID and their chat session, so I'm not going to bother with this. I think that the implementation of sending a new "question" for their matching task shouldnt be a problem by just doing a simple
setTimeout(function(){ io.emit('newQuestion', {picture: randomPicture})}, 120000);
The string match should be another task. This could be easily done in javascript. As you are just trying to match complete strings, I would go about this as follows. Suppose you have a chat string1 and chat string2 corresponding to user1 and user2. From there you will need to cross reference all words in string1 with string2. For this string1 has to be broken apart.
var string1 = "apple half";
var string2 = "an apple";
var string1Split = string1.split(" ");
var match = false;
for(i=string1Split.length; i--;) {
result = str.match(/string1Split[i]/g);
if(result.length > 0 && match == false) {
match = true;
}
}
if(match) {
//add points to it
}
I'm not sure if you have to replace the quotes ' " ' in your string1Split[i] but thats just a matter of doing another regexp.
It sounds like you could use arrays to store the tags, as they are sent to the server. At the same time you could check those incoming tags against the other user's stored tags.
I'd recommend redis for something like this. It's super fast, and it doesn't seem like you need to store data for long periods of time or keep in any kind of complicated data structure.
Related
I'm building a chess-related application using nodejs. I've been trying to use chess.js as much as I can but I think I've hit a roadblock in terms of functionality. Before extending that functionality, I wanted to make sure that there wasn't another tool that can do what I need.
I'm looking for a way to convert a PGN string into a list of FEN moves. I was hoping to use load_pgn() in chess.js to load the moves into the object and then loop over each move and invoke the fen() function to output the current FEN. However, chess.js doesn't seem to have a way to walk through the moves in a game. Unless I'm missing something.
I'd rather not have to get into parsing strings, but will if I have to. Any suggestions?
Solution:
also see efirvida's answer below for a solution
Something like this (untested) seems to work. The function accepts a Chess object created with chess.js that already has a PGN loaded into it.
function getMovesAsFENs(chessObj) {
var moves = chessObj.history();
var newGame = new Chess();
var fens = [];
for (var i = 0; i < moves.length; i++) {
newGame.move(moves[i]);
fens.push(newGame.fen());
}
return fens;
}
Take a look to the github page .load_pgn link
var chess = new Chess();
pgn = ['[Event "Casual Game"]',
'[Site "Berlin GER"]',
'[Date "1852.??.??"]',
'[EventDate "?"]',
'[Round "?"]',
'[Result "1-0"]',
'[White "Adolf Anderssen"]',
'[Black "Jean Dufresne"]',
'[ECO "C52"]',
'[WhiteElo "?"]',
'[BlackElo "?"]',
'[PlyCount "47"]',
'',
'1.e4 e5 2.Nf3 Nc6 3.Bc4 Bc5 4.b4 Bxb4 5.c3 Ba5 6.d4 exd4 7.O-O',
'd3 8.Qb3 Qf6 9.e5 Qg6 10.Re1 Nge7 11.Ba3 b5 12.Qxb5 Rb8 13.Qa4',
'Bb6 14.Nbd2 Bb7 15.Ne4 Qf5 16.Bxd3 Qh5 17.Nf6+ gxf6 18.exf6',
'Rg8 19.Rad1 Qxf3 20.Rxe7+ Nxe7 21.Qxd7+ Kxd7 22.Bf5+ Ke8',
'23.Bd7+ Kf8 24.Bxe7# 1-0'];
chess.load_pgn(pgn.join('\n'));
// -> true
chess.fen()
// -> 1r3kr1/pbpBBp1p/1b3P2/8/8/2P2q2/P4PPP/3R2K1 b - - 0 24
something like
moves = chess.history();
var chess1 = new Chess();
for (move in moves){
chess1.move(move);
fen = chess1.fen()
}
(Not really an answer; just a comment that needs extra formatting.)
Your getMovesAsFENs function might also be written as:
function getMovesAsFENs(chessObj) {
return chessObj.history().map(function(move) {
chessObj.move(move);
return chessObj.fen();
});
}
Perhaps it doesn't matter to you, but this appeals to my sense of neatness.
Here is an end to end answer with some ES6 sugar added in:
const Chess = require('chess.js').Chess;
const chess1 = new Chess();
const chess2 = new Chess();
const startPos = chess2.fen();
const pgn = `1.e4 c5 0-1`;
chess1.load_pgn(pgn);
let fens = chess1.history().map(move => {
chess2.move(move);
return chess2.fen();
});
//the above technique will not capture the fen of the starting position. therefore:
fens = [startPos, ...fens];
//double checking everything
fens.forEach(fen => console.log(fen));
"However, chess.js doesn't seem to have a way to walk through the
moves in a game. Unless I'm missing something.".
You are right (as I have read the entire library multiple times by now). And everything that needs to see back into the history is basically undoing and then redoing the moves, without some sort of real navigation integrated (it is an interesting choice to have it solved this way, with the pros of it being lighting fast for some tasks, but with the cons of being a real pain for other seemingly easier tasks like the one you need).
Disclaimer (I wrote the following tool), I have been creating a tool (isepic-chess.js) for the past 5+ years, something similar to chess.js and I think it's slowly getting there... The library stores the moves history in an object with information like (fen, from_square, to_square, san, etc.), and also have some kind of "cursor" with the move index and some move-navigation helpers.
So with isepic-chess.js you can just call the board method board.fenHistoryExport() to get the FEN list after you parse the PGN game:
var example_pgn = `[Event "m1 London"]
[Site "?"]
[Date "1861.07.??"]
[Round "9"]
[White "Kolisch, Ignatz"]
[Black "Anderssen, Adolf"]
[Result "0-1"]
[Annotator "JvR"]
[SetUp "1"]
[FEN "5r1k/pp4pp/3r3q/8/3PpP1P/1P2NbP1/PB1Q3K/R7 b - - 0 30"]
[PlyCount "13"]
[EventDate "1861.??.??"]
30... Rxf4 $1 {Anderssen starts fireworks.} 31. Qe1 (31.gxf4 $2 Qxh4+ 32.Kg1
Rg6+) 31... Rg6 (31...Rxh4+ $1 32.gxh4 Rg6 $1) 32. Bc1 (32.Ng2 $1) 32... Rxh4+
$1 33. gxh4 Qf4+ 34. Kh3 Bg2+ $1 35. Nxg2 Qf3+ 36. Kh2 Qxg2# { Anderssen won
the match by this mate (+4, =2, -3).} 0-1`;
var board = Ic.initBoard({
pgn : example_pgn
});
console.log(board.fenHistoryExport());
There is a more complete node.js example in the README.md with the const {Ic} = require("isepic-chess"); import thingy to have it running in node.js.
I have a function that is meant to remove items from a Collection if a certain field does not pass a validation check (either email or phone, but that's not important in this context). Problem is that a regular expression is relatively slow, and I have lists of 1 million+ items.
My function
public HashSet<ListItemModel> RemoveInvalid(HashSet<ListItemModel> listItems)
{
string pattern = (this.phoneOrEmail == "email")//phoneOrEmail is set via config file
?
//RFC 5322 compliant email regex. see http://www.regular-expressions.info/email.html
#"[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*#(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?"
:
//north-american phone number regex. see http://stackoverflow.com/questions/12101125/regex-to-allow-only-digits-hypens-space-parentheses-and-should-end-with-a-dig
#"(?:\d{3}(?:\d{7}|\-\d{3}\-\d{4}))|(?:\(\d{3}\)(?:\-\d{3}\-)|(?: \d{3} )\d{4})";
Regex re = new Regex(pattern);
if (phoneOrEmail == "email")
{
return new HashSet<ListItemModel>(listItems.Where(x => re.IsMatch(x.Email,0)));
}
else
{
return new HashSet<ListItemModel>(listItems.Where(x => re.IsMatch(x.Tel, 0)));
}
}
This takes way too long to execute. Is there a faster way of returning a subset that contains only valid emails/phone numbers?
I need to come up with something that is lightning quick. My other operations usually take only a couple of seconds on 700k+ items, but this method is taking forever and I hate that. I will be experimenting with a series of LINQ .Contains(x,y,z) checks, but in the meantime, I'd like some input from people who are smarter than me.
I've using Kohana for a couple of weeks. One thing I noticed is that Kohana is missing eager loading (as far as I know). Let's say I have the following tables.
Subjects
id
name
Chapters
id
subject_id
name
Videos
id
chapter_id
name
When a user opens a subject page, I want to display all the chapters and videos. With ORM, I can do
$tutorials = ORM::factory('subject')->where('id','=', 1)->find();
foreach($tutorials as $tutorial)
{
$chapters = $tutorial->chapters->find_all();
foreach($chapters as $chapter)
{
$videos = $chapter->videos->find_all();
}
}
The above code is not efficient since it makes too many queries.
I thought about using join or database query builder, but both of them do not return a model object as their results. I also looked into with(), but it seems like it only works with one-to-one relationship.
using join on an ORM object returns an OPM object, but it doesn't return the data from the joining tables.
What would be my best option here? I would like to minimize # of queries and also want to get ORM objects a result. Whatever it would be, should return all the columns from tutorials, chapters, and videos.
First of all, your code is excess. ORM method find() returns 1 Model_Subject object. See
$chapters = ORM::factory('subject', 1)->chapters->find_all();
foreach($chapters as $chapter)
{
$videos = $chapter->videos->find_all();
}
With DB builder you can make just 2 requests. First get array of all chapters ids:
$chapters = DB::select('id')
->from('chapters')
->where('subject_id', '=', '1')
->execute()
->as_array(NULL, 'id');
Second - get all videos by ids as Model_Video object
$videos = DB::select('id')
->from('videos')
->where('chapter_id', 'IN', $chapters)
->as_object('Model_Video')
->execute()
->as_array();
So I guess you want something like this.
$videos = ORM::factory('Video')
->join(array('chapters', 'chapter'), 'LEFT')->on('video.chapter_id', '=', 'chapter.id')
->join(array('subjects', 'subject'), 'LEFT')->on('chapter.subject_id', '=', 'subject.id')
->where('subject.id', '=', $id)
->find_all();
Come to think of it, if the video belongs_to chapter belongs_to subject, try the following using with():
$videos = ORM::factory('Video')
->with('chapter:subject') // These are the names of the relationships. `:` is separator
// equals $video->chapter->subject;
->where('subject.id', '=', $id)
->find_all();
With things like this it often helps to think 'backwards'. You need the videos on that subject so start with the videos instead of the subject. :)
EDIT: The drawback of the second function is that it is going to preload all the data, it might be shorter to write but heavier on the server. I'd use the first one unless I need to know the subject and chapter anyway.
We are working on integrating Solr 3.6 to an eCommerce site. We have indexed data & search is performing really good.
We have some difficulties figuring how to use Predictive Search / Auto Complete Search Suggestion. Also interested to learn the best practices for implementing this feature.
Our goal is to offer predictive search similar to http://www.amazon.com/, but don't know how to implement it with Solr. More specifically I want to understand how to build those terms from Solr, or is it managed by something else external to solr? How the dictionary should be built for offering these kind of suggestions? Moreover, for some field, search should offer to search in category. Try typing "xper" into Amazon search box, and you will note that apart from xperia, xperia s, xperia p, it also list xperia s in Cell phones & accessories, which is a category.
Using a custom dictionary this would be difficult to manage. Or may be we don't know how to do it correctly. Looking to you to guide us on how best utilize solr to achieve this kind of suggestive search.
I would suggest you a couple of blogpost:
This one which shows you a really nice complete solution which works well but requires some additional work to be made, and uses a specific lucene index (solr core) for that specific purpose
I used the Highlight approach because the facet.prefix one is too heavy for big index, and the other ones had few or unclear documentation (i'm a stupid programmer)
So let's suppose the user has just typed "aaa bbb ccc"
Our autocomplete function (java/javascript) will call solr using the following params
q="aaa bbb"~100 ...base query, all the typed words except the last
fq=ccc* ...suggest word filter using last typed word
hl=true
hl.q=ccc* ...highlight word will be the one to suggest
fl=NONE ...return empty docs in result tag
hl.pre=### ...escape chars to locate highlight word in the response
hl.post=### ...see above
you can also control the number of suggestion with 'rows' and 'hl.fragsize' parameters
the highlight words in each document will be the right candidates for the suggestion with "aaa bbb" string
more suggestion words are the ones before/after the highlight words and, of course, you can implement more filters to extract valid words, avoid duplicates, limit suggestions
if interested i can send you some examples...
EDITED: Some further details about the approach
The portion of example i give supposes the 'autocomplete' mechanism given by jquery: we invoke a jsp (or a servlet) inside a web application passing as request param 'q' the words just typed by user.
This is the code of the jsp
ByteArrayInputStream is=null; // Used to manage Solr response
try{
StringBuffer queryUrl=new StringBuffer('putHereTheUrlOfSolrServer');
queryUrl.append("/select?wt=xml");
String typedWords=request.getParameter("q");
String base="";
if(typedWords.indexOf(" ")<=0) {
// No space typed by user: the 'easy case'
queryUrl.append("&q=text:");
queryUrl.append(URLEncoder.encode(typedWords+"*", "UTF-8"));
queryUrl.append("&hl.q=text:"+URLEncoder.encode(typedWords+"*", "UTF-8"));
} else {
// Space chars present
// we split the search in base phrase and last typed word
base=typedWords.substring(0,typedWords.lastIndexOf(" "));
queryUrl.append("&q=text:");
if(base.indexOf(" ")>0)
queryUrl.append("\""+URLEncoder.encode(base, "UTF-8")+"\"~1000");
else
queryUrl.append(URLEncoder.encode(base, "UTF-8"));
typedWords=typedWords.substring(typedWords.lastIndexOf(" ")+1);
queryUrl.append("&fq=text:"+URLEncoder.encode(typedWords+"*", "UTF-8"));
queryUrl.append("&hl.q=text:"+URLEncoder.encode(typedWords+"*", "UTF-8"));
}
// The additional parameters to control the solr response
queryUrl.append("&rows="+suggestPageSize); // Number of results returned, a parameter to control the number of suggestions
queryUrl.append("&fl=A_FIELD_NAME_THAT_DOES_NOT_EXIST"); // Interested only in highlights section, Solr return a 'light' answer
queryUrl.append("&start=0"); // Use only first page of results
queryUrl.append("&hl=true"); // Enable highlights feature
queryUrl.append("&hl.simple.pre=***"); // Use *** as 'highlight border'
queryUrl.append("&hl.simple.post=***"); // Use *** as 'highlight border'
queryUrl.append("&hl.fragsize="+suggestFragSize); // Another parameter to control the number of suggestions
queryUrl.append("&hl.fl=content,title"); // Look for result only in some fields
queryUrl.append("&facet=false"); // Disable facets
/* Omitted section: use a new URL(queryUrl.toString()) to get the solr response inside a byte array */
is=new ByteArrayInputStream(solrResponseByteArray);
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
Document doc = dBuilder.parse(is);
XPathFactory xPathfactory = XPathFactory.newInstance();
XPath xpath = xPathfactory.newXPath();
XPathExpression expr = xpath.compile("//response/lst[#name=\"highlighting\"]/lst/arr[#name=\"content\"]/str");
NodeList valueList = (NodeList) expr.evaluate(doc, XPathConstants.NODESET);
Vector<String> suggestions=new Vector<String>();
for (int j = 0; j < valueList.getLength(); ++j) {
Element value = (Element) valueList.item(j);
String[] result=value.getTextContent().split("\\*\\*\\*");
for(int k=0;k<result.length;k++){
String suggestedWord=result[k].toLowerCase();
if((k%2)!=0){
//Highlighted words management
if(suggestedWord.length()>=suggestedWord.length() && !suggestions.contains(suggestedWord))
suggestions.add(suggestedWord);
}else{
/* Words before/after highlighted words
we can put these words inside another vector
and use them if not enough suggestions */
}
}
}
/* Finally we build a Json Answer to be managed by our jquery function */
out.print(request.getParameter("json.wrf")+"({ \"suggestions\" : [");
boolean firstSugg=true;
for(String suggestionW:suggestions) {
out.print((firstSugg?" ":" ,"));
out.print("{ \"suggest\" : \"");
if(base.length()>0) {
out.print(base);
out.print(" ");
}
out.print(suggestionW+"\" }");
firstSugg=false;
}
out.print(" ]})");
}catch (Exception x) {
System.err.println("Exception during main process: " + x);
x.printStackTrace();
}finally{
//Gracefully close streams//
try{is.close();}catch(Exception x){;}
}
Hope to be helpfull,
Nik
This might help you out.I am trying to do the same.
http://solr.pl/en/2010/10/18/solr-and-autocomplete-part-1/
I run my bot in a public channel with hundreds of users. Yesterday a person came in and just abused it.
I would like to let anyone use the bot, but if they spam commands consecutively and if they aren't a bot "owner" like me when I debug then I would like to add them to an ignored list which expires in an hour or so.
One way I'm thinking would be to save all commands by all users, in a dictionary such as:
({
'meder#freenode': [{command:'.weather 20851', timestamp: 209323023 }],
'jack#efnet': [{command:'.seen john' }]
})
I would setup a cron job to flush this out every 24 hours, but I would basically determine if a person has made X number of commands in a duration of say, 15 seconds and add them to an ignore list.
Actually, as I'm writing this answer I thought of a better idea.. maybe instead of storing each users commands, just store the the bot's commands in a list and keep on pushing until it reaches a limit of say, 15.
lastCommands = [], limit = 5;
function handleCommand( timeObj, action ) {
if ( lastCommands.length < limit ) {
action();
} else {
// enumerate through lastCommands and compare the timestamps of all 5 commands
// if the user is the same for all 5 commands, and...
// if the timestamps are all within the vicinity of 20 seconds
// add the user to the ignoreList
}
}
watch_for('command', function() {
handleCommand({timestamp: 2093293032, user: user}, function(){ message.say('hello there!') })
});
I would appreciate any advice on the matter.
Here's a simple algorithm:
Every time a user sends a command to the bot, increment a number that's tied to that user. If this is a new user, create the number for them and set it to 1.
When a user's number is incremented to a certain value (say 15), set it to 100.
Every <period> seconds, run through the list and decrement all the numbers by 1. Zero means the user's number can be freed.
Before executing a command and after incrementing the user's counter, check to see if it exceeds your magic max value (15 above). If it does, exit before executing the command.
This lets you rate limit actions and forgive excesses after a while. Divide your desired ban length by the decrement period to find the number to set when a user exceeds your threshold (100 above). You can also add to the number if a particular user keeps sending commands after they've been banned.
Well Nathon has already offered a solution, but it's possible to reduce the code that's needed.
var user = {};
user.lastCommandTime = new Date().getTime(); // time the user send his last command
user.commandCount = 0; // command limit counter
user.maxCommandsPerSecond = 1; // commands allowed per second
function handleCommand(obj, action) {
var user = obj.user, now = new Date().getTime();
var timeDifference = now - user.lastCommandTime;
user.commandCount = Math.max(user.commandCount - (timeDifference / 1000 * user.maxCommandsPerSecond), 0) + 1;
user.lastCommandTime = now;
if (user.commandCount <= user.maxCommandsPerSecond) {
console.log('command!');
} else {
console.log('flooding');
}
}
var obj = {user: user};
var e = 0;
function foo() {
handleCommand(obj, 'foo');
e += 250;
setTimeout(foo, 400 + e);
}
foo();
In this implementation, there's no need for a list or some global callback every X seconds, instead we just reduce the commandCount every time there's a new message, based on time difference to the last command, it's also possible to allow different command rates for specific users.
All we need are 3 new properties on the user object :)
Redis
I would use the insanely fast advanced key-value store redis to write something like this, because:
It is insanely fast.
There is no need for cronjob because you can set expire on keys.
It has atomic operations to increment key
You could use redis-cli for prototyping.
I myself really like node_redis as redis client. It is a really fast redis client, which can easily be installed using npm.
Algorithme
I think my algorithme would look something like this:
For each user create a unique key which counts the commands consecutively executed. Also set expire to the time when you don't flag a user as spammer anymore. Let's assume the spammer has nickname x and the expire 15.
Inside redis-cli
incr x
expire x 15
When you do a get x after 15 seconds then the key does not exist anymore.
If value of key is bigger then threshold then flag user as spammer.
get x
These answers seem to be going the wrong way about this.
IRC Servers will disconnect your client regardless of whether you're "debugging" or not if the client or bot is flooding a channel or the server in general.
Make a blanket flood control, using the method #nmichaels has detailed, but on the bot's network connection to the server itself.