How to get a list of all documents that contain the search string in their document id in Firestore's collection? [duplicate] - node.js

Is it possible to query a firestore collection to get all document that starts with a specific string?
I have gone through the documentation but do not find any suitable query for this.

You can but it's tricky. You need to search for documents greater than or equal to the string you want and less than a successor key.
For example, to find documents containing a field 'foo' staring with 'bar' you would query:
db.collection(c)
.where('foo', '>=', 'bar')
.where('foo', '<', 'bas');
This is actually a technique we use in the client implementation for scanning collections of documents matching a path. Our successor key computation is called by a scanner which is looking for all keys starting with the current user id.

same as answered by Gil Gilbert.
Just an enhancement and some sample code.
use String.fromCharCode and String.charCodeAt
var strSearch = "start with text here";
var strlength = strSearch.length;
var strFrontCode = strSearch.slice(0, strlength-1);
var strEndCode = strSearch.slice(strlength-1, strSearch.length);
var startcode = strSearch;
var endcode= strFrontCode + String.fromCharCode(strEndCode.charCodeAt(0) + 1);
then filter code like below.
db.collection(c)
.where('foo', '>=', startcode)
.where('foo', '<', endcode);
Works on any Language and any Unicode.
Warning: all search criteria in firestore is CASE SENSITIVE.

Extending the previous answers with a shorter version:
const text = 'start with text here';
const end = text.replace(/.$/, c => String.fromCharCode(c.charCodeAt(0) + 1));
query
.where('stringField', '>=', text)
.where('stringField', '<', end);
IRL example
async function search(startsWith = '') {
let query = firestore.collection(COLLECTION.CLIENTS);
if (startsWith) {
const end = startsWith.replace(
/.$/, c => String.fromCharCode(c.charCodeAt(0) + 1),
);
query = query
.where('firstName', '>=', startsWith)
.where('firstName', '<', end);
}
const result = await query
.orderBy('firstName')
.get();
return result;
}

If you got here looking for a Dart/Flutter version
Credit to the java answer by Kyo
final strFrontCode = term.substring(0, term.length - 1);
final strEndCode = term.characters.last;
final limit =
strFrontCode + String.fromCharCode(strEndCode.codeUnitAt(0) + 1);
final snap = await FirebaseFirestore.instance
.collection('someCollection')
.where('someField', isGreaterThanOrEqualTo: term)
.where('someField', isLessThan: limit)
.get();

I found this, which works perfectly for startsWith
const q = query(
collection(firebaseApp.db, 'capturedPhotos'),
where('name', '>=', name),
where('name', '<=', name + '\uf8ff')
)

The above are correct! Just wanted to give an updated answer!
var end = s[s.length-1]
val newEnding = ++end
var newString = s
newString.dropLast(1)
newString += newEnding
query
.whereGreaterThanOrEqualTo(key, s)
.whereLessThan(key, newString)
.get()

Related

How to change code to prevent SQL injection in typeorm

I am writing code using nestjs and typeorm.
However, the way I write is vulnerable to SQL injection, so I am changing the code.
//Before
.where(`userId = ${userId}`)
//After
.where(`userId = :userId` , {userId:userId})
I am writing a question because I was changing the code and couldn't find a way to change it for a few cases.
//CASE1
const query1 = `select id, 'normal' as type from user where id = ${userId}`;
const query2 = `select id, 'doctor' as type from doctor where id = ${userId}`;
const finalQuery = await getConnection().query(`select id, type from (${query1} union ${query2}) as f limit ${limit} offset ${offset};`);
//CASE2
...
.addSelect(`CASE WHEN userRole = '${userRole}' THEN ...`, 'userType')
...
//CASE3 -> to find search results in order of accuracy
...
.orderBy(`((LENGTH(user.name) - LENGTH((REPLACE(user.name, '${keyword.replace( / /g, '', )}', '')))) / LENGTH('${keyword.replace(/ /g, '')}'))`,
'ASC')
...
//CASE4
let query = 'xxxx';
let whereQuery = '';
for(const i=0;i<5;i++)
whereQuery += ' or ' + `user.name like '%${keyword}%'`
query.where(whereQuery)
I cannot use parameter in the select function.
In the above case, I am wondering how to change it.
Is it ok to not have to modify the select code?

Is there a way to replace several occurences in a HTML file in one line?

Please can you tell me how to make shorter this code. i wanna do it in one line.
var htmlstring = "{{1}}Hello {{2}}, clic here !";
var firststep = htmlstring.replace('{{1}}', "http://google.fr");
var secondstep = htmlstring.replace('{{1}}', "http://google.fr");
var thirdstep = secondstep.replacee('{{2}}', "Mister");
In short,
I have it:
{{1}}Hello {{2}}, clic here !
I wanna have this at the end:
http://google.frHello Mister, clic here !"
Sure, use String.replace and regexp:
const html = '{{1}}Hello {{2}}, click here !'
const url = 'http://google.fr';
const name = 'Mister';
const output = html.replace(/\{\{1\}\}/g, url).replace(/\{\{2\}\}/g, name);
I have no Idea how you can replace both {{1}} and {{2}} in one go, but you can replace both occurences of {{1}} by using this function:
String.prototype.replaceAll = function(search, replacement) {
var target = this;
return target.replace(new RegExp(search, 'g'), replacement);
};
Use it like this:
var htmlstring = '{{1}}Hello {{2}}, clic here !';
htmlstring.replaceAll("{{1}}", "http://google.fr");
htmlstring.replaceAll("{{2}}", "Mister");
ES6 Update:
You can now use an even better function for the replaceAll() method:
String.prototype.replaceAll = function(search, replacement) {
return this.split(search).join(replacement);
};
This will utilize .split() to first make an array without the search value and then .join() that Array with the replacement:
String.prototype.replaceAll = function(search, replacement) {return this.split(search).join(replacement);};
let htmlstring = `{{1}}Hello {{2}}, click here !`;
console.log(htmlstring.replaceAll("{{1}}","https://google.fr").replaceAll("{{2}}", "Mister"));

How to search records in all directory except one in Marklogic

I am trying to search XML's in my database using marklogic-java api with following restrictions :
1) A particular XML tag must contain my given value for ex : tradeId must equal to what I pass
2) Results must lie in collections provided by me
3) Results can lie anywhere in marklogic database except in one particular directory
I don't have the solution to point 3. Few of the records meeting my first two criteria have a URI starting with /TRADES/* and so i want to search everywhere except under directory "/TRADES".
This is how my code looks :
public static List<DocumentPage> getResults() throws KeyManagementException, NoSuchAlgorithmException,
KeyStoreException, CertificateException, IOException {
String tradeId = "XYZ";
DatabaseClient client = getDBClient();
QueryManager queryMgr = client.newQueryManager();
XMLDocumentManager xmlMngr = client.newXMLDocumentManager();
StringHandle rawHandle = new StringHandle();
rawHandle.withFormat(Format.XML).set(getQueryToFetchMessagesByTradeId(tradeId));
RawQueryByExampleDefinition querydef = queryMgr.newRawQueryByExampleDefinition(rawHandle);
querydef.setCollections("/messages/processed", /messages/toBeProcessed");
querydef.setDirectory("/");
return getDocumentPageList(querydef, client);
}
private static String getQueryToFetchMessagesByTradeId(String tradeId) {
String query = "<q:qbe xmlns:q=\"http://marklogic.com/appservices/querybyexample\">\n" + "<q:query>\n<q:word>"
+ "<tradeId tradeIdScheme=" + "\"http://www.abcd.com/internalid/trade-id\"" + ">" + tradeId
+ "</tradeId></q:word>\n" + "</q:query>\n" + "</q:qbe>";
return query;
}
Any help is highly appreciated.
I don't see an option to compose metadata (directory) queries in query by example syntax. So you'll have to use Structured Queries instead. I find them more readable in Java anyway since they don't require string concatenation. Use StructuredQueryBuilder like so:
StructuredQueryBuilder qb = new StructuredQueryBuilder();
StructuredQueryDefinition querydef =
qb.and(
qb.containerQuery(
qb.element(new QName("http://www.abcd.com/internalid/trade-id", "tradeId")),
qb.and(
qb.term( tradeId ),
qb.value(
qb.elementAttribute(
qb.element(new QName("http://www.abcd.com/internalid/trade-id", "tradeId")),
qb.attribute("tradeIdScheme")
),
"http://www.abcd.com/internalid/trade-id"
)
)
),
qb.not(
qb.directory(1, "/TRADES/")
)
);
querydef.setCollections("/messages/processed", "/messages/toBeProcessed")
return getDocumentPageList(querydef, client);

Distinct values in Azure Search Suggestions?

I am offloading my search feature on a relational database to Azure Search. My Products tables contains columns like serialNumber, PartNumber etc.. (there can be multiple serialNumbers with the same partNumber).
I want to create a suggestor that can autocomplete partNumbers. But in my scenario I am getting a lot of duplicates in the suggestions because the partNumber match was found in multiple entries.
How can I solve this problem ?
The Suggest API suggests documents, not queries. If you repeat the partNumber information for each serialNumber in your index and then suggest based on partNumber, you will get a result for each matching document. You can see this more clearly by including the key field in the $select parameter. Azure Search will eliminate duplicates within the same document, but not across documents. You will have to do that on the client side, or build a secondary index of partNumbers just for suggestions.
See this forum thread for a more in-depth discussion.
Also, feel free to vote on this UserVoice item to help us prioritize improvements to Suggestions.
I'm facing this problem myself. My solution does not involve a new index (this will only get messy and cost us money).
My take on this is a while-loop adding 'UserIdentity' (in your case, 'partNumber') to a filter, and re-search until my take/top-limit is met or no more suggestions exists:
public async Task<List<MachineSuggestionDTO>> SuggestMachineUser(string searchText, int take, string[] searchFields)
{
var indexClientMachine = _searchServiceClient.Indexes.GetClient(INDEX_MACHINE);
var suggestions = new List<MachineSuggestionDTO>();
var sp = new SuggestParameters
{
UseFuzzyMatching = true,
Top = 100 // Get maximum result for a chance to reduce search calls.
};
// Add searchfields if set
if (searchFields != null && searchFields.Count() != 0)
{
sp.SearchFields = searchFields;
}
// Loop until you get the desired ammount of suggestions, or if under desired ammount, the maximum.
while (suggestions.Count < take)
{
if (!await DistinctSuggestMachineUser(searchText, take, searchFields, suggestions, indexClientMachine, sp))
{
// If no more suggestions is found, we break the while-loop
break;
}
}
// Since the list might me bigger then the take, we return a narrowed list
return suggestions.Take(take).ToList();
}
private async Task<bool> DistinctSuggestMachineUser(string searchText, int take, string[] searchFields, List<MachineSuggestionDTO> suggestions, ISearchIndexClient indexClientMachine, SuggestParameters sp)
{
var response = await indexClientMachine.Documents.SuggestAsync<MachineSearchDocument>(searchText, SUGGESTION_MACHINE, sp);
if(response.Results.Count > 0){
// Fix filter if search is triggered once more
if (!string.IsNullOrEmpty(sp.Filter))
{
sp.Filter += " and ";
}
foreach (var result in response.Results.DistinctBy(r => new { r.Document.UserIdentity, r.Document.UserName, r.Document.UserCode}).Take(take))
{
var d = result.Document;
suggestions.Add(new MachineSuggestionDTO { Id = d.UserIdentity, Namn = d.UserNamn, Hkod = d.UserHkod, Intnr = d.UserIntnr });
// Add found UserIdentity to filter
sp.Filter += $"UserIdentity ne '{d.UserIdentity}' and ";
}
// Remove end of filter if it is run once more
if (sp.Filter.EndsWith(" and "))
{
sp.Filter = sp.Filter.Substring(0, sp.Filter.LastIndexOf(" and ", StringComparison.Ordinal));
}
}
// Returns false if no more suggestions is found
return response.Results.Count > 0;
}
public async Task<List<string>> SuggestionsAsync(bool highlights, bool fuzzy, string term)
{
SuggestParameters sp = new SuggestParameters()
{
UseFuzzyMatching = fuzzy,
Top = 100
};
if (highlights)
{
sp.HighlightPreTag = "<em>";
sp.HighlightPostTag = "</em>";
}
var suggestResult = await searchConfig.IndexClient.Documents.SuggestAsync(term, "mysuggestion", sp);
// Convert the suggest query results to a list that can be displayed in the client.
return suggestResult.Results.Select(x => x.Text).Distinct().Take(10).ToList();
}
After getting top 100 and using distinct it works for me.
You can use the Autocomplete API for that where does the grouping by default. However, if you need more fields together with the result, like, the partNo plus description it doesn't support it. The partNo will be distinct though.

How to get the attribute value of xml using LINQ to XML?

I have the following xml schema.
<Rooms>
<Room RoomNumber="room1" EMAIL="ssds#dsfd.com" dsfdd=""/>
<Room RoomNumber="room2" EMAIL="ssds#sdd.com" dsfdd=""/>
</Rooms>
I have to return Email address based on the input(input to program is room number).
How i can achieve this using LINQ to XML?
Try this:
var xml = XElement.Parse("<Rooms>"+
"<Room RoomNumber=\"room1\" EMAIL=\"ssds#dsfd.com\" dsfdd=\"\"/>"+
"<Room RoomNumber=\"room2\" EMAIL=\"ssds#sdd.com\" dsfdd=\"\"/>"+
"</Rooms>");
string room = "room1"; //input
var email = xml.Elements("Room")
.Where(c => c.Attribute("RoomNumber").Value == room)
.Select(c => c.Attribute("EMAIL").Value).FirstOrDefault();
var doc = XDocument.Load(myXmlFilePath);
// or doc = XDocument.Parse(myXmlString);
string roomNumber = "room1";
var emailQuery = from room in doc.Root.Elements("Room")
where (string)room.Attribute("RoomNumber") == roomNumber
select (string)room.Attribute("EMAIL");
Then, with a query like that you can get results:
// if there is always only one <Room> with given roomNumber
var email = emailQuery.First();
// otherwise
var emails = emailQuery.ToList();

Resources