(Kinda solved) Solr search with special characters - search

I'm trying to make symbols in searches not relevant.
If we see the following field value has a "perr" Token at indexation, as well as the value in query.
Why isn't a query like this returning anything? (extra fields remove for brevety)
<lst name="params">
<str name="q">per-ro</str>
<str name="qt">ri_dismax_mixed_es</str>
<str name="indent">true</str>
<str name="wt">xml</str>
<str name="debugQuery">true</str>
<str name="_">1651060917537</str>
</lst>
...
<result name="response" numFound="0" start="0">
</result>
<lst name="debug">
<lst name="queryBoosting">
<str name="q">per#ro</str>
<null name="match"/>
</lst>
<str name="rawquerystring">per-ro</str>
<str name="querystring">per-ro</str>
<str name="parsedquery">(+DisjunctionMaxQuery((... | exactLoose_es_PRODUCTO_DESCRIPCION_DETALLADA_es_str:"per (ro perr)"^8.0 | ... </str>
<str name="parsedquery_toString">+(... | exactLoose_es_PRODUCTO_DESCRIPCION_DETALLADA_es_str:"per (ro perr)"^8.0 | ...</str>
<lst name="explain"/>
<str name="QParser">DisMaxQParser</str>
QT ri_dismax_mixed_es (extra fields remove for brevety)
<requestHandler class="solr.SearchHandler" name="ri_dismax_mixed_es" >
<lst name="defaults">
<str name="facet.method">enum</str>
<str name="defType">dismax</str>
<str name="echoParams">explicit</str>
<int name="rows">20</int>
<float name="tie">0.1</float>
<str name="qf">... exactLoose_es_PRODUCTO_DESCRIPCION_DETALLADA_es_str^8 ...</str>
<str name="pf">... exactLoose_es_PRODUCTO_DESCRIPCION_DETALLADA_es_str^8 ...</str>
<int name="ps">100</int>
<str name="fl">...</str>
<str name="mm">1</str>
<str name="facet">true</str>
<int name="facet.limit">-1</int>
<int name="facet.mincount">1</int>
<str name="sort">score desc</str>
<str name="q.alt">*:*</str>
</lst>
<lst name="appends">
<str name="facet.field">hierarchy_es_front</str>
...
</lst>
<arr name="last-components">
<str>elevator_es</str>
</arr>
</requestHandler>

Related

Get a Buried xml int inside node using vBA

Evening all,
I have had some help here before and hope that you can help again.
Previously I was trying to extract two bits of information from the following xml - which I have tagged in the xml. From the help on these boards I have worked up this code that extracts this.
I have since needed to extract a third bit of information which corresponds to Data Item 2 - i.e. it's color but I've been trying to use the same method but keep getting an error saying object doesn't support this. I've done a lot of googling and testing before coming back but just can't get it to work. Any help/advice/solution would be really appreciated.
Working code for Data Item 1 and 2:
Sub ReadXML()
Call fnReadXMLByTags
End Sub
Function fnReadXMLByTags()
Dim mainWorkBook As Workbook
Set mainWorkBook = ActiveWorkbook
mainWorkBook.Sheets("Sheet1").Range("A:A").Clear
Set oXMLFile = CreateObject("Microsoft.XMLDOM")
XMLFileName = "C:\Users\xxx\Documents\TestFile.xml"
oXMLFile.Load (XMLFileName)
Set slotNodes = oXMLFile.SelectNodes("/instrument/member/list/obj/member/string")
End Function
XML with Tags for Data 1,2 and data 3 which I can't reach
<?xml version="1.0"?>
<instrument>
<string name="name" value="TEST" wide="true"/> <!--I GETTHIS FIRST DATA 1-->
<member name="slotvisuals">
<int name="ownership" value="1"/>
<list name="obj" type="obj">
<obj class="USlotVisuals" ID="882527840">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="CUSTOM ART1" wide="true"/>
<string name="description" value="CUSTOM ART1 DESCRIPTION" wide="true"/>
<int name="group" value="0"/>
</obj>
<obj class="USlotVisuals" ID="45186017184">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="GROUP 2" wide="true"/>
<string name="description" value="GROUP 2" wide="true"/>
<int name="group" value="1"/>
</obj>
<obj class="USlotVisuals" ID="882712304">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="GROUP 3" wide="true"/>
<string name="description" value="GROUP 3" wide="true"/>
<int name="group" value="2"/>
</obj>
<obj class="USlotVisuals" ID="44402087248">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="GROUP 4" wide="true"/>
<string name="description" value="GROUP 4" wide="true"/>
<int name="group" value="3"/>
</obj>
</list>
</member>
<member name="slots">
<int name="ownership" value="1"/>
<list name="obj" type="obj">
<obj class="PSoundSlot" ID="2271687808">
<obj class="PSlotThruTrigger" name="remote" ID="2276282784">
<int name="status" value="144"/>
<int name="data1" value="0"/>
</obj>
<obj class="PSlotMidiAction" name="action" ID="2268706176">
<int name="version" value="600"/>
<member name="noteChanger">
<int name="ownership" value="1"/>
<list name="obj" type="obj">
<obj class="PSlotNoteChanger" ID="2277009712">
<int name="channel" value="-1"/>
<float name="velocityFact" value="1"/>
<float name="lengthFact" value="1"/>
<int name="minVelocity" value="0"/>
<int name="maxVelocity" value="127"/>
<int name="transpose" value="0"/>
<int name="minPitch" value="0"/>
<int name="maxPitch" value="127"/>
</obj>
</list>
</member>
<member name="midiMessages">
<int name="ownership" value="1"/>
</member>
<int name="channel" value="-1"/>
<float name="velocityFact" value="1"/>
<float name="lengthFact" value="1"/>
<int name="minVelocity" value="0"/>
<int name="maxVelocity" value="127"/>
<int name="transpose" value="0"/>
<int name="maxPitch" value="127"/>
<int name="minPitch" value="0"/>
<int name="key" value="-1"/>
</obj>
<member name="sv">
<int name="ownership" value="2"/>
<list name="obj" type="obj">
<obj class="USlotVisuals" ID="909444640">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="CUSTOM ART1" wide="true"/>
<string name="description" value="CUSTOM ART1 DESCRIPTION" wide="true"/>
<int name="group" value="0"/>
</obj>
<obj class="USlotVisuals" ID="43544745280">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="GROUP 2" wide="true"/>
<string name="description" value="GROUP 2" wide="true"/>
<int name="group" value="1"/>
</obj>
<obj class="USlotVisuals" ID="44505641184">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="GROUP 3" wide="true"/>
<string name="description" value="GROUP 3" wide="true"/>
<int name="group" value="2"/>
</obj>
<obj class="USlotVisuals" ID="44402092688">
<int name="displaytype" value="1"/>
<int name="articulationtype" value="0"/>
<int name="symbol" value="73"/>
<string name="text" value="GROUP 4" wide="true"/>
<string name="description" value="GROUP 4" wide="true"/>
<int name="group" value="3"/>
</obj>
</list>
</member>
<member name="name">
<string name="s" value="THIS IS SLOT 1" wide="true"/> <!--AND THEN THIS -DATA2-->
</member>
<int name="color" value="1"/> <!--BUT I CANT GET THIS DATA3-->
</obj>
</list>
</member>
<member name="controller">
<int name="ownership" value="1"/>
</member>
</instrument>
This worked for me - your problem "DATA3" item is the last one in the second loop
Sub fnReadXMLByTags()
Dim oxmlfile As Object, els, el
Set oxmlfile = CreateObject("Microsoft.XMLDOM")
oxmlfile.LoadXML ([A1]) 'loading XML from cell
Set els = oxmlfile.SelectNodes("/instrument/member/list/obj/member/string")
Debug.Print els.Length
For Each el In els
Debug.Print el.getattribute("name"), el.getattribute("value")
Next el
Set els = oxmlfile.SelectNodes("/instrument/member/list/obj/int")
Debug.Print els.Length
For Each el In els
Debug.Print el.getattribute("name"), el.getattribute("value")
Next el
End Sub
If you only want the int where name = color then you can use:
Set els = oxmlfile.SelectNodes("/instrument/member/list/obj/int[#name=""color""]")

Parsing keyed/valued nested JSON at different levels using TypeScript in Node

I got some data that I have to parse at different levels to fetch it and keep a similar structure in the way it is formated. I'm using TypeScript in the Node environment (Google Cloud) which I'm using for my current project. I'm quite new to this language and environement, I'm sicking guidance on the language feature aspect not on the data processing part.
The data that I have as input is in XML and is presented as the following:
<item>
<key xsi:type="xsd:string">intermedite_key</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">Key_parse_first_level_1</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">intermedite_key</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">Key_parse_second_level_1</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:int">intermedite_key</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">Key_1</key>
<value xsi:type="xsd:string">DataToParse</value>
</item>
<item>
<key xsi:type="xsd:string">Key_2</key>
<value xsi:type="xsd:string">DataToParse</value>
</item>
</value>
</item>
</value>
</item>
<item>
<key xsi:type="xsd:string">Key_parse_second_level_2</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:int">intermedite_key</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">Key_parse_third_level_1</key>
<value xsi:type="xsd:string">DataToFetch</value>
</item>
<item>
<key xsi:type="xsd:string">Key_parse_third_level_2</key>
<value xsi:type="xsd:string">DataToFetch</value>
</item>
</item>
</value>
</item>
</value>
</item>
</value>
</item>
</value>
</item>
<item>
<key xsi:type="xsd:string">Key_parse_first_level_2>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">intermedite_key</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">Key_parse_second_level_1</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:int">intermedite_key</key>
<value xsi:type="ns2:Map">
<item>
<key xsi:type="xsd:string">Key_parse_third_level_1</key>
<value xsi:type="xsd:string">DataToFetch</value>
</item>
<item>
<key xsi:type="xsd:string">Key_parse_third_level_2</key>
<value xsi:type="xsd:string">DataToFetch</value>
</item>
</value>
</item>
</value>
</item>
</value>
</item>
</value>
</item>
<item>
... Other items
</item>
</value>
</item>
I have managed to write this piece of code which works to parse the first level but at the second level:
/*Converts the XMl into a an object with 'fast-xml-parser'*/
const getRawResponseJSON = parser.parse(XMLDataToConvert);
const FisrtLevel = getRawResponseJSON['item']['value']['item']
Object(FisrtLevel).forEach((FisrtLevelValues: any) => {
const SecondLevel = Object.entries(FisrtLevelValues.value.item.value.item)
SecondLevel.forEach((SecondLevelValues: any) => {
console.log("value - " + SecondLevelValues);
});
});
Those are the console out-puts for Key_parse_second_level_1 :
value - 0,[object Object]
value - 1,[object Object]
And for Key_parse_second_level_2:
value - value,[object Object]
It seems that Key_parse_second_level_1 values aren't "aligned" with Key_parse_second_level_2 and when I go deeper I don't have the expected results for keys and values. Therefore I have different questions, was using Object classes a good idea for this case (I was enticed to use them for the keys and entries methods) ? Would an Array class have been better to use ?
I solved my issue by doing this :
const SecondLevel = Object.entries(FisrtLevelValues.value.item.value.item)
Object.keys(SecondLevel).forEach((SecondLevelKeys: any) => {
let SecondLevelValues: any;
if(String(SecondLevelKeys).match("value")){
return;
} else if (String(SecondLevelKeys).match("key")){
SecondLevelValues = SecondLevel['value']['item']['value']['item'];
} else if (String(SecondLevelKeys).match(/[0-9]/)) {
SecondLevelValues = SecondLevel[SecondLevelKeys]['value']['item']['value']['item'];
}
Object.keys(SecondLevelValues).forEach((ThirdLevelKeys) => {
const getKey = SecondLevelValues[ThirdLevelKeys]['key'];
const getValues = SecondLevelValues[ThirdLevelKeys]['value'];
...
});
});
At the SecondLevel the if is used to filter out the "value" key because Object.key() gave two results, "value" and "key" (which is the sole needed for my cause). The last else if filters keys by number because when having multiple items Object.key() gave me numbers as item index.
In case it helps someone.

Search over all fields on Solr

I'm trying to implement search over all fields on Solr. Exhttp://localhost:8983/solr/sample-items/select?q=oscar
but no result returns. However, If I specify the field it works fine Ex http://localhost:8983/solr/sample-items/select?q=name:oscar
I try to use copyField like following but not working:
on managed-schema file
<field name="Designation" type="text_general" stored="true"/>
<field name="Location" type="text_general" stored="true"/>
<field name="_root_" type="string" docValues="false" indexed="true" stored="true"/>
<field name="_version_" type="plong" indexed="false" stored="true"/>
<field name="age" type="plongs" stored="true"/>
<field name="id" type="string" multiValued="false" indexed="true" required="true" stored="true"/>
<field name="name" type="text_general" stored="true"/>
<field
name="_text_"
type="text_general"
indexed="true"
stored="true"
multiValued="true"
/>
.
.
.
<fields>
<copyField source="Designation" dest="_text_" maxChars="256"/>
<copyField source="name" dest="_text_" maxChars="256"/>
<copyField source="Location" dest="_text_" maxChars="256"/>
</fields>
on solrconfig.xml
file
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
</lst>
<initParams path="/update/**,/query,/select,/tvrh,/elevate,/spell,/browse">
<lst name="defaults">
<str name="df">_text_</str>
</lst>
</initParams>
Any advice?
To be able to search from >1 fields (or over all fields) by default , you need to perform following two configurations.
In schema file (e.g. managed_schema) create copy field and copy other field's data into this copy field. Example is as follows.
...
<field name="_text_" type="text_general" multiValued="true" indexed="true" stored="false"/>
...
<copyField source="directed_by" dest="_text_" maxChars="256"/>
<copyField source="genre" dest="_text_" maxChars="256"/>
...
In solrconfig.xml , update initiParam tag to make above created copy field as default search field for search handlers. Example is as follows.
<initParams path="/update/**,/query,/select,/tvrh,/elevate,/spell,/browse">
<lst name="defaults">
<str name="df">_text_</str>
</lst>
</initParams>
Sample Query and output :
ShubhangiP:~ spardeshi$ curl -XGET http://localhost:8983/solr/films/select?q=Gary
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"Gary"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"/en/45_2006",
"directed_by":["Gary Lennon"],
"initial_release_date":"2006-11-30T00:00:00Z",
"genre":["Black comedy",
"Thriller",
"Psychological thriller",
"Indie film",
"Action Film",
"Crime Thriller",
"Crime Fiction",
"Drama"],
"name":".45",
"_version_":1645282402834055168}]
}}
From details provided in question, in managed_schema file collector field is used as copy field for three other fields named Designation , Name and Location.
But not sure if you made collector field as default search field.
You can also think of using the edismax query parser. Here the link for the same. the edismax query parser .Here you can provide the qf parameter. It provides you to boost hits in each field differently. For example qf=field1^4 field2 will give hits in the field1 field four times more weight than hits in the field2 field.
Search across multiple fields, specifying (via boosts) how important each field is relative each other:
http://localhost:8983/solr/techproducts/select?q=video&defType=edismax&qf=features^20.0+text^0.3
You can boost results that have a field that matches a specific value:
http://localhost:8983/solr/techproducts/select?q=video&defType=edismax&qf=features^20.0+text^0.3&bq=cat:electronics^5.0

Why is Apache Solr not Returning any Results?

After creating a Solr core named "test", I loaded in an XML file using the following:
bin\solr.cmd create -c test
java -jar -Dc=test -Dauto example\exampledocs\post.jar C:\Example\data.xml
Here are the contents of the file, data.xml.
data.xml
<add>
<doc>
<field name="field_name_1">Value One</field>
<field name="field_name_2">Value Two</field>
<field name="field_name_3">Value Three</field>
</doc>
<doc>
<field name="field_name_1">Value Four</field>
<field name="field_name_2">Value Five</field>
<field name="field_name_3">Value Six</field>
</doc>
<doc>
<field name="field_name_1">Value Seven</field>
<field name="field_name_2">Value Eight</field>
<field name="field_name_3">Value Nine</field>
</doc>
</add>
The following query does not return a result. Why is Solr not returning anything for this query?
http://localhost:8983/solr/test/select?q=Value
Result:
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"Value"}},
"response":{"numFound":0,"start":0,"docs":[]
}}

Azure media Service Stitching Landscape and portrait videos

Im trying to stitch together 2 videos, the lead video being landscape and the second being portrait.
However Azure flips the portrait video sideways automatic. is there a way to stop this behaviour? and have the portrait part have black bars to make up the aspect ratio.
The second video is user generated so i have no control as to what size or orientation it will be.
Update 1:
Pre processing the Portrait video through the media service and then stitching the resulting file seems to work. but that makes it a 2 step operation. is there a faster way or is that the solution?
Update 2:
Yes the ones that are flipped are from smartphones
XML
<?xml version="1.0" encoding="utf-16"?>
<Preset
Version="4.0">
<Job />
<MediaFile
DeinterlaceMode="AutoPixelAdaptive"
ResizeQuality="Super"
NormalizeAudio="True"
AudioGainLevel="1"
VideoResizeMode="Stretch">
<Sources>
<Source>
</Source>
<Source
MediaFile="%1%">
</Source>
</Sources>
<OutputFormat>
<MP4OutputFormat
StreamCompatibility="Standard">
<AudioProfile Condition="SourceContainsAudio">
<AacAudioProfile
Codec="AAC"
Channels="2"
BitsPerSample="16"
SamplesPerSecond="44100">
<Bitrate>
<ConstantBitrate
Bitrate="128"
IsTwoPass="False"
BufferWindow="00:00:00" />
</Bitrate>
</AacAudioProfile>
</AudioProfile>
<VideoProfile Condition="SourceContainsVideo">
<MainH264VideoProfile
BFrameCount="3"
EntropyMode="Cabac"
RDOptimizationMode="Quality"
HadamardTransform="True"
SubBlockMotionSearchMode="Quality"
MultiReferenceMotionSearchMode="Quality"
ReferenceBFrames="False"
AdaptiveBFrames="True"
SceneChangeDetector="True"
FastIntraDecisions="False"
FastInterDecisions="False"
SubPixelMode="Quarter"
SliceCount="0"
KeyFrameDistance="00:00:05"
InLoopFilter="True"
MEPartitionLevel="EightByEight"
ReferenceFrames="4"
SearchRange="128"
AutoFit="True"
Force16Pixels="False"
FrameRate="0"
SeparateFilesPerStream="True"
SmoothStreaming="False"
NumberOfEncoderThreads="0">
<Streams
AutoSize="False">
<StreamInfo
Size="1280, 720">
<Bitrate>
<ConstantBitrate
Bitrate="4500"
IsTwoPass="False"
BufferWindow="00:00:05" />
</Bitrate>
</StreamInfo>
</Streams>
</MainH264VideoProfile>
</VideoProfile>
</MP4OutputFormat>
</OutputFormat>
</MediaFile>
</Preset>
Can you please try updating your preset as follows:
<?xml version="1.0" encoding="utf-16"?>
<Presets Rotation="Auto">
<Preset
Version="5.0">
...
</Preset>
</Presets>

Resources