I am new to apache-nutch and want to crawl few questions on stackoverflow. My urls/seed.txt has following data:-
/questions/58763948/setting-a-list-item-is-converting-it-into-a-tuple
/questions/58763947/start-up-eclipse-an-error-has-occured-see-the-log-file
/questions/58763946/problem-with-the-proxy-using-zap-docker-image-gitlab
/questions/58763945/how-to-select-unique-random-data-based-on-percent-in-sql
/questions/58763943/probelm-with-using-filter-function-to-remove-missing-values-form-a-dataset
/questions/58763942/flutter-keep-data-in-textfield-after-setstate
/questions/58763941/are-receipts-generated-by-google-play-api-v2-and-the-latest-version-v3-compatibl
/questions/58763940/how-to-add-eventhandler-to-popupmenuitem-in-flutter
/questions/58763938/how-to-solve-electron-and-grpc-version-problem-in-angular-project
...
Is there any property which i can include in nutch-site.xml to add https://stackoverflow.com before every url in seed.txt. I don't want to change seed.txt since file is large
No, there is no such configuration property. It's possible to do this by adding the following rule to regex-normalize.xml:
<regex>
<pattern>^/</pattern>
<substitution>https://stackoverflow.com/</substitution>
</regex>
Also make sure that the plugin urlnormalizer-regex is included in the property plugin.includes.
Related
I need to repair an XML file in TypeScript and I cannot find any info on it, since everyone who posts something like this has different needs. I'd like to be pointed in the right direction here.
I have an XML request as shown below. It is autogenerated by node-soap when given JSON. Using the WSDL, node-soap attempts to fill in the namespace prefixes for each property. The problem is, it gets them wrong a lot.
In this example below, q106 should be replaced with hep3.
<soap:Envelope q15="some-good-url" q106="some-good-url-1" q98-"some-good-url-2>
...
<q98:SalesOrderAuditInfo>
<q15:ConfirmedBy xsi:nil="true"></q15:ConfirmedBy>
<q15:ConfirmedDate>0001-01-01T00:00:00</q15:ConfirmedDate>
<q15:CreatedBy>
<q106:ID>103</q106:ID>
<q106:Value>System, System</q106:Value>
</q15:CreatedBy>
<q15:CreatedDate>2022-10-26T00:43:13.413</q15:CreatedDate>
<q15:SalesOrderType>Standard</q15:SalesOrderType>
</q98:SalesOrderAuditInfo>
I know which namespace prefixes are bad because I have a sample request that was supplied to me. It's just XML. It looks like this:
<soap:Envelope hep="some-good-url" hep1="some-good-url-1" hep2-"some-good-url-2 hep3="some-good-url-3">
...
<hep2:SalesOrderAuditInfo>
<hep:ConfirmedBy xsi:nil="true"></hep:ConfirmedBy>
<hep:ConfirmedBy:ConfirmedDate>0</hep:ConfirmedByConfirmedDate>
<hep:CreatedBy>
<hep3:ID>103</hep3:ID>
<hep3:Value>System, System</hep3:Value>
</hep:CreatedBy>
<hep:CreatedDate>0</hep:CreatedDate>
<hep:SalesOrderType>Standard</hep:SalesOrderType>
</hep2:SalesOrderAuditInfo>
Here is the part that really matters. The Sample Request is the entire possible request body. The supplied request (with incorrect namespaces) is a subset of the sample request. I need to loop through each of the elements in the supplied request, and check to make sure the URL of that element matches the URL of the respective element in the sample request.
So in this example, loop through each element of supplied request. Start with SalesOrderAuditInfo. It's namespace URL is some-good-url-2. If we check the Sample Request, we can see that SalesOrderAuditInfo also corresponds to some-good-url-2.
Continue until we hit the ID tag. This has a namespace set to some-good-url-1. If we check the Same ID (inside of CreatedBy, inside of SalesOrderAuditInfo), we can see the namespace should actually be set to some-good-url-3. So we replace q106 with hep3.
I also need to take all of the namespaces defined in the Sample Request Envelope and move them into the supplied request envelope so that this new hep3 will be defined.
At this point, I need to edit the namespace prefix. In this example, q106:ID would be replaced be the string hep3:ID. Same with all of the closing tags.
Which library can I use to accomplish this in XML? Is anyone familiar with node-soap screwing these namespaces up and know of any fix?
I am using node-soap v0.43
I am sure I am not the first to encounter this, but I was unable to find a solution while Googling.
I am trying to add a drop-down list to my document. At the top of my model I am adding these namespaces:
use \DocuSign\eSign\Model\List;
use \DocuSign\eSign\Model\ListItem;
When doing so I get this error because "List" is a reserved word in PHP.
A PHP Error was encountered
Severity: Parsing Error
Message: syntax error, unexpected List (T_LIST), expecting identifier (T_STRING)
Filename: models/Docusign_model.php
Line Number: 19
I tried changing the name of the class from List to Elist but then I got errors from ObjectSerializer that it could not find Elist:swaggerType.
What am I missing on how to add a list to my document?
Thom
#thom I think this is really a "PHP" parsing question as is answered here for you Parse error: syntax error, unexpected (T_STRING), expecting variable (T_VARIABLE)
So I think the $ missing is your real issue as discussed in the referenced article above and below from PHP Manual.
http://www.php.net/manual/en/language.oop5.basic.php
Recommend you look at GIT example from SDK using CustomFieldList at https://github.com/docusign/docusign-php-client/blob/ccc86ac37334f34728361d73b2f8c4592225b8d2/src/Model/CustomFieldsEnvelope.php
excerpt
protected static $swaggerTypes = [
'list_custom_fields' => '\DocuSign\eSign\Model\ListCustomField[]',
'text_custom_fields' => '\DocuSign\eSign\Model\TextCustomField[]'
];
http://www.php.net/manual/en/language.oop5.basic.php
Also, maybe the first place to validate if you even need a specific "use" is by reviewing this PHP sample code from a good friend Ergin https://gist.github.com/Ergin008/d4a8b9210fbea41414b0
As I see it with most of the DocuSign SDK's, you have the client and specific services you want to use per excerpt below:
// Download PHP client: https://github.com/docusign/DocuSign-PHP-Client
require_once './DocuSign-PHP-Client/src/DocuSign_Client.php';
require_once './DocuSign-PHP-Client/src/service/DocuSign_RequestSignatureService.php';
require_once './DocuSign-PHP-Client/src/service/DocuSign_ViewsService.php';
Regardless if I am right or wrong, let us know if this helped you go in the right direction :-)
I am deploying FIWARE security GEs (i.e., Wilma, AuthzForce, Keyrock) in my computer. Security level 2 (Basic Authorization) is working well, but now I need security level 3 (Advanced Authorization) using XACML.
Long story short, I want a tutorial of implementation security level 3. However, as far as I know, any tutorial or document about security level 3 does not exist.
For now, I create my policy with PAP's API, and change 'custom_policy' option in config.js from 'undefined' to 'policy.js'. And then I create 'policy.js' file into 'PEP/policies', but don't change anything compared with its template file because I don't know what this code does exactly. I think I should make XACML Request form using 'xml' variable. But in my case, PEP gives me the error when I make the XACML Request using 'xml' variable, and return this variable. Here is my error of PEP:
Error: Root - Error in AZF communication <?xml version="1.0" encoding="UTF-8" standalone="yes"?><error xmlns="http://authzforce.github.io/rest-api-model/xmlns/authz/S" xmlns:ns2="http://www.w3.org/2005/Atom" xmlns:ns3="http://authzforce.github.io/core/xmlns/pdp/5.0" xmlns:ns4="http://authzforce.github.io/pap-dao-flat-file/xmlns/properties/3.6"><message>Invalid parameters: cvc-elt.1: Cannot find the declaration of element 'Request'.</message></error>
And here is my 'getPolicy' code (XACML Request) in policy.js. I just made very simple request whether response is permit or not because I'm not sure what I'm doing at that time.:
exports.getPolicy = function (roles, req, app_id) {
var xml = xmlBuilder.create('Request', {
'xmlns': 'urn:oasis:names:tc:xacml:3.0:core:schema:wd-17',
'CombinedDecision': 'false',
'ReturnPolicyIdList': 'false'})
.ele('Attributes', {
'Category': 'urn:oasis:names:tc:xacml:1.0:subject-category:access-subject'});
So, anyone can give me any information about implementation of security level 3?
Upgrade to Wilma 6.2 (bug fixing).
Reuse the code from lib/azf.js which is known to work, and adapt the Request content to your needs. The variable is wrongly called XACMLPolicy there, but don't be mistaken, this is an actual XACML Request. This is using xml2json package to convert the JSON to XML, whereas in your code you seem to use a different one, xmlbuilder maybe? You didn't paste the full code - where does this xmlBuilder variable come from? - so I'm just guessing.
If you are indeed using xmlbuilder package and want to stick with it, I notice that in the example using namespaces, the xmlns attribute is put in a different way:
var xmlBuilder = require('xmlbuilder');
var xml = xmlBuilder.create('Request', { encoding: 'utf-8' })
.att('xmlns', 'urn:oasis:names:tc:xacml:3.0:core:schema:wd-17')
.att('CombinedDecision': 'false')
.att('ReturnPolicyIdList': 'false')
.ele('Attributes', {'Category': 'urn:oasis:names:tc:xacml:1.0:subject-category:access-subject'});
Maybe this makes a difference, I didn't check.
Also feel free to create an issue with your question on Wilma's github to get help from the dev team. (I am not one of them but we've worked together for AuthzForce integration.)
The error you are getting is really
Invalid parameters: cvc-elt.1: Cannot find the declaration of element
'Request'.
This is a simple XML validation issue. You need to make sure that the XACML request you send contains the right namespace declaration.
You'll see there is another question on this topic here.
Can you paste your XACML request so we can tell whether it is valid?
I need to clear/reset the external ID on a record in NetSuite, but nothing I do is working.
Some of the InventoryItem records are incorrectly mapped to records in another system. I have an application that can sync up the two systems, but I need to clear NetSuite's external IDs first.
Responses don't really need to be SOAP-specific. If you know how to do it with some specific NetSuite/SuiteTalk client, that might point me in the right direction.
What I've Tried
First up, I tried using the nullFieldList... but maybe it doesn't work because externalId is an attribute, not an element?
<messages:update>
<messages:record internalId="7777" xsi:type="accounting:InventoryItem">
<core:nullFieldList xsi:type="core:NullField">
<core:name>externalId</core:name>
</core:nullFieldList>
</messages:record>
</messages:update>
The external ID is just a string, so I tried just setting it to blank. Didn't work either.
<messages:update>
<messages:record internalId="7777" xsi:type="accounting:InventoryItem">
<accounting:externalId></accounting:externalId>
</messages:record>
</messages:update>
I even tried setting the external ID to 0, but I get back a "not unique identifier" error
<messages:update>
<messages:record internalId="7777" xsi:type="accounting:InventoryItem">
<accounting:externalId>0</accounting:externalId>
</messages:record>
</messages:update>
Other Info
I'm using NetSuite's SOAP API v.2013_1
When I say "it doesn't work", I mean: after I do the update, I get a success response similar to the following:
<readResponse>
<platformCore:status isSuccess="true" xmlns:platformCore="urn:core_2013_1.platform.webservices.netsuite.com"/>
<record internalId="7777" externalId="42" xsi:type="listAcct:InventoryItem" xmlns:listAcct="urn:accounting_2013_1.lists.webservices.netsuite.com">
<!-- snip -->
</record>
</readResponse>
If you are using scripts in netsuite you can run a scheduled script to clear records in NS by loading each record and setting the externalid to '' using the following simple code:
var rec= nlapiLoadRecord(type,id);
rec.setFieldValue('externalid','');
nlapiSubmitRecord(rec);
This seemed to work for me in my enviornment which was on 2015.2.
Unfortunately my understanding is that once you set an externalid you cannot clear it, you can set it to another value, but not back to null. I have experienced this both using SuiteScript as well as a Boomi process that uses the 2014.1 endpoint. This may have changed in the recent releases, as I have not tried it recently with SuiteScript nor with a newer endpoint.
You can eliminate the externalId on a record once it's been set. Here's an example using the NetSuite gem:
ns_customer = NetSuite::Records::Customer.get external_id: 'ext_id'
ns_customer.external_id = ''
ns_customer.update
Here's the corresponding XML for update:
<env:Body>
<platformMsgs:update>
<platformMsgs:record xsi:type="listRel:Customer" platformMsgs:internalId="199113" platformMsgs:externalId=""/>
</platformMsgs:update>
</env:Body>
I have had to attempt this before as well. I know the pains you describe. I ended up putting a "-" in front of my external ID to unlink it between my systems. You can do this in SOAP or even as a simple one time csv upload. As it was one time, I did csv.
How to get the current id that had been used when editing and viewing a record?
I am currently following the instruction by Alfonso Secas that he posted in this topic..
http://www.grocerycrud.com/forums/topic/1326-how-to-use-grocery-crud-and-image-crud-together/page-2
And now on the part where he said "When editing a record, get the current ID from $this->getStateInfo(); to compose the iframe's source url."
It seems like getStateInfo doesn't exist yet so it shows an error "Fatal error: Call to undefined method Examples::getStateInfo() "
http://www.grocerycrud.com/documentation/options_functions/getState
this will do what u want . give it a try
u need to use the CRUD object to call this function instead of $this ..
You have some options:
Primary key
getStateInfo()->get_primary_key
Last uri segment
end($this->uri->segments)
Does not work in list action.
As a workaround you can use one of CodeIgniter's methods to get id from page url
https://www.codeigniter.com/userguide3/libraries/uri.html
You need to take last segment from URI
$segmentsCount = $this->uri->total_segments();
$itemID = intval($this->uri->segment($segmentsCount));
p.s. it is not best solution but it can do the trick