Sorry in advance for the novel, I've tried to distill this down into something short and sweet, but find it impossible. I'm really hoping someone can shed some light on this problem.
The project: to build a set of utilities to make using the CDC Wonder API easier.
The Wonder API has several endpoints, in the format of /D140 or /D76, for example.
The endpoint must be hit with a POST request, with a set of query parameters sent as an xml string as a POST data parameter.
The query parameter xml string is complicated. The basic format is
<parameter>
<name></name>
<value></value>
</parameter>
</request-parameters>
The names and values are best understood by going to the user-facing query portal. Each one of the inputs (forms, checkboxes, radio buttons, etc) corresponds to a query parameter. Each parameter name is in the form of a code, such as D76.V9 or O_V27fmode. The value may also be a code, or occasionally could be plaintext, such as *All*.
Each endpoint requires a slightly different set of parameters, and the coding for the values is also different. The CDC does not provide documentation on which parameters each endpoint expects. Their documentation specifically tells you that the best way to find this information is by examining the source code of the user-facing query portal, where each element has its name and value embedded as an attribute.
I've found that you can also retrieve these parameters by opening up the dev tools network tab, submitting a filled out query form, and, after the new page loads with your results, checking the POST call's header form data.
One of the utilities I'm trying to build is to scrape the query portal page and programmatically extract the names and values of all parameters from the HTML source code. Seems simple enough and I think I know how to do it.
But, here's the rub: the URL of the user-facing query portal is the same as the URL for the endpoint that you call. In a web browser, going to this URL (https://wonder.cdc.gov/controller/datarequest/D76) sends you to the query form that you fill out. But when you use that same URL in a request in Python, it expects an xml string of query parameters, otherwise you will get an error.
Is it possible to use the requests library to hit this user portal? I tried setting the referrer to https://wonder.cdc.gov/ucd-icd10.html, which is a page you're redirected to before you can get to the query portal, and you must agree to the terms and conditions. I also tried pulling the form data out of the network tab and sent it as an xml string with my POST request, and got a 500 back -- it's expecting the query parameters (including a specific parameter agreeing to the terms of use, as outlined in the documentation). Here's an example of the code I'm using:
xml_string = '''<query-parameters>
<parameter><name>stage</name> <value>about</value> </parameter> <parameter> <name>saved_id</name> <value></value> </parameter> <parameter> <name>action-I Agree</name> <value>I Agree</value> </parameter> </query-parameters> '''
data = {'request_xml': xml_string}
response = requests.post('https://wonder.cdc.gov/controller/datarequest/D76')
print(response.text)
print(response)
Really curious to know how to crack this nut, what am I missing here? How should I be approaching this problem?
Long time after you asked the question but I think the issue is with the post request is made and also the contents of your data variable.
Also just a note that it should be 'result-parameters' instead of 'query-parameters' in your xml_string.
It does seem that the documentation has improved a lot:
https://github.com/alipphardt/cdc-wonder-api/blob/master/CDC%2BWONDER%2BAPI%2BExample.ipynb
Here's how I think you can fix this:
xml_string = \
'''<query-parameters>
<parameter>
<name>stage</name>
<value>about</value>
</parameter>
<parameter>
<name>saved_id</name>
<value></value>
</parameter>
</query-parameters> '''
url = 'https://wonder.cdc.gov/controller/datarequest/D76' ## or D140
param_name = 'request_xml'
url = "https://wonder.cdc.gov/controller/datarequest/D76"
response = requests.post(url, data={"request_xml": xml_string, "accept_datause_restrictions": "true"})
Related
How do we compose a get url to get an envelope or envelopes that contain a given custom field. I tried to compose a get url as one shown below but not getting the desired results (i tried to follow the documentation here -> https://developers.docusign.com/esign-rest-api/reference/Envelopes/Envelopes/listStatusChanges but I think i am not passing the custom field to the query string correctly
https://demo.docusign.net/restapi/v2.1/accounts/ACCOUNT_ID_NUMBER/envelopes?from_date=2019-12-05&folder_types=sentitems&status=completed&custom_field=[SignerEmail=john.doe#email.com]
You're using the right API call. Try it without the square brackets and encode the value.
Eg
... &custom_field=SignerEmail%3Djohn.doe%40email.com
The above is untested. If it doesn't work, we'll dig into it further
Important: are you trying to find envelopes that have a custm_envelop_field (metadata) that you created and you named "SignerEmail"?? If so, the the above is the way to go.
If you're looking for envelopes where a signing recipient's email is john#doe.com, try the general search as Inbar suggests. If it doesn't work, you may need another tactic.
Suggest you read here:
https://developers.docusign.com/esign-rest-api/guides/concepts/envelopes/search
the parameter is called search_text and would search everything including custom fields. There's no need to separately specify custom fields.
Something like this:
https://demo.docusign.net/restapi/v2.1/accounts/ACCOUNT_ID_NUMBER/envelopes?search_text=[SignerEmail=john.doe#email.com]
I need to programmatically find the path to the Room & Resources database, preferably accessible thru a HTTP request. Just placing a HTTP request and getting the path/filename returned somewhere in the response would be sufficient.
And/Or are there any assumptions that can be made in regards to the file/path-name of Rooms & Resources? So, at the very least, I can provide an educated guest to the end user.
Since the room & resource database may be clustered - a way to determine this would also be appreciated and preferably thru a HTTP request.
NO changes can be made to the environment to get this information - this means no programming in the users mailfile or any other design changes.
The question is in regard to IBM Domino 8.5+
/J
To get the information "somewhere in the response" is quite easy: Just use the hidden ($Users)- View to get that information:
http://servername/names.nsf/($Users)/Resource%2FOrga?OpenDocument
As you see, you have to encode the slashes in the resource name with %2F
The respons will be a website, that contains Server AND FilePath in two fields:
<input name="MailServer" type="hidden" value="YourServer/Org">
<input name="MailFile" type="hidden" value="pathtoresourcedatabase.nsf">
In that document there is only the "main" server. Information about clustering would have to be obtained by looking up the MailServer- Value (in a canonicalized form) in the view ($Servers):
http://servername/names.nsf/($Servers)/CN=YourServer%2FO=Org?OpenDocument
In the result there will be a field called ClusterName:
<input name="ClusterName" type="hidden" value="YourClusterName">
And with that information you could lookup the clustermembers using the view called ($Clusters), but there you have to parse the information, as this is not a single document... Just check the output of:
http://servername/names.nsf/($Clusters)?OpenView
If you need to get a "better parseable" result, then use ReadViewEntries instead of OpenView, then the result is XML (or json, if you add another parameter &outputformat=JSON)
Theres a view for rooms and resources in the names.nsf. You can probably query this view with an http request and parse the result, to find out the path to the resource databases.
BTW. it is possible to have multiple resource database.
Page parameter (in Seam) or GET parameter (general) are often mentioned as a proper means to transfer information from one view to another. But obviously it is not a good idea to have sensitive data in the url, e.g //myserver/show.jsf?userId=12, since it is easy to manipulate these params and look at data someone is not permitted to look at.
So far I've been using what examples and literature show (couse until now was not important):
<s:link..>
<f:param value="#{user.id}" name="userId" />
</s:link>
in the JSF-file and in the according target page.xml
<param name="userId" value="#{userHome.userId}" />
I am interested in two things (still kind of new to Seam):
1) What different possible strategies of securing unpermitted access, to e.g. different user accounts, are you using, if you want to stick to page parameter? I'm sure some of you have been facing that challenge already. And what are the pros and cons of these strategies.
2) I want to make use of the Seam EntityHome objects here and there in the project since it is a comfortable handling of entities and kind of a DAO structure - but how to efficiently work with HomeObjects if not using page parameter?
Would appreciate to some thoughts and experiences from you guys. Thanks a lot.
josh
GET parameters are not inherently unsafe, all REST services rely on data being put in the URL. Parameters (GET or POST) are unsafe if your user detail page, in your example, does not check if you actually have access to user account "12". Also, don't think POST parameters are any harder to manipulate than GET parameters.
So, your code should check if you are entitled to view sensitive data. To handle unauthorized access, you can just throw an org.jboss.seam.security.AuthorizationException in the setUserId() method if the user is setting an ID he is not entitled to. Launching this exception makes Seam follow the exception handling mechanism described in pages.xml (by default it redirects to the /error.xhtml page with an error message).
#In Identity identity; // The standard Seam Identity component
#In Long sessionUserId; // You should outject this during user login
public void setUserId(Long userId) {
// Grant access is user is an admin or his id is the same as the one
// he is trying to set. Otherwise, exception.
if (!identity.hasRole('admin') && !sessionUserId.equals(userId)) {
throw new AuthorizationException("Not authorized");
}
this.userId = userId;
}
I'm planning on implementing a google site search (and paying for it so I can get access to the XML). One thing I am wondering about is the possibility to use custom meta tags in it.
I've heard yes from colleagues but nothing confirmed. Searching for an answer has given nothing (maybe because you cant?)
Anybody knows?
Edit: I want to be able to retrieve those meta tags from the search result to be able to provide different styling for different types of pages.
Yes, you can do this, although not, as far as I know, using meta tags. Instead, google allows you to put custom attributes inside your HTML, which can then be retrieved from the XML you get back from Google Custom Search results API.
Take a look at
http://code.google.com/apis/customsearch/docs/snippets.html for more info on how this works. The Structured Data section outlines the specific ways you can stick custom metadata inside your page so Google will hand it back to you along with each search result. The easiest seems to be a PageMap, which is an HTML comment inside your HEAD, like this:
<!--
<PageMap>
<DataObject type="document">
<Attribute name="title">The Biomechanics of a Badminton Smash</Attribute>
<Attribute name="author">Avelino T. Lim</Attribute>
<Attribute name="description">The smash is the most explosive and aggressive stroke in Badminton.
Elite athletes can generate shuttlecock velocities of up to 370 km/h.
To perform the stroke, one must understand the biomechanics involved,
from the body positioning to the wrist flexion. </Attribute>
<Attribute name="page_count">25</Attribute>
<Attribute name="rating">4.5</Attribute>
<Attribute name="last_update">05/05/2009</Attribute>
<Attribute name="thumbnail">http://www.example.com/papers/sic.png" width="627" height="167" />
</DataObject>
</PageMap>
-->
If you absolutely must use META tags, the Bing Search API does support indexing all meta tags whose NAME property starts with "Search.", e.g.
<meta name="Search.MyCustomProp" content="Hola mi amigo Google, I want my META!">
Then in the Bing Results API you can pull out all these values using the WebResult.SearchTag property. When issuing the request, you'll either need the WebRequest.SearchTags Property or you can just add text to your search query, e.g. poodle meta:search.MyCustomProp(amigo) searches for the word "poodle" on pages where the search.MyCustomProp meta tag contains the word "amigo". Using the Bing API is free as long as you stay under 7 queries per second and adhere to some other restrictions detailed here.
Sorry to be talking so much about Bing, feel free to ignore that part-- but I have had reasonably positive experiences using the Bing API at a previous job.
Can you detail more about the meta tags you want to use. You can add meta tags and it should not matter to Google.
I am working on a new web app in lotus/domino. I am newer to lotus/domino programming, so forgive me for not knowing something simple.
What I am trying to do is display a table of information, with one of the columns containing a link. The link is formatted like [http://server/app.nsf/form?openform&ideaNum=1&var2=foo2]
How can I retrieve the information from the url, to get information from a view or a document?
So far I have tried to access the query string in the webQueryOpen event, but lotus runs the agent before the DOM writes to the browser, the query_string isn't available.
I am not sure how else to get information from one form to another in lotus.
Thanks for the help.
-Kris
If you're passing multiple params in the query_string, you can get Domino to do a bit more of the work for you. There's an Function command - UrlQueryString - that can format the params into a list, delimiting at the "&".
E.g. server/app.nsf/form?openform&ideaNum=1&var2=foo2
becomes a list:
openform
ideaNum=1
var2=foo2
To use it, on the destination form create a multi-value text field called something like "QryStringList".
Set it to be computed when composed, and set it's value to #UrlQueryString.
In your WebQueryOpen agent, you can access the params as doc.QryStringList(0), docQryStringList(1) and so on, if you're using Lotusscript.
I'm fairly sure I put this together using info from the IBM/ldd forum, but the exact source is lost in the mists of time and memory, so apologies if I've copied someone's post.
Just as a follow up, I found that you are able to specify an on form variable named the same as QUERY_STRING. The variable is actually a reserved term for Lotus, so Lotus pre-fetches the variable before running the information out the browser.
Due to the pre-fetching, you are able to access the url information during the webQueryOpen event.
I hope this helps someone else looking for the same type of functionality.
-Kris