I am trying to scrape a oracle adf faces rich client webpage but I am not getting the best of luck, I login automatically using node.js request module but after that I can't get to any other page with request. I get stuck on redirects, the loop script or simply don't get information I expect to.
I am using Wireshark to view every page and the way it handles, I recreate the page to match headers and even size but everytime the framework denies me access.
Before you ask, it's legal and I am not breaking any terms of service. Just trying to make a web api to speed up a process. I have used phantomjs with casperjs but get stuck on ajax calls that don't show on page and php curl but it's much easier with java.
Any suggestions are really really appreciated.
My bad on this one, wireshark was displaying fields as truncated, if you want to see the full field you need to right click the packet and click follow TCP stream, rich clients have very long posts generated by the framework behind the rich client and it appears I was missing about half of them when I did the calls.
Related
I want to stress test JSF application (using Primefaces) with Jmeter and I'm facing a strange problem.
Said application saves some textual field and one image field. The workflow of application is that, on image upload control (primefaces) image is stored in session and on save button click application saves textual data as well as image data (from session).
Now the problem is this: I made two post request - one with image data and 2nd with textual data - but the page can't simulate saving.
Is there any way to simulate the process in jmeter?
Given you send the same requests as browser does you should be able to replicate the browser behaviour, just make sure to:
Properly build HTTP Request sampler(s)
Pay attention to HTTP Headers
Correlate dynamic parameters like JSF ViewState
With regards to point 1 it should be sufficient to just record the requests using JMeter's HTTP(S) Test Script Recorder, just make sure to copy the file(s) you're uploading to the "bin" folder of your JMeter installation, this way JMeter will be able to properly capture the requests. See Recording File Uploads with JMeter article for more details.
Points 2 and 3 - cross check the requests which are being sent from browser using browser developer tools and JMeter's View Results Tree listener - the requests need to be exactly the same apart from dynamic parameters which need to be correlated
And don't forget to add HTTP Cookie Manager to your Test Plan, it should deal with JSESSIONID and other cookies
I am looking for a way to get complete data from the network of my own website with nodejs, exactly like in the network tab of the chrome devtool.
But, surprinsingly, I cannot find any information about it.
What I need is actually to get data from requests made by an iframe located on my webpage. But the tricky thing is that these specific requests don`t pass by my server. Is there a way to access these requests as I would in the network tab of chrome?
The only way that I found is to create a chrome plugin, it takes some effort but then I could catch any request.
I have a Windows Phone 8 app. My app needs to POST some data to a web page that I have on my server. Please note, I am trying to POST to a web page, not a web service. The reason that I need to POST to a web page is because 1) I'm trying to render some contents in a web page that is currently launched via a WebBrowserTask 2) I am passing a large amount of data to the web page.
From what I can tell, the WebBrowserTask only allows "GET". Now, I'm totally lost in regards to what to do. Is there a was to POST data via a WebBrowserTask? If not, is there a way to serialize my data as JSON and cram it into the query string? I know that's ugly. At the same time, I'm not sure what else to do.
Thank you
Try adding a WebBrowser control then make your request through either WebClient or HttpWebRequest. Once you get the html response use the NavigateToString(string) method to display the result.
But you can simply use the Navigate(uri) method if you only need to specify some query string params.
How can I perform a post through the chrome extention, lets say I want to send the current tab page title to a webpage
You can do POST XHRs from chrome extensions to any URL, as long as you have host permissions defined in your manifest. See these docs.
In a chrome extension the best way to try and do what i think you want is via a content script see documentation a word of warning however pinging your server with a POST request every time someone with your extension installed opens a web page is going to be extremely heavy going on your servers especially if you have a lot of installs. A possible solution is to use the content script to keep tally of the sites a user visits and save this data in a HTML5 database (wich chrome supports) then using background.html sending the data at given intervals in bulk with an AJAX request, this will significantly cut down the number of times your server is pinged.
I'm trying to write a userscript/Chrome extension to capture JSON data being sent while using a web service so that I can reformat it and display selected portion on page. Currently the JSON is sent as the application loads (as I've observed from watching traffic with Fiddler 2). Is my only option to request the JSON again or is capture possible? As I'm not providing a code example, a requested answer is even some guidance on what method / topic to research or if I'm barking up the wrong tree.
No easy way.
If it is for a specific site you might look into intercepting and overwriting part of a code which sends a request. For example if it is sent on a button click you can replace existing click handler with your own implementation.
You can also try to make a proxy for XMLHttpRequest. Not sure if this even possible, never seen a working example. You can look at some attempts here.
For all these tasks you probably would need to run your javascript code out of sandboxed content script to be able to access parent page variables, so you would need to inject <script> tag with your code right into the page from a content script: