I am using intern Runner for creating reports. I have successfully created default reports but I want to create report in JSON format. Can anyone guide me ?
There isn't a built-in JSON reporter at this time, so the quickest way to get JSON output would be to write one. Luckily, reporters are very simple. For example, check out the Console reporter. Basically, a reporter is just a class with method names corresponding to Intern reporter events (like suiteEnd and testPass). As those methods are called, the reporter creates its report. A JSON reporter could just add results to a report object and then write that to a file in a runEnd handler.
Related
I am using the test plugin for VS 2012 (although have just installed 2013), and need to know:
Is it possible to have a parameter pass a different value from a selected list while load testing?
I have used the sample load test located here: http://www.visualstudio.com/get-started/load-test-your-app-vs and created a new web test that meets my needs as below.
I have a simple journey recorded that is an email registration web page. The journey is essentially completing name & address, email, conf email, password, conf password. On submission of the form, a verification email is sent.
I need to check that this process can handle around 3000 users. The email to actually send the verification has been hardcoded for test purposes, but I need a unique email to submit the form. I would essentially like to run 3000 test cases through, and just change the email address each time.
What is the best way to do this?
The simple answer is do a web search for data driving (or data driven) Visual Studio web performance tests. You should find many articles and tutorials.
In more detail:
Outline of how to data drive a test
Firstly, Visual Studio distinguishes different types of test. A Load Test is a way of running individual test cases many times, as if by many simultaneous users, gathering data about the test executions and producing a report. The test cases that a load test can execute include Web Performance Tests and Coded UI Tests; both of these can be data driven.
Data driving a Web Performance Test requires a data source. The data can be CSV, XML, Spreadsheet, database and in TFS. I will describe using CSV.
Create a CSV file, containing something similar to the following. Note that the top line of field names is required and those names are used within the test.
Name,Email,Telephone
Fred,fred#example.com,0123 456789
George,george#example.com,0123 456790
Harry,harry#example.com,0123 456791
See also CodedUI test does not read data from CSV input file for some notes CSV file creation.
Open the test project in Visual Studio and open the .webtest file for the test. Use the context (right-click) menu of the top node of the test, ie the test's name (or use the corresponding icon) and select "Add data source ...". Follow the prompts to add the CSV file into the project.
Within the Web Performance Test expand the request to show the form parameters or query string or whatever that is to use the data. View the properties panel of the relevant field and select the appropriate property, in many cases it is the Value property. Click the little triangle for choosing a value for the property. The popup should show the data source, expand the items shown and select the required field. After selecting the field the property will show a value such as {{DataSource1.FileName#csv.Email}}. The doubled curly braces ({{ and }}) indicate the use of a context parameter. All the used data source fields are available as context parameters. All of the data source fields can be made available by altering the Select Columns property of the data source file. Data source field can be used as part of a property value by using values such as
SomeText{{DataSource1.FileName#csv.Email}}AndMoreText
Data source access methods
The data from the datasource can be read and used in four ways. The default is Sequential. Other orders are selected using Solution Explorer to access the properties of the file (eg FileName#csv). The Access Method property can be set to one of:
Sequential data is read sequentially through the file. After the last line of the file is read, the first line of the file will be next line to be read. Thus each line may be read more than once.
Random data is read randomly.
Unique data is read sequentially through the file. After the end of the file is read the test will not be executed again. Thus each line in can only be read once.
Do not move cursor automatically intended for more complex tests where the cursor is moved via calls from plugins.
A web test may use more than one data source file. These files may have different access methods. For example one file containing login names and passwords could be accessed Sequentially and another file with other data could be accessed Randomly. This would allow each login to try many different sets of the other data.
Data sources and loops
Web performance tests may contain loops. The properties of a loop include Advance data cursors. This allows, for example, a data source file to contain items to be found and added to a shopping basket such that each loop iteration adds a new item.
Is it possible to send a list of parameters to a test case / step in soapUI? I want to perform some database validation, and I would like to be able to pass a list of the expected values to the test case and / or step that performs the validation.
I know it's possible to send single parameters ("properties") to a test case, but I don't think that's good enough for this use case. My idea is to write a single "test case" that performs the validation that can be called from other test cases, and they pass in the values they expect to find in the database.
If you can generate the list of Random parameters through Groovy script then the best solution is: Through groovy script set the property field's value and pass this value into request using property transfer. The property field value will change at each and every run of the groovy script.
Now run the request in loop in groovy script.
So we can able to run the one test request multiple times with different parameter values.
We have a similar issue. We need to run multiple XML-files as source files for the sequence of requests. We got about 15 steps in the test, and the only thing that changes is the initial XML.
We solved this by using groovy to set a propperty to a comma seperated list, then pick the first element, remove it from the list, and run the tests. In the end, we return to the "pick first element from list"-step if the propperty is not empty.
This is a goto implementation of a basic loop, and we would prefer to do this differently, but we have not figured out how to (we run soapUI via maven2).
If you are using soapUI Pro, you can create a data source step > request step > dataSource loop step.
dataSource step can take excel, XML, grid as the soure. You need to create a property and select the type of the source.
The next step is running the request.
Then you should create a dataSource loop step. This steps returns to the dataSource until running all the request.
Please check soapui documentation
if you are not using soap Pro version, creating a groovy script is the solution as described with the previous answer.
We had our own custom logger in a C# program and now are trying to port to log4net.
In our app, there is further structure to what would normally go into %message. It may contain requestid, associated users, and other structure where requestid and user have internal significance to the program.
The hope is to ultimately be able to search on the fields inside %message, say requestid so we can collect all log entries with the same requestid for example.
Does log4net assist in anyway in creating own custom fields? The reason we ask is that currently the entire %message is logged as one string by default.
Any other suggestions on how to provide further formatting for %message? Otherwise we would have to pre-format %message inside our own code as, say, a CSV format
You can use event context to add additional structured data to a log entry:
http://www.beefycode.com/post/Log4Net-Tutorial-pt-6-Log-Event-Context.aspx
Depending on what kind of information you want to log you may need to create a wrapper that accepts additional parameters or else you have to write verbose code like this:
log4net.ThreadContext.Properties["myInformation"] = yourAdditionalInformation;
log.Info("info message");
Other information can be calculated and thus can be set once (for instance on application start up). Have a look at the calculated context properties in the above tutorial.
I am implementing logging in our application using ETW.
I used ecmangen.exe to create the manifest with a set of custom events, templates, tasks and keywords.
I used mc.exe to compile and generated corresponding C# class for use in my code.
I used this generated class in our code to emit events.
I imported the manifest via wevtutil and captured events via event trace session (Win2K8R2 - Performance - Data Collector Sets)
When I open the generated trace file via Event Viewer, I see the events I was expecting. But I found that the custom task categories and keywords are showing in their numeric form, instead of their names.
Should these values be translated? Or am I going about this the wrong way?
I'm trying to use SOAPUI (4.0) to do load testing, and I want to have each SOAP request be different, with some attribute values and element values in the requests being populated from (for example) a text file.
The SOAP message is going to be the same for each request, except for the several attribute values and element values.
The SOAP message includes an unsigned SAML assertion, and that has some attributes that need potentially be different for each SOAP request. Among these attributes there's one called "IssueInstant" which is basically a date/timestamp string, and an "Id" attribute, which is a unique string per request.
Ideally, I'd like to be able to populate that "Id" attribute value from the text file.
I've been able to populate the IssueInstant automatically in SOAPUI, by including a small piece of Groovy code, to get the current date/time, re-format it, then store that in a property. This Groovy code is in the startup script in the SOAPUI testcase.
In the body of the SOAP message, I have a Subject element that I want to have populated from the text file.
After the IssueInstant, Id, and Subject are populated, I want SOAPUI to send the request.
So, for example, say the text file has:
id0001,cn=foo1,dc=whatever,dc=com
id0002,cn=foo2,dc=whatever,dc=com
id0003,cn=foo3,dc=whatever,dc=com
Then, when I run the SOAPUI load test, I'd like the first request to have Id=id0001 and subject cn=foo1,dc=whatever,dc=com, the second request to have Id=id0002 and subject cn=foo2,dc=whatever,dc=com, and the third request to have Id=id0003 and subject cn=foo3,dc=whatever,dc=com, and then the load test loops back through those 3 sets of values until it ends.
The thing that I'm having a hard time understanding is how to step through the file in the Groovy code and how the Groovy code is suppose to know which line in the text file is the next line from which to build the properties?
I hope that this explanation of what I'm looking for is clear enough. If not, please let me know, and I hope that someone can help.
In soapUI Pro there exist so called DataSource and DataSource Loop test steps. They are used for looping through a set of test data. For example a text file. If you have the ability to use soapUI Pro, I recommend you to have a look at this: http://soapui.org/Data-Driven-Testing/functional-tests.html
Otherwise you have to load the file via groovy.
how to step through the file in the Groovy code
I'm sure you'll find some code snippets via Google.
how the Groovy code is suppose to know which line in the text file is the next line from which to build the properties
Create a test case property with initial value 1. Always after reading a line increment the value by 1. By reading this property in your groovy code you always know which line to read.