Updating can performance test script e.g. with LoadRunner can take a lot of time and be quite frustrating. If there has been some updates with the applications, you usually have to run the script and then find out what has to be changed, update and run again and so on. Does anyone have some concrete best practices how to ease this updating inferno? One obvious thing is good communication with developers.
It depends on the kind of updates. If the update is dramatic, like adding new fields for user to fill in, then, someone has to manually touch up the test scripts.
If, however, the update is minor, for example, some changes to the hidden fields or changes to the internal names of user-facing fields, then it's possible to write a script that checks the change and automatically updates the test script.
One of the performance test platforms, NetGend, automatically takes care of the hidden fields and the internal names of user-facing fields so it's very easy to create a script to performance-test a HTML form. Tester only needs to fill in the values that he/she would have to enter using a browser, so no correlation is necessary there. Please send me a message if you need to know more about it.
There are many things you can do to insulate your scripts from build to build variability. The higher up the OSI stack you go the lower the maintenance charge, but the higher the resource cost for the virtual user type. Assuming changes are limited to page level resources and a few hidden fields here and there for web sites or applications, then you can record in HTML mode. You blast the EXTRARES sections as the page parser in HTML mode will automatically parse the page and load the page resources even without an explicit reference - It can be a real pain to keep these sections in synch if you have developers who are experimenting quite a bit.
Next up, for forms which have a very high velocity in terms of change consider the use of a web_custom_request() for the one form. You can use correlation statements to pick up all of the name|value pairs as needed and build the form submit dynamically. There will be a little bit more up front work for this but you should have pay offs at around the fourth changed build where you would normally have been rebuilding some scripts.
Take a look at all of the hosts referenced in your code. Parameterize all of these items. I have a template that I use for web virtual users which pairs a default value and the ability to change any of the host names via the control panel extra attributes section. Take a look at the example for lr_get_attrib_string() for how you might implement the pickup and pair that with a check for NULL and a population with a default value in your code
This is going to seem counter intuitive, but comment your script heavily for changes that are occurring often so you know where to take the extra labor change up front to handle a more dynamic data set.
Almost nothing you do with any tool can save you from struuctural changes in the design and flow of the app, such as the insertion of a new page in the workflow, but paying attention to the design on the high change pages, of which there are typically a small number, can result in a test code with a very long life.
Of course if your application is web services based then there is a natual long life to the use of exposed public services. Code may change on the back end of the service, but typically the exposed public interface is very stable.
Related
I am searching for a way to determine how a job was initiated on the HPCC cluster.
There are several ways to submit a job. For example:
1- a manual submission via the ECL IDE / ECL Watch
2- an external cron submission
3- an ECL submission of dynamically built code
4- if a file lands in a directory, it triggers a submission
etc.
I can retrieve some important information by executing a STD.System.Workunit.WorkunitList,
but I cannot find any function that would give me an attribute indicating the source of
that submission.
HPCC is a data-centric platform and ECL reflects that approach. So I am attempting to build
a matrix that defines the code in relation to that data. A product is technically a bunch
of data (files) that is the result of source input -> scrub and transformation processes -> to
the final base files. Then those files are then prepped / indexed for external use:
1- Roxie queries
2- PowerBI
3- webpage
4- reports ftp'd or emailed
etc.
I want to build this matrix that defines (by product) the initiating job(s), where they were initiated, any schedule (?), the associated input/output files (flagging whether they are source/intermediate/base/output). I am trying to design this so that the matrix can be dynamically built, because as we all know:
(1) nowhere does this type of documentation exist so that if someone new comes in to work on
a product, they can go and see the scope and life cycle of the data,
(2) nobody likes to document,
(3) the second any manual documentation is actually created and saved; it is out of sync with reality
So far, the design will be a collection of files (defined by the level of detail) which would
then be JOINed together to yield the final matrix. Not sure if this would end up as a PowerBI report or a webpage...still tossing that around. Still, this might prove to be something useful for
anyone using HPCC who wants a 30,000 ft view of their product.
I have attempted to programmatically scan a WUID output, looking for the necessary attributes but I have had little success.
I appreciate any assistance / comments.
No matter what component submits ECL to execute on the platform, they all ultimately end up going through the same WsWorkunits API, which is the public SOAP / REST interface.
While some client applications will leave a fingerprint so you can deduce where it came from, it is not a foolproof mechanism...
For Example: In http://play.hpccsystems.com:8010/esp/files/index.html#/workunits/W20221115-075604/xml you can see the ECL IDE appends some meta information into the Workunit (it stores the IDE version number in the "Application" section)
Let say a company has an internal program called Service Checklist.
Because the program named Service Checklist, the programmer creates a name on variables, pages, database tables, stored procedures and anything else based on the program name.
For example :
Variable named ListServiceChecklist or ListSC
A page namedCreateServiceChecklist or CreateSC
Stored Procedure named GetServiceChecklist or GetSC
Table named ServiceChecklist
This program has been used for years, and new small improvements added every year.
Then someday the manager decided to rename the program from Service Checklist to E-Checklist, and he needs it to be done fast. So if a text that is shown to the user contain words Service Checklist, it must be changed to E-Checklist
For example :
Text Menu List Service Checklist must be changed to List E-Checklist
Alert message Success create Service Checklist changed to Success create E-Checklist
So the programmer started to change all the text from Service Checklist to E-Checklist on the front end side. But what about all the variables, pages name, database tables, stored procedures, etc?
If the programmer leaves it, other programmers will be confused as the name doesn't make any sense anymore.
If the programmer renames all the variables, etc, it will take a lot of time and a huge pain in the ass.
As a programmer what's the best way to solve this problem?
Like most things in programming, it depends. There isn't a one rule fits all.
Ideally everything gets changed to help code readability but in many cases the amount of time this would take doesn't justify the benefits.
You have to take into consideration the amount of time it will take to do these updates and measure it against how much time it will save you in the future.
It also depends on the size of the project, how many times the code gets updates, how much time the team has to work on this type of things and probably a few others I can't think of right now.
I am an automated-test engineer and call it occupational hazard, I keep thinking towards making automated testing efficient and easy.
Couple of weeks back, I was thinking, in fact just wondering, if we could introduce a new process where I force designers/developers to provide me a wireframe for every single or unique screen in the web application.
Example: In a hypothetical web application, if home page displays differently for different profiles like associates, approvers, auditors and managers, I want a wireframe for each of those profiles. But if there are 2 managers (data combinations) in the organisation, I don't expect wireframe for these 2 managers as just data is changing and structure would remain same from automated-testing perspective.
This helps in automation in the following manner:
Say a sprint starts and it's decided that 2 new features shall be developed in this sprint.
Wireframes for these 2 new features shall be provided to the whole team at the begining of the sprint say in sprint-planning meeting.
Using these wireframes I would automate my testcases and when the actual screens are deployed, I would just run my automated-testcases. Of course they will fail due to some minor issues but they will be easy fixes. This is way better than not starting automation until the actual screen are deployed.
If this approach is possible, I will later demand for id and name properties of user controls (like links, buttons, edit boxs, check boxes, radio buttons etc) along with wireframes. I am aware that some teams are already sharing id and names properties of user controls with their automated-testing team upfront so that they can automate the testcases even before actual screens are deployed. You could say my idea is little enhanced version of that.
It sounded impractical so I stopped thinking towards it but later felt like I should ask the community if it's possible? if yes, what's the overhead on the overall design/development team.
Some developers reading this might think that pffhhh... we are packed with stuffs already and this would be an overhead. But trust me. Early collaboration with testers and automate-testers would really help the quality of the application. So if the overhead is well within the limit, we could think towards introducing this process.
Please let me know your views.
This is a rather generic question, but I'll try to be as precise as possible:
quite often I'm asked by customers for proper implementations of LotusScript's
continue = false
in Notes' Query* events. One quite common situation is a form's QueryOpen event where we actually can stop the process of opening the document in question based on some condition, e.g based on the response from a user dialog.
For some Xpages events like querySaveDocument there are quite obvious solutions, whereas with others I only can recommend re-thinking the entire logic like preventing code execution at a much earlier stage. But of course most people in question would prefer a generic approach like "re-write those codes using...". And - to be honest - I'd like to know myself ;)
I'm more or less familiar with the Xpages / JSF lifecycle, but have to admit that I don't have a proper idea how I could stop execution at any given phase.
As always, any hint is welcome.
EDIT (to clarify my question, but also in response to Tim's answer below):
It's not just the QuerySave but also the QueryModeChange and QueryRecalc that somehow need to be transformed together ith an existng application's logic but that don't have their equivalent in the Xpages logic. Are both concepts (forms based and xpages based) just too different at this point?
As an example think of a workflow application where we need to check certain conditions before we allow opening an existing doc in edit mode for a potential author. In my Notes client application I add some code to 2 events, i.e. QueryOpen, where I check the "mode" arg, and 2nd QueryModeChange, where I check the current doc mode. In both cases I can prevent the doc from being edited by adding my continue = false, if necessary. Depending on the event the doc will either not change its mode, or not open at all.
With an Xpage I can use buttons for changing a doc's edit mode, and I can "hide" those buttons, or just add some checking code or whatever.
But 17 years of Domino consulting have tought me at least one lesson: there'll always be users that'll find the hidden ways to reach their goals. In our case they might find out that a simple modification of the page's URL will finally allow them to edit the doc. To prevent this I could maybe use the "beforeRenderResponse" event, I assume. But then, beforeRenderResponse is also called in other situations as well, so that we have to investigate the current situation first. Or I could make sure that users don't have author rights unless the situation allows it.
Again, not a huge problem, but when making the transition from a legacy Notes application this means re-thinking its entire logic. Which makes the job more tediuos, and especilly more expensive.
True? Or am I missing some crucial parts of the concept?
Structure your events as action groups and, when applicable, return false. This will cause all remaining actions in the group to be skipped.
For example, you could split a "Save" button into two separate actions:
1.
// by default, execute additional actions:
var result = true;
/* execute some logic here */
if (somethingFailed) {
result = false;
}
return result;
Replace somethingFailed with an evaluation based on whatever logic you have in place of the block comment to determine whether it's appropriate to now save the document.
2.
return currentDocument.save();
Not only does the above pattern cause the call to save() to be skipped if the first action returns false, but because save(), in turn, returns a boolean, you could theoretically also add a third action as a kind of postSave event: if the save is successful, the third action will automatically run; if the save fails, the third action will be automatically skipped.
All queryModeChange logic should be moved to the readonly attribute of a panel (or the view root of an XPage or Custom Control) containing all otherwise editable content... you would basically just be flipping the boolean: traditionally, queryModeChange would treat false (for Continue) as an indication that the document should not be edited (although this also forces you to check whether the user is trying to change from read to edit, because if you forgo this check, you're potentially also preventing a user from changing the mode back to read when it's already in edit), whereas readonly should of course return true if the content should not be editable.
Since the queryModeChange approach was nearly always an additional layer of "fig leaf" security, in XPages it's far better to handle this via actual security mechanisms; the readonly attribute is explicitly intended for enforcing security. Additionally, in lieu of using readonly, you could instead use the acl complex property that is also available for panels, XPages, and Custom Controls to provide different permissions to different subsets of users; anyone with a certain role, for instance, would automatically have edit, whereas the level for the default entry can be computed based on item values indicating the current "status" and/or "assignee". With either (or both) of these mechanisms in place, it doesn't matter what the user does to the URL... the relevant components cannot be editable if the container is read only. They could even try to hack in by running JavaScript in Chrome Developer Tools, attempting to emulate the POST requests that would be sent if they could edit the content... the data they send will still not get pushed back to the model, because the targeted components are read-only by virtue of the attributes of their container.
Attempting to apply all Notes client patterns directly to the XPages context is nearly always an exercise in frustration -- and, ultimately, futility. While I won't divulge specifics here, I (and some of the smartest people I know) learned this lesson at great cost. While users may say (and even believe) that they want exactly what they already have... if they did, they would be keeping what they already have, not paying you to turn it into something else. So any migration from a Notes client app to an XPage "equivalent" is your one opportunity to revisit the reason the code used to do what it did, and determine whether that even makes sense to retain within the XPage, based not only on the differential between Notes client and XPage paradigms, but also on any differential between what the users' business process was when the Notes client app was developed and what their process is now.
Omitting this evaluation guarantees that the resulting app will be running code it doesn't need to and fail to make the most of the target platform.
queryRecalc is a perfect example of this: typically, recalculation was blocked to optimize performance when the user's desktop and network resources were responsible for performing complex and/or network-intensive recalculations. In XPages this all happens on the server, so a network request from the browser that returns a page where everything has changed is typically no more expensive for the end user than a page where nothing has changed (unless there's an extreme differential in the amount of markup that is actually sent). Unless the constituent components are bound to data that is expensive for the server to recalculate, logical blocking of recalculation offers little or no performance benefit for the user. Furthermore, if you're trying to block recalculation in an event, you're too late: XPages uses a "lifecycle" that consists of 6 phases, so by the time your event code runs, any recalculation you're trying to block has already occurred. So, if the reason for blocking recalculation was to optimize performance, implement a scope caching strategy that ensures you're only pulling fresh data when it makes sense to do so, and the end user experience will be sufficiently performant without trying to prevent the entire page from recalculating. If, on the other hand, queryRecalc was being used as another fig leaf (something has changed, but we don't want to show the user the updates yet), that logic should definitely be revisited to determine whether it's still applicable, still (if ever) a good idea, and which portions of the platform are now the best fit for meeting the business process objectives.
In summary, use the security mechanisms unique to XPages for locking down portions or all of a page, and use the memory scopes that we didn't have in the Notes client to ensure the application performs well. Porting an event that used to contain this logic to an XPage event that continues to contain this logic will likely fail to produce the desired result and squander some of the benefits of migrating to XPages.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
What should a good BugTracking tool be capable of?
Although there is a large set of features that a bug tracker can have I feel like it is a little overkill and was considering rolling out my own solution. With that being said I didn't want to remove any core functionality that might be used frequently with existing solutions.
The ones I can think of so far:
- creating bugs
- assigning bugs
- closing bugs
- adding description to the bug
Thanks!
Communication between the developer and the user.
Ability for the user to assign certain bits of information such as severity (how much that bug relates to them).
Ability for the developer to override that priority and, if possible, give a reason.
Ability to assign tasks to a developer.
Ability to sort between bug, enhancement, and feature request. The difference between an enhancement and feature request is very subtle but VERY important.
Ability to attach files (such as screen shots)
Ability to have custom fields (such as being able to select which OS, which service pack level, application version, etc).
Ability to have custom user profiles which also give detailed information about their hardware. It's also nice to be able to have the users phone number (if they are on your LAN) so you can ask questions, if needed.
Privacy. Some items, such as security exploits or information that deals with financial information, will need to be kept secret. Even OSS does this from time to time until they can get a patch ready. Everyone has their own rules.
Ability to show the changes between revisions so you can email out a Change Log so users know what you have and have not done.
Reminders about which items are left undone and are assigned to you / unassigned at all.
That's all I can think of...
A good search engine.
It's amazing how many bug tracking products that cost thousands of dollars get this horribly wrong.
Without a really decent search your bug tracking is more like a "bug logging" - log and forget - system which is pretty much useless.
create a bug
close a bug
this is sufficient for closure over the life-cycle of a 'bug' entity. Whether it is enough features for your purpose is another matter.
Take a look at the features of Mantis, choose the features that you need, calculate how long it would take you to write them, and then spend your time on something more useful unless you absolutely have to create your own. ;-)
For most systems like a bug tracking one, it's usually not the creation or editing of the data that makes the system useful. It all comes down to how easily you can navigate through the information to 'add-value' on top of just collecting the data.
Think about the people who will use the system, the programmers, managers, etc. For each group of people, what type of information will make it worth their while to come back to the system over and over again. How can you make it easier for them to get this information?
Collecting information is easy, adding value to it is the hard part.
Paul.
A bug tracker is nothing more than a list of things that need to be done.
It can be as simple as a text file in the software's directory to a fully fledged bug tracker with hundreds of users.
Start with what you need to work with, then expand as needed.
Use Jira, you'll be in good hands.
Here are some important features:
Assign priority to bug (e.g. critical, major, medium, minor, trivial)
Assign bug to a specific release in which it will be fixed
Watcher functionality (so you can be e-mailed when the status changes)
Workflow (i.e. who is working on it, what's the status)
Categorization, Prioritization, and Standardization.
And an easy way to query it so that you can reap the rewards of your hard work on the above three.
Also, make sure whatever you do is extensible! We always decide to add/edit our bug templates during the project depending on needs/fires.
There are a lot of great solutions out there, you probably don't need to roll your own.. But either way you're going to have to make the same decisions. We use a solution that allows us to roll our own templates, so at the beginning of every project we revisit this same discussion.
FWIW: When we rolled our own request tracking system, we built it around procmail and our existing internal web authentication system because we wanted it to be extremely unobtrusive to use: we just send e-mails to the developers (using group aliases if we want) and add a "[t]" to the subject to open a ticket. The recipients get a modified e-mail with the original request and an additional link to the web page that displays the ticket and allows them to close it with 1 mouse click. So the most common tasks are performed through the e-mail client (opening, requesting more information, replying, ...), although there is also a simple web interface for searching etc.
It took only a few hours to write and after more than 34000 request tickets in 7 years or so, I guess it's OK to claim that it has only the essential core features:
create a ticket (by e-mail with marked subject)
close a ticket (clicking on the link in the e-mail, then clicking on "done")
all communication goes over e-mail, not through a web interface(!)
people who were recipients or sender of the original e-mail (opening ticket) are notified about closed tickets ("Subject: <old subject> closed by <someone>" + link to ticket in the body, enough information for most people so they don't have to go look which ticket/bug that was etc.)
a simple web interface provides a search function for own/open/sent/team tickets
Notable absent features that might be needed for a bigger development team / more intense software development:
flexible status for the tickets (dupe, wontfix, reopened etc.)
priorities
reassigning tickets explicitly (in our dev team, the e-mail just gets resent to the unlucky guy who has to do it)
adding comments to the ticket that don't get sent to everyone
assigning the bug to a particular version of the software
YMMV, but it has worked very well for us so far, both for bugs and for simple requests that the sender wants to keep track of.
Define bug.
Thinking about that will most likely make you realize that you're gonna spend a lot of time "rolling your own".
This might be a little beyond what you had in mind, but for me, integration with source control is a must-have. To be able to view the diffs between versions associated with a bug/issue is very handy.
Please please please don;t spend much time "rolling your own". Your time is better spent researching and learning to use real tracking systems.
Some to look at
Trac, Bugzilla and FogBugz. The last one has free hosted solution for small (one or two man shops?) companies.
SO has lots of threads about this topic.
Try not to roll your own unless it is just a word doc or a spreadsheet. Any time you spend making your own is a TOTAL waste.
EDIT
Since you won't be dissuaded, then I'll maybe add some things others have not mentioned.
You need reporting functionality - users need to be able to run queries and they should be able to select the fields they want to "view".
Workflow/lifecycle of a defect is also a good feature. (basically a state machine of the states the defect will go through. ) In fact, this is a useful exercise for you to define all your use cases and functionality. Given that you are in college and did not start out as aa CS major, I doubt you will come up with many on your own. Take some time to browse the feature lists and demos of existing products.
Ability for emails to be sent to various interested parties.
Anonymous users able to see a SPECIFIC defect that they entered
Different access levels and authorities (admin, manager, developer, tester, end-user)
Our bug tracking system is one of the two essential links between my company and our customers ("live" product reviews where existing customers are encouraged to suggest improvements and user interface tweaks being the other).
A bug tracking system must, first and foremost, encourage trackable "dialogs" with your customers. It must answer the question "Have you fixed the problem (defined broadly) that I have been having yet?"
It must have (in no particular order):
A short description of the problem or feature request (the title)
Room for an extended description
The ability to attach files/images (screenshots)
The ability to prioritize bugs/features
The ability to categorize entries as bugs, features, inquiry, etc.
The ability to assign bugs/features to areas (UI, database, documentation, etc.)
he ability to assign bugs/features to products (we track bugs on five products)
The ability to assign bugs/features to releases ("to be fixed in version 5.1")
The ability to assign bugs/features to people (developers/writers)
The ability to assign bugs/features to customers (reporters)
The ability to re-assign to a different person (developer)
The ability to Resolve bugs/features (mark them as finished and ready for testing)
The ability to mark resolution status (fixed, won't fix, can't reproduce, etc.)
The ability to Close bugs/features (take them off list after resolution & testing)
The ability to Reopen bugs/features (restore to "Open" if testing fails)
The ability to inform customers the bug has been resolved (e.g. via email)
Date and Time stamp on every step (Open, Resolve, Close, Re-open)
The ability to report on the number of Open bugs! (how close to release are we?)
The ability to show bug reports versus resolutions
The ability to search on bugs/features by date, priority, product, person, etc.
The ability to list and sort bugs for easy scanning!
Those are the things that we typically use in our system (FogBugz). While this may seem like a long list, we really do use every feature that I've listed here!