Considering me a newbie into this but if I want to develop an engine to extract only validations from a technical documents like functional specification. Usually the validations are quick to identify.
If this can be done somehow I can use it for further automation.
I checked and few frameworks are available like
https://opennlp.apache.org/
http://nlp.stanford.edu/
I did few POC's as well , but designing an intelligent engine having generic rules is where I am getting blocked.
Any pointers will be helpful ...
For extracting only validation you need to have the grammar or similar like structure maybe a regular expression. If you dont have ANY idea what type of validation you might face then you can follow un-supervised learning approach. Designing any generic rule is not as easy as it sound so you need to work extra hard for this.
Related
I am mediation developer, I want to know about which languages are used in telecommunication for mediation.
Comptel Eventlink is quite a sophisticated mediation solution and it has multiple components and multiple ways to configure and extend it.
It includes custom environments to extend and build operator specific functionality, for example the Business Logic Tool which provides a graphical UI to build business logic.
There are also many different data aggregation, filtering etc rules which can be configured.
So, a lot of the skills you would need to work with a solution like this will be specific to the solution itself. There does not seem to be many courses offered at this time although you may find some in the more general Nokia courses as Nokia now owns Comptel. There also seems to be a good collection of Comptel manual online if you search in places like kupdf.net.
You can use Perl, C, C++ and Comptel's Business Logic Tool to program mediation based on EventLink.
Can you give me suggestions of graph libraries that are best to develop Rete algorithm.
I'm using .net 4.0
I found QuickGraph but I'm not sure if it's useful in this case.
I'm not a C# dev, but I've implemented rete in another language. You want a directed acyclic graph algorithm, start looking here on github. Or perhaps here. However, you can get away with a simpler data structure with a visitor. And, if you haven't I'd read Doorenbos, 1995, which will walk you through how to implement the whole thing.
Well, I agree with Chase. I have built a rules engine using Composite and Visitor, and its working absolutely flawless. Composite helps in organizing rules in a hierarchy (nesting) and Visitor helps you draw unlimited operations like evaluators, visualizers etc. I'd suggest building a truth logic first using composite and visitor and then wrap it up with expression parsing, where expressions are represented as text, maybe XML nodes, which naturally has the hierarchical structure to represent nesting of rules. Best is that you can version expression based rules.
At the moment, I have to work with XACML. As there doesn't seem to be an editor to fit my needs, and as writing documents in it is a real pain, I wonder if I could not create some sort of DSL to make creating documents easier (are less error-prone). Is this possible with XText? I have a feeling it's possible but quite hard to do (especially for someone who doesn't know XText ;-)).
Getting rid of manually edited XML files is a typical use case for Xtext. The tedious part is the syntax definition itself. As soon as you have an idea how your files should look like, it's usually straight forward to get a working prototype with Xtext. What sort of concerns do you have?
I am looking for some resources pertaining to the parsing and understanding of English (or just human language in general). While this is obviously a fairly complicated and wide field of study, I was wondering if anyone had any book or internet recommendations for study of the subject. I am aware of the basics, such as searching for copulas to draw word relationships, but anything you guys recommend I will be sure to thoroughly read.
Thanks.
Check out WordNet.
You probably want a book like "Representation and Inference for Natural Language - A First Course in Computational Semantics"
http://homepages.inf.ed.ac.uk/jbos/comsem/book1.html
Another way is looking at existing tools that already do the job on the basis of research papers: http://nlp.stanford.edu/index.shtml
I've used this tool once, and it's very nice. There's even an online version that lets you parse English and draws dependency trees and so on.
So you can start taking a look at their papers or the code itself.
Anyway take in consideration that in any field, what you get from such generic tools is almost always not what you want. In the sense that the semantics attributed by such tools is not what you would expect. For most cases, given a specific constrained domain it's preferable to roll your own parser, and do your best to avoid any ambiguities beforehand.
The process that you describe is called natural language understanding. There are various algorithms and software tools that have been developed for this purpose.
I need do write an expert systems that should aid user in picking up best mobile phone operator. It should be very simple and not based on languages/libaries such as CLISP or JESS. So I need to write it all from the ground up.
Do you know some books or online tutorials that explains how this can be done?
What I really need to get to know is how to represent knowledge and facts.
Any help would be much appreciated.
If you get any of the good texts on AI, there will be a section on expert systems; you can, if forced, work it out from there and implement your own.
The basic idea is really fairly simple: you have a collection of rules in "if-then" form that represent inferences, or4 implications. Like, for example:
IF blood temperature > 41°C
THEN patient.has-fever := TRUE
IF patient has wet-sounding breathing
THEN patient.has-pneumonia
IF patient.has-fever AND patient.has-pneumonia
THEN CONCLUDE bacterial pneumonia. ACTION prescribe Augmentin
In other words, you have a bunch of rules, and you evaluate the rules until you get to a conclusion. There's a lot more to is (forward or backward chaing and that kind of thing) which you can read about in thed pretty decent Wikipedia article.
I'm puzzled why you can't use an existing rule engine though -- there are a number of them, for most languages, usually under pretty liberal licenses. That's really an easier route unless this is a homework problem or something.
Prolog is well suited to writing rule-based systems (a pretty standard approach to expert systems development). P# compiles to C#, which may meet your needs - and it's free.
More information on P#.
The basis rationale, and mathematical proof, for the PROLOG language, should help you understand most of the concepts you will need to address, if not provide the final language you need to use to implement it.
I couldn't find a link to the original implementation, but it would not help you much anyway. Alain Colmerauer's early work on logic programming should be helpfull.
[EDIT] Sorry, duplicate...
I would vote for some implementation of Prolog or CLIPS, depending if backward or forward chaining logic best suits the problem. Instead of re-implementing either of these, spend the time working out how to integrate them with your environment.
Jess is a good choice but you should read the book "Jess in action" as a first step.