Where should I look for documentation or hints for different kinds of layers available in pycaffe? and their attributes?
I want to define an architecture which uses batch-normalization, but I'm lost, in the examples there is only convolution, fully connected, max-pooling and relu layers. nothing more.
Pycaffe is just a wrapper/interface for caffe.
Check here for the list of layers available in caffe.
Find the layer you are interested in (e.g. batch_norm_layer.hpp). Open the header file and check the method type(). The return value of this method (e.g. "BatchNorm") is what you are looking for and is used in pycaffe.
Read the comments at the top of each header file. It explains layer details and their input parameters.
As a supplementary approach, take a look at layer implementations. Check the LayerSetUp method and look for param or this->layer_param_. You can find the exact parameters and their names.
The parameters' names are the same as those in the network description (.prototxt) files. Find an example and you will figure out the parameters.
Related
I am new to this field, this question might look dumb to some of you but please bear with it.
I have created a keras model which works well. I can save it in .hdf5 format using model.save("model_name.hdf5) which is good.
Main question is if there is any other format i can save my model in so that it can be used in c++/android/JavaScript. Is this even possible to do so?? If you are thinking why I am asking for all 3 languages, its because I have 3 projects each of them use respective language.
Thanks for any help in advance.
The answer depends on what you are going to save and use in another language.
If you just need the architecture of the model to be saved, you may save it as JSON, which can later be used in any other platform and language you are going to use.
model_json = model.to_json()
If you also need the weights and biases, I do not know any specific tool, but you can simply read the stored data in python, create a multidimensional array, and then save it in a file appropriate for any of the languages you need. For example, the weights of the second layer can be found in model.layers[2].get_weights().
Finally, if you want to run the model in another language, you need to implement the for-loops that make the processing. You might find some conversion tools for your target language. For example, for C, this tool can help.
I'm looking at the CAN layers contrib and its documentation
I want to use the SignalPacket solution to implement a few messages described in DCB's but this raises a question. How can I easily make use of the CAN layer and then bind my custom SignalPacket based on the identifier field in the CAN layer, specifically how do I handle getting the 8 byte payload data that today is included in the data field of the CAN class, into my custom SignalPacket instance?
Alternatives:
1: Modify the CAN layer, removing the payload, then binding my custom SignalPacket to the CAN layer based on the identifier?
2: Leave the CAN layer untouched and in some way access the parent layer's data using pkt.underlayer to populate the custom SignalPacket?
UPDATE:
Answered my own question after finding a solution from the maintainers, see separate reply below
Remaining question: Has anyone used the #2 type of solution in any other case and care to describe how it would look?
Polybassa has an answer to the question on the scapy github, proposing two alternatives #1 + a version of the existing solution with the CAN type and the custom type.
https://github.com/secdev/scapy/issues/2252
We have a system that allows users to answer a question as free text and we want to check whether their answer contains any of our predefined topics. These topics will be defined prior to answers being submitted.
We tried to use a method similar to spam detection, but this is only good for determining whether something is true/false, incorrect/correct. We need the response to say which of the predefined topics a piece of text contains. Is there an algorithm that would solve this problem?
Maybe you will try to use "bag of words" for feature extraction and "naive Bayes classifier with multinomial model" for classification.
In this page this described more detail link.
You could also try explicit semantic analysis (ESA)[1][2]. Given a set of documents that represent concepts (in your case your topics) you can train a model and given any new sentence as input you can get a ranked list of the closest concepts "evoked" by that sentence. Of course this assume you have a document with some text describing every concept you want to identify (that's why the most common thing to do is to use Wikipedia pages as concepts), but if this is the case you could give it a try.
[1] https://en.wikipedia.org/wiki/Explicit_semantic_analysis
[2] http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf
I Use a CQRS thin read layer to provide denormalized lists/reporting data for the UI.
In some parts of my application I want to provide a search box so the user can filter through the data.
Lucene.NET is my full text search engine of choice at the moment, as I've implemented it before and am very happy with it.
But where does the searching side of things fit in with CQRS?
I see two options but there are probably more.
1] My Controller can pass the search string to a search layer (Lucene.NET) which returns a list of ID that I can then pass to the CQRS read layer. The read layer will take these IDs and assemble them into a WHERE ID IN (1,2,3) clause, ultimately returning a DataTable or IEnumerable back to the controller.
List<int> ids = searchLayer.SearchCustomers("searchString");
result = readLayer.GetCustomers(ids);
2] My thin read layer can have searching coded directly into it, so I just call
readLayer.GetListOfCustomers("search string", page, page1);
Remember that using CQRS doesn't mean that you use it in every part of your application. Slicing an application into smaller components allows using various architectural principles and patterns as they make sense. A full text search API might be one of these components.
Without knowing all the details of your application, I would lean towards a single entry point from your controller into your search layer (your option #2 above). I think your controller shouldn't know that it needs to call layer #1 for full-text enabled searching and layer #2 for just regular WHERE clause type searching.
I would assume you would have two different 'contexts' (e.g. SQLContext and LuceneContext), and these would be dependencies injected into your ReadLayer. Your read layer logic should then make the decision on when to use LuceneContext and when to use SQLContext; your controller won't know and shouldn't know.
This also allows you to swap out LuceneContext for MongoContext if there's a compelling reason in the future without your controller knowing or having to change.
And if you use interfaces for everything (e.g. ISearchContext is implemented by LuceneContext and MongoContext), then really the only change for swapping out contexts is in your IoC container's initialization/rules.
So, go with option #2, inject in your dependencies, have your controller just work through your read layer, and you should be good to go. Hopefully this helps!
I want to recognize named entities in a specific field (e.g. baseball). I know there are tools available like StanfordNER, LingPipe, AlchemyAPI and I have done a little testing with them. But what I want them to be is field specific as I mentioned earlier. How this is possible?
One approach may be to
Use a general (non-domain specific) tool to detect people's names
Use a subject classifier to filter out texts that are not in the domain
If the total size of the data set is sufficient and the accuracy of the extractor and classifier good enough, you can use the result to obtain a list of people's names that are closely related to the domain in question (e.g. by restricting the results to those that are mentioned significantly more often in domain-specific texts than in other texts).
In the case of baseball, this should be a fairly good way of getting a list of people related to baseball. It would, however, not be a good way to obtain a list of baseball players only. For the latter it would be necessary to analyse the precise context in which the names are mentioned and the things said about them; but perhaps that is not required.
Edit: By subject classifier I mean the same as what other people might refer to simply as categorization, document classification, domain classification, or similar. Examples of ready-to-use tools include the classifier in Python-NLTK (see here for examples) and the one in LingPipe (see here).
Have a look at smile-ner.appspot.com which covers 250+ categories. In particaul, it covers a lot of persons/teams/clubs on sports. May be useful for your purpose.