I think the documentation only explains how to use the model through an API but that does not allow much flexibility nor automation. For example, I do not know how to test my model against some popular benchmarks from HuggingFace.
The general flow of fine tuning Open AI models consists of creating an account, having a valid API key and then uploading the data for fine tuning using the CLI tool, as described here: https://beta.openai.com/docs/guides/fine-tuning
Then to test against question answering benchmarks, like SQuAD you simply dowload the dataset, create a script that takes the questions (see below json snippet) and feeds to your model by calling the API as described here (using curl): https://beta.openai.com/docs/api-reference/making-requests
"question": "What century did the Normans first gain their separate identity?",
"id": "56ddde6b9a695914005b962c",
"answers": [
{
"text": "10th century",
"answer_start": 671
},
{
"text": "the first half of the 10th century",
"answer_start": 649
},
{
"text": "10th",
"answer_start": 671
},
{
"text": "10th",
"answer_start": 671
}
],
"is_impossible": false
Related
So Vosk-api is a brilliant offline speech recogniser with brilliant support, however with very poor (or smartly hidden) documentation, at the moment of this post (14 Aug, 2020)
The question is: is there any kind of replacement of google-speech-recognizer feature, which allows additional transcription improvement by speech adaptation?
E.g.
"config": {
"encoding":"LINEAR16",
"sampleRateHertz": 8000,
"languageCode":"en-US",
"speechContexts": [{
"phrases": ["weather"]
}]
}
For Google this config means that phrase weather will have more priority, with respect to, say, whether which sounds the same.
Or class tokens?
I understand that it may not be implemented in Vosk for python3, but still...
Here are references:
https://cloud.google.com/speech-to-text/docs/class-tokens
https://cloud.google.com/speech-to-text/docs/speech-adaptation
You can follow this document for information on Vosk model adaptation:
https://alphacephei.com/vosk/adaptation
Basically there are 4 levels:
Update small model with list of words to recognize
Update small model offline with the language model from texts
Update language model and the dictionary inside the big model
Finetune acoustic model on your data
The process is not fully automated, but you can ask in the group for help.
I am trying to run batch tests for my LUIS app. The entities labelled as List Entity are displayed as False Positive inside the chart.
I have already gone through the below question and I believe this issue has already been resolved. But I still have issues with the list entity type. I have not included list entities inside the batch file.
LUIS Batch Testing for Entity is Not Working
In the documentation, it is stated that:
"Only the machine-learned entities are used: simple, hierarchical (parent-only), and composite entities. Do not add non-machine-learned entities because they are always found either through regular expressions, or explicit text matches"
I have used the following syntax inside batch test file (no labelled entites):
[
{
"text": "example utterance goes here",
"intent": "intent name goes here",
"entities": []
}
]
Do I have to exclude all the utterances containing the list entity while creating the batch file ? If so how can I test the Intent prediction for those utterances using batch tests?
Do I have to exclude all the utterances containing the list entity while creating the batch file ?
Correct.
If so how can I test the Intent prediction for those utterances using batch tests?
You'll need to test them manually.
Per the docs,
Entity types allowed: only machined-learned entities of simple and composite. Batch testing is only useful for machined-learned intents and entities.
The reason for this is that batch testing is meant to test scenarios where you've made changes to your app and you want to ensure it still works. Detection of non-machine-learned entities is never going to change, so they don't really need to be batch-tested (other than maybe the first time).
That being said, I understand you'll likely still want the ability to do this. I recommend opening a feature request on the LUIS UserVoice
FHIR
is a standard for health care data exchange, published by HL7®.
The DocumentReference
Provides metadata about the document so that the document can be
discovered and managed
Through the Provenance one can
describe entities and processes involved in producing and delivering
or otherwise influencing that resource
Nearly 80 percent of clinical information in electronic health
records(EHRs) is "unstructured" and in a format that health
information technology systems cannot use.
It is therefore natural to apply computer techniques to automatically generate structured data from the medical records. For that there are several implementations available both on the market and also fully open source. For example cTAKES, CLAMP, NOBLE, ClarityNLP and others are all freely available solutions targeting this task.
They all address the specific need of generating structured data from unstructured medical notes, however they all deliver the structure using their own format, that eventually could be converted into FHIR.
However, a central problem is on how to represent the Provenance of the extracted information, since FHIR is - to the best of my knowledge - missing the way of connecting to the precise location within the DocumentReference object of where the information has been extracted from , with which technology, and which is the level of "quality" of the extracted information.
Before submitting a Change Request https://gforge.hl7.org/gf/project/fhir/tracker/?action=TrackerItemBrowse to the FHIR normative, it is recommended to expose the issue to the widest community and the stackoverflow.com is one of the main recommended channels.
For this purpose I am hereby looking forward opinions on the matter, and namely on how to specify the provenance of FHIR Resources generated by applying NLP over medical narratives. For example, taking an example from the Adverse Event Corpus of Gurulingappa et al https://doi.org/10.1016/j.jbi.2012.04.008 ,
10030778|Intravenous azithromycin-induced ototoxicity.|ototoxicity|43|54|azithromycin|22|34
123456789012345678901234567890123456789012345678901234567890
1 2 3 4 5
The question is how to represent into FHIR that such drug induced problem has been extracted from the specific bytes positions 22-34 (drug) and 43-54 (problem) from the text (the Title of the paper 1999 in this example).
{
"resourceType": "AdverseEvent",
"id": "example",
"actuality": "actual",
"category": [
{
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/adverse-event-category",
"code": "product-use-error",
"display": "Product Use Error"
}
]
}
],
"event": {
"coding": [
{
"system": "http://snomed.info/sct",
"code": "9062008",
"display": "Ototoxicity (disorder)"
}
],
"text": "10030778|Intravenous azithromycin-induced ototoxicity."
},
"subject": {
"reference": "Patient/example"
},
"date": "1999-02-29T00:00:00+00:00",
"seriousness": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/adverse-event-seriousness",
"code": "Non-serious",
"display": "Non-serious"
}
]
},
"severity": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/adverse-event-severity",
"code": "mild",
"display": "Mild"
}
]
},
"recorder": {
"reference": "Pharmacotherapy. 1999 Feb;19(2):245-8."
},
"suspectEntity": [
{
"instance": {
"reference": "Azithromycin"
}
}
]
}
Currently the FHIR standard does not allow to represent the precise byte position, the quality of the extraction, and the method used to perform it.
Excellent discussion. The FHIR Provenance so far today is focused on the most likely need of Provenance. An important Principle of FHIR is that we focus on the most needed functionality before we focus on things that are not likely to be implemented or used. This is not to say those things not yet supported are not likely to be implemented or used, but rather simply pointing out the Prioritization of what gets addressed.
So, looking at your use-case, I first ask if it is realistic that a consumer of some FHIR Resource (e.g. Observation) will care deeper about Provenance than a gross statement that the Observation was extracted from a specific Document. That is to ask if it is important to have recorded the Provenance details any deeper than is available today? This is not to say that academically it is interesting, it surely is logical. But how useful is it, especially since it will be very expensive to record this level of detail.
Lets assume there is some reasonable, although small, need for this. The FHIR specification enables anyone to define extensions anywhere. So you could define an extension to the Provenance resource that supports your use-case. Likely some extension on the Provenance.entity. If there is some reasonable cohort that need this extension, it could be defined in a publicly accessible Implementation Guide using StructureDefinition, and be registered on the fhir.org as such. If this seems even more useful, these extensions could be added to the FHIR specification within the Provenance resource.
However I am not convinced that this is a widely needed functionality beyond those extension based mechanisms. Especially since this need is only needed with DocumentReference based Binary that are not structured.
The good news is that Provenance is simply at FMM of 3, and we do expect to try to get it normative in the next R5 release. So now is the time to have these excellent discussions.
This question already has answers here:
MongoDB relationships: embed or reference?
(10 answers)
Mongoose populate vs object nesting
(1 answer)
Closed 4 years ago.
I have a personal project about writing articles.
I do in nosql with mongoose. For my API i use nodejs + express.
But i'm fluent with sql, but i'm begginer in Nosql...
My problem is about my articles with categories.
I want set 2 level of categories like that :
Category 1
Subcategorie 1
Article
SubCategorie 2
Article
SubCategorie 3
Article
Category 2
Subcategorie 4
Article
SubCategorie 5
Article
SubCategorie 6
Article
Then my articles are populate in subCategories.
I started to set 3 documents :
Categorie
SubCategorie
Articles
Like this way, i can populate my articles in SubCategorie, and populate my SubCategorie in Categorie.
But i'm not sure than is the best way to use mongoose and nosql in general. I need an other point of view.
I want respect the best practices obviously.
To be simple i'll need to :
Display all articles from SubCategory
Display all articles from Category
Display Category and SubCategory on the page of one article.
I would permit to use only the existent category as well.
Any suggestion ?
Thanks a lot ;)
As a general rule for document oriented databases, you should structure your documents according to the access.
If you want to have a page that renders a full article with the names of the category and subcategory, you could structure articles like that:
{
"_id": ObjectId("5b0f9961076337823360d072"),
"category": "Gadgets",
"subcategory": "TVs",
"title": "New TV from ACME",
"text": "Lorem Ipsum ....",
"published": ISODate("2018-05-31T06:42:41.270Z")
}
This way you can fetch all data for the article page with on query.
If you wanted to show all articles for one category and subcategory, you could filter on those fields and need only one query. In that case you should create indices on those fields to speed up the queries.
You could (but it depends on how you want to access your categories) save your categories and subcategories in one collection. Those documents could be structured like that:
{
"_id": ObjectId("5b0f9961076337823360d072"),
"name": "Gadgets",
"subcategories": ["TVs", "Computers", "Phones"]
}
But perhaps that structure does not fit your needs. In that case you could have two collections. It always depends on the implementation of your application.
You will notice, that there are redundancies in the data structure. The name of the categories are saved multiple articles. But that is ok. You have to make sure in your application that the saved data is correct by validating the input before saving.
The redundancies prove another problem: If you want to rename a cateogry, you would have to update each document that has the category with the changed name. But those updates usually happen not often. Document design is a tradeoff. Fast fetching, because you only access one collection or even only one document. Tedious and slow updates. But with a good document structure you can minimize the slow operations and maximize the fast ones.
As I've been taught personally, as opposed to relational, the main point of NoSQL is originally to avoid having to join tables with each other, as much as possible. This is typically a trade-off between data replication (ACID properties) and ease of update VS. query efficiency.
In practice, this would consist in nesting one document in the other. e.g, each Article would contain the Category data of which it belongs (but this could be implemented the other way around depending on your needs).
Article {
name: string,
content: string,
category: Category { name: ... }
}
The following is only my very personal opinion; but I'm not a huge fan of Mongoose in that matter.
It does add up an abstraction layer which is indeed reassuring for developers coming from the relational world, but if not used with extreme care, you may easily fall back into doing relational with a tool not designed for that.
Edit: Much better example from mbuechmann above.
I think you can go with this approach. There will be 2 collections
Article
Categories
Categories collections will have for example following types documents.
{
"_id" : ObjectId("5a7030519697334c17afe3e6"),
"parent_id" : ObjectId("5a7030469697334c17afe3e5"),
"name" : "cat 1",
"is_subtype" : true,
"sub_types" : [
ObjectId("5a7030969697334c17afe3e9"),
ObjectId("5a70309c9697334c17afe3ea")
],
}
Here you can now populate parent category and their child category by using their Id. Also In Article you can populate category information by category id.
And Article will have following types of documents
{
"_id": ObjectId("5a7030519697334c17afe3e6"),
"category_id": ObjectId("5a7030469697334c17afe3e5"),
"text": "article 1",
"slug": "article-1"
}
I hope this is what you're looking for.
Basically I have a web application that I'm building that will store scores grouped by a specific key ("Class"). "Class" can be 9A,9B,9C,9D, and the full document will be something akin to:
{
"Class": "9A",
"Course": "XX",
"Category": "Acat",
"Score": xxxx // integer
}
Basically I only want to retrieve the top 10 results for each class / course, and I'm not sure what the best solution here will be. Atm I have a couple methods I can think of, but neither seem like good practice to me:
Using the async module or promises (Q.all), perform one query for each class in parallel.
Select all the results, and limit it in a loop, then send the data once I have 10 results for each.
Option A seems better of the two, but not ideal. There could not be a better solution, but figured I'd look and see if there is one.
Cheers!