I have installed sphinxbase, sphinxtrain and pocketsphinx in Linux (Ubuntu). Now I am trying to train data with speechcorps,transcriptions, dictionary etc obtained from VOXFORGE. (My etc and wav folder's data is obtained from VOXFORGE)
As I am new so I just want to train data and get some results with few line of transcripts and few wav files. let say 10 wav file and 10 transcript lines cosponsoring to it. Like this person in doing in this video
but when I run sphinxtrain then I am getting error.
Estimated Total Hours Training: 0.07021431623931
This is a small amount of data, no comment at this time
If I do CFG_CD_TRAIN= no I dont know what it means.
What changes I need to make? So I am able to remove this error.
PS: I can not add more data because I want to see some results first for my better understanding the whole scenario.
Not enough data for the training, we can only train CI models
You need at least 30 minutes of audio data to train CI models. Alternatively, you can set CFG_CD_TRAIN to "no".
Related
So I am using the keras module to create a facial recognition program but I have hit one problem, my computer can`t compute all the answers at once so I change the data to smaller amount and calculate each part until it is close to 100% accuracy. My data is constantly being trained with different data e.g. Happy face, Sad face and Confused face the code is then trained with this data then another set of data Angry face, Lonely face and Amazed face and the code is trained with this data. The two datasets are run at different times but both produce a h5 file with the data they have collected. How can I combine these two or more files into one singular file. I am guessing that the model may have to be retrained with the h5 files and then produce a singular 5h file but I do not know. Anyone know how to combine two trained models saved in h5 files?
The code below shows where I train and save the model before changing the data and rerunning the code.
model.fit(train_generator, steps_per_epoch=int(train / batch_size), epochs=epochs, callbacks=[checkpoint, lr_scheduler])
I just want little guidance that there are 3 IOB file dev, test & train.
Dev has 1 million lines.
Test has 4 million lines.
Train has 30 million.
I am currently just converting dev file as of now because i wasn't sure whether is there any error or not in it.
(the IOB format is correct) It's been over 3 hours as of now can idea will this file work or shall I use something else.
I am fine-tuning a bert model using spacy in google colab the Runtime hardware chosen is GPU and the , and for reference I have followed this article:
https://towardsdatascience.com/how-to-fine-tune-bert-transformer-with-spacy-3-6a90bfe57647
I have followed the exact steps of the article.
I am not familiar with NLP domain neither do I have profound knowledge of pipelining. Can someone please help regarding this, it's really important.
Below i would attach the image regarding time and the statement executed for conversion.
Image showing time elapsed and command executed
I am trying make an app which will detect traffic sign from video's frames.I am using yolo-tensor by following steps from https://github.com/thtrieu/darkflow .
I need to know how can I train this model with my data-set of images of traffice signs?
If you're using Darkflow on Windows then you need to make some small adjustments to how you use Darkflow. If cloning the code and using straight from the repository then you need to place python in front of the commands given as it is a python file.
e.g. python flow --imgdir sample_img/ --model cfg/yolo-tiny.cfg --load bin/yolo-tiny.weights --json
If you are installing using pip globally (not a bad idea) and you still want to use the flow utility from any directory just make sure you take the flow file with you.
To train, use the commands listed on the github page here: https://github.com/thtrieu/darkflow
If training on your own data you will need to take some extra steps as outlined here: https://github.com/thtrieu/darkflow#training-on-your-own-dataset
Your annotations need to be in the popular PASCAL VOC format which are a set of xml files including file information and the bounding box data.
Point your flow command at your new dataset and annotations to train.
The best data for you to practice is PASCAL VOC dataset. There are 2 folders you need to prepare for the training. 1 folder with images and 1 folder with xml files(annotation folder), 1 image will need 1 xml file (have the same name) content all the basic informations (object name, object position, ...). after that you only need to choose 1 predefine .cfg file in cfg folder and run the command follow:
flow --model cfg/yolo-new.cfg --train --dataset "path/to/images/folder" --annotation "path/to/annotation/folder"
Read more the options supported by darkflow to optimize more the training process.
After spending too much time on how to train custom data set for object detection
Prerequisite :
1:training environment : a system with at least 4gb GPU or you can use AWS / GCP pre-configured cloud machine with cuda 9 installation
2: ubuntu 16.04 os
3: images of the object you want to detect. images size should not be too much large it will create out of memory issue in dataset training
4: labelling tool many are available like LabelImg/ BBox-Label-Tool i used is also good one
I tried python project dataset-generator also but result of labelling using dataset generator was not efficient in real time scenarios
My suggestion for training environment is to use AWS machine rather than spend time in local installation of cuda and cudnn even though you are able to install cuda locally but if you are not having GPU >= 4 gb you will not be able to train many times it will break due to out of memory issue
solutions to train data set :
1: train ssd_mobilenet_v2 data set using tensorflow object detection api
this training output can be use on both android , ios platform
2: use darknet to train data set which required pascal VOC data format of labelling , for that labelIMG can do the job of labelling very good
3: retrain that data weights which comes as output from darknet with darkflow
I have been working with CMUsphinx for Turkish language speech to text for couple months. I have succeeded to run a train on a 100 hours of sound. My target was to use the resulting Acoustic Model with Sphinx3 decoder. However Sphinx3 decoder cannot decode my test wav files. Then I have noticed that sphinxtrain runs pocketsphinx_batch in the end of training for testing the model.
So, I started working on poscketsphinx. I am at a point where pocketsphinx batch cannot decode a wav file (actually it only produces ııı nothing else) but pocketsphinx continuous produces more meaningful output with the same file (e.g. 10 correct words out of 15 words).
I guess I am missing some configuration steps. I have an compressed archive in this link
which includes the Acoustic and language models, dictionary and wav files I try to decode.
I am asking to get help for being able to use my model with Sphinx3 and Pocketsphinx_batch.
Thank you.
Fortunately I found the problem. It was feature vectors which are produced by sphinx_fe. I was creating them with default values. After reading the make_feats.pl and sphinxtrain.cfg files, I created feature vectors compatible with the Acoustic Model. Sphinxtrain.cfg has the lifter parameter as 22, but if we use sphinx_fe with default values lifter is 0, which means no lifter. I created mfc with lifter value 22 then it worked.
I am trying to train a binary classifier using libsvm.
My data quantity is very large and I need to know of any way I can divide the input data into different files and input to the train function.
So basically I know this :
svm-train train file
I wonder if there's a way to do:
svm-train train_file1 train_file2 train_file3.....
Does anyone know any way to do this??
From the FAQ's of libsvm
For large problems, please specify enough cache size (i.e., -m). You may train only a subset of the data. You can use the program subset.py in the directory "tools" to obtain a random subset.