I'm currently working on a project design where I will create a program/model to translate my native dialect to English, I'm asking is there any books or anything that can you recommend to me in creating my project.
On the NLP side of things there's this course: Natural Language Processing with spaCy & Python - Course for Beginners and this older course: Natural Language Processing (NLP) Tutorial with Python & NLTK on Free Code Camp, which is generally a good place to start. Their courses provide in depth explanations of concepts and provide good examples.
On the translation side of things, the DeepL translator is easy to use in multiple languages and offers a free api. It also offers and incredibly easy to use python library if that's the language you intend to use (which you should because python is the best out there for NLP).
I hope this helps, but dennlinger is right - you shouldn't typically ask broad recommendation questions on StackOverflow!
Good day. I am currently working on Machine Translation (Speech-(Text--Text)-Speech) with our local dialects and I already have the speech and text corpus. However, I am facing a problem in recording the speech as input and transcribing it to a text file because the modules available for speech recognition did not cover our dialects, mostly it just supports English and other major languages.
Is there anyone who know how I can fix it? I would be honored to accept your valuable suggestions and it will help me a lot on my studies. Thanks!
To work on text-to-speech for unusual dialects is a big challenge since frequently the audio models do not exist and have to be created from scratch. A good place to start is with one of the tutorials from http://voxforge.org. At this site you will find not only the tutorials involving a number of audio decoders and model generators but also a useful forum where students of various languages other than English have found solutions related to their own dialect problems.
A general plan might be as follows: build a simple English model following the examples given to get used to the terminology and concepts and process involved. Given your inevitable success with English, you can then turn your possession of a native corpus to advantage by building models for your own dialect. It is a reasonable goal and has been done many times before. Be warned however that to get good recognition across a broad vocabulary you will need a very comprehensive corpus data set.
I'm looking for an open source library to detect the spoken language used in an audio file, such as a wav file.
I tried CMU sphinx , but was not able to understand how to use it for language detection? Can someone please help?
If you are willing to learn another toolkit, you should consider Kaldi [1]. It is an open-source speech recognition toolkit with a speaker recognition system (which uses similar models as a language identification system) in the trunk and an experimental language Identification setup in the sandbox language_id. After checking out the repository, you can switch to the LID sandbox with svn switch ^/sandbox/language_id. The LID examples are in egs/lre07.
Whichever toolkit you use, I recommend an i-Vector based system instead of a phonotactic system. An i-Vector based system will be easier to setup as it doesn't require transcripts, and significantly faster, since it avoids decoding.
You can try CMU sphinx in all phone decode mode.
Train models for languages you wish to identify.
Pick language for which hypothesis score is best
I'm interested in creating a visual programming language which can aid non-programmers(like children) to write simple programs, much like Labview or Simulink allows engineers to connect functional blocks together without the knowledge of how they are internally built. Is this called programming by demonstration? What are example applications?
What would be an ideal platform which can allow me to do this(it can be a desktop or a web app)
Check out Google Blockly. Blockly allows a developer to create their own blocks, translations (generators) to virtually any programming language (or even JSON/XML) and includes a graphical interface to allow end users to create their own programs.
Brief summary:
Blockly was influenced by App Inventor, which itself was based off Scratch
App Inventor now uses Blockly (?!)
So does the BBC microbit
Blockly itself runs in a browser (typically) using javascript
Focused on (visual) language developers
language independent blocks and generators
includes a Block Factory - which allows visual programming to create new Blocks (?!) - I didn't find this useful myself...except for understanding
includes generators to map blocks to javascript/python
e.g. These blocks:
Generated this code:
See https://developers.google.com/blockly/about/showcase for more details
Best wishes - Andy
The adventure on which you are about to embark is the design and implementation of a visual programming language. I don't know of any good textbooks in this area, but there are an IEEE conference and refereed journal devoted to this field. Margaret Burnett of Oregon State University, who is a highly regarded authority, has assembled a bibliography on visual programming languages; I suggest you start there.
You might consider writing to Professor Burnett for advice. If you do, I hope you will report the results back here.
There is Scratch written by MIT which is much like what you are looking for.
http://scratch.mit.edu/
A restricted form of programming is dataflow (aka. flow-based) programming, where the application is built from components by connecting their ports. Depending on the platform and purpose, the components are simple (like a path selector) or complex (like an image transformator). There are several dataflow systems (just I've made two), some of them has no visual editor, some of them are just a part of a bigger system, and there're some which don't even mention the approach. (Did you think, that make, MS-Excel and Unix Shell pipes are some kind of this?)
All modern digital synths based on dataflow approach, there's an amazing visual example: http://www.youtube.com/watch?v=0h-RhyopUmc
AFAIK, there's no dataflow system for definitly educational purposes. For more information, you should check this site: http://flowbased.org/start
There is a new open source library out there: TUM.CMS.VPLControl. Get it here. This library may serve as a basis for your purposes.
There is Snap written by UC Berkeley. It is another option to understand VPL.
Pay attention on CoSpaces Edu. It is an online platform that enables the creation of virtual worlds and learning experiences whilst providing a more flexible approach to the learning curriculum.
There is visual coding named "CoBlocks".
Learners can animate and code their creations with "CoBlocks" before exploring and sharing them in mobile VR.
Also It is possible to use JavaScript or TypeScript.
If you want to go ahead with this, the platform that I suggest is the one used to implement Scratch (which already does what you want, IMHO), which is Squeak Smalltalk. The Squeak environment was designed with visual programming explicitly in mind. It's free, and Smalltalk syntax can learned in half an hour. Learning the gigantic class library may take just a little longer.
The blocks editor which was most support and development for microbit is microsoft makecode
Scratch is a horrible language to teach programming (i'm biased, but check out Pipes Visual Programming Language)
What you seem to want to do sounds a lot like Functional Block programming (as in functional block programming language IEC 61499 and other VPLs for mechatronics development). There is already a lot of research into VPLs so you might want to make sure that A) what your are trying to do has an audience and B) what you are trying to do can be done easily.
It sounds a bit negative in tone, but a good place to start to test the plausibility of your idea is by reading Davor Babic's short blog post at http://blog.davor.se/blog/2012/09/09/Visual-programming/
As far as what platform to use - you could use pretty much anything, just make sure it has good graphic libraries (You could use Java with Swing - if you like pain - or Python with TKinter) just depends what you are familiar with. Just keep in mind who you want to eventually launch the language to (if its iOS, then look at using Objective-C, etc.)
I have started working on a project which requires Natural Language Processing. We have do the spell checking as well as mapping sentences to phrases and their synonyms. I first thought of using GATE but i am confused on what to use? I found an interesting post here which got me even more confused.
http://lordpimpington.com/codespeaks/drupal-5.1/?q=node/5
Please help me decide on what suits my purpose the best. I am working a web application which will us this NLP tool as a service.
You didn't really give much info, but try this: http://www.nltk.org/
I don't think NLTK does spell checking (I could be wrong on this), but it can do parts of speech tagging for text input.
For finding/matching synonyms you could use something like WordNet http://wordnet.princeton.edu/
If you're doing something really domain specific: I would recommend coming up with your own ontology for domain specific terms.
If you are using Python you can develop a spell checker with Python Enchant.
NLTK is good for developing Sentiment Analysis system too. I have some prototypes of the same too
Jaggu
If you are using deep learning based models, and if you have sufficient data, you can implement task specific models for any purpose. With the development of deep leaning based languages models, you can used word embedding based models with lexicon resources to obtain synonyms and antonyms. You can also follow the links below to obtain more resources.
https://stanfordnlp.github.io/CoreNLP/
https://www.nltk.org/
https://wordnet.princeton.edu/