I am quite new working with High-Performance Computing (HPC). In the one I am working I cannot install any packages from the internet, I don't even have access to the internet.
The support I have is very limited when I contact them. I am developing a small application and I need the last version of pandas. They have told me that to get the last version I need to do this:
module load lang/Anaconda3
sources/resources/conda/miniconda3/bin/activate
conda activate SigProfilerExtractor
This work but I want to create my own environment. I know how to do this on my own computer but in the HPC I do not know how to do this? I know that the packages must be somewhere but how can I install them in my environment from where they live?
And second question: They have tools and packages located in many different environments so it is very difficult to find out when the future tool I will use is. Not all environments have useful names and the manual they provided is not updated. If I need the tool mytool, how can I find it?
I'm working with autodesk forge, because I have a project for the University, but I have a small problem now.
I read that the IFC Model Derivative processor is currently based on the Navisworks importer, but a better implementation is using Revit instead. As I work mostly with IFC Files this could make a big difference.
I made a small application with the help of the forge-apis npm package, which enables me to easily integrate the Forge APIs.
The Problem is that I don't see an option to use the new Model Derivative IFC pipeline. Is there a way to implement it in my application?
As this point, you'll need to make requests (e.g. using request package). When we completely move to Revit pipeline you can resume with the forge-api package.
My question pertains to distributing a python application the proper way. i have done research, but the answers appear to be vague and usually instruct the user to upload projects to the python package index. I simply want to make sure i research the proper methods instead of steering in the wrong direction.
I have been writing a stock analysis application with stored text files, and a dozen or so python scripts that interact with each other. Outside computers are to communicate with my servers through a random port (not the internet ports). all the imports work properly and the communication works properly, but i have no clue how to approach the distribution/setup or the GUI.
As an application with dozens of python scripts, should i write each script as an executable via pyInstaller, or should i add a python environment to the application to install python on all the user systems?
I need to add 'README', and 'LICENSE' to the application. What is the point of set up? is that a resource so python knows what is going on or does it actually setup the application? Does it need a URL, and what is the point of using the URL? Do i need anything else for the setup tools?
How do i package the entire application to download on user systems? I want the software to be uploaded via a URL link or from a USB. Do i have to make a tar file? i can do it automatically by writing my own python setup script, but i believe python has an easier way. Do i put a function in the setup to send an executable file to the desktop as a controller so the users don't open all the folders? How do i install from a link on a website?
For a graphical interface for the users on python applications, do most programmers use tkinter, or python integration into HTML, CSS? I do not want this application to be live in a website, but rather as a desktop application. Do i need to use a framework, such as django, to implement python into my html code?
I apologize if this is all unorganized. Ive always coded my own programs and never distributed or put a GUI on any of the scripts, so i just want to make sure i approach the task the right way.
I am building a chat bot with rasa-nlu. I went through the tutorial and I have built a simple bot. However, I need lots of training data for building a chat bot that is able to book a taxi. So I need data to build a specific bot.
Is there a repository, or corpus, for booking a taxi?
Or is there a way to generate this kind of dataset?
This is a blog post from one of the founders of Rasa and I think it's got some really excellent advice. I think you're going about it the wrong way asking for a pre-built training set. Start it yourself, then add friends, etc until you've built a training set that works best for your bot.
Put on your robot costume
Beyond that the Rasa docs have this under improving model performance
When the rasa_nlu server is running, it keeps track of all the
predictions it’s made and saves these to a log file. By default log
files are placed in logs/. The files in this directory contain one
json object per line. You can fix any incorrect predictions and add
them to your training set to improve your parser.
I think you'll be surprised how far you can get with just the training set you can come up with yourself.
Good luck on finding the corpus, but either way hope these links and snippets helped.
One method of doing this is, head over to LUIS.AI
Login using Office 365, Make your own Taxi Booking App, by giving in Intents and Utterances like below:
Now after training and publishing the model, download the corpus like below:
Now, after downloading the corpus, it will look something like this:
Install RASA NLU, I have Windows 8.1 on my machine, so the steps are as follows:
These are the steps to configure RASA:
First install:
Anaconda 4.3.0 64-bit Windows for installing Python 3.6 interpreter: https://repo.continuum.io/archive/Anaconda3-4.3.0-Windows-x86_64.exe
&
Python Tools for Visual Studio 2015: https://ptvs.azureedge.net/download/PTVS%202.2.6%20VS%202015.msi
Next, install the following packages in this order in administrative mode in command prompt:
Spacy Machine Learning Package: pip install -U spacy
Spacy English Language Model: python -m spacy download en
Scikit Package: pip install -U scikit-learn
Numpy package for mathematical calculations: pip install -U numpy
Scipy Package: pip install -U scipy
Sklearn Package for Intent Recognition: pip install -U sklearn-crfsuite
NER Duckling for better Entity Recognition with Spacy: pip install -U duckling
RASA NLU: pip install -U rasa_nlu==0.10.4
After installing all the above packages successfully, make a spaCy configuration file which will be read by RASA, like as follows:
{
"project": "Travel",
"pipeline": "spacy_sklearn",
"language": "en",
"num_threads": 1,
"max_training_processes": 1,
"path": "C:\\Users\\Kunal\\Desktop\\RASA\\models",
"response_log": "C:\\Users\\Kunal\\Desktop\\RASA\\log",
"config": "C:\\Users\\Kunal\\Desktop\\RASA\\config_spacy.json",
"log_level": "INFO",
"port": 5000,
"data": "C:\\Users\\Kunal\\Desktop\\RASA\\data\\FlightBotFinal.json",
"emulate": "luis",
"spacy_model_name": "en",
"token": null,
"cors_origins": ["*"],
"aws_endpoint_url": null
}
Next, Make a directory structure like this:
data folder -> Will contain all LUIS formatted corpus
models -> Will contain all trained models
logs -> Will contain active learning logs and RASA framework logs
Like this,
Now, make batch file scripts for Training and Starting RASA NLU Server.
Make a TrainRASA.bat by Notepad or Visual Studio Code and write this:
python -m rasa_nlu.train -c config_spacy.json
pause
Now make a StartRASA.bat by Notepad or Visual Studio Code and write this:
python -m rasa_nlu.server -c config_spacy.json
pause
Now train and start RASA Server by clicking on the batch file scripts that you just now made.
Now, everything is ready, just fire up chrome and issue a HTTP GET request to your enpoint /parse
Like: http://localhost:5000/parse?q=&project=
You will get a JSON response that corresponds to LUISResult class of Bot Framework C#.
Now handle the business logic you want to perform after doing that.
Alternatively, You can take a look at RASA Core, it was mainly built for this purpose.
RASA Core, which uses machine learning to build dialogs instead of
simple if-else statements.
The below link contains datasets relevant for commercial chatbot applications ('human-machine' dialogues). It's a fairly comprehensive collection of both human-human and human-machine text dialogue datasets, as well as audio dialogue datasets. https://breakend.github.io/DialogDatasets/
We did face the same problem while trying to build a love relationship coach bot. Long story short, we decided to create a simple tool to collect data from our friends, our colleagues or people on Mechanical Turk: https://chatbotstrap.io.
The idea is to create polls like this one: https://chatbotstrap.io/en/project/q5pimyskbhna2rm?language=en&nb_scenarios=10
and send them to anyone you know. With that solution, we were able to build a dataset of more than 6000 sentences divided in 10 intents in a few days.
The tool is free as long as you agree that the dataset constructed with it can be opensourced. They are also payed plans if you prefer to be the sole beneficiary of the data you collect.
I'm working on a project where my XNA 4.0 powered 3D engine needs to load external fbx models input by the user, in run time rather than in the default compile time way.
I understand XNA is built to bundle/process complex resources compile time to make the runtime smaller, but as I only need to target Windows I wonder if it is possible to load models with textures externally, and if so, how?
Yes, as #Andrew mentioned, using the built in content pipeline would require a developer install so that the content pipeline is available. Of course, you can parse it yourself and pull out the information at runtime to avoid that dependency. There are people out there doing it ... for example, the guys at sandswept studios have an API to do this, and are willing to discuss commercial agreements (just contact them):
http://thunderfist-podium.blogspot.com/2008/09/fbx-and-xna-part-1-fbx-format-and.html
I found the solution here:
http://create.msdn.com/en-US/education/catalog/sample/winforms_series_2