I've been trying to setup Minio server in Distributed Mode using 2 nodes, but everytime I tried, I always get error "All serverPools should have same deployment ID expected xxx, got yyy".
I'm setting up minio on Ubuntu servers.
I followed the instruction in Minio official docs here, but I can't find any mention of this error or any tutorial to make the Deployment ID the same.
Does anybody know what this is or how to make the deployment ID the same?
Thanks!
I've typically seen this happen when users attempt to do something similar to the following:
Start MinIO with minio server http://minio.example.net/mnt/disk-{1...4}
Later try to 'migrate' to distributed mode as minio server http://minio-{1...4}.example.net/mnt/disk-{1...4}
The second command is an entirely different topology, and results in a new deployment ID. When MinIO checks the existing backend disks, it sees that there is existing metadata, with a deployment ID that was generated based on the original topology. It then throws an error.
We would need to know quite a bit more about what you are attempting here - whether this is a fresh deployment, what MinIO version you are using, what the startup command was, etc, before being able to debug any further. But the above would be my guess as to what the issue is.
If this is a fresh deployment and you have no data to be concerned with, you can completely clean the backend drives of all data - including the .minio.sys folder at each path - and go from there. If you keep having the same issue with completely empty backend drives, then that is a little more unusual, and might be better suited as a Github issue so we can try to track that down further.
Related
Ive made a database from sqlite and i uploaded that with my github to Heroku but its only getting the data from the database and not changing it. No errors, just not working. When i am testing it on my pc it works fine.
I'm not sure why you're only able to get data and not change it. If you can share an example of how you're getting and setting, I might be able to help you get it going temporarily.
I learned, however, that SQLite only offers temporary storage on Heroku:
Why is SQLite a bad fit for running on Heroku?
Disk backed storage
SQLite runs in memory, and backs up its data store in files on disk. While this strategy works well for development, Heroku’s Cedar stack has an ephemeral filesystem. You can write to it, and you can read from it, but the contents will be cleared periodically. If you were to use SQLite on Heroku, you would lose your entire database at least once every 24 hours.
Even if Heroku’s disks were persistent running SQLite would still not be a good fit. Since SQLite does not run as a service, each dyno would run a separate running copy. Each of these copies need their own disk backed store. This would mean that each dyno powering your app would have a different set of data since the disks are not synchronized.
Instead of using SQLite on Heroku you can configure your app to run on Postgres.
I then followed their instructions for setting up Postgre. It's worth reading through the instructions, but the gist of it to use the Heroku CLI:
From the section Provisioning Heroku Postgres:
"Use the heroku addons command to determine whether your app already has Heroku Postgres provisioned"
If heroku-postgresql doesn’t appear in your app’s list of add-ons, you can provision it with the following CLI command: heroku addons:create heroku-postgresql:hobby-dev
As of writing, the hobby tier is free. Read about plans here.
This command adds an environment variable to your project named DATABASE_URL.
I'm using keyv by Luke Childs. I installed its companion #keyv/postgres. (I also uninstalled my sqlite stuff.)
I used the newly added DATABASE_URL environment variable to wire into the keyv steps linked above:
const Keyv = require('keyv');
const keyv =
process.env.NODE_ENV !== "production"
? new Keyv()
: new Keyv(process.env.DATABASE_URL);
I haven't found the best solution yet for developing/testing Postgre locally. Heroku Postgre requires SSL for connecting remotely (when your app is running locally). In the code block above, you'll see that I'm initializing Keyv without a database while developing locally (new Keyv()).
From here, if I need to verify the DB storage, I can set up a PostgreDB for developing locally, but I also imagine that it's possible to connect to the Heroku Postgre using SSL. If you or anyone has a solution they like for this step, please let me know.
#T. Rotzooi, I'm three months late to your question, but perhaps this explanation could help future people. I haven't found any other resources discussing this issue that you and I both encountered.
I am currently tracking my MLflow runs to a local file path URI. I would also like to set up a remote tracking server to share with my collaborators. One thing I would like to avoid is to log everything to the server, as it might soon be flooded with failed runs.
Ideally, I'd like to keep my local tracker, and then be able to send only the promising runs to the server.
What is the recommended way of copying a run from a local tracker to a remote server?
To publish your trained model to a remote MLflow server you should use 'register_model' API. For example, if you are using spacy flavor of MLflow you can use as below, where 'nlp' is the trained model:
mlflow.spacy.log_model(spacy_model=nlp, artifact_path='mlflow_sample')
model_uri = "runs:/{run_id}/{artifact_path}".format(
run_id=mlflow.active_run().info.run_id, artifact_path='mlflow_sample'
)
mlflow.register_model(model_uri=model_uri, name='mlflow_sample')
Make sure that the following environment variables should be set. In below example S3 storage is used:
SET MLFLOW_TRACKING_URI=https://YOUR-REMOTE-MLFLOW-HOST
SET MLFLOW_S3_BUCKET=s3://YOUR-BUCKET-NAME
SET AWS_ACCESS_KEY_ID=YOUR-ACCESS-KEY
SET AWS_SECRET_ACCESS_KEY=YOUR-SECRET-KEY
I have been interested in a related capability of copying runs from one experiment to another for a similar reason, ie set one area for arbitrary runs and another into which the results for promising runs that we move forward with are moved. Your scenario with separate tracking server is just the generalization of mine. Either way, apparently there is not a feature for this capability built-in to Mlflow currently. However, the mlflow-export-import python-based tool looks like it may cover both our use cases, and it cites usage on both Databricks and the open-source version of Mlflow, and it appears current as of this writing. I have not tried using this tool yet myself though - if/when I try it I'm happy to jot a follow-up here saying whether it worked well for this purpose, and/or anyone else could do same. Thanks and cheers!
I am trying to avoid using a DB in my simple RESTful app.
I created a "posts.txt" file which has all the posts in it, the app reads from this file and creates the posts array (JSON.parse).
The problem is, when I "git push heroku master" it, the "posts.txt" in heroku gets overriden and thus I lose all the posts created by guests.
I tried to .gitignor this file but it seems I just do it worng (or that I didn't understand the idea of "untracking" a file).
What can I do in order to prevent the overriding (I just don't want to push a new "posts.txt" into heroku every time)?
Due to your Heroku app potentially being run on multiple servers over time, there is no way to ensure that your posts.txt file will remain consistent overtime. Also as you make changes, and as you have noted, it can easily get overwritten.
Heroku can terminate your application and start it on another server as needed. Almost like a server-less type setup.
That means there is no real way to ensure a stable data persistence on Heroku without some type of database layer.
Great point mentioned in the comments that I forgot to mention. The file will also be deleted after cycling because the filesystem is ephemeral. You can find more information about file uploads missing/deleted on Heroku's site here.
One other thing about this is even you are using some type of VPS or something like that, you'd still run into the problem of how to sync the posts down to your local machine during development and ensuring that stays in sync. Database layer is for sure the way to go here.
I've just installed Neo4j 1.8.2 onto Azure by following this step-by-step process...
http://de.slideshare.net/neo4j/neo4j-on-azure-step-by-step-22598695
Unfortunately, when I browse to http://:7474/webadmin Fiddler says Error 10061 - No connection could be made because the target machine actively refused it.
I've followed the instructions exactly and haven't received any errors.
Any help much appreciated.
So, I think I got to the bottom of this. I think it was due to the size of compute / VM I was creating. It looks like the problem is caused when running on Extra Small instances. I created a new installation using a Small instance and everything now works :).
Try setting the server to accept connections form all hosts, and maybe use a newer Neo4j, say 1.9.4
http://docs.neo4j.org/chunked/stable/security-server.html#_secure_the_port_and_remote_client_connection_accepts
The way the VM Depot image is set up, it's pre-configured to allow all hosts to connect, and the Neo4j server will auto-start. The only thing you need to take care of, when constructing your VM, is to open an Input Endpoint, with any public port you want (preferably 7474 to stay true to Neo4j) and internal port 7474.
Note that the UI changed a bit since the how-to was published: You can specify the endpoint as the last step before creating your virtual machine. Other than that, the instructions should be the same. And... once the VM is up and running (it'll take about 5-10 minutes), you just visit http://yourservicename.cloudapp.net:7474 and you should see the web admin. Note: this is not the same as your vm name. If you named your VM something like 'neo' then you do not want http://neo:7474 or http://neo.cloudapp.net:7474. You need to use your cloud service name (you had to create a name for the service when you deployed the VM.
I've deployed that image several times in demos, and just tried again right now to make sure nothing wonky happened. Worked perfectly.
I have been struggling to do the tutorial, https://www.windowsazure.com/en-us/develop/nodejs/tutorials/web-app-with-mongodb/, which basically makes a simple node.js application that has access to a Mongo DB. I keep running into the following issue when launching the program locally with the command Start-AzureEmulator:
"No connection could be made because the target machine actively refused it 127.0.0.1:27017"
I tried various ports and configurations with no success. Oddly enough, when I run mongodb.exe, the database launches without hiccup (this is just through the command line not within the Azure Emulator). I have also tried reinstalling all of the tools multiple times. It seems I am at a loss of what to do next.
Have any of you experienced this problem or have been able to complete this tutorial?
As a side note, do any of you know any cloud providers that allow the use of sockets with node.js? This is one of the main reasons I am trying to use Azure.
I assume you've followed the instructions step by step and haven't modified anything yet?
I note from the screenshot below, that the sample tries to open Mongo at 127.255.0.1:27017, not 127.0.0.1:27017:
I suggest checking your Azure services' URL's in case you're looking for Mongo on the wrong address.