All,
I started the mlflow server as below. I do see the backend store containing the expected metadata. However, the artifact folder is empty despite many runs.
> mlflow server --backend-store-uri mlflow_db --default-artifact-root
> ./mlflowruns --host 0.0.0.0 --port 5000
The mlflow ui has the below message for the artifacts section:
No Artifacts Recorded
Use the log artifact APIs to store file outputs from MLflow runs.
What am I doing wrong?
Thanks,
grajee
Turns out that
"--backend-store-uri mlflow_db" was pointing to D:\python\Pythonv395\Scripts\mlflow_db
and
"--default-artifact-root ./mlflowruns" was pointing to D:\DataEngineering\MlFlow\Wine Regression\mlflowruns which is the project folder.
I was able to point both the output to one folder with the following syntax
file:/D:/DataEngineering/MlFlow/Wine Regression
In case you want to log artifacts to your server with local file system as object storage, you should specify --serve-artifact --artifact-destination file:/path/to/your/desired/location instead of just a vanilla path.
Related
I have a google cloud run flask application named "HelloWorld1" already up and running however i need to create a second flask application. I followed the below steps as per documentation:
1- On "Cloud Shell Editor" clicked "<>Cloud Code" --> "New Application" --> "Cloud Run Application Basic Cloud Run Application .."-->"Python (Flask): Cloud Run", provide and new folder and application is created.
2- When i try to run it using "Run on Cloud Run Emulator" i get the following error:
Starting to run the app using configuration 'Cloud Run: Run/Debug Locally' from .vscode/launch.json...
To view more detailed logs, go to Output channel : "Cloud Run: Run/Debug Locally - Detailed"
Dependency check started
Dependency check succeeded
Starting minikube, this may take a while...................................
minikube successfully started
The minikube profile 'cloud-run-dev-internal' has been scheduled to stop automatically after exiting Cloud Code. To disable this on future deployments, set autoStop to false in your launch configuration /home/mian/newapp/.vscode/launch.json
Update initiated
Update failed with error code DEVINIT_REGISTER_BUILD_DEPS
listing files: file pattern [requirements.txt] must match at least one file
Skaffold exited with code 1.
Cleaning up...
Finished clean up.
I tried following:
1- tried to create different type of application e.g django instead of flask however always getting the same error
2- tried to give full path of [requirements.txt] in docker settings, no luck.
Please if someone help me understanding why i am not able to run a second cloud run Flask app due to this error?
It's likely that your Dockerfile references the 'requirements.txt' file, but that file is not in your local directory. So, it gives the error that it's missing:
listing files: file pattern [requirements.txt] must match at least one file
I am currently using nodejs that is deployed in ebs on aws. I have a function that will write a pdf and then email it off but it says the file path can't be found. I've verified the project file seems to be /var/app/current/, but changing the reference of the file path doesn't seem to remove the error. Any idea how to go about fixing this?
The /var/app/current/ does not exist initially. Its only created at the very last stage of your deployment.
The deployment happens in /var/app/staging/ folder, and at the very last, once everything finishes, /var/app/staging/ is moved into /var/app/current/.
Thus, I would not recommend using absolute paths in your project or config files. Its better to use relative path or container_commands for config scripts:
The specified commands run as the root user, and are processed in alphabetical order by name. Container commands are run from the staging directory, where your source code is extracted prior to being deployed to the application server.
So we are currently moving away from our current deployment provider: Beanstalk, which is great but we are on the top tier and we keep running out of space or hitting our repository limits. So we are moving away so please do not suggest any other SaaS provider.
I personally use Gitlab for my own projects and a few company projects and it's amazing we use a self hosted version on our local server in our company building.
We have CI setup and currently are using the following deployment code (I have minified the bits just to the deployment for development) - this uses the shell executer for deploying as we deploy to an existing linux server.
variables:
HOSTNAME: '<hostname>'
USERNAME: '<username>'
PASSWORD: '<password>'
PATH_DEV: '/path/to/www'
# Define the stages (we can add as many as we want)
stages:
# - build
- deploy
# The code for development deployment
deploy_dev:
stage: deploy
script:
- echo "Deploying to development environment..."
- rm .gitlab-ci.yml
- rsync -urltvz --filter=':- .gitignore' --exclude=".git" -e "sshpass -p"$PASSWORD" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" * $USERNAME#$HOSTNAME:$PATH_DEV
- echo "Finished deploying."
environment:
name: Development
url: http://dev.domain.com
only:
- envdev
The Problem:
When we use the above code to deploy it's perfect and works really well, and it deploys all the code after optimisation etc, but we have found a little bug here.
When you delete a file then the rsync command will not delete the file, now I did some searching and found the --remove flag you can add, and it worked - but it deleted all the user uploaded content as well. Now I added the .gitignore in to the filtering, so it would ignore some the files in their (which are usually user generated) or configuration files or/and libraries (npm, etc.). This is fine until a user started uploading files using the media manager in our framework which stores in a folder that is not in the .gitignore file and it can't because it contains other files, as we also add our own files in there so they're editable by the user, so now I am unsure how to manage this.
What we are looking for is a CI setup, which will upload file changes to the server, so it would search through the latest commits, and find the latest files that have been changed and then push only them files up. Of course I would like to do this with the Gitlab CI still, so any ideas examples or tutorials would be amazing.
Thanks in advance.
~ Danny
May it helps: https://github.com/banago/PHPloy
Looks this tool designed for php project, but I think it can use other web deployment.
how it works:
PHPloy stores a file called .revision on your server. This file contains the hash of the commit that you have deployed to that server. When you run phploy, it downloads that file and compares the commit reference in it with the commit you are trying to deploy to find out which files to upload. PHPloy also stores a .revision file for each submodule in your repository.
I added a scripts.config file to .ebextensions at the root of my Node app deployed in beanstalk.I did not see the tags for the EC2 instances in the console. Nor did I see any mention of 1_add_tags in beanstalk logs. What did I do wrong and how do I find out if the commands in the script.config were called at all!
The config file in .ebextensions is as follows ....
01_add_tags:
command: ec2-create-tags $(ec2-metadata -i | cut -d ' ' -f2) --tag Environment=Production --tag Name=Proxy-Server --tag Application=something
env:
EC2_HOME: /opt/aws/apitools/ec2
EC2_URL: https://ec2.ap-southeast-2.ama...
JAVA_HOME: /usr/lib/jvm/jre
PATH: /bin:/usr/bin:/opt/aws/bin/
Cheers,
Prabin
Amazon's answer to the problem. (This worked for me) ...
You can utilise the ebextensions to execute certain commands on instance boot.
Supposing that you want to implement this on Linux based containers. I have formulated a sample config file for you and attached to this case.
Please follow below guidelines :
In the AWS Management console, check the IAM Role/Instance profile used by beanstalk. By default it uses "aws-elasticbeanstalk-ec2-role". Add permissions for this role to create new tags (ec2:CreateTags).
If you do not have ".ebextensions" folder at the root of your application or the "WEB-INF" folder, then create the folder.
Modify the key value pairs in the config file. Multiple pairs are separated by a space.
A sample snippet is as below:
{
"container_commands": {
"01_add_tags": {
"command": "aws ec2 create-tags --resources $(GET http://169.254.169.254/latest/meta-data/instance-id) --tags Key=ClientName,Value=testClient Key=NewTag,Value=new-value --region us-east-1"
}
}
}
Add the modified config file in the ".ebextensions" folder.
Upload this version to beanstalk. It should launch new instances and execute the config file.
Please give it sometime, preferably till the instances pass EC2 instance status checks. Refresh the page for the additional tags to be displayed.
Please note that we are using "Container_commands" instead of "Command" used in the blog.
Container Commands run after the application and web server have been set up and the application version file has been extracted, but before the application version is deployed. This is important as these commands have access to environment variables such as your AWS security credentials set by the instance-profile.
I would recommend you to go through the restrictions for AWS Resources tagging mentioned at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-restrictions
I would like to highlight that maximum number of tags per resource is 10.
Also check the table for tagging support for certain resource. For example, currently tagging is not supported for ELB.
I had the similar problem where I tried to install libjpeg using the ./ebextensions/foo.config file. I tried everything but was never able to find a good solution.
I was able to solve it though, by setting up a completely new Elastic Beanstalk Application and then deploying my same version on the new instance instead. When I did this everything was installed perfectly and working fine.
Check out my answers here:
https://stackoverflow.com/a/23109410/2335675
https://stackoverflow.com/a/23131959/2335675
Hope this fixes your issues as well.
I am running my NodeJS project on DotCloud. Sadly, DotClouds deployment is "project-intrusive" that is it requires a supervisord.conf file to reside in the app-root. My deployment setup looks like this (using git repos).
project-deploy.git/prod/dotcloud.yml
project-deploy.git/prod/project -> project.git
(/prod/project use project.git as a submodule to access the code)
Now, my though of this is that I eventually would end up having different environments like this, e.g. dev, test and stage. The dev environment wouldn't even have a dotcloud.yml file since it is expected to run everything locally.
Well this works pretty well. But the problem is the supervisord.conf file which is just for deployment to dotcloud, now it resides in the project.git repo, but it doesn't belong there since it is just for deployment.
Are there any modules or NodeJS scripts that let you put deployment configuration files elsewhere, and maybe even specify what the target environment is, e.g. node deploy.js --production, or something like that?
There is a way to get rid of supervisord.conf. Assuming that you want to run e.g. node app.js, you can put the following in dotcloud.yml:
www:
type: nodejs
process: node app.js
Now, of course, it doesn't solve the problem of the dotcloud.yml file itself; but at least it reduces clutter a little bit -- removing it from the approot.