I am using the command: dotnet "myfile.dll"
Initially it was giving me this error: The user's home directory could not be determined. Set the 'DOTNET_CLI_HOME' environment variable to specify the directory to use.
Now after messing around with it, I have moved my files to c:/mydir, and it is giving this error: Failed to initialize CoreCLR, HRESULT: 0x80070057. I found this, but isn't c:/mydir a drive root?
Couple of things I noted:
I am able to run the .dll fine in a different directory.
Both directories contain same files.
The reason I want to run it in c:/mydir is because I am using AWS CodeDeploy, and that is where it copies the files (as far as I know; and the other directories are just the old versions where the files get copied from).
These issues are not linked (the first one I get from running a automated shell script after installation, and the second I get from manually trying to launch the .dll).
If someone could help me solve either one of these issues it would be greatly appreciated.
Try adding Environment=DOTNET_CLI_HOME=/temp to your Service declaration in your .service file. Example syntax:
[Service]
...
Environment=VARNAME=VARCONTENTS
So the actual like would look like this
Environment=DOTNET_CLI_HOME=/temp
I am trying to use Google Cloud Platform (specifically, the Vision API) for Python with AWS Lambda. Thus, I have to create a deployment package for my dependencies. However, when I try to create this deployment package, I get several compilation errors, regardless of the version of Python (3.6 or 2.7). Considering the version 3.6, I get the issue "Cannot import name 'cygrpc'". For 2.7, I get some unknown error with the .path file. I am following the AWS Lambda Deployment Package instructions here. They recommend two options, and both do not work / result in the same issue. Is GCP just not compatible with AWS Lambda for some reason? What's the deal?
Neither Python 3.6 nor 2.7 work for me.
NOTE: I am posting this question here to answer it myself because it took me quite a while to find a solution, and I would like to share my solution.
TL;DR: You cannot compile the deployment package on your Mac or whatever pc you use. You have to do it using a specific OS/"setup", the same one that AWS Lambda uses to run your code. To do this, you have to use EC2.
I will provide here an answer on how to get Google Cloud Vision working on AWS Lambda for Python 2.7. This answer is potentially extendable for other other APIs and other programming languages on AWS Lambda.
So the my journey to a solution began with this initial posting on Github with others who have the same issue. One solution someone posted was
I had the same issue " cannot import name 'cygrpc' " while running
the lambda. Solved it with pip install google-cloud-vision in the AMI
amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2 instance and exported the
lib/python3.6/site-packages to aws lambda Thank you #tseaver
This is partially correct, unless I read it wrong, but regardless it led me on the right path. You will have to use EC2. Here are the steps I took:
Set up an EC2 instance by going to EC2 on Amazon. Do a quick read about AWS EC2 if you have not already. Set one up for amzn-ami-hvm-2018.03.0.20180811-x86_64-gp2 or something along those lines (i.e. the most updated one).
Get your EC2 .pem file. Go to your Terminal. cd into your folder where your .pem file is. ssh into your instance using
ssh -i "your-file-name-here.pem" ec2-user#ec2-ip-address-here.compute-1.amazonaws.com
Create the following folders on your instance using mkdir: google-cloud-vision, protobuf, google-api-python-client, httplib2, uritemplate, google-auth-httplib2.
On your EC2 instance, cd into google-cloud-vision. Run the command:
pip install google-cloud-vision -t .
Note If you get "bash: pip: command not found", then enter "sudo easy_install pip" source.
Repeat step 4 with the following packages, while cd'ing into the respective folder: protobuf, google-api-python-client, httplib2, uritemplate, google-auth-httplib2.
Copy each folder on your computer. You can do this using the scp command. Again, in your Terminal, not your EC2 instance and not the Terminal window you used to access your EC2 instance, run the command (below is an example for your "google-cloud-vision" folder, but repeat this with every folder):
sudo scp -r -i your-pem-file-name.pem ec2-user#ec2-ip-address-here.compute-1.amazonaws.com:~/google-cloud-vision ~/Documents/your-local-directory/
Stop your EC2 instance from the AWS console so you don't get overcharged.
For your deployment package, you will need a single folder containing all your modules and your Python scripts. To begin combining all of the modules, create an empty folder titled "modules." Copy and paste all of the contents of the "google-cloud-vision" folder into the "modules" folder. Now place only the folder titled "protobuf" from the "protobuf" (sic) main folder in the "Google" folder of the "modules" folder. Also from the "protobuf" main folder, paste the Protobuf .pth file and the -info folder in the Google folder.
For each module after protobuf, copy and paste in the "modules" folder the folder titled with the module name, the .pth file, and the "-info" folder.
You now have all of your modules properly combined (almost). To finish combination, remove these two files from your "modules" folder: googleapis_common_protos-1.5.3-nspkg.pth and google_cloud_vision-0.34.0-py3.6-nspkg.pth. Copy and paste everything in the "modules" folder into your deployment package folder. Also, if you're using GCP, paste in your .json file for your credentials as well.
Finally, put your Python scripts in this folder, zip the contents (not the folder), upload to S3, and paste the link in your AWS Lambda function and get going!
If something here doesn't work as described, please forgive me and either message me or feel free to edit my answer. Hope this helps.
Building off the answer from #Josh Wolff (thanks a lot, btw!), this can be streamlined a bit by using a Docker image for Lambdas that Amazon makes available.
You can either bundle the libraries with your project source or, as I did below in a Makefile script, upload it as an AWS layer.
layer:
set -e ;\
docker run -v "$(PWD)/src":/var/task "lambci/lambda:build-python3.6" /bin/sh -c "rm -R python; pip install -r requirements.txt -t python/lib/python3.6/site-packages/; exit" ;\
pushd src ;\
zip -r my_lambda_layer.zip python > /dev/null ;\
rm -R python ;\
aws lambda publish-layer-version --layer-name my_lambda_layer --description "Lambda layer" --zip-file fileb://my_lambda_layer.zip --compatible-runtimes "python3.6" ;\
rm my_lambda_layer.zip ;\
popd ;
The above script will:
Pull the Docker image if you don't have it yet (above uses Python 3.6)
Delete the python directory (only useful for running a second
time)
Install all requirements to the python directory, created in your projects /src directory
ZIP the python directory
Upload the AWS layer
Delete the python directory and zip file
Make sure your requirements.txt file includes the modules listed above by Josh: google-cloud-vision, protobuf, google-api-python-client, httplib2, uritemplate, google-auth-httplib2
There's a fast solution that doesn't require much coding.
Cloud9 uses AMI so using pip on their virtual environment should make it work.
I created a Lambda from the Cloud9 UI and from the console activated the venv for the EC2 machine. I proceeded to install google-cloud-speech with pip.That was enough to fix the issue.
I was facing same error using goolge-ads API.
{
"errorMessage": "Unable to import module 'lambda_function': cannot import name'cygrpc' from 'grpc._cython' (/var/task/grpc/_cython/init.py)","errorType": "Runtime.ImportModuleError","stackTrace": []}
My Lambda runtime was Python 3.9 and architecture x86_64.
If somebody encounter similar ImportModuleError then see my answer here : Cannot import name 'cygrpc' from 'grpc._cython' - Google Ads API
I currently am using this guide to package up my project wasp. However currently everything lives inside of the wasp file.
That's not ideal. I would rather have all the classes in separate files so it can be more effectively managed. I have the series of files needed in the debian directory. But I'm not sure how to configure the packaging to package multiple files.
Is there a way to change my packaging to package more than just the one script file?
I'm not a debian package or Python expert, but one way would be to copy the various source files to another location (outside of /usr/bin), and then have /usr/bin/wasp call out to them.
Say you put all of your python code in src/ in the root of your repo. In the debian/install file, you'd have:
wasp usr/bin
src/* usr/lib/wasp/
You'd then just need /usr/bin/wasp to call some entry point in src. For example,
#!/usr/bin/python3
import sys
sys.path.append('/usr/lib/wasp/')
import wasp # or whatever you expose in src
# ...
Again, I don't know the best practices here (either in directory or python usage) but I think this would at least work!
I am working on changing machines from the RHEL world over to the debian/ubuntu world, and I am struggling a bit with a packaging problem. I am trying to build a package for Ubuntu 16.4.
I've got an very old pre-compiled application that can only listen through xinetd. I am creating a binary only package similar to what this person was doing: I need my Debian rules file to simply copy files to it's target. I simply need to copy pre-compiled files into directories.
I have no problem getting files in /opt and in /var/log, however I have been trying to get the dpkg to copy the needed setup file into /etc/xinetd.d/
So I have a debian/package.install file something like this:
opt/oldapplication-3.10/* opt/oldapplication-3.10/
var/log/* var/log/
etc/xinetd.d/oldapplication /etc/xinetd.d
The xinetd setup file never makes it to xinetd.d, and trying to look at the dpkg install with debug doesn't give me any hints. The file is definitely in the tarball, it just simply never gets moved.
Looking through the different dh helper applications, I can't see anything that fits, and google does nothing to illuminate the problem.
Do I have to simply move the file over in a postinst script? Is that the only way to solve this, or is there a more "debian" way to do this by creating a file in the dpkg's debian directory? Is there a more generic setup I should be doing to put files into /etc?
Thanks.
I need to get the content of archives and then I want to uncompress the selected one - but I dont want to uncompress the archives to know what's in it. I'd like to list and uncompress at least zip and rar, but (if that's possible) I don't want to be limited to only these two.
Can you advise good npm modules or other projects to achieve this?
Here's what I came up with:
zip
I found node-zip can only unzip files, but not list archive content.
rar
The best solution seems node-rar, but I can't install it on Windows.
node-uncompress This does what it says: It's an "Command-line wrapper for uncompressing various file types." So there is again no possibility to list archive content.
Currently I try to get node-uncompress to list files and hopefully it must never run cross-platform.
Solution:
I am now using 7zip with the node module node-7z instead of trying to get every archive working on its own. The corresponding site is: https://www.npmjs.com/package/node-7z
This library uses the OS independent archive manager 7zip. On Windows 7za is used. "7za.exe (a = alone) is a standalone version of 7-Zip". I've tested it on Windows and Ubuntu and it works great.
Update:
At Windows: Somehow I just got it working by adding 7za to the Path variables - not by adding 7za.exe to the "the same directory of your package.json file." like the description says.
Update 2:
On Windows 7za, that's referred in the node-7z post, cannot handle .rar-archives. So I'm using the "casual" 7-zip instead of 7za.exe. I just renamed the commanline 7z.exe to 7za.exe and added the 7-zip folder to the Path Variables.