I am trying to use gcloud in order to deploy a nodejs app. I have followed this tutorial.
When I run:
gcloud preview app deploy
I have 2 possible issue:
First :
Beginning deployment... If this is your first deployment, this may
take a while...failed. ERROR: gcloud crashed (ResponseNotReady)
Or sometimes the process go further :
Verifying that Managed VMs are enabled and ready. ERROR: gcloud
crashed (ResponseNotReady)
The log look like that :
File "C:\Program Files (x86)\Google\Cloud
SDK\google-cloud-sdk\bin..\lib\third_party\httplib2__init__.py",
line 1308, in _conn_request
response = conn.getresponse()
File "C:\python27_x64\lib\httplib.py", line 1119, in getresponse
raise ResponseNotReady() ResponseNotReady 2016-02-22 15:13:00,937 ERROR root gcloud crashed (ResponseNotReady):
Thank you !
Seems to be some connection instability... I made it without any change but retrying like an hundred times.
Related
i have a arm template that contains a customscriptextension that is meant to install an app (Sophos AV and Notepad++) from a storage account.
the deployment is picking up the storageaccount correctly and copying the sophos.ps1 file from the blob storage account and adding it to the VM.
i can find the file in this location
C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.10.12\Downloads\0
the log file for the customscriptextension shows that sophos file is downloaded.
[INFO] DownloadFiles: fileUri = https://stgacctname.blob.core.windows.net/scripts/sophos.ps1
[INFO] DownloadFiles: fileDownloadPath = C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.10.12\Downloads\0
[INFO] Waiting for all async file download tasks to complete...
[INFO] DownloadFiles: asynchronously downloaded file to fileDownloadLocation = Downloads\0\sophos.ps1
[INFO] Files downloaded. Asynchronously executing command: 'SecureCommand_0'
[INFO] Command execution task started. Awaiting completion...
but the install of this file is failing and spitting out this error
**[ERROR] Command execution finished, but failed because it returned a non-zero exit code of: '1'. Command execution may have been interrupted by a VM restart**
[INFO] {"sequenceNumber":0,"totalDuration":21317,"status":"error","code":1,"filesDownloaded":1,"durationOfFileDownload":336,"durationOfDownloadHashing":0,"statusFileAccessed":5,"operation":"enable","operationDuration":123,"operationResult":"success","operationSizeOfOutput":42,"operationSizeOfError":0,"sha256HashOfCommandToExecute":"HASHNUMBER","downloadSummary":{"downloadUris":["https://stgacctname.blob.core.windows.net/scripts/sophos.ps1"],"fileHashes":[{"fileName":"sophos.ps1","sha256Hash":"HASHNUMBER"}]}}
the sophos.ps1 file contains the following
Set-ExecutionPolicy Bypass
cmdkey /add:stgacctname.file.core.windows.net /user:AZURE\stgacctname /pass:sttgacctkey
\\stgacctname.file.core.windows.net\fileshare\SophosSetup.exe --quiet --devicegroup="Test servers"
\\stgacctname.file.core.windows.net\fileshare\npp.8.1.4.Installer.x64.exe /S
when i open the PS1 file and run it as admin it runs and installs sophos without any issues.
so question is...why is it failing here?
my thoughts are that the PS1 file is not being run as admin... its running as a standard user so its failing.. but thats just pure speculation.
can anybody shed some light on what could be happening?
let me know if you need any more info...
in the mean time i will keep digging around....
cheers!
Recently I have needed to add web sockets to my backend application currently hosted on Google App Engine (GAE) standard environment. Because web sockets are a feature only available in GAE's flexible environment, I have been attempting a redeployment but with little success.
To make the change to a flexible environment I have updated the app.yaml file from
runtime: nodejs10
env: standard
to
runtime: nodejs
env: flex
While previously working in the standard environment, now with env: flex when I run the command gcloud app deploy --app-yaml=app-staging.yaml --verbosity=debug I get the following stack trace:
Do you want to continue (Y/n)? Y
DEBUG: No bucket specified, retrieving default bucket.
DEBUG: Using bucket [gs://staging.finnsalud.appspot.com].
DEBUG: Service [appengineflex.googleapis.com] is already enabled for project [finnsalud]
Beginning deployment of service [finnsalud-staging]...
INFO: Using ignore file at [~/checkouts/twilio/backend/.gcloudignore].
DEBUG: not expecting type '<class 'NoneType'>'
Traceback (most recent call last):
File "/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 982, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 809, in Run
resources = command_instance.Run(args)
File "/google-cloud-sdk/lib/surface/app/deploy.py", line 115, in Run
return deploy_util.RunDeploy(
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 669, in RunDeploy
deployer.Deploy(
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 428, in Deploy
source_files = source_files_util.GetSourceFiles(
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/source_files_util.py", line 184, in GetSourceFiles
return list(it)
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/util/gcloudignore.py", line 233, in GetIncludedFiles
six.ensure_str(upload_directory), followlinks=True):
File "//google-cloud-sdk/lib/third_party/six/__init__.py", line 884, in ensure_str
raise TypeError("not expecting type '%s'" % type(s))
TypeError: not expecting type '<class 'NoneType'>'
ERROR: gcloud crashed (TypeError): not expecting type '<class 'NoneType'>'
In this stack trace, it mentions an error in google-cloud-sdk/lib/googlecloudsdk/command_lib/util/gcloudignore.py so I had also reviewed my .gcloudignore file but was unable to find anything out of place:
.gcloudignore
.git
.gitignore
node_modules/
In an attempt to work around this bug I tried removing my .gcloudignore file which resulted in a different error, but still failed nevertheless:
Do you want to continue (Y/n)? Y
DEBUG: No bucket specified, retrieving default bucket.
DEBUG: Using bucket [gs://staging.finnsalud.appspot.com].
DEBUG: Service [appengineflex.googleapis.com] is already enabled for project [finnsalud]
Beginning deployment of service [finnsalud-staging]...
DEBUG: expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
File "/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 982, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 809, in Run
resources = command_instance.Run(args)
File "/google-cloud-sdk/lib/surface/app/deploy.py", line 115, in Run
return deploy_util.RunDeploy(
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 669, in RunDeploy
deployer.Deploy(
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 428, in Deploy
source_files = source_files_util.GetSourceFiles(
File "/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/source_files_util.py", line 184, in GetSourceFiles
return list(it)
File "/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/util.py", line 165, in FileIterator
entries = set(os.listdir(os.path.join(base, current_dir)))
File "/usr/local/Cellar/python#3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/posixpath.py", line 76, in join
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
ERROR: gcloud crashed (TypeError): expected str, bytes or os.PathLike object, not NoneType
Thinking maybe this was an error relating to the version of my CLI I have also run the following commands to try and update:
gcloud app update
gcloud components update
Unfortunately, this had no change on the output.
I have noticed that when I run this command with the app.yaml env value set to flexible, there are no updates to the logging section on google cloud and no changes to the files uploaded to the project's storage bucket. To me, this indicates that the crash is occurring in the CLI before any communication to the google cloud services is made. If this is correct, then it seems unlikely that the cause of the error would be related to a bad configuration on google cloud and must be related to something (software or configuration) on my local machine.
I have also tried using the 'Hello World' app.yaml configuration on the flexible environments 'Getting Started' page to rule out a configuration error my own application's app.yaml but this also had no change on the output.
Finally, if at any point I change env: flex back to env: standard then the issue does disappear. Unfortunately, as stated above, this won't work for deploying my web sockets feature.
This has gotten me thinking that possibly the error is due to a bug with the gcloud cli application. However, if this were the case, I would have expected to see many more bug reports for this issue by others whom are also using the GAE's flexible environment.
Regardless, given this stack trace points to code within the gcloud cli, I have opened a bug ticket with google which can be found here: https://issuetracker.google.com/issues/176839574
I have also seen this similar SO post, but it is not the exact error I am experiencing and remains unresolved: gcloud app deploy fails with flexible environment
If anyone has any ideas on other steps to try or methods to overcome this issue, I would be immensely grateful if you drop a note on this post. Thanks!
I deployed a nodejs application using the Quickstart for Node.js in the standard environment
Then I changed the app.yaml file from :
runtime: nodejs10
to
runtime: nodejs
env: flex
Everything worked as expected.
It might be related to your specific use case.
Surprisingly, this issue does seem to be related to a bug in the gcloud cli. However, there does seem to be a workaround.
When a --appyaml flag is specified for a deployment to the flex environment, then the CLI crashes with the messages outlines in my question above. However, if you copy your .yaml file renaming to app.yaml (the default) and delete this --appyaml flag when deploying then the build will proceed without errors.
If you have also experienced this error, please follow the google issue as I am working with the google engineers to be sure they reproduce and eventually fix this bug.
Broken app.yaml
runtime:nodejs14
Fixed app.yaml
runtime: nodejs14
I am dead serious. And :
glcoud info --run-diagnostics
was ZERO HELP.
Once I did this the "ERROR: gcloud crashed (TypeError): expected string or bytes-like object" went away.
I guess "colon + space" is part of the spec:
Why does the YAML spec mandate a space after the colon?
I am trying to deploy a sample python app which I got from another tutorial. However, the deployment fails as below:
gcloud app deploy Beginning deployment of service [default]... ERROR:
gcloud crashed (FileNotFoundError): [Errno 2] No such file or
directory:
'/Users/nileshdeshmukh/Desktop/Training/Python/FlaskIntroduction-master/env/.Python'
My app.yaml file is as below:
runtime: python3
env: standard
runtime_config:
python_version: 3
I have all dependencies copied in env/bin but the build process is looking for env only..
I think the problem would be solved if the deployment process looks at env/bin, but don't know how to force it to look at given path
The runtime_config setting is for App Engine flex only and isn't needed for App Engine Standard. You can safely remove it.
As per the error, you should ensure that all your dependencies are self-contained and shipped with your app or listed in your requirements.txt file.
Be careful, some gcloud commands use .gitignore file to prevent sending useless file to Cloud for building your app.
You can override this behavior by creating a .gcloudignore file. Same syntax as git ignore but take into account only by gcloud commands and not by git. By the way you can differentiate the file to send to the cloud and file to send to git
I'm trying to use bitbucket pipelines with my project. I use Nodejs.
When I'm running gcloud app deploy manually from Mac or Windows - it works fine, deploy successfully finishes. But from bitbucket pipelines it fails with Error Response: [13] An internal error occurred.
Here is the stack trace which I've got by running gcloud app deploy --verbosity=debug:
Updating service [default]...
.DEBUG: Operation [apps/my-project/operations/a68a837e-edcf-4987-83db-d9b47f4309ae] not complete. Waiting to retry.
.......DEBUG: Operation [apps/my-project/operations/a68a837e-edcf-4987-83db-d9b47f4309ae] complete. Result: {
"metadata": {
"target": "apps/my-project/services/default/versions/20180915t180908",
"method": "google.appengine.v1.Versions.CreateVersion",
"user": "bitbucket-works#my-project.iam.gserviceaccount.com",
"insertTime": "2018-09-15T18:09:46.693Z",
"endTime": "2018-09-15T18:09:49.655Z",
"#type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1"
},
"done": true,
"name": "apps/my-project/operations/a68a837e-edcf-4987-83db-d9b47f4309ae",
"error": {
"message": "An internal error occurred.",
"code": 13
}
}
failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred.
Traceback (most recent call last):
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 839, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 770, in Run
resources = command_instance.Run(args)
File "/tmp/google-cloud-sdk/lib/surface/app/deploy.py", line 90, in Run
parallel_build=False)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 625, in RunDeploy
flex_image_build_option=flex_image_build_option)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 431, in Deploy
extra_config_settings)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 207, in DeployService
poller=done_poller)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 315, in WaitForOperation
sleep_ms=retry_interval)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 254, in WaitFor
sleep_ms, _StatusUpdate)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 316, in PollUntilDone
sleep_ms=sleep_ms)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/core/util/retry.py", line 229, in RetryOnResult
if not should_retry(result, state):
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 310, in _IsNotDone
return not poller.IsDone(operation)
File "/tmp/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 184, in IsDone
encoding.MessageToPyValue(operation.error)))
OperationError: Error Response: [13] An internal error occurred.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred.
My bitbucket-pipelines.yml:
pipelines:
default:
- step:
name: Test and deployment
image: node:8.9
script: # Modify the commands below to build your repository.
- npm install
- npm test
- npm build
# Downloading the Google Cloud SDK
- curl -o /tmp/google-cloud-sdk.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-216.0.0-linux-x86_64.tar.gz
- tar -xvf /tmp/google-cloud-sdk.tar.gz -C /tmp/
- /tmp/google-cloud-sdk/install.sh -q
- source /tmp/google-cloud-sdk/path.bash.inc
# Setup
- echo $GCLOUD_CLIENT_SECRET > client-secret.json
- gcloud auth activate-service-account --key-file client-secret.json
- gcloud config set project $GCLOUD_PROJECT
- gcloud -q app deploy --verbosity=debug
My .gcloudignore:
.gcloudignore
.git
.gitignore
# Node.js dependencies:
node_modules/
design/
I found some solution on StackOverflow and Google, but nothing works. I've tried different versions of Google Cloud SDK - but the result is the same.
Thanks for any help in advance.
Finally, I've solved it.
TL;DR:
Add Cloud Build Editor role for your bitbucket member on IAM & admin page.
I've updated GCloud version up to 218 and it started to show more information about the error (not just internal error occurred):
OperationError: Error Response: [13] Permission to create cloud build is denied. 'Cloud Build Editor' role is required for the deployment: https://cloud.google.com/cloud-build/docs/securing-builds/configure-access-control#permissions.
The problem is lack of permissions for bitbucket member on GCloud. Seems like it's a recently added requirement, 'cause no one manual has information about.
To solve the issue you should add Cloud Build Editor role for your bitbucket member on IAM & admin page.
Hope it helps.
I'm using a Node Express app with App Engine. Everything's been deploying fine, but I must have had an interrupted deployment because now when I try to deploy using this:
gcloud preview app deploy .
I get this:
Error 409: --- begin server output ---
Another transaction by user <username> is already in progress for app:
s~<appId>, version: 1. That user can undo the transaction with "appcfg
rollback".
--- end server output ---
ERROR: (gcloud.preview.app.deploy) Command failed with error code [1]
I can't find an appcfg file to rollback, nor any documentation about this issue for a node app.
You can use gcloud preview app deploy . --force
which will implicitly do a rollback (to clear the deploy lock) before attempting the deployment.
Use the appcfg.py rollback command.
Here, create a file as app.yaml and a directory WEB-INF in which there will be two files named as web.xml and appengine-web.xml.