I am implementing Continuous Integration into my Laravel workflow, and while going through the basic I came across a sample project on Gitlab where (1.) Laravel Envoys was used to write tasks related to how the app should be deployed and then (2.) bootstrapping the process using Gitlab CI.
I got a bit confused, it seems to me that the part (bellow) where you define the tasks using Enovy is easily replicable when defining jobs inside the .gitlab-ci.yml file, which makes the use of Envoy redundant:
...
#setup
$repository = 'git#gitlab.example.com:<USERNAME>/laravel-sample.git';
$releases_dir = '/var/www/app/releases';
$app_dir = '/var/www/app';
$release = date('YmdHis');
$new_release_dir = $releases_dir .'/'. $release;
#endsetup
...
#task('update_symlinks')
echo "Linking storage directory"
rm -rf {{ $new_release_dir }}/storage
ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage
echo 'Linking .env file'
ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env
echo 'Linking current release'
ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current
#endtask
...
I would be grateful if someone could correct me if I'm wrong, or explain what benefits Envoy could bring into the Gitlab Continuous Integration workflow.
You are correct that the example shell script could be easily implemented in either a .gitlab-ci.yml file or an Envoy.blade.php file (so 'no', Envoy is not necessary for gitlab-ci deployments of Laravel apps.) I see three main reasons why a user might choose to have their deployment tasks in Envoy over gitlab:
Familiarity
Laravel developers are likely to have more familiarity with the languages that Envoy uses for deployment (PHP and the Blade syntax) than they are with the languages that gitlab uses (Yaml formatting with gitlab's pipeline syntax.)
Keeping the less familiar .gitlab-ci.yml file simple and adding the bulk of the complexity to the more familiar Envoy file may save the developer time.
Portability
Some developers may want the option to switch between CI platforms. By keeping the gitlab-ci file simple and having the bulk of the deployment logic in the Envoy file, the developer could switch to another CI server, like Jenkins, without needing to rewrite the deployment code. (Or, as in a case that I've seen, the developer may be using both gitlab-ci and Jenkins to build their software. Using Envoy would mean more shared code between the two CI platforms.)
Existing Stack
Envoy Task Runner uses software that is already required for Laravel deployment (PHP and Composer.) Gitlab, on the other hand, requires the installation of gitlab-runner on a machine in order to deploy.
Related
For all the other stackoverflow questions, it seems like people are asking either about a private npm git repository or about a different technology stack. I'm pretty sure I can use a private npm registry with GAE Flexible, but I was wondering if it was possible with the Standard version?
From the GAE standard docs, doesn't seem like it is possible. Anyone else figure out otherwise?
Google marked this feature request as "won't fix, intended behavior" but there is a workaround.
Presumably you have access to the environment variables during the build stage of your CI/CD pipeline. Begin that stage by having your build script overwrite the .npmrc file using the value of the environment variable (note: the value, not the variable name). The .npmrc file (and the token in it) will then be available to the rest of the CI/CD pipeline.
For example:
- name: Install and build
env:
NPM_AUTH_TOKEN: ${{ secrets.PRIVATE_REPO_PACKAGE_READ_TOKEN }}
run: |
# Remove these 'echo' statements after we migrate off of Google App Engine.
# See replies 14 and 18 here: https://issuetracker.google.com/issues/143810864?pli=1
echo "//npm.pkg.github.com/:_authToken=${NPM_AUTH_TOKEN}" > .npmrc
echo "#organizationname:registry=https://npm.pkg.github.com" >> .npmrc
echo "always-auth=true" >> .npmrc
npm install
npm run compile
npm run secrets:get ${{ secrets.YOUR_GCP_PROJECT_ID }}
Hat tip to the anonymous heroes who wrote replies 14 and 18 in the Issure Tracker thread - https://issuetracker.google.com/issues/143810864?pli=1
If you have a .npmrc file checked in with your project's code, you would be wise to put a comment at the top, explaining that it will be overwritten during the CI/CD pipeline. Otherwise, Murphy's Law dictates that you (or a teammate) will check in a change to that .npmrc file and then waste an unbounded amount of time trying to figure out why that change has no effect during deployment.
So we are currently moving away from our current deployment provider: Beanstalk, which is great but we are on the top tier and we keep running out of space or hitting our repository limits. So we are moving away so please do not suggest any other SaaS provider.
I personally use Gitlab for my own projects and a few company projects and it's amazing we use a self hosted version on our local server in our company building.
We have CI setup and currently are using the following deployment code (I have minified the bits just to the deployment for development) - this uses the shell executer for deploying as we deploy to an existing linux server.
variables:
HOSTNAME: '<hostname>'
USERNAME: '<username>'
PASSWORD: '<password>'
PATH_DEV: '/path/to/www'
# Define the stages (we can add as many as we want)
stages:
# - build
- deploy
# The code for development deployment
deploy_dev:
stage: deploy
script:
- echo "Deploying to development environment..."
- rm .gitlab-ci.yml
- rsync -urltvz --filter=':- .gitignore' --exclude=".git" -e "sshpass -p"$PASSWORD" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" * $USERNAME#$HOSTNAME:$PATH_DEV
- echo "Finished deploying."
environment:
name: Development
url: http://dev.domain.com
only:
- envdev
The Problem:
When we use the above code to deploy it's perfect and works really well, and it deploys all the code after optimisation etc, but we have found a little bug here.
When you delete a file then the rsync command will not delete the file, now I did some searching and found the --remove flag you can add, and it worked - but it deleted all the user uploaded content as well. Now I added the .gitignore in to the filtering, so it would ignore some the files in their (which are usually user generated) or configuration files or/and libraries (npm, etc.). This is fine until a user started uploading files using the media manager in our framework which stores in a folder that is not in the .gitignore file and it can't because it contains other files, as we also add our own files in there so they're editable by the user, so now I am unsure how to manage this.
What we are looking for is a CI setup, which will upload file changes to the server, so it would search through the latest commits, and find the latest files that have been changed and then push only them files up. Of course I would like to do this with the Gitlab CI still, so any ideas examples or tutorials would be amazing.
Thanks in advance.
~ Danny
May it helps: https://github.com/banago/PHPloy
Looks this tool designed for php project, but I think it can use other web deployment.
how it works:
PHPloy stores a file called .revision on your server. This file contains the hash of the commit that you have deployed to that server. When you run phploy, it downloads that file and compares the commit reference in it with the commit you are trying to deploy to find out which files to upload. PHPloy also stores a .revision file for each submodule in your repository.
How to configure GitLab CI to store the SSTATE_DIR and the DL_DIR between several jobs? Currently, bitbake rebuilds the complete project every time, which is very time consuming. So i would like to use the sstage again. I tried caching, but building time increases effectively, due to the big zip/unzip overhead.
I don't even need a shared sstate between several projects, just a method to store the output between jobs.
I'm using Gitlab 11.2.3 with a shell executor as runner.
Thanks a lot!
In version 11.10, GIT_CLEAN_FLAGS was added, which could be used to achieve what you want to do with the shell executor.
For completeness: when using the docker executor, this can be achieved by using docker volumes, which are persistent across builds.
If you're only using one runner for this, you could potentially use GIT_STRATEGY: none, which would re-use the project workspace for the following job; relevant documentation. However, this wouldn't be extremely accurate in case you have multiple jobs running which requires the runner, as it could dilute the repository, if jobs are started from across different pipelines.
Another way, if you're still using one runner, is you could just copy the directories out and back into the job you require.
Otherwise, you may potentially be out of luck, and have to wait for the sticky runners issue.
You can reuse a shared-state cache between jobs simply as follows:
Specify the path to the sstate-cache directory in the .yml file of your
gitlab-ci pipeline. An example fragment from one of mine:
myrepo.yml
stages:
...
...
variables:
...
TCM_MACHINE: buzby2
...
SSTATE_CACHE: /sstate-cache/$TCM_MACHINE/PLAT3/
PURGE_SSTATE_CACHE: N
...
In my case /sstate-cache/$TCM_MACHINE/PLAT3/ is a directory in the docker container
that runs the build. This path is mounted in the docker container from a
permanent sstate cache directory on the build-server's filesystem, /var/bitbake/sstate-cache/<machine-id>/PLAT3.
PURGE_SSTATE_CACHE is overridable by a private variable
in the pipeline schedule settings so that I can optionally delete the cache for a squeaky clean
build.
Ensure that the setting of SSTATE_CACHE is appended to the bitbake conf/local.conf
file of the build, e.g.
.build_image: &build_image
stage: build
tags:
...
before_script:
...
script:
- echo "SSTATE_DIR ?= \"$SSTATE_CACHE\"" >> conf/local.conf
...
Apply the same pattern to DL_DIR if you use it.
Variables you use in the .yml file can be overriden by gitlab-ci trigger
or schedule variables. See Priority of variables
I have GitLab running on computer A, development environment (Visual studio Pro) on computer B and Windows Server on computer C.
I set up GitLab-Runner on computer C (Windows server). I also set up .gitlab-ci.yml file to perform build and run tests for ASP.NET Core application on every commit.
I don't know how can I get code on computer C (Windows server) so I can build it (dotnet msbuild /p:Configuration=Release "%SOLUTION%"). It bothers me that not a single example .gitlab-ci.yml I found on net, doesn't pull code form GitLab, before building application. Why?
Is this correct way to set-up CI/CD:
User create pull request (a new branch is created)
User writes code
User commit code to branch from computer B.
GitLab runner is started on computer C.
It needs to pull code from current branch (CI_COMMIT_REF_NAME)
Build, test, deploy ...
Should I use common git command to get the code, or is this something GitLab runner already do? Where is the code?
Why no-one pull code from GitLab in .gitlab-ci.yml?
Edited:
I get error
'"git"' is not recognized as an internal or external command
. Solution in my case was restart GitLab-Runner. Source.
#MilanVidakovic explain that source is automatically downloaded (which I didn't know).
I just have one remaining problem of how to get correct path to my .sln file.
Here is my complete .gitlab-ci.yml file:
variables:
SOLUTION: missing_path_to_solution #TODO
before_script:
- dotnet restore
stages:
- build
build:
stage: build
script:
- echo "Building %CI_COMMIT_REF_NAME% branch."
- dotnet msbuild /p:Configuration=Release "%SOLUTION%"
except:
- tags
I need to set correct variable for SOLUTION. My dir (where GitLab-Runner is located) currently holds this folder/files:
- config.toml
- gitlab-runner.exe
- builds/
- 7cab42e4/
- 0/
- web/ # I think this is project group in GitLab
- test/ # I think this is project name in GitLab
- .sln
- AND ALL OTHER PROJECT FILES #Based on first look
- testm.tmp
So, what are 7cab42e4, 0. Or better how to get correct path to my project structure? Is there any predefined variable?
Edited2:
Answer is CI_PROJECT_DIR.
I'm not sure I follow completely.
On every commit, Gitlab runner is fetching your repository to C:\gitlab-runner\builds.. on the local machine (Computer C), and builds/deploys or does whatever you've provided as an action for the stage.
Also, I don't see the need for building the source code again. If you're using Computer C for both runner and tests/acceptance, just let the runner do the building and add Artifacts item in your .gitlab-ci.yaml. Path defined in artifacts will retain your executables on Computer C, which you are then able to use for whatever purposes.
Hope it helps.
Edit after comment:
When you push to repository, Gitlab CI/CD automatically checks your root folder for .gitlab-ci.yaml file. If its there, the runner takes over, parses the file and starts executing jobs/stages.
As soon as the file itself is valid and contains proper jobs and stages, runner fetches the latest commit (automatically) and does whatever script item tells it to do.
To verify that everything works correctly, go to your Gitlab -> CI / CD -> Pipelines, and check out whats going on. You should see something like this:
Maybe it would be best if you posted your .yaml file, there could be a number of reasons your runner is not picking up the code. For instance, maybe your .yaml tags are not matching what runner is created to pick up etc.
I made a website using Node.js as the server. As I know, the node.js file should start working by typing commands in terminal, so I'm not sure if Github Pages supports node.js-hosting. So what should I do?
GitHub pages host only static HTML pages. No server side technology is supported, so Node.js applications won't run on GitHub pages. There are lots of hosting providers, as listed on the Node.js wiki.
App fog seems to be the most economical as it provides free hosting for projects with 2GB of RAM (which is pretty good if you ask me).
As stated here, AppFog removed their free plan for new users.
If you want to host static pages on GitHub, then read this guide. If you plan on using Jekyll, then this guide will be very helpful.
We, the Javascript lovers, don't have to use Ruby (Jekyll or Octopress) to generate static pages in Github pages, we can use Node.js and Harp, for example:
These are the steps. Abstract:
Create a New Repository
Clone the Repository
git clone https://github.com/your-github-user-name/your-github-user-name.github.io.git
Initialize a Harp app (locally):
harp init _harp
make sure to name the folder with an underscore at the beginning; when you deploy to GitHub Pages, you don’t want your source files to be served.
Compile your Harp app
harp compile _harp ./
Deploy to Gihub
git add -A
git commit -a -m "First Harp + Pages commit"
git push origin master
And this is a cool tutorial with details about nice stuff like layouts, partials, Jade and Less.
I was able to set up github actions to automatically commit the results of a node build command (yarn build in my case but it should work with npm too) to the gh-pages branch whenever a new commit is pushed to master.
While not completely ideal as i'd like to avoid committing the built files, it seems like this is currently the only way to publish to github pages and should work for any frontend Node.js app (or app built with a frontend framework like React or Vue) that can be served as static files.
I based my workflow off of this guide for a different react library, and had to make the following changes to get it to work for me:
updated the "setup node" step to use the version found here since the one from the sample i was basing it off of was throwing errors because it could not find the correct action.
remove the line containing yarn export because that command does not exist and it doesn't seem to add anything helpful (you may also want to change the build line above it to suit your needs)
I also added an env directive to the yarn build step so that I can include the SHA hash of the commit that generated the build inside my app, but this is optional
Here is my full github action:
name: github pages
on:
push:
branches:
- master
jobs:
deploy:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout#v2
- name: Setup Node
uses: actions/setup-node#v2-beta
with:
node-version: '12'
- name: Get yarn cache
id: yarn-cache
run: echo "::set-output name=dir::$(yarn cache dir)"
- name: Cache dependencies
uses: actions/cache#v2
with:
path: ${{ steps.yarn-cache.outputs.dir }}
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- run: yarn install --frozen-lockfile
- run: yarn build
env:
REACT_APP_GIT_SHA: ${{ github.SHA }}
- name: Deploy
uses: peaceiris/actions-gh-pages#v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./build
Alternative solution
The docs for next.js also provides instructions for setting up with Vercel which appears to be a hosting service for node.js apps similar to github pages. I have not tried this though and so cannot speak to how well it works.
No, You cannot publish on Github pages. Try Heroku or something like that. You can only deploy static sites on github pages. You can't deploy a server on github pages.
No,
GitHub allows hosting only static websites(having only HTML, CSS, javascript).
Dynamic websites(having databases, servers, and all) can't be hosted as a Github page.
And node.js app is a server-based website, we can't host it on Github.
You can try Heroku, Openshift to host your website.
ahm. Yep, as most answer says. Github Pages only process html and css and a front-end JS.
But you can use JS framework like Gatsby which is mainly known to generate static purely static files, it gathers the data on compilation.
Then use that generated folder as the directory of the site.
I would like to add that it IS very much possible, as I am doing it right now. Here's how I'm doing it:
(I'm going to assume you have a package and/or directory ready to publish.)
In the root of your package.json, add
"homepage": "https://{pages-endpoint}/{repo}",
Where the pages-endpoint is the blah.github.io endpoint you specified in the Settings -> Pages portion of your repository, and repo is the name of your repository.
Then make sure you npm install --global gh-pages --save-dev. You need the --global to ensure the bin file is on your PATH and --save-dev should add it as a dependency in your package.json
After that, just npm run build && gh-pages -d build. The -d specifies your output build directory. The standard is build, but mine was public. If it's different, just change it.
Lastly, make sure in the Settings -> Pages section, you select gh-pages as the branch to host and leave the directory as / (root). Once it's built, your site should be available at your github.io endpoint.
Happy Dev-ing!
It's very simple steps to push your node js application from local to GitHub.
Steps:
First create a new repository on GitHub
Open Git CMD installed to your system (Install GitHub Desktop)
Clone the repository to your system with the command: git clone repo-url
Now copy all your application files to this cloned library if it's not there
Get everything ready to commit: git add -A
Commit the tracked changes and prepares them to be pushed to a remote repository: git commit -a -m "First Commit"
Push the changes in your local repository to GitHub: git push origin master