This post relates to a rapidly changing event.
I simply want to connect to my GitHub account. When I do it, I get this error message as a small red pop up on the upper right side of the page:
Items could not be retrieved, Internal server error.
As of May 25, 2022, at 19:52 UTC, GitHub integration has been re-enabled:
We are happy to report that the GitHub integration is re-enabled! You can now reconnect with GitHub and restore your Heroku pipeline functionality, including Review Apps, with newly generated tokens.
You can connect to GitHub immediately or wait for the enhanced integration as described in this blog post. To re-establish your GitHub connection now, please follow these instructions.
Here is what the referenced blog post says about "enhanced integration":
In an effort to improve the security model of the integration, we are exploring additional enhancements in partnership with GitHub, which include moving to GitHub Apps for more granular permissions and enabling RFC8705 for better protection of OAuth tokens. As these enhancements require changes by both Heroku and GitHub, we will post more information as the engagement evolves.
No timeline is mentioned for availability of the enhanced integration.
Between April 15 and May 25, 2022, Heroku's GitHub integration feature was disabled while Heroku investigated a security breach. During that time, deploying was still possible via other means, most notably via git push.
To mitigate impact from potentially compromised OAuth tokens, we will revoke over the next several hours all existing tokens from the Heroku GitHub integration. We are also preventing new OAuth tokens from being created until further notice. Your GitHub repositories will not be affected in any way by this action.
Which Heroku features have become non-operative due to the removal of the Heroku-GitHub integration?
Enabling review apps
Creating (automatic and manual) review apps
Deploying (automatic and manual) review apps
Deploying an app from GitHub (either manual or automatic)
Heroku CI cannot create new runs (automatically or manually) or see GitHub branch list
Heroku Button: unable to create button apps from private repositories
ChatOps: unable to deploy or get deploy notifications
Any app with a GitHub integration may be affected by this issue. To address specific integration issues, please open a case with Heroku Support
Migrating from GitHub deployment to Git deployment
At 2022-04-21 23:53 UTC, Heroku provided extended instructions for migrating from GitHub-based deployment to Git-based deployment:
While our customers remain unable to reconnect to GitHub via the Heroku dashboard, we wanted to share a supplement to the code deployment methods previously provided. For instructions on how to change your deployment method from GitHub to Heroku Git, please refer to the following Help article: How to switch deployment method from GitHub to Heroku Git with all the changes/app code available in a GitHub repo.
This is due to an issue reported at their status portal, here.
For now, the solution is to use another pushing strategy.
The best one, for now, is using their remote with Heroku CLI. The steps for it are:
1. Install the Heroku CLI if not yet installed
Further information about installation and setup can get here
2. Perform login with the heroku login command
You will be prompted with your default browser window for completing the login at the Heroku portal.
3. Assuming you already have your app set up at Heroku (if not, please follow this), you just need to add a new remote for your Git repository with Heroku CLI.
Run heroku git:remote -a example-app - substitute "example-app" with your app name.
4. git remote -v to check if the remote has been set successfully
You should see something like this appear as a response:
heroku https://git.heroku.com/your-app-name.git (fetch)
heroku https://git.heroku.com/your-app-name.git (push)
5. Push your branch to the new heroku remote
git push heroku your_branch_name
6. You should see the progress of your deployment being displayed in the terminal
The reference for this answer has been taken from here, which can also be used if further information other than the one provided in this answer is needed.
I'm in the same situation, and, as others stated, it's due to a Heroku security issue. In the meantime, you can deploy your code by using the Heroku CLI.
So, on the Heroku web dashboard, select Heroku Git:
Then set up the Heroku CLI with heroku login.
Finally, if your repository already exists on GitHub, you need to add a new remote by running:
heroku git:remote -a your_app_name
git push heroku master
You can find more information about this solution in the official documentation.
It is just a temporary thing, and more details about this issue are here.
You could push to both GitHub and Heroku at once for a temporary solution:
git push -u origin <branch>
git push heroku <branch>
I see the previous answers, but since I was facing an issue with review-apps (PR apps), mostly you will be working with different branches in that case, so here is a solution for pushing your stuff other than the (master/main) branch to Heroku.
First make sure your remote origin is set up correctly
heroku git:remote -a your_awesome_app
You can also confirm it by git remote -v and you should see your origin pointing to your Heroku application.
git remote -v
heroku https://git.heroku.com/your_awesome_app.git (fetch)
Here origin name is heroku.
If you want to deploy your changes from the main branch
git push heroku main
If you want to push your changes from feature branch (other than the* main branch)
Then push your feature branch to Heroku using the below command
git push heroku feature:main
heroku - is your origin name (confirm your origin name with git remote -v
feature - is your current branch which is other than main/master branch (check your branch name with git branch or git status)
For those who are using this integration for deployment purposes this, I suggest you use the deploy to a Heroku GitHub action here:
Deploy to Heroku
That way, you do not have to make significant changes to your deployment workflow.
I had the same issue. I already had cli installed.
git remote
Output:
heroku
origin
git remote -v
Output:
heroku https://git.heroku.com/YOUR-APP.git (fetch)
heroku https://git.heroku.com/YOUR-APP.git (push)
origin https://github.com/GitUserName/yourRepo.git (fetch)
origin https://github.com/GitUserName/yourRepo.git (push)
Verify using the Git branch, if your branch is named main. For example, you would do:
git push heroku main
For me it is
git push heroku master
Now push your local changes
git push heroku master
Output:
To https://git.heroku.com/YOUR-APP.git
! [rejected] master -> master (non-fast-forward)
error: failed to push some refs to 'https://git.heroku.com/YOUR-APP.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
I do not care that it is different than its remote. I've been developing locally and did not realize the automatic Git deploys had been failing. I care about the local changes:
git push heroku master -f
Now the deployed application is working as expected.
If you are getting an error while trying to push, it may be due to having different branch names for development vs deployment. In such a case, follow the instruction below;
If you are developing on a branch and deploying via Git you must run:
git push heroku <branchname you're developing on>:<branch you're deploying from>
This article goes into details on the behavior:
Duplicate Build Version Detected
Here's the easy work pattern I have used for Heroku. This is intended to help others who may not have gone through this before. I used this previously (2014-5) and had to set it up again last night to push.
First add the Heroku remote:
git remote add heroku https://git.heroku.com/YOUR-APP.git
As GitHub is often "origin" (git push origin...), this adds another remote destination, "heroku" (git push heroku...)
git remote
Output:
heroku
origin
My pattern for code/git/pushing:
Local development is the same. Push to GitHub, merge, and nothing changes.
Set your deploy to "Heroku git" as #a-chris outlines.
To push to Heroku, simply push the correct branch to the newly added "heroku" remote source. I use the --force option to dismiss any possibility of conflict. Unless you have been using Heroku Git and branching previously, there should only be one branch - typically "master" to use.
This will trigger a deploy. You can watch or review in the dashboard as well as the terminal. Treat the new "heroku" source as a directory to dump code to promote and not a repository you want to keep history, etc. A second-class citizen in this particular pattern.
I push from my local terminal now instead of auto-deploy or via the dashboard button. If your organisation is large, I recommend controlling access. Many developers may not have experience juggling multiple destination repositories or to catch an accidental push.
To trigger a local push, be sure your master (or whatever) is up to date...watch your commit hashes!
This will set you up to follow advice such as BR19_so and others.
Henrique Aron's answer is working for the local machine.
For a remote SSH server, you will face an IP mismatch error.
To resolve it:
Log in to the Heroku website
Go to account settings
Reveal the API keys in the panel
In the server CLI, type heroku login -i
Input email, use the API key as the password, and you can follow the rest of the steps of a Git remote push
I am using a 2014 MacBook Pro with macOS 11.6.5 (Big Sur) version for command
brew tap heroku/brew && brew install heroku
I was getting an error and a message to update to the latest Xcode. The latest Xcode needed the latest macOS, which I could not upgrade to (I needed a 2015 MacBook Pro or newer). For a 2014 MacBook Pro, I was able to Install Xcode_13.2.1.xip and was able to run. Now I can push updates to Heroku.
brew tap heroku/brew && brew install heroku
There is an update regarding this issue. You can now enable GitHub integration as mentioned in the update.
If it doesn't work, you can try removing the App connection to GitHub (disconnect) and adding the connection again. Adding the connection again can also be done in incognito mode because sometimes an issue occurs related to cookies (mentioned here).
We are using Git for a website project where the develop branch will be the source of the test server, and the master branch will serve as the source for the live, production site. The reason being to keep the git-related steps (switching branches, pushing and pulling) to a minimum for the intended user population. It should be possible for these (not extremely technical) users to run a script that will merge develop into master, after being alerted that this would be pushed to live. master cannot be modified by normal users, only one special user can do the merge.
This is where I'm not sure how to integrate this identity change into my code below:
https://gist.github.com/jfix/9fb7d9e2510d112e83ee49af0fb9e27f
I'm using the simple-git npm library. But more generally, I'm not sure whether what I want to do is actually possible as I can't seem to find information about this anywhere.
My intention would be of course to use a Github personal token instead of a password.
Git itself doesn't do anything about user or permission management. So, the short answer is, don't try to do anything sneaky. Rather, use Github's user accounts they way they were intended.
What I suggest is to give this special user their own Github account, with their own copy of the repo. Let's say the main repo is at https://github.com/yourteam/repo, and the special repo is at https://github.com/special/repo.
The script will pull changes from the team repo's develop branch, and merge this into it's own master branch and push to https://github.com/special/repo.
Then, it will push its changes to the team's master branch. This step can optionally be a forced push, since no one else is supposed to mess with master, anyway. (In case someone does, using a forced push here means they have to fix their local repo to match the team repo later on, rather than having the script fail until someone fixes the team repo.)
At the same time, your CI software will notice that master has changed at https://github.com/special/repo, and will publish as you normally would. This is the linchpin: the CI doesn't pay attention to the team repo, so although your team has permission to change it, those changes don't make it into production.
This special user will need commit access to the team repo, in addition to its own GitHub repo. The easiest way is probably to use an SSH key, and run the git push command from the script, rather than trying to use the GitHub API.
I downloaded and started authoring environment (crafter-cms-authoring.zip)
Created site backed by remote git repo as described in: Create site based on a blueprint then push to remote bare git repository
Created a content type, new page.
Published everything
Now, I would expect, that I can see my changes in the remote repo. But all I can see are the initial commits from the 2. step above. No new content type, no new page, no branch "live". (The content items are however visible in the local repo)
What is missing?
Edit:
Since Creafter can by set up in many ways, in order to clarify my deployment scenario, I am adding deployment diagram + short description.
There are 3 hosts - one for each environment + shared git repo.
Authoring
This is where studio is located and content authors make changes. Each change is saved to the sandbox local git repository. When a content is published, the changes are pulled to the published local git repository. These two local repos are not accessible from other hosts.
Delivery
This is what provides published content to the end user/application.
Deployer is responsible for getting new publications to the delivery instance. It does so by polling (periodically pulling from) specific git repository. When it pulls new changes, it updates the local git repository site, and Solr indexes.
Gitlab
This hosts git repository site. It is accessible from both - Authoring and Delivery hosts. After its creation, the new site is pushed to this repo. The repo is also polled for new changes by Deployers of Delivery instances.
In order for this setup to work, the published changes must somehow end up in Gitlab's site repo, but they do not (the red communication path from Authoring Deployer to the Gitlab's site)
Solution based on #summerz answer
I implemented GitPushProcessor and configured new deployment target in authoring Deployer, adding mysite-live.yaml to /opt/crafter-cms-authoring/data/deployer/target/:
target:
env: live
siteName: codelists
engineUrl: http://localhost:9080
localRepoPath: /opt/crafter-cms-authoring/data/repos/sites/mysite/published
deployment:
pipeline:
- processorName: gitPushProcessor
remoteRepo:
url: ssh://path/to/gitlab/site/mysite
I think you might have confused push with publish.
On Publishing
Authoring (Studio) publishes to Delivery (Engine) after an approval workflow that makes content go live. Authoring is where content (and code if you like) is managed and previewed safely, then that is published to the live delivery nodes for delivery to the end-user.
On DevOps
A site's local git repository can be pushed/pulled to/from remote repositories. This means:
Code can flow from a developer's workstation to Studio (via a github, gitlab, bitbucket etc.) <== this is code moving forward (and can flow via environments like QA, Load Testing, etc.)
Content can flow back, from Studio to the developer's local workstation in a similar manner <== this is content moving backward (you can have production content on your laptop if you want)
When code flows forward from a developer to Studio, that's when Studio pulls from the remote git repo.
When content flows backward from Studio to the developer, that's when Studio pushes to the remote git repo.
Documentation
A good bird's eye view of the architecture of the system relating to publishing can be found here: http://docs.craftercms.org/en/3.0/developers/architecture.html
A good article that explains the DevOps workflow/Git stuff is here: http://docs.craftercms.org/en/3.0/developers/developer-workflow.html
Update based on the expanded question
My new understanding based on your question is: You can't allow the deployers in Delivery to access Authoring's published repo to poll due to some constraint (even over SSH and even with limits on the source IP). You'd like to use GitLab as a form of content depot that's accessible as a push from Authoring and pull from Delivery.
If my understanding is correct, I can think of two immediate solutions.
Set up a cron job in authoring to push to GitLab periodically.
You'll need to add GitLab as a remote repo in published and then set up a cron like this:
* * * * * git --git-dir /opt/crafter/data/repos/sites/{YOUR_SITE}/published/.git push 2>&1
Test it out by hand first, then cron it.
Write a deployer processor that can push content out to an end-point upon a change or, wait for the ticket: https://github.com/craftercms/craftercms/issues/2017.
Once this is built, you'll need to configure another deployer in Authoring that will push to GitLab.
In either case, beware not to update things in GitLab since you're using published and not sandbox. (See DevOps notes above to learn why.)
My requirement is whenever developer try to do check-in existing GitLab repository then before doing check-in in repository,build should trigger (Jenkins build) and Junit test case should run on new check-in and if passes then it should go forward and will allow developer to do check-in in main repository.
I am not sure but is pre-hook commit can achieve this requirement?
While you could achieve this with pre-commit hooks, it's more common to do so with post-commit hooks on the server-side.
You can achieve this by operating a branch based workflow, there are multiple to choose from - I would recommend reading through this guidance by Atlassian.
Developers will create branches from a 'main' branch (often master, but can be a 'dev' branch working towards a release for instance), then develop code on that branch. They will then push their branch and commits to the remote repository (GitLab). When ready to merge into the main branch, your developers can open a merge request onto the main branch.
On GitLab you can setup a webhook to trigger Jenkins builds when a push event occurs. I would recommend this guide to guide you through it.
In the GitLab project settings you can require a passing build before merge requests are allowed to merge.
Furthermore, your understanding of Git seems incorrect - check in is not a term used in Git. Please take a look at the Git documentation. In Git a developer creates commits against a local copy of the repository, then pushes these to a remote repository (GitLab/GitHub etc.). There is no direct equivalent of the 'check in' used in various centralised version control systems e.g. SVN.
I have a public gitlab project here
https://gitlab.com/parmentelat/minisim2
I tried to add a corresponding project in readthedocs.io, so that a new commit being pushed onto gitlab triggers a doc rebuild on readthedocs
I do this routinely with projects hosted at github and it's really easy - at least under my setup - since readthedocs shows me an updated list of github repos right away, and everything goes smoothly after that.
When trying to import this gitlab project under readthedocs though, I have to chose 'Import manually' as my gitlab projects would not show up.
(In the 'connected services' of my readthedocs settings page, I could find a way to connect to github and to bitbucket; gitlab does not seem supported)
Fair enough, I try this manual import, but at that point no matter how I try to spell the project's URL and what method (git or https) I try to use for importing the project, I get this error message
This repository doesn't have a valid webhook set up. That means it won't be rebuilt on commits to the repository.
You can resync your webhook to fix this.
is what I am trying to do doable at all ?
do I need to do something specific on the gitlab side
thanks for any hint
You can manually set the webhook on gitlab.com:
Click the settings icon for your project
Select "Integrations"
Enter the above URL, select "Push events" and "Enable SSL verification"
Click "Add Webhook"
That should do it.