Getting a list of changed files in BitBucket Pipelines - bitbucket-pipelines

I know that I can limit specific steps to "condition: changesets: includePaths:" That's not what I'm trying to do.
I know that I can get a list of modified files in the last commit with git diff. That, unfortunately, isn't sufficient.
What I would like to do is get a list of all of the files which have been modified in this branch since last push. For instance, imagine that you do the following:
Create a branch.
Commit A.
Push.
Commit B.
Commit C.
Commit D.
Push.
I want a list of the files modified in all of B, C and D so that I can pass them to my testing scripts and evaluate all of the files which have been modified since the last time the pipeline was able to run.
With pull-requests, I think I can do this by diffing BITBUCKET_COMMIT and BITBUCKET_PR_DESTINATION_BRANCH. But for pipeline steps which I want to run simply any time commits are pushed, I can't figure out how to get the changeset I want.

Related

Remove one specific empty git commit with command-line tool

For a nodejs command-line tool I add an empty commit to a repo and then want to remove it later.
Later I have at least 3 commits. The first one is a merge commit, the second one is the empty one I created and the third one is likely one from another now merged repo. Now that my tool has done it's task I want to remove the empty commit.
git rebase --onto emptyCommitID^ emptyCommitID
resulted in: fatal: Does not point to a valid commit 'emptyCommitID^'
(since the ID is the correct one I assume the commit is invalid due to it being empty)
git rebase --keep-base --onto thirdCommit^ headCommit
resulted in fatal: cannot combine '--keep-base' with '--onto'
trying rebase -i HEAD~3 after the tool had done it's main job resulted in:fatal: invalid upstream 'HEAD~3' which might be due to either the empty commit or the merged unrelated histories idk.
I do not want to use git filter-branch --prune-empty, because the tool shall leave other potentially empty commits untouched.
(The tool is for merging repos with unrelated histories. I create the empty commit so that files are staged when merged instead of committed directly which also happens when the --no-commit flag is set in an just initialized repo without prior commits)
maybe it is possible to solve this with git rebase --interactive, but I had the described problem with the invalid upstream and view this as very difficult to implement with a command line tool, mostly using exec to execute it's commands. Do you know a solution?
I think git rebase --onto emptyCommitID^ emptyCommitID should work.
fatal: Does not point to a valid commit 'emptyCommitID^' means that the emptyCommitID has no parent. It contradicts that the second one is the empty one and the first one is its parent.

How to get all commits in a repository along with the corresponding branch name?

I am faced with a problem of getting all commits in a repository with the branch name too along with the commit ID . While there exists one endpoint that lists all the commits (https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/commits) what this API does not give is the branch name along with the commit ID . If I call the branches endpoint : /2.0/repositories/{workspace}/{repo_slug}/refs/branches/{name} I can get only the latest commit and not all commits in the branch . To do any kind of mapping I would need to call each branch and then another loop to call each commit within a branch this causes the code to fail as I exceed the no. of requests allowed . I need some solutions to tackle this problem .
I am writing a python script that calls these two api endpoints in two loops and generating a list of lists out of this
You can use the file history option provided by bitbucket.
And Branch name by default is master unless you change it in your properties file.
https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/filehistory/%7Bcommit%7D/%7Bpath%7D

How can I prevent jobs from running based on the files changed compared with master?

I have some gitlab jobs in my pipleline which are slow and I'd like to prevent them from running when the changes will not affect the job's outcome.
This is what I have tried:
run_tests:
stage: checks
script:
- cargo test
except:
- master
- tags
only:
changes:
- "**/*.rs"
- "**/Cargo.toml"
- "**/Cargo.lock"
This sort of works. If a merge request has multiple commits, this job will not run on any of the commits after the first one, unless a Rust source file has changed.
But the job will still always run on the first commit of the branch, even if there are no changes to Rust source files between this branch and master. Even worse, if the tests fail on the first commit, subsequent commits might skip the tests so broken code could get merged.
How can I change this filter so that the diff is done against the target branch of the merge request?
Hidden quite deep in the documentation for only: changes, there is this snippet:
Without pipelines for merge requests, pipelines run on branches or tags that don’t have an explicit association with a merge request. In
this case, a previous SHA is used to calculate the diff, which
equivalent to git diff HEAD~. This could result in some unexpected
behavior, including:
When pushing a new branch or a new tag to GitLab, the policy always evaluates to true.
When pushing a new commit, the changed files are calculated using the previous commit as the base SHA
Which is the scenario you are currently experiencing, where the job always runs on the first commit of the new branch.
For this to work as you want, I believe you need to add only: merge_requests, however this would require changing your entire pipeline workflow to integrate only: merge_requests to your pipeline.
From the documentation:
With pipelines for merge requests, it’s possible to define a job to be
created based on files modified in a merge request.
In order to deduce the correct base SHA of the source branch, we
recommend combining this keyword with only: [merge_requests]. This
way, file differences are correctly calculated from any further
commits, thus all changes in the merge requests are properly tested in
pipelines.
Various issues from GitLab:
https://gitlab.com/gitlab-org/gitlab/-/issues/11427
https://gitlab.com/gitlab-org/gitlab/-/issues/27875

error: failed to push some refs to 'git#github.com:<name>/<project>.git'

Hi I'm a total newbie using git I read a few StackOverflow Q&A on this error but didn't understand how my problem relates to others with the same error message. So I didn't dare to chance it and loose 6 hours of work that I need to push to my remote GitHub repo.
.
1.) The Beginning
So my code in the local working directory got broken and I couldn't figure out what the problem was. So I wanted git to go back to the latest working version of my project. I then found this tutorial and managed to get my local project to go back to a working version.
https://www.git-tower.com/learn/git/faq/restore-repo-to-previous-revision
$ git reset --hard <SHA-1 HASH NUMBERS>
.
2.) Trying to push
These are the commands I usually follow when I want to push local code to my remote repo.
# Pushing Changes - Staging
$ git diff
$ git status
$ git add -A
$ git status
$ git commit -m "Modified multiply function."
# Pushing Changes - Repository
$ git pull origin master
$ git push origin master
.
3.) The error
I don't understand what the error message is trying to tell me. Can somebody help me by showing what sequence of commands I need to run to be able to push code like usual again?
penguin#linux$ git push origin master
To git#github.com:<name>/<project>.git
! [rejected] master -> master (non-fast-forward) error: failed to push some refs to 'git#github.com:<name>/<project>.git'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Let's address this rather long comment (which I should do in another comment, but this won't fit at all so I'm answering a different question, namely, "how does detaching and reattaching HEAD work"):
Tnx torek I managed to force away that error with the link you provided. I also had some detached HEAD issues which I got rid by following Razan Paul's post here Fix a Git detached head? But all my changes while in detached HEAD mode was lost after the fixes. I have copies so I can just recreate them manually I guess. So in the future when I want to go back to the latest working version on my local working directory. What procedure will suite my situation best? I see so many different suggestions I don't know what's best for me
(incidentally, when addressing a comment to someone in particular you may want to use the #<name> syntax so they get alerted).
What you need is a proper Git book. The best free one is, in my opinion, the Pro Git 2 book. (There are not many, if any, other free ones so "best" could be vacuously true here. :-) ) A shorter, non-book reference that I find helpful is Think Like (a) Git, because Git is built atop some very simple bits of graph theory—but unless you know this, and until you realize this, you're stuck in this xkcd cartoon.
(I even have my own start on a book, but I have very little time to work on it these days.)
There are many work-flows you can use in Git. The first thing to remember, though, is that Git's "unit of storage" as it were—the thing it desperately tries to protect, so that you can get it back—is the commit. That's why you were having trouble pushing: you were telling some other Git: throw away some commits. Git does not want to do that.
Commits are uniquely identified by their hash IDs. These hash IDs are not very useful to humans, so Git gives us things like branch names to remember particular IDs—but in some cases you may have to resort to raw IDs, which you can cut and paste with your mouse, for instance.
Git's desire to keep commits around means that when you make commits using a detached HEAD, Git tries to keep those too. It will do so for, by default, at least 30 days. You can find the hash IDs using git reflog. If you use git reset --hard to make Git "forget" commits that were on a branch (rather than a detached HEAD), Git keeps those IDs around for at least 30 days as well, on the branch name's reflog. There's one reflog for each branch name, and one for HEAD itself.
Last, an attached HEAD—attached being the opposite of detached—is when you are, as git status will say, "on" some particular branch:
$ git status
On branch master
...
In this case, the branch name master is how Git actually identifies which commit you have checked out right now; and the name HEAD is simply attached to the name master.
To make this all work right, Git works backwards.
A brief introduction to graphs
To really comprehend what all this means, you need to draw the commit graph. Remember that each commit has its own unique hash ID—those big ugly 40-character-long strings of a hexadecimal number, like 5be1f00a9a701532232f57958efab4be8c959a29—but that's kind of unweildy, so you might want to just use single letters for small drawings:
A <-B <-C <--master
This is a pretty-new repository with just three commits in it. We made A first, then we made B, then we made C. (We'll run out of names for commits when we get to our 27th commit, so you can see why Git uses longer hash IDs.)
Since Git works backwards, the name master identifies, not commit A, but rather commit C. We say that the name master points to C.
Commit C, being our third commit, has inside it the name—the hash ID—of our second commit, commit B. We say that C's parent commit is B. So commit C points to B. Likewise, B has inside it the hash ID of commit A, so B points back to A.
Commit A was the first commit. It can't point back to a previous commit, so it just doesn't. This makes commit A special: it's a root commit. Every non-empty repository has at least one of these, the first commit ever made.
When you run git log, Git will start with your current commit—commit C, here—and work backwards: it shows you C, and then since C points back to B, Git shows B too. Since B points back to A, Git shows A as well; but A is a root commit, so Git can stop.
How branches grow
Since we're on master, let's make a new commit D. We'll do whatever we want with the source, git add files, and run git commit to create D.
Where will D point back to? Well, obviously it has to point back to C. So Git makes D have a parent commit of C:
A <-B <-C <-D
Git's final step here is to change the name master so that it holds the hash ID of commit D, giving us this picture:
A <-B <-C <-D <--master
Note that all the arrows necessarily point backwards. Moreover, all the arrows that are inside commits are frozen for all time: D points back to C, never anywhere else. The only arrow that changes is the one coming out of the branch name! So I tend to draw them without the internal arrows, to save space and make it fit better:
A--B--C--D <-- master
This is another key to understanding Git: branch names move, over time. Branches acquire new commits, and the names point to the last commit on the branch—what Git calls the tip commit. Git uses this tip commit to find the next-older commit for that branch: the tip's parent. Git uses that commit to find its previous commit, and so on, all the way back to the root commit.
The name HEAD normally just identifies a branch name
Let's complicate-up the above repository by adding a branch coming out of C:
A--B--C--D <-- master
\
E <-- develop
Here, we now have two branches. The name develop identifies commit E, whose parent is C. If we now run:
git checkout master
we'll be on branch master, as git status will say; if we create a new commit F now, its parent will be D. If we instead git checkout develop and create a new commit F, its parent will be E instead. So Git needs to know: which branch are we on? This is where we need to draw in the name HEAD:
A--B--C--D <-- master (HEAD)
\
E <-- develop
or:
A--B--C--D <-- master
\
E <-- develop (HEAD)
When your HEAD is attached like this, it's attached to a branch name. When you make new commits, Git will change the branch name so that it points to the new commit you just made.
A "detached HEAD" simply holds the commit hash ID directly
If you find yourself in detached HEAD mode, it's usually because you checked out an older commit directly. For instance, let's check out commit B, while keeping everything we have:
A--B <-- HEAD
\
C--D <-- master
\
E <-- develop
I had to bend the graph lines around to fit HEAD in, but now HEAD points directly to commit B.
If we now make a new commit F, its parent will be B, and Git will make HEAD point directly to F:
A--B--F <-- HEAD
\
C--D <-- master
\
E <-- develop
Note that whenever Git creates a new commit, the commit to which HEAD points—directly, if detached, or indirectly, if attached to a branch name—also changes! The new commit's parent is always whatever was HEAD just a moment ago, and now HEAD is the new commit, which is once again the current commit.
(This is also why parents don't know their children's hash IDs, but a child does know its own parent's ID. Git has to work backwards, because children know their parents: the parent commits exist when the child is created; but parent commits—which are read-only and frozen in time—don't know what children they will acquire in the future.)
As soon as we re-attach HEAD, though, we "lose" the ID of commit F:
A--B--F <-- ???
\
C--D <-- master (HEAD)
\
E <-- develop
Who remembers F's ID? The answer is: only the reflogs.
If you want to hang on to commit F, you should find its ID in your reflogs and attach a name to it—a branch or tag name, usually:
A--B--F <-- newbranch
\
C--D <-- master (HEAD)
\
E <-- develop
Now newbranch remembers the ID of F. If you'd made two commits, F and G, while you had a detached HEAD, you'd have to find the later one somehow and make sure your name points to G, so that you have:
G <-- newbranch
/
A--B--F
\
C--D <-- master (HEAD)
\
E <-- develop
and not:
G <-- ???
/
A--B--F <-- newbranch
\
C--D <-- master (HEAD)
\
E <-- develop
This is why making a lot of commits on a detached HEAD is not a great idea: it can be really hard to tell, from hash IDs, which commit is the tip-most.
The git it clearly mentioning that the current branch in which your working is not updated with remote repository as others have committed their code to that repository.
First you need to take a pull of remote repository and then commit your code
NOTE : Here origin is your remote repository location which you
might have added
Eg: Considering that your working on master branch
git pull origin master /* To update with the remote repository */
git push origin master /* To push your updated code */

GIT: git checkout --ours still showing "both modified" [duplicate]

This question already has an answer here:
git checkout --ours does not remove files from unmerged files list
(1 answer)
Closed 3 years ago.
I'm trying to resolve merge conflict in some files
both modified: myFile.h
I ran this command:
git checkout --ours myFile.h
after that I ran:
git status
It shows this:
both modified: myFile.h
Why still shows "both modified" ?
Some git checkout commands resolve a merge conflict, taking the version you checked out, and some don't. This is one of the cases that don't.
You must therefore mark the conflict as resolved manually, with: git add myFile.h.
Why this is the way it is
Merging (the action, i.e., merge-as-a-verb) is done through Git's index (also called the staging area or sometimes the cache). The index has one entry for each file that will go into the next commit you make. Normally, that one entry holds one file—but the index has four slots per entry, which are numbered. Slot zero (0) is the normal "no conflict, file is ready to commit" slot. Slots 1, 2, and 3 are used only during a merge conflict, and contain the merge base version (slot 1), --ours version (slot 2), and --theirs version (slot 3).
If slot zero is filled, the other three slots are empty. If any of the other slots are filled, slot 0 is empty and the file is in "conflicted" state. (It's possible to have just two of the other three slots filled, as is the case for add/add, modify/delete, and rename/delete conflicts.)
When you run git checkout commit -- path,1 Git copies a file from the given commit into slot zero, then from slot zero to the work-tree. Copying to slot zero wipes out slots 1-3 if they're filled, so the file is now resolved!
But, when you run git checkout --ours -- path, Git doesn't have to write anything to index slot zero, it can just get the file's contents from slot 2. So it copies from slot 2 to the work-tree, and the file is not resolved.
Note that this means you can do git checkout HEAD -- path to extract the file from the HEAD commit, writing to slot zero and thus resolving as well as writing to the work-tree. This is subtly different in another way, though. Suppose that during the merge, Git decided that the file was renamed as well as being modified. It's taken the rename into account: the file's new name is evil/zorg instead of the old name evil-zorg. If you git checkout --ours, Git will extract the old (HEAD) version of evil-zorg under the new name evil/zorg. If you git checkout HEAD, Git won't find the file under the new name!
(This is another case of Git letting implementation details show through—or, equivalently, of cramming too much stuff into one command.)
1The reason for the -- is to handle a file whose name looks like an option. (For instance, what if the file's name is --theirs?) If the path part doesn't look like an option, you don't need the --. It's a good habit to pick up, though: use the -- every time and you won't be surprised someday when your file name does resemble an option.

Resources