Is there any way to do a dry run of xgettext on source files, in order to simply check if there are any differences compared to the current .pot file?
I have set up a Github workflow that will run xgettext on source files any time a change to a source file is pushed to the repository. The result is that often the change to the source file didn't change the translation strings, so the only difference in the resulting .pot file is the Creation date, which gets updated every time xgettext is run. This makes for unnecessary commits, and triggers unnecessary webhook calls to my weblate instance which picks up on an "updated" .pot file, and winds up generating its own Pull Request with an "updated" .pot file.
If there were a way to do a dry run and first check if there are any actual differences in the strings, I could avoid unnecessary commits and PRs from polluting my repo. Any ideas?
I was able to add a filter to my Github workflow, checking whether there are any significant changes besides a simple update to the value of POT-Creation-Date in the .pot file. I added an id to the step that takes care of running xgettext, then after running xgettext I save the count of significant lines changed in the pot file to a variable that will be accessible to the next step:
- name: Update source file translation strings
id: update_pot
run: |
sudo apt-get install -y gettext
xgettext --from-code=UTF-8 --add-comments='translators:' --keyword="pgettext:1c,2" -o i18n/litcal.pot index.php
echo "::set-output name=POT_LINES_CHANGED::$(git diff -U0 | grep '^[+|-][^+|-]' | grep -Ev '^[+-]"POT-Creation-Date' | wc -l)"
Then I check against this variable before running the commit step:
- name: Push changes # push the output folder to your repo
if: ${{ steps.update_pot.outputs.POT_LINES_CHANGED > 0 }}
uses: actions-x/commit#v4
with:
# The committer's email address
email: 41898282+github-actions[bot]#users.noreply.github.com
# The committer's name
name: github-actions
# The commit message
message: regenerated i18n/litcal.pot from source files
etc.
Related
I have changed line of a sql file file. But the diff only shows the diff colour without any change code.
the line is: #enabled=0, before the change we had 1 instead of 0.
without the gitattribute
*.sql text diff
I get the error message that file suppressed by a .gitattributes entry or the file's encoding is unsupported.
[this is the link of the image of my git diff] (https://i.stack.imgur.com/bgMvv.png)
You need to check your git status (assuming the #enabled=0 was done on your workstation)
Check if:
the file is indeed Test/Scripts/ScriptsIgnoredOnImport.sql
there is any local commit which would not have been pushed yet.
The file on GitHub can also tell you more, by typing b (which triggers the file "blame" view on GitHub).
As shown here, you can then "View blame prior to this change" and see if your #enabled= was visible then.
As noted by torek, you could have a difference in encoding as well.
As I mentioned in "How do I determine file encoding?", you can (even in a simple CMD on Windows), check the encoding of your current file with:
git show :your/file.sql | file -
# compare it with the previous version
git show #~your/file.sql | file -
How can i get the last merged branch name in git from the remote
Tried
git log --first-parent --merges -1 --oneline
But not getting the branch name
Please help
In general, you cannot.
A merge commit may have, as its commit message, text of the form merge branch foo or merge branch foo of someurl, but to read that message, you must obtain the commit from the remote. Even so, there's no guarantee that branch foo exists any more, or that the name means anything if it does exist. Merging really works by commit hash IDs, not by branch names; branch names may evanesce.
If you think you need a branch name here, you are probably doing something wrong. Branch names do make sense here in other version control systems, but in Git, depending on them is unwise.
Here is the command you need to give (change the branch name from origin/master to whichever branch you're checking merges for):
git log --merges origin/master --oneline --grep='^Merge' -1 | grep -oe "[^']*[^']" | sed -n 2p
I had quite a bit of a hard time trying to solve this issue.
I had to get this information for my CircleCi pipeline, and to solve my problem I used the GitHub cli.
Specifically the pr list command: https://cli.github.com/manual/gh_pr_list
The following command gives you all the information of the last PR you merged into main
gh pr list -s merged -B main -L 1
Then you need to manipulate the string and get only the branch name. Which is the text before "MERGED"
I took it splitting the string and taking the penultimate element.
I have multiple websites structured (simplified) as follows under a single GIT repository:
/
/site-1
/site-1/index.js
/site-1/about.js
/site-1/package.json
/site-1/node_modules(not versioned)
/site-1/package-lock.json(not versioned)
/site-2
/site-2/index.js
/site-2/about.js
/site-2/package.json
/site-2/node_modules(not versioned)
/site-2/package-lock.json(not versioned)
/site-3
/site-3/index.js
/site-3/about.js
/site-3/package.json
/site-3/node_modules(not versioned)
/site-3/package-lock.json(not versioned)
I did some amendments in /site-1/index.js, /site-1/package.json and added a file /site-1/changes.md.
The changes were done in 2 separate git commit in a feature branch called feature/carousel.
I want to apply the same changes in /site-2 and /site-3.
I've tried the following:
git format-patch master -o patches to retrieve the new diff in this feature branch with regards to master branch, but i was unable to apply the diff in /site-2 and /site-3.
diff -ruN site-1 site-2 > PatchFile1 to generate a consolidated diff, but it takes into account files that have been modified in /site-2 as well and its not a generic diff that can be applied directly to /site-3
Any idea how to achieve this?
You can use git apply with the --directory option to apply a patch to another main directory, as explained here:
https://blog.soltysiak.it/en/2017/08/how-to-use-git-patch-system-to-apply-changes-into-another-folder-structure/
First, create a patch file changes.patch based on the changes applied to directory site-1. Then you can do the following:
git apply -p1 --directory='site-2' changes.patch
git apply -p1 --directory='site-3' changes.patch
-p1 removes the site folder from the patch headers, which is the main part that differs between the different directories.
--directory='site-2' will cause the site-2 prefix to be added to each header in the patch
I'm working on a Java app that uses multiple APIs and would like to keep the API tokens out of the public GitLab repository. The app is packaged and deployed to a remote server and I don't know how to make the tokens available without including them in the GitLab repository otherwise.
Is there a way I can restrict the viewing of a file (or part of it) to sort of "redact" these tokens? Or should I go about it a different way?
Don't put API keys in your repo. Inject them into your use of the repo via environment variables which are maintained by your deployment system. If you deployment system doesn't have that ability, you probably need to change it. It doesn't need to be too complicated - for example change it to deploying your code from git, then copying a .env file into place separately. If your deployment mechanism only lets you use git repos, you could put your env vars into a separate repo that is kept private.
I have a similar situation with injecting google-services.json file into an Android application. Long story short we have multiple environments our app targets, and the production environment file must, somehow, reside in the build pipelines (either, committed or something).
As pointed by the previous response having this information committed in the main repo is not ideal. Developers could accidentally use the production environment while testing for example.
How we solved this
First, documentation. All those files (google-services.json and similar others) are ignored in git and the developer documentation states you must add your own.
Second, the CI build pipelines. We are also using GitLab, and we store those files as base64 encoded strings in CI variables, controlling then access to those variables via the protected tags/branches mechanism GitLab offers.
Serializing the files
There are two steps involved in here. First serialize an actual file in base64. Second, de-serialize the file from base64 into its appropriate location.
base64 --wraps=0 google-services.json (wraps option prevents line wrapping if done in console directly.). Then store the output in a GitLab CI variable.
In the .gitlab-ci.yml file do the inverse to inject the file.
echo $VAR_NAME | base64 -d > where/you/need/the/file.
You then control the appropriate environment to use via the $VAR_NAME variable.
An example of this an be found at https://gitlab.com/snippets/1926611. This case is for an xml file with the Google Maps API key, but the process is identical.
You can create a variable in you GitLab project settings. The variable can be used in your .gitlab-ci.yml file.
For example,
create a variable named GOOGLE_SERVICE_JSON and set the value to the base64 format of the file content. You can get it by command base64 google-services.json
update your .gitlab-ci.yml file, decode the GOOGLE_SERVICE_JSON value to google-services.json file like this
assembleDebug:
stage: build
script:
- echo ${GOOGLE_SERVICE_JSON} | base64 -d > app/google-services.json
- ./gradlew assembleDebug
artifacts:
paths:
- app/build/outputs/
You can also use this method to encode the keystore file to a variant and decode it to a file in pipeline build.
Here is a full example
image: openjdk:8-jdk
variables:
ANDROID_COMPILE_SDK: "28"
ANDROID_BUILD_TOOLS: "28.0.3"
ANDROID_SDK_TOOLS: "6609375_latest"
before_script:
- echo ANDROID_COMPILE_SDK ${ANDROID_COMPILE_SDK}
- echo ANDROID_BUILD_TOOLS ${ANDROID_BUILD_TOOLS}
- echo ANDROID_SDK_TOOLS ${ANDROID_SDK_TOOLS}
- apt-get --quiet update --yes
- apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1
- wget --quiet --output-document=android-sdk.zip https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS}.zip
- unzip -d android-sdk-linux android-sdk.zip
- export ANDROID_SDK_ROOT=$PWD/android-sdk-linux
- export SDK_MANAGER="${ANDROID_SDK_ROOT}/tools/bin/sdkmanager --sdk_root=${ANDROID_SDK_ROOT}"
- echo y | ${SDK_MANAGER} "platforms;android-${ANDROID_COMPILE_SDK}" >/dev/null
- echo y | ${SDK_MANAGER} "platform-tools" >/dev/null
- echo y | ${SDK_MANAGER} "build-tools;${ANDROID_BUILD_TOOLS}" >/dev/null
- export PATH=$PATH:${ANDROID_SDK_ROOT}/platform-tools/
- chmod +x ./gradlew
# temporarily disable checking for EPIPE error and use yes to accept all licenses
- set +o pipefail
- echo y | ${SDK_MANAGER} --licenses
- set -o pipefail
stages:
- build
assembleDebug:
stage: build
script:
- echo ${GOOGLE_SERVICE_JSON} | base64 -d > app/google-services.json
- echo ${KEY_STORE_PROP} | base64 -d > app/keystore.properties
- echo ${STORE_FILE} | base64 -d > app/keystore.jks
- ./gradlew assembleDebug
artifacts:
paths:
- app/build/outputs/
assembleRelease:
stage: build
script:
- echo ${GOOGLE_SERVICE_JSON} | base64 -d > app/google-services.json
- echo ${KEY_STORE_PROP} | base64 -d > app/keystore.properties
- echo ${STORE_FILE} | base64 -d > app/keystore.jks
- ./gradlew assembleRelease
artifacts:
paths:
- app/build/outputs/
We develop in a mixed environment - some people work on Macs and some work on Linux. This has proven to be a bit of a challenge at times, as those people who work on Linux are used to having their filesystems be case sensitive, so there's no issue committing (accidentally or otherwise) multiple files differing just by case. (e.g. FileName.ext versus filename.ext)
However, when the people on Macs go to check out the repository, having a case-insensitive filesystem means that the two files - differing only in case - overwrite each other and cause general havoc.
I know that there are various git settings to help people on case-insensitive filesystems work better with case changes (e.g. core.ignorecase), but these don't solve the issue where there's two different files in the repository, only differing by case.
I realize that the only way to fix it is to make sure the Linux people don't commit the two files differing only in case in the first place. -- Is there some setting in git which will pop up a warning or error if a user on a case-sensitive filesystem attempts to commit file(s) which would be confused with each other on a case-insensitive filesystem?
There's nothing built in (although there should be, no doubt). What you can do is provide a pre-commit hook that verifies that all names are OK and prevents the commit if not.
This hook only needs to be run on the Linux box (although making it work on Linux and Mac is easy, it's just Windows with its default impoverished toolbox that is problematic). You might want to add it to a side branch and give the Linux folks instructions on setting it up.
You may want to check branch names as well, as in git pre-commit or update hook for stopping commit with branch names having Case Insensitive match. (Interesting: the answer on this question is my own; I had forgotten it.)
First, let's write a "check for case conflict" function. This is just a matter of sorting with case-folding (so that "helloworld" and "helloWorld" are placed adjacent to each other), then using uniq -di to print any duplicate (after case-folding) strings, but no non-duplicates:
sort -f | uniq -di
If this produces any output, these are the "bad names". Let's capture the output in a temporary file and check its size, so we can print them to standard output as well:
#! /bin/sh
TF=$(mktemp)
trap "rm -f $TF" 0 1 2 3 15
checkstdin() {
sort -f | uniq -di > $TF
test -s $TF || return 0 # if $TF is empty, we are good
echo "non-unique (after case folding) names found!" 1>&2
cat $TF 1>&2
return 1
}
Now we just need to use it on files that will be committed, and perhaps on branch names as well. The former are listed with git ls-files, so:
git ls-files | checkstdin || {
echo "ERROR - file name collision, stopping commit" 1>&2
exit 1
}
You can fancy this up to use git diff-index --cached -r --name-only --diff-filter=A HEAD to check only added files, allowing existing case collisions to continue, and/or try to check things across many branches and/or commits, but that gets difficult.
Combine the above two fragments into one script (and test) and then simply copy it to an executable file named .git/hooks/pre-commit.
Checking branch names is a bit trickier. This really should happen when you create the branch name, rather than when you commit to it, and it's impossible to do a really good job on the client—it has to be done on a centralized server that has a proper global view.
Here is a way to do it on the server in a pre-receive script, in shell script rather than in Python (as in the linked answer). We still need the checkstdin function though, and you might want to do it in an update hook rather than a pre-receive hook, since you don't need to reject the entire push, just the one branch name.
NULLSHA=0000000000000000000000000000000000000000 # 40 0s
# Verify that the given branch name $1 is unique,
# even IF we fold all existing branch names' cases.
# To be used on any proposed branch creation (we won't
# look at existing branches).
check_new_branch_name() {
(echo "$1"; git for-each-ref --format='%(refname:short)' refs/heads) |
checkstdin || {
echo "ERROR: new branch name $1 is not unique after case-folding" 1>&2
exit 1 # or set overall failure status
}
}
while read oldsha newsha refname; do
... any other checks ...
case $oldsha,$refname in
$NULLSHA,refs/heads/*) check_new_branch_name ${refname#refs/heads/};;
esac
... continue with any other checks ...
done