Git/Jenkins/Windows (not ordered) line ending issue - linux

I'm playing with a Qt project in Windows. Actually I'm preparing it to build on Linux also. The problem is that the build.sh script (below) inflates 1 byte per line (*) somewhere in the process which makes it fail when it gets executed by the end user.
*I'm comparing the size in the developer machine against the size in the end user machine. Between these, Jenkins (running on Windows) get things from git server and packs a tar.gz file (using cygwin), which the end user gets after all.
#!/bin/bash
qmake Project.pro
make
make clean
How should I approach this issue?
May I provide more information?

To me it looks like git Jenkins is using on Windows has crlf conversion enabled - it will take Linux line endings \n and convert them to \n\r. The symptom you usually see is
/bin/bash^M: bad interpreter: No such file or directory
You should probably set core.autocrlf = input, or at least prevent automatic conversion on script, binary, archives, etc...
See here for more details: https://help.github.com/articles/dealing-with-line-endings/#platform-all. I know it suggests using core.autocrlf = true for Windows, but you are actually using Cygwin on Windows, and you are packing scripts for Cygwin so you probably want core.autocrlf = input (no conversion).

Related

There are some changes opening in the linux of a git repo. But the repo is latest in windows

My pc is a dual system with Linux Mint and Windows 11.
The git repo always has some changes in Linux.
Every time I need to git stash and git stash drop.
I don't know how to descript this situation with google.
Help pls. Thanks.
The problem is that you have different line ending settings between Linux and Windows. What you're seeing in Linux is that your files have carriage returns on the end of them, which is the Windows line ending.
The easiest way to solve this is to add this to .gitattributes:
* text=auto
Then, on an otherwise empty working tree, add that file, run git add --renormalize ., and commit. That will cause the files to be renormalized as having LF line endings in the index.
If you're going to be working across OSes, then you'd also do well to add eol=lf to the end of that line in .gitattributes so that files are always handled with LF endings. You'll also need to configure your tools on Windows to emit LF endings instead of CRLF endings. That will prevent line ending differences in the working tree across systems.

Windows Linux Subsystem - File permissions when edited outside bash [duplicate]

As the title suggests, if I paste a c file written somewhere else into the root directory of the Linux Subsystem, I can't compile it.
I did a test where I made two differently titled hello world programs: one in vi that I can get into from the bash interface, and one elsewhere. When I compiled the one made in vi, it worked fine. Trying to do so for the one made elsewhere (after pasting it into the root directory), however, resulted in this:
gcc: error: helloWorld2.c: Input/output error
gcc: fatal error: no input files
compilation terminated
Any help with this would be much appreciated.
Do not change Linux files using Windows apps and tools!
Assuming what you meant by "paste a C file written somewhere else into the root directory of the Linux subsystem" is that you pasted your file into %localappdata%\lxss, this is explicitly unsupported. Files natively created via Linux syscalls in this area have UNIX metadata, which files natively created with Windows tools don't have.
Use /mnt/c (and the like) to access your Windows files from Linux; don't try to modify Linux files from Windows.
Quoting from the Microsoft blog linked at the top of this answer (emphasis from the original):
Therefore, be sure to follow these two rules in order to avoid losing files, and/or corrupting your data:
DO store files in your Windows filesystem that you want to create/modify using Windows tools AND Linux tools
DO NOT create / modify Linux files from Windows apps, tools, scripts or consoles
You cannot copy (by default, who knows how Windows bash is set up!) files into the root directory! Your gcc error is say "no input files", so the copy has most likely failed. Copy the files to your home directory instead, for instance:
cp helloWorld2.c ~/
instead of:
cp helloWorld2.c /

git compatibilty setting on windows for using linux server over windows share

I've installed the latest version from git-scm.com on my Windows 8.1 System. My Dev Server is a Linux Server, but I'm connected to this server using a windows share.
After the git initalization, I've started to "add" files to the repositiory and received this message on the windows command line, for every file:
warning: LF will be replaced by CRLF in inc/src/tpl/default.smarty.
The file will have its original line endings in your working
directory.
While installing git I selected option 1 from this options:
Checkout Windows-style, commit Unix-style line endings
Checkout as-is, commit Unix-style line endings
Checkout as-is, commit as-is
So I'm working on windows, commiting over a windows share to a linux server, and I will also push these changes to another linux server.
Now I'm a little bit unsure about this setting because I'm working on a linux server. The only windows "Thing" is my command line and that I'm editing the files using a windows share.
I would like to know, if this is the right setting for git, or if I'm doing it wrong.
Thanks
I'm not 100% clear on your setup, but here is what is happening. The files you are adding have Unix-style line endings (i.e. they end with LF). Since you picked option 1, git is just warning you that when you check these files out on your Windows box, they will have Windows-style line endings (i.e. they will end with CRLF).
Normally this is nothing to worry about, but you mention that you "will also push these changes to another linux server." If by this you mean a git push, then no worries; the line endings will be normalized to Unix-style in the repository. However, if you mean that you copy the files from your Windows box to a Linux box, then you will eventually end up copying files with Windows-style line endings to the Linux box, which you probably don't want to do. If that is the case, you are probably better off with option 3. Note that if you edit these files on your Windows box, you may even want to go option 2, so that you will always have Unix-style line endings in the repository.

InnoSetup: "The volume for a file has been externally altered"

InnoSetup appears to be corrupting my executable when compiling the setup project.
Executing the source file works fine, but executing the file after installation produces Win32 error 1006 "The volume for a file has been externally altered".
I've tried disabling compression and setting various flags, to no avail.
Has anyone experienced this?
UPDATE
Okay there's been some twists to the situation:
At the moment, I can even manually copy a working file to the location it is installed to and get "The volume for a file...". To be clear: I uninstall the application, create the same folder and paste the files there and run.
UPDATE 2
Some more details for those that want it:
The InnoSetup script is compiled by FinalBuilder using output from msbuild, also executed by FinalBuilder, running on my machine with XP SP3. The executable is a C# .Net assembly compiled in configuration Release|AnyCPU. The file works when executed in the folder the Install Script takes it from. It produces the same behaviour on an XP Virtual Machine. The MD5 hashes of the source file and the installed file are the same.
Ok, I just received this same error. I have a config which my executable uses. I looked in my folder a million times - but finally notice the config file was zero length. I corrected the config and the error stopped occurring.
Check the simplest things first... good lucK!
ERROR_FILE_INVALID
1006 (0x3EE): The volume for a file has been externally altered so that the opened file is no longer valid.
I suspect you're having this issue after moving the files to a network share. It seems to me that what's happening is you have an open file-handle - possibly to a temporary file you are creating - and then some other process (perhaps running on a different host) is coming along and renaming or deleting that file or its' parent directory tree.
So my advice is:
Try installing to a local directory
Run after an anti-virus scan, in
safe-mode or on a different machine
to see if there isn't some
background nasty changing
volume/directory properties while
your program is running.
Make sure the program itself isn't doing anything weird with the volume or directory tree you're working with.
Never seen that before. I've got a few questions and suggestions:
- Are you signing the EXE during the compile of the setup? If so, try leaving that part out.
- WHat OS are you installing on or does it happen on all machines you've tried?
- Run the install with the /LOG="c:\install.log" option and post the log. It might show something happening during install.
- Run a byte compare or MD5 check on the source EXE and the installed EXE. Are they the same? Do they have the same version resource?

SVN Error: Can't convert string from native encoding to 'UTF-8'

I've got a post-commit hook script that performs a SVN update of a working copy when commits are made to the repository.
When users commit to the repository from their Windows machines using TortoiseSVN they get the following error:
post-commit hook failed (exit code 1) with output:
svn: Error converting entry in directory '/home/websites/devel/website/guides/Images' to UTF-8
svn: Can't convert string from native encoding to 'UTF-8':
svn: Teneriffa-S?\195?\188d.jpg
The file in question above is: Teneriffa-Süd.jpg notice the accented u. This is because the site is German and the files have been spelt in German.
When executing a update on the working copy at the Linux command-line no errors are encountered. The above error only exists when the post-commit hook is executed via a commit by a Windows SVN client.
Questions:
Why would SVN try to change the encoding of a file?
Are filenames allowed to contain chars that are outside the Windows standard ASCII ones?
Update:
It turns out that the file in question's filename correctly displays as Teneriffa-Süd.jpg when viewed from a Windows machine (via Samba) but when I view the filename from the Linux server (using SSH and PuTTY) where the file resides I get Teneriffa-Süd.jpg
Yet another example:
$ svn update
svn: Error converting entry in directory '.' to UTF-8
svn: Can't convert string from native encoding to 'UTF-8':
$ export LC_CTYPE=en_US.UTF-8
$ svn update
(... and all is fine now)
It does not change the encoding of the file. It changes the encoding of the filename (to something that every client can hopefully understand).
Allowed by whom ? NTFS uses 16-bit code points, and Windows can expose the file names in various encodings, based on how you ask for it (it will try to convert them to the encoding you ask for). Now... That bit (how you ask) depends on the specific svn client you use. It sounds to me like a bug in TortoiseSVN.
Edit to add:
Ugh. I misunderstood the symptoms. the svn server stores everything in utf-8 (and it seems that it did that successfully).
The post-commit hook is the bit that fails to convert from UTF-8. If I understand what you're saying correctly, the post-commit hook on the server triggers an svn update to a shared drive (the svn server therefore starts an svn client to itself...) ? This means that the configuration that needs to be fixed is the one for the client on the server.
Check the LANG / LC_ALL on the environment executing the svn server.. As it happens, the hooks are run in a vacuum environment (see Tip). So you should set the variable in the hook itself.
See also this page for info on how svn handles localisation
If Error is -
[abc#288832-web3 public_html]$ svn update
svn: Error converting entry in directory 'images' to UTF-8
svn: Valid UTF-8 data
(hex: 46 65 6e 65 72 62 61 68)
followed by invalid UTF-8 sequence
(hex: e7 65 2b 46)
Then do this.
[abc#288832-web3 public_html]$ printf "\x46\x65\x6e\x65\x72\x62\x61\x68\n"
Fenerbah
(This means that the system has some file name starting with "Fenerbah" in that folder.)
[abc#288832-web3 public_html]$ cd images
[abc#288832-web3 images]$ rm -rf Fenerbahçe+Forma+2.jpg
So you can see that there is a special character in the name and it is not supported by SVN.
put this in your post-commit
export LANG=xxxxx (your lang)
Just use the following line in your script before executing any svn command.
User appropriate language codes, in following example I used japanese
export LC_ALL=ja_JP.UTF8
Don't forget to generate those locales in your system
(as root)
example for Ru
locale-gen ru_RU.CP1251
locale-gen ru_RU.UTF-8
dpkg-reconfigure locales
It changes the encoding to a location-neutral encoding in case someone with a different encoding checks it out.
Of course. But it's not "Windows" ASCII (Windows actually uses some strange encoding like CP1251 or so).
The best way to fix this is to make sure that your system uses UTF-8 whenever possible (check $LANG).
It seems that all LC_ varables need .UTF8 at the end. For example, I happened to have LC_ALL, LC_TIME, and LC_CTYPE defined. After setting LC_CTYPE the problem was not solved, so I needed to type LC_ALL as well and then it worked:
LC_ALL=en_US.UTF-8
LC_TIME=en_DK.UTF-8
LC_CTYPE=en_US.UTF-8
In order to avoid the problem again, I copied the file to a different name, removed the old one from svn, added new one to svn, and send a message to a collaborator not to do this.
I got a similar problem when running "svn add" on a directory, but the solution was different. I couldn't see the "hex" digits using printf (actually no hex output was shown by svn), but this command allowed me to see the results, and fix it:
LC_ALL=C svn add probealign
I think, in general, sticking LC_ALL=C before your command allows you to see the offending files... and is a lot easier than pasting in a lot of \x72 stuff (which apparently may not be available).
For information, I got this error on commit native encoding to 'UTF-8'with a windows client tortoise svn,
when my URL of repository was :
http://x.x.x.x/svn/myrepos
I changed my URL of repository for :
svn://x.x.x.x/myrepos
and now all is perferct.
I think this information will be useful to some.
In my case, I had the setting in ~/.subversion/config as below
log-encoding = ...
Commenting it worked.

Resources