How can I create a directory link that will survive git submodule update?
Given the following example directory structure:
proj
|-- lib3
| `-- submod
| `-- lib
`-- lib
I created a soft link from proj/lib/ to proj/lib3/submod/lib using something like the following command:
brad#bradpc:~/proj/lib$ ln -s ../lib3/submod/lib submodlib
creating the following directory structure:
proj
|-- lib3
| `-- submod
| `-- lib
`-- lib
`-- submodlib
However, running git submodule update destroys my link. I was under the impression that a soft link is a logical link to a relative path, so removing the object and re-adding it would keep the link intact. How can I achieve the desired effect?
A soft link made with ln -s should behave like you intended. As I understand it, if you do a git submodule update some part of your directory proj/lib3/submod/lib gets deleteted and recreated. That means there's no difference in that, than manually do a rm proj/lib3/submod and after that a mkdir -p proj/lib3/submod/lib for example.
I tested this manually (rm and mkdir) on my openSuse Linux installation and the soft link was still fine after recreating the directory structure.
In which OS enviroment do you work? Perhaps it's not a true softlink.
Related
I am having 3 small problems which are inter-related.
1.I recently created my own virtual environment ,I want to export that environment to my friend's system so that he can run my environment's main program in one tap.
2.Also where to put main driver python code file in venv so that it can easily be executed in other system.
3.I used open() to read a text file ,but i am not sure what must be its directory so that it can be worked on other(any) system ,i am currently storing it within my venv
What I tried:
1.It is completed so I exported it to other system and (but i am not sure which folder to select so it can be operated on other Window),I copied my_venv directly and pasted it in other system.
2.I stored it within my_venv/main.py
3.I tried open(r'.vmy_env/text.txt','r').
You cannot actually move your virtual environment from one system to your friend's system. What you can do instead is :
Create a src folder inside the virtual environment folder and keep all the code files and necessary files related to the project inside this source folder.
Use pip freeze command in order to obtain all the installation details. Store all these details inside a file like requirements.txt. You can do it either by manually copy-pasting or by using output redirection.
Now that we have the requirements and basic structure down, make a virtual environment in another (your friend's) system. Ask him to place the src folder in the exact same place as you did. Then ask him to install all the dependencies using pip (you can follow this link)
Then he should be good to go with the project execution. Another helpful link can be this which shows how to use the above mentioned steps.
A suggested folder structure can be something like this :
+
|
|---- src
| |
| |---- main.py
| |
| |---- data
| | |
| | |---- dataset1.csv
| | |---- dataset2.csv
| | +
| |
| |---- utils
| | |
| | |---- helper.py
| | |
| | +
| |
| |---- requirements.txt
| |
| +
+
Here the main.py is the driver code and all other directories will contain helper/utility functions and classes.
Some good practices while managing Project's folder structure is :
Keep all the code or data files inside the source folder (here src).
Make use of relative paths instead of absolute paths. You can make use of os module in order to do the same. Since it would eliminate the need of modifying the code every time you run it on a different machine or operating system.
Never copy the venv folder. It's only the src folder we need.
Using version control system is a big plus when it comes to effective project management and collaboration. So try looking into git
If you could share your current folder structure then I could help you out more precisely.
Im looking for a tool that would convert a given directory into a text based directory in form of:
im working with macOS, maybe there is a browser based tool for this?
./directory
|
+-- subdirectory1/
| |
| + fileA.md
|
+-- subdirectory2/
|
+ fileA.md
+ ...
There is a Unix tool that will do this called tree. This tool will output the directory tree structure of a given folder. It is a command line tool which means that you will have to use the terminal to get your results. Typing tree -d ~ will, for example, output the tree structure of you home directory.
Although it is not included by default on MacOS you can install it yourself. You can download and compile the source on their homepage (link) or use a package manager like HomeBrew to install it (brew install tree).
I'm struggling how to setup streams for the following scenario:
I have a library project (//libX) using typical mainline, release and dev streams.
However, I want to have dev streams for separate products (//libX/projectA) that use this library. These products have different directory structures, and I want to map //libX/main/... to a subfolder //libX/projectA/extern/libX/....
For example, my lib is structured like this:
//libX/main
/bin
/src
/tests
readme.txt
And my projects are something else altogether, but use my lib
//libX/projectA
/documentation
/extern
/libX
/bin
/src
/tests
readme.txt
/MaxSDK
/source
/tools
config.xml
The closest I've had it to work was like this:
Paths: share ...
Remaps: ... extern/libX/...
But the remaps only seem to fix the file locations on the local machine. Using the above remap settings, the libX files end up at the same root as projectA files.
Can the above scenario work with streams or I should go back to branch specs?
Thanks!
Your project shouldn't be a child stream of your lib stream -- putting it in its own stream depot seems less confusing:
//libX/main
/bin
/src
/tests
readme.txt
//libX/projectA (child of //libX/main)
/bin
/src
/tests
readme.txt
//projectA/main
/documentation
/extern
/libX (mirror of //libX/projectA)
/bin
/src
/tests
readme.txt
/MaxSDK
/source
/tools
config.xml
And you'd get this structure by doing:
Stream: //projectA/main
Paths:
share ...
import extern/libX/... //libX/projectA/...
Unfortunately there are some limitations with this approach -- if your libX Paths aren't a trivial share ... then the import won't pick it up correctly since the import path depotPath syntax imports a depot path, not a stream path. With a normal import you also can't make changes to libX/projectA from within this stream -- you can use import+ to permit this, but I've seen enough problems with import+ that my inclination would be to make this my workflow when changing the library:
p4 switch //libX/projectA
(make changes)
p4 submit
p4 switch //projectA/main
although this assumes that the library is modular enough (with its own unit tests that cover your project's use case etc) that you can do that work in it independently.
I am a user of a docker iamge on a server. On this server the caffe framework is installed on the "shared" side so that every docker image is able to access this framework without needing it to install themselves.
However, I want to make changes in one python script (e.g the draw_net.py) which is located in a python folder. I now want to copy the files to my own workspace and continue working with them, and only change the draw_net.py but make this file use the compiled data (otherwise I would have to compile it my own, which is a little painful). However since I do not own these files I am not able to change them, therefore I wanted to copy one file into my container and change it, but the dependencies must be still kept up, this is where I run into errors. I do not want to recompile anything on my image, I just want to change one python script a little.
The structure looks like the following:
bin/
dev/
home/
lib64/
mnt/
proc/
run/
srv/
tmp/
var/
boot/
etc/
lib/
media/
opt/
caffe/
build/
lib/
_caffe.so
data/
docker/
examples/
matlab/
python/
caffe/
__init__.py
__init__.pyc
pycaffe.py
pycaffe.pyc
draw_net.py
src/
cmake/
distribute/
docs/
include/
models/
scripts/
tools/
root/
workspace/
myfolder/
sbin/
sys/
usr/
Now I want the files from /opt/caffe/python/ , especially the draw_net.py in my own docker image are, so in /root/workspace/myfolder.
A simple cp command resulted in the following errors
cp: cannot create symbolic link 'cpcaffe/build' Operation not
supported
cp: cannot create symbolic link 'cpcaffe/python/caffe/_caffe.so': Operation not supported
Executing gives me the expected reseult that it cannot located the moduel _caffe , which is located in /opt/caffe/build/lib/_caffe.so
I tried to add the line
import os
os.chdir('/opt/caffe/python');
at the beginning of the script but this did not change anything.
I guess that I somehow to make a symbolic link by myself to the original version, but I am a little newbish around this topic and could need some pointers.
I have root access to all folders under root/ , all other folders are shared folders where I only have read rights.
I am new to git and I keep all my git repos in one folder inside an NTFS disk so that both Windows and Linux partitions can see it. The problem is that executable files in that folder can't be given execution permissions in Linux.
For the repos that contain executable files, I've been working in a directory of the Linux partition, so that I can execute them, and then once I'm done editing them, I copy them back to the relevant repo folder to keep version-controlling them.
Side note (for GolezTrol):
When I say executables, I mean, for example, *.cpp or *.java files. If I want to compile those on an NTFS partition, I will get an error if I try to run the executable. So I can't work in the NTFS folder, I have to work in the Linux partition. Think for example, the workspace folder in Eclipse. I can't make any app work if I place it in the NTFS folder. So I have to move it to the Linux partition, edit my files, and then copy the relevant files back to the repo folder in the NTFS partition. I'm not tracking the executables, I'm tracking the files that produce those executables. I hope this clears it up.
I was wondering if there is a way to keep the version-control of those files in their repo folder, while the files live in the Linux partition, so that I don't have to copy them back when I'm finished editing, and I can still see them from Windows. Something like a link to the executables folder that lives in the repos folder, maybe (it would probably be nice to be able to edit them from Windows too, as if they really lived in the repo folder, but I guess that won't be possible).
I searched SO and found a similar question. I tried the first answer, although it's not exactly what I'm aiming for, but it didn't work for me. I placed the .git file in my executables folder, containing:
gitdir: path/to/repo/in/NTFS/disk/.git
and then initialized the repo, but I got an error:
$ git init
fatal: Not a git repository: path/to/repo/in/NTFS/disk/.git
So I went back to the repo folder (in the NTFS disk), did git init and came back to the executables folder. Now I get a different error:
$ git init
fatal: Invalid gitfile format: path/to/executables/folder
I think I didn't understand the answer.
Regarding the second answer, I couldn't understand how to implement it either.
So, wrapping up, my question is... how can I have the executable files in a folder in the Linux partition but version control them from the repo folder in the NTFS disk? Would it be possible to make Windows see those files?
P.S. I would like to avoid (if possible) getting into mounting or editing fstab files.
P.S2. This question is different to the one I linked. I'm thinking more of a sort of link to the executables folder from the repo folder. It would be ideal to see the files from Windows too. If that is not possible, I'm open to other answers, since the answers in the linked question didn't work for me. Had those answers solved my problem, I wouldn't be asking this question.
Update:
My executables directory:
$ tree -L 1 -aF --dirsfirst
.
├── file.txt
└── .git
0 directories, 2 files
Contents of file.txt: file
Contents of .git: gitdir: /media/admin/DATA/github/af-62/.git
My NTFS directory (that is, /media/admin/DATA/github/af-62):
$ tree -L 1 -aF --dirsfirst
.
├── .git/
├── .gitignore
└── LICENSE.md
1 directory, 2 files
(Please don't pay too much attention to the files, they're placeholder files for now.)
The -a option shows hidden files, as those whose name starts with a dot, and the --dirsfirst shows directories first, so in the first case, .git is at the end while in the second .git is at the beginning.
In the NTFS directory:
$ git log
commit f8f7e3bac01f51e5d819a31e28c1a42c181b0407
Author: private <user#example.com>
Date: Wed Jul 15 11:50:44 2015 +0200
First commit.
$ git checkout /path/to/executables/folder/file.txt
fatal: /path/to/executables/folder/file.txt: '/path/to/executables/folder/file.txt' is outside repository
In the executables directory:
$ git init
fatal: Invalid gitfile format: /path/to/executables/folder/.git
$ git checkout file.txt
fatal: Invalid gitfile format: .git
$ git checkout /media/admin/DATA/github/af-62/LICENSE.md
fatal: Invalid gitfile format: .git
According to jvdm's linked resource, "[...the .git file...] It must point to a valid Git bare repository", but /media/admin/DATA/github/af-62/ is a valid repository, isn't it? What am I doing wrong?
You need to tell git that /path/to/executables/folder is the working
dir of the git repository /path/to/repo/in/NTFS/disk/.git, one way
to do it:
cd /path/to/executables/folder
echo gitdir: /path/to/repo/in/NTFS/disk/.git > .git
Now notice that a git status (if /path/to/executables/folder was
empty) will show all your files as deleted. Then you can:
git checkout .
To checkout them into your working dir.