This is related to a previous question but is more specifically about implementation.
To recap the previous question, I have a Pyinstaller app with 3rd party dependencies. As #AKX answered, the license text from those dependencies should be included in the Pyinstaller distribution.
Pyinstaller makes no attempt to handle this.
Therefore I am looking for a way to automate this process.
My project uses a virtual env, and the dependencies are located at:
\venv\Lib\site-packages
One option would be to recursivley search through this folder, and find any files that match a specific pattern (e.g LICENSE.txt). However, some of the packages do not ship with a license file, and instead just name the license in their metadata. For example, I am using openpyxl
there is a folder
\venv\Lib\site-packages\openpyxl-3.0.0-py3.7.egg-info
which contains a PKG-INFO file. This lists the license (MIT/Expat) but does specifically contain the license text. In this case would it be acceptable to include a copy of the PKG-INFO file, or some copy of the standard MIT license text?
Has anyone faced this issue and implemented a solution in an elegant way? #AKX (in the previous question linked to a tool which outputs the license information as text/json etc.. but doesn't actually attempt to move any files.
If anyone has a similar question, this is the solution I implemented, which seems to work pretty well for me:
At the top of my .spec file I added this
matches = ["LICENSE.txt","METADATA","PKG-INFO"]
lics = []
print("Find 3rd party dependency license files")
for root, dir, files in os.walk("venv\Lib\site-packages"):
for file in files:
if file in matches:
src = f"{root}/{file}"
dest = f"licenses/{os.path.basename(root)}"
lics.append((src,dest))
print(f"\tLicense file: {root}/{file}")
print(f"{len(lics)} dependency licenses found. Copying to /license folder in distribution")
which builds a list of tuples containing the source of the license or metadata, and the destination to copy it to in the distribution.
To do the actual copying, I made use of the datas keyword in the Analysis object, and simply expanded the list using *lics.
This could be adapted to include more matches or be smarter and parse the files, or something like that. It is also probably a bit wasteful and copies some files multiple times, but this seems like a minimal overhead for a tiny text file.
If anyone sees this and has suggestions or comments then I'd be happy to be further informed on this subject!
Related
I am attempting to convert parts of an Android app to iOS using Doppl, and I am getting a strange result: Doppl keeps trying to pull in android.arch.lifecycle:reactivestreams, even though I don't want it to.
Specifically, in app/build/j2objcSrcGenMain/android/arch/lifecycle/, there is a reactivestrams/ subdirectory with R.h and R.m files in it. This seems to make Xcode cranky and may explain why I had some oddities with pod install.
My app/build.gradle has compile "android.arch.lifecycle:reactivestreams:$archVer", because my activity is using LiveDataReactiveStreams.fromPublisher(). However:
The activity is not in the translatePattern (and since its code is not showing up in app/build/j2objcSrcGenMain/, I have to assume that the translatePattern is fine)
I do not have a doppl statement related to reactivestreams, because there does not appear to be a Doppl conversion of this library (nor should it be needed here)
AFAIK, nowhere else in this app am I referring to LiveDataReactiveStreams, which AFAIK is the one-and-only public class from the reactivestreams library
So, the questions:
What determines whether Doppl creates R.h and R.m files for some dependency? It's not the existence of a doppl statement, as I have doppl statements for a lot of other dependencies (RxJava, RxAndroid, Retrofit) and those do not get R.h and R.m files. It's not whether the dependency is referenced from generated code, as my repository definitely uses RxJava and Retrofit, yet there are no R files for those.
How can I figure out why Doppl generates R.h and R.m for reactivestreams?
Once I get this cleared up... do I re-run pod install, or is there some other pod command to refresh an existing pod with a new implementation?
Look into 'app/build/generated/source/r/debug' and confirm there's an R.java being created for the architecture component. It'll be under 'android/arch/lifecycle/reactivestrams'.
I think there are 2 problems here.
Problem 1
Somehow Doppl/J2objc is of the opinion that this file should be transpiled. It could be either that 'translatePattern' matches with it, or that something in the shared code is referencing it. If you can't figure out which, please post a comment and I'll try to help (or post in slack group).
Problem 2
Regardless of why that 'R.java' is being sucked into the translate step, because of how stock J2objc is configured, the code is being generated with package folders instead of creating One Big Name. That generated file should be called 'AndroidArchLifecycleReactivestramsR.h' (and AndroidArchLifecycleReactivestramsR.m). Xcode really doesn't like package folders. That's why there's a slightly custom J2ojbc being used with Doppl, so we can have files with big names instead of folders.
In cases where you intentionally use package names that match with what J2objc considers to be "system" classes, you need to provide a header mapping file to force long names. The 'androidbase' doppl library needs to add a lot of files that are in the 'android' package, which J2objc considers "system". We override those names in the mapping file.
build.gradle
https://github.com/doppllib/core-doppl/blob/master/androidbase/build.gradle#L19
mapping file
https://github.com/doppllib/core-doppl/blob/master/androidbase/src/main/java/androidbase.mappings
I screwed up.
In my dopplConfig, I have:
translatePattern {
include '**/api/**'
include '**/arch/**'
include '**/RepositoryTest.java'
}
In this case, **/arch/** not only matches my arch package, but also the arch package from the Architecture Components.
Ordinarily, this would not matter, because the Architecture Components source code is not in my project. But, R.java gets generated, due to resources, and the translatePattern includes generated source code in addition to lovingly hand-crafted source code. So, that's where my extraneous Objective-C was coming from.
Many thanks to Kevin Galligan for his assistance with this, out on the #newbiehelp Doppl Slack channel!
I realise this is a newb question but I've been racking my brains for hours.
So I want to use the STK (Synthesis Toolkit) to generate sine waves etc. I've downloaded the source files from https://ccrma.stanford.edu/software/stk/download.html
I unzipped the tar.gz using 7zip.
I opened up the demo.cpp project file (under /projects/demo/demo.cpp) and whenever I try to compile it, I receive lots of errors, all starting with "undefined reference to"
I have set the compiler search directory to include the root folder of the stk kit (unzipped as "stk-4.5.0"). My understanding is that I also have to find file for the linker, that is of the type .lib? Is that correct? I haven't been able to find a .lib file to link to.
I feel like this is a simple fix - what am I missing?
You've downloaded a source , not the compiled library which might be used by linker (.lib file) There are three solutions:
1. Compile whole src folder of your download ( see instructions on their website). Frankly, I've tried to do that recently but failed, especially under Windows although I don't have any experience in this so you can try yourself.
2. Use precompiled library available in some Linux repositories. I had success with Slacko Puppy linux. Package simply adds libstk.a (linux static library) to /usr/lib and stk headers to /usr/include/stk (notice you have to add exactly that path to compiler, since /usr/lib is not enough). This gave me best results.
3. If you use only several classes, you can copy their .cpp files directly to your project source folder. Remember to add parent classes, too (see STK site -> Classes). Again I recommend that you give the compiler path to include folder you've unzipped, but copying only selected headers should also work.
I've found this article helpful (see also A1): http://www.learncpp.com/cpp-tutorial/a3-using-libraries-with-codeblocks/
Hope I've helped
I'm working on implementing a build system using scons for a somewhat large software project. There is a directory structure which separates the code for individual libraries and programs into their own directories. With our existing make system, I can do a "make clean" in a single program directory and it will only clean the files associated with the source in that directory. If I do an "scons -c" though, it recognizes that the program depends on a slew of libraries that are in sibling (or cousin) directories and cleans all of the files for those as well. This is not what I want since I then have to rebuild all of these libraries which can take several minutes.
I have tried playing with the "NoClean()" command, but have not gotten it to work in the way I need. Given the size of the code base and complexity of the directory structure, I can't realistically have a NoClean() line for every file in every library.
Is there any way to tell scons to ignore any dependencies above the current directory when doing a clean (i.e. scons -c) ?
I'd love to have a good answer to this myself.
The only solution that I can offer for now is that you get Noclean working.
So in your library, you should have something like this
lib_objs = SharedObject(source_list)
mylib = SharedLibrary('libname', lib_objs)
So for this we want to protect the library and the sources from being cleaned.
NoClean([mylib, lib_objs])
Notice that I had to split the building of the object files from the library because I want to be able to pass them to NoClean as well.
Try using the target name when cleaning.
scons -c aTargetName
You can use the SCons Alias() function to simplify the target name and to also group several target names into one alias.
With this approach you'll have to add an alias in each appropriate subdir, which isn't necessarily a bad thing :)
Is there a kind of "include" directive in RPM spec? I couldn't find an answer by googling.
Motivation: I have a RPM spec template which the build process modifies with the version, revision and other build-specific data. This is done by sed currently. I think it would be cleaner if the spec would #include a build-specific definitions file, which would be generated by the build process, so I don't need to search and replace in the spec.
If there is no include, is there an idiomatic way to do this (quite common, I believe) task?
Sufficiently recent versions of rpmbuild certainly do support %include:
%include common.inc
Unfortunately, they aren't very smart about it -- there is no known set of directories, in which it will look for the requested files, for example. But it is there and variables are expanded, for example:
%include %{_topdir}/Common/common.inc
RPM does not support includes.
I have solved similar problems with either m4 macro processor or by just concatenating parts of spec (when the "include" was at the beginning).
If you only need to pass a few variables at build time, and not include several lines from another file, you can run
rpmbuild --define 'myvar SOMEVALUE' -bb myspec.spec
and you can use %myvar in the spec.
I faced this same issue recently. I wanted to define multiple sub-packages that were similar, but each varied just slightly (they were language-specific RPMs). I didn't want to repeat the same boiler-plate stuff for each sub-package.
Here's a generic version of what I did:
%define foo_spec() %{expand:%(cat '%{myloc}/main-foo.spec')}
%{foo_spec bar}
%{foo_spec baz}
%{foo_spec qux}
The use of %{expand} ensures that %(cat) is only executed a single time, when the macro is defined. The content of the main-foo.spec file is then three times, and each time %1 in the main-foo.spec file expands to each of bar, baz and qux, in turn, allowing me to treat it as a template. You could easily expand this to more than one parameter, if you have the need (I did not).
For the underlying issue, there maybe two additional solutions that are present in all rpm versions that I am aware of.
Subpackages
macro and rpmrc files.
Subpackages
Another alternative (and perhaps the "RPM way") is to use sub-packages. Maximum RPM also has information and examples of subpackages.
I think the question is trying to structure something like,
two spec files; say rpm_debug.spec and rpm_production.spec
both use %include common.spec
debug and production could also be client and server, etc. For the examples of redefining a variable, each subpackage can have it's own list of variables.
Limitations
The main advantage of subpackages is that only one build takes place; This may also be a disadvantage. The debug and production example may highlight this. That can be worked around using strip to create variants or compiling twice with different output; perhaps using VPATH with Gnu Make). Having to compile large packages and then only have simple variations, like with/without developer info, like headers, static libraries, etc. can make you appreciate this approach.
Macros and Rpmrc
Subpackages don't solve the problem of structural defines that you wish for an entire rootfs hierarchy, or larger collection of RPMs. We have rpmbuild --showrc for this. You can have a large amount of variables and macros defined by altering rpmrc and macros when you run rpm and rpmbuild. From the man page,
rpmrc Configuration
/usr/lib/rpm/rpmrc
/usr/lib/rpm/redhat/rpmrc
/etc/rpmrc
~/.rpmrc
Macro Configuration
/usr/lib/rpm/macros
/usr/lib/rpm/redhat/macros
/etc/rpm/macros
~/.rpmmacros
I think these two features can solve all the problems that %include can. However, %include is a familiar concept and was probably added to make rpm more full-featured and developer friendly.
Which version are you talking about? I currently have %include filename.txt in my spec file and it seems to work just like the C #include directive.
> rpmbuild --version
RPM version 4.8.1
You can include the *.inc files from the SOURCES directory (%_sourcedir):
Source1: common.inc
%include %{SOURCE1}
In this way they will go automatically into SRPMS.
I've used scripts (name your favorite) to take a template and create the spec file from that. Also, the %files tag can import a file that is created by another process, e.g. Python's bdist-rpm.
In this instance I'm using c with autoconf, but the question applies elsewhere.
I have a glade xml file that is needed at runtime, and I have to tell the application where it is. I'm using autoconf to define a variable in my code that points to the "specified prefix directory"/app-name/glade. But that only begins to work once the application is installed. What if I want to run the program before that point? Is there a standard way to determine what paths should be checked for application data?
Thanks
Thanks for the responses. To clarify, I don't need to know where the app data is installed (eg by searching in /usr,usr/local,etc etc), the configure script does that. The problem was more determining whether the app has been installed yet. I guess I'll just check in install location first, and if not then in "./src/foo.glade".
I dont think there's any standard way on how to locate such data.
I'd personally do it in a way that i'd have a list of paths and i'd locate if i can find the file from anyone of those and the list should containt the DATADIR+APPNAME defined from autoconf and CURRENTDIRECTORY+POSSIBLE_PREFIX where prefix might be some folder from your build root.
But in any case, dont forget to use those defines from autoconf for your data files, those make your software easier to package (like deb/rpm)
There is no prescription how this should be done in general, but Debian packagers usually installs the application data somewhere in /usr/share, /usr/lib, et cetera. They may also patch the software to make it read from appropriate locations. You can see the Debian policy for more information.
I can however say a few words how I do it. First, I don't expect to find the file in a single directory; I first create a list of directories that I iterate through in my wrapper around fopen(). This is the order in which I believe the file reading should be done:
current directory (obviously)
~/.program-name
$(datadir)/program-name
$(datadir) is a variable you can use in Makefile.am. Example:
AM_CPPFLAGS = $(ASSERT_FLAGS) $(DEBUG_FLAGS) $(SDLGFX_FLAGS) $(OPENGL_FLAGS) -DDESTDIRS=\"$(prefix):$(datadir)/:$(datadir)/program-name/\"
This of course depends on your output from configure and how your configure.ac looks like.
So, just make a wrapper that will iterate through the locations and get the data from those dirs. Something like a PATH variable, except you implement the iteration.
After writing this post, I noticed I need to clean up our implementation in this project, but it can serve as a nice start. Take a look at our Makefile.am for using $(datadir) and our util.cpp and util.h for a simple wrapper (yatc_fopen()). We also have yatc_find_file() in case some third-party library is doing the fopen()ing, such as SDL_image or libxml2.
If the program is installed globally:
/usr/share/app-name/glade.xml
If you want the program to work without being installed (i.e. just extract a tarball), put it in the program's directory.
I don't think there is a standard way of placing files. I build it into the program, and I don't limit it to one location.
It depends on how much customising of the config file is going to be required.
I start by constructing a list of default directories and work through them until I find an instance of glade.xml and stop looking, or not find it and exit with an error. Good candidates for the default list are /etc, /usr/share/app-name, /usr/local/etc.
If the file is designed to be customizable, before I look through the default directories, I have a list of user files and paths and work through them. If it doesn't find one of the user versions, then I look in the list of default directories. Good candidates for the user config files are ~/.glade.xml or ~/.app-name/glade.xml or ~/.app-name/.glade.xml.