How can I extract static libs containing repeated object files? - linux

I'm trying to build a big static library merging two static libraries. In moment I'm using the 'ar' command, extracting objects, for example, from 'a.a' and 'b.a' and then reassembling these objects using 'ar' again:
$ ar x a.a
$ ar x b.a
$ ar r merged.a *.o
Unfortunately it isn't working for my purpose, since a.a has inside different objects with the SAME NAME. The 'ar' command is extracting the repeated objects and replacing the already extracted ones with the same name. Even with the same name, these objects have different symbols, so I get undefined references since some symbols are being missed together with the replaced files.
I have no access to the original objects and already tried 'ar xP' and 'ar xv' and lots of 'ar stuff'. Does anyone can help me showing how to merge these libs?
Thanks in advance.

I tried 'ar p', but talking to a friend it was decided the following python solution could be better. Now it's possible to extract the repeated object files.
def extract_archive(pathtoarchive, destfolder) :
archive = open(pathtoarchive, 'rb')
global_header = archive.read(8)
if global_header != '!<arch>\n' :
print "Oops!, " + pathtoarchive + " seems not to be an archive file!"
exit()
if destfolder[-1] != '/' :
destfolder = destfolder + '/'
print 'Trying to extract object files from ' + pathtoarchive
# We don't need the first and second chunk
# they're just symbol and name tables
content_descriptor = archive.readline()
chunk_size = int(content_descriptor[48:57])
archive.read(chunk_size)
content_descriptor = archive.readline()
chunk_size = int(content_descriptor[48:57])
archive.read(chunk_size)
unique_key = 0;
while True :
content_descriptor = archive.readline()
if len(content_descriptor) < 60 :
break
chunk_size = int(content_descriptor[48:57])
output_obj = open(destfolder + pathtoarchive.split('/')[-1] + '.' + str(unique_key) + '.o', 'wb')
output_obj.write(archive.read(chunk_size))
if chunk_size%2 == 1 :
archive.read(1)
output_obj.close()
unique_key = unique_key + 1
archive.close()
print 'Object files extracted to ' + destfolder + '.'

You have to extract the objects from the static library (the library which contains the duplicated objects)
Then you have to build a new library from the extracted objects.
The new library will contain ONLY ONE instance of the duplicated objects.
You have to use the ar t command to produce the lists of the objects from the two libraries (the original-with duplicates one and the new one - without duplicates).
Then use e.g vimdiff to check the differences between the two list.
Write down all the differences.
Then extract only those objects (step 6) objects from the original library, using the command ar x my_original_lib.a object.o
Then rename the produced extracted object to any name you like
Then use the command ar m my_original_lib.a object.o to rearrange the object.o
Then use the same command, as step 7, and you will extract the second object.o
Give a different name to the newly extracted object
Use both of them to build the new library.
The method holds for ANY number of duplicated objects in the static library. Just use the step 9 and 7 repeatedly to extract all the dublicates

You can rename objects -- their name does not mean anything during linking. This should work:
mkdir merge-objs &&
cd merge-objs &&
ar x ../a.a &&
for j in *.o; do mv $j a-$j; done &&
ar x ../b.a &&
ar r ../merged.a *.o &&
cd .. && rm -rf merge-objs

A C++ code which merges many libraries into a single new one, without overwriting possible duplicated objects is here: http://bazaar.launchpad.net/~paulsepolia/+junk/arbet/files/head:/0025_arbet_FINAL/

Related

How do I make the output shapefile from a query be named by its corresponding value in the loop using Python?

aprx = arcpy.mp.ArcGISProject("CURRENT")
m = aprx.listMaps("Map")[0]
for lyr in m.listLayers("POP_ACS17"):
if lyr.supports("DEFINITIONQUERY"):
for value in listnumbers.split(','):
lyr.definitionQuery = "COLUMN=" + num
output_path=r'C:/ArcGIS/Projects/Value_by_Value/'+ num
arcpy.FeatureClassToShapefile_conversion('POP_ACS17',output_path)
How do i create a shapefile name in the folder based on the query result, and not the same name "POP_ACS17" multiple times? Instead something like num + "_POPACS17" and so on. When i type
arcpy.FeatureClasstoShapefile_conversion('POP_ACS17', num + "_"+ outputh_path)
it does not work. i get an error message
ERROR 000732: Output Folder: Dataset C:/ArcGIS/Projects/Value_by_Value/1.0/1.0_PCT_POP_ACS17 does not exist or is not supported
All I want to do is just have my output renamed by the corresponding value in the loop underscore and the default name that it already gives which is 'POP_ACS17'
so i want
1.0_POP_ACS17.shp in folder 1.0
2.0_POP_ACS17.shp in folder 2.0
3.0_POP_ACS17.shp in folder 3.0
if not then
POP_ACS17_1.0 in folder 1.0
POP_ACS17_2.0 in folder 2.0
POP_ACS17_3.0 in folder 3.0
and so on... It does not matter which order as long as I know how this can be done.
Instead of
arcpy.FeatureClassToShapefile_conversion('PCT_POP_ACS17', output_path)
do.....
arcpy.FeatureClassToFeatureClass_conversion("_PCT_POP_ACS17", output_path, value+"_PCT_POP_ACS17.shp")
make sure you add .shp otherwise each output will appear in your project file.

History of previously opened m-files in MATLAB

Is anyway to find history of previously opened m-files in MATLAB R2014b from 2 or 3 months ago? (a list of name of files and paths)
Matlab R2014b stores its recent files in:
%APPDATA%\MathWorks\MATLAB\R2014b\MATLAB_Editor_State.xml
It's a .xml file so it's easy to load and parse with xmlread. I'm not very familiar with xml parsing syntax, but here is how to get information about files (to be adapted to your needs of course):
function [recentFiles] = GetRecentFiles()
%[
% Opens editor's state file
filepart = sprintf('MathWorks\\MATLAB\\R%s\\%s', version('-release'), 'MATLAB_Editor_State.xml');
filename = fullfile(getenv('APPDATA'), filepart);
document = xmlread(filename);
% Get information about 'File' nodes
recentFiles = struct([]);
fileNodes = document.getElementsByTagName('File');
for fni = 1:(fileNodes.getLength())
attributes = fileNodes.item(fni-1).getAttributes(); % Careful, zero based indexing !
for ai = 1:(attributes.getLength())
% Get node attribute
name = char(attributes.item(ai-1).getName()); % Zero based + need marshaling COM 'string' type
value = char(attributes.item(ai-1).getValue()); % Zero based + need marshaling COM 'string' type
% Save in structure
name(1) = upper(name(1)); % Just because I prefer capital letter for field names ...
recentFiles(fni).(name) = value;
end
end
%]
end
This returns a structure like this:
recentFiles =
1x43 struct array with fields:
AbsPath
LastWrittenTime
Name
NB: I've tried to type in matlab command window matlab.desktop.editor.*, but seems there's nothing regarding recent files (anyway there are a lot of interesting things to manipulate the editor from the command line)
Last answer waIs really helpful. I've just modified it to read and open the recent tab files. This works on Matlab R2013a:
function [recentFiles] = recover_tabs()
%[
% Opens editor's state file
filepart = sprintf('MathWorks\\MATLAB\\R%s\\%s', version('-release'), 'MATLAB_Editor_State.xml');
filename = fullfile(getenv('APPDATA'), filepart);
document = xmlread(filename);
% Get information about 'File' nodes
recentFiles = struct([]);
fileNodes = document.getElementsByTagName('File');
for fni = 1:(fileNodes.getLength())
attributes = fileNodes.item(fni-1).getAttributes(); % Careful, zero based indexing !
for ai = 1:(attributes.getLength())
% Get node attribute
name = char(attributes.item(ai-1).getName()); % Zero based + need marshaling COM 'string' type
value = char(attributes.item(ai-1).getValue()); % Zero based + need marshaling COM 'string' type
% Save in structure
name(1) = upper(name(1)); % Just because I prefer capital letter for field names ...
recentFiles(fni).(name) = value;
end
end
% loop to access files in the tab history
for j=1:length(recentFiles)
arquivo = [recentFiles(j).AbsPath '\' recentFiles(j).Name];
% if exists, then open
if exist(arquivo, 'file') == 2
open(arquivo);
end
end
%]
end
Base in the answer by CitizenInsane, but for any Matlab version.
To find the .xml file in any Matlab version, use prefdir:
>> prefdir
ans = '/Users/user/Library/Application Support/MathWorks/MATLAB/R2018a'
MATLAB_Editor_State.xml will be stored there. Therefore the fuction would be:
function [recentFiles] = GetRecentFiles()
% Opens editor's state file
filepart = sprintf([ prefdir '/MATLAB_Editor_State.xml']);
filename = fullfile(getenv('APPDATA'), filepart);
document = xmlread(filename);
% Get information about 'File' nodes
recentFiles = struct([]);
fileNodes = document.getElementsByTagName('File');
for fni = 1:(fileNodes.getLength())
attributes = fileNodes.item(fni-1).getAttributes(); % Careful, zero based indexing !
for ai = 1:(attributes.getLength())
% Get node attribute
name = char(attributes.item(ai-1).getName()); % Zero based + need marshaling COM 'string' type
value = char(attributes.item(ai-1).getValue()); % Zero based + need marshaling COM 'string' type
% Save in structure
name(1) = upper(name(1)); % Just because I prefer capital letter for field names ...
recentFiles(fni).(name) = value;
end
end
In R2018b you can increase the Most recently used file list in Preferences > Editor/Debugger. I like the methods above, but they do not work if you're working across machines (e.g., using Github). I coded a solution that uses the modified file date from the machine, instead of relying on MATLAB itself.

Error in using os.path.walk() correctly

So I created this Folder C:\TempFiles to test run the following code snippet
Inside this folder i had two files -> nd1.txt, nd2.txt and a folder C:\TempFiles\Temp2, inside which i had only one file nd3.txt
Now when i execute this code:-
import os,file,storage
database = file.dictionary()
tools = storage.misc()
lui = -1 # last used file index
fileIndex = 1
def sendWord(wrd, findex): # where findex is the file index
global lui
if findex!=lui:
tools.refreshRecentList()
lui = findex
if tools.mustIgnore(wrd)==0 and tools.toRecentList(wrd)==1:
database.addWord(wrd,findex) # else there's no point adding the word to the database, because its either trivial, or has recently been added
def showPostingsList():
print("\nPOSTING's LIST")
database.display()
def parseFile(nfile, findex):
for line in nfile:
pl = line.split()
for word in pl:
sendWord(word.lower(),findex)
def parseDirectory(dirname):
global fileIndex
for root,dirs,files in os.walk(dirname):
for name in dirs:
parseDirectory(os.path.join(root,name))
for filename in files:
nf = open(os.path.join(root,filename),'r')
parseFile(nf,fileIndex)
print(" --> "+ nf.name)
fileIndex+=1
nf.close()
def main():
dirname = input("Enter the base directory :-\n")
print("\nParsing Files...")
parseDirectory(dirname)
print("\nPostings List has Been successfully created.\n",database.entries()," word(s) sent to database")
choice = ""
while choice!='y' and choice!='n':
choice = str(input("View List?\n(Y)es\n(N)o\n -> ")).lower()
if choice!='y' and choice!='n':
print("Invalid Entry. Re-enter\n")
if choice=='y':
showPostingsList()
main()
Now I should Traverse the three files only once each, and i put a print(filename) to test that, but apparently I am traversing the inside folder twice:-
Enter the base directory :-
C:\TempFiles
Parsing Files...
--> C:\TempFiles\Temp2\nd3.txt
--> C:\TempFiles\nd1.txt
--> C:\TempFiles\nd2.txt
--> C:\TempFiles\Temp2\nd3.txt
Postings List has Been successfully created.
34 word(s) sent to database
View List?
(Y)es
(N)o
-> n
Can Anyone tell me how to modify the os.path.walk() as such to avoid the error
Its not that my output is incorrect, but its traversing over one entire folder twice, and that's not very efficient.
Your issue isn't specific to Python 3, it's how os.walk() works - iterating already does the recursion to subfolders, so you can take out your recursive call:
def parseDirectory(dirname):
global fileIndex
for root,dirs,files in os.walk(dirname):
for filename in files:
nf = open(os.path.join(root,filename),'r')
parseFile(nf,fileIndex)
print(" --> "+ nf.name)
fileIndex+=1
nf.close()
By calling parseDirectory() for the dirs, you were starting another, independant walk of your only subfolder.

SCONS: How do I carry on an action on a target in place

Let's say I want to strip all the debug symbols in the shared libraries that I build whiling keeping the original file name.
I tried to add an command in the method:
def mySharedLibrary(self, *args, **kwargs):
# do some common work for every shared library like add a soname or append some lib files to LIBS parameter
target = SharedLibary(*args, **kwargs)
target = env.Command(target,target, "objcopy --strip-debug ${SOURCE}")
return target
I get this error: two different method was given to the same target,
I guess it's because the two targets returned by env.Command and SharedLibrary are exactly the same name.
Any ideas to do this?
Thanks in advance!
I had the same problem and got the same error. What I had to do was to create an intermediate target/library. The intermediate and final targets each had their own library name, so SCons doesnt get confused.
You could probably do something like the following:
env.SharedLibrary(target = 'namePreStrip', source = 'yourSource')
env.Command(target = 'name', source = 'namePreStrip', 'objcopy...')
I used objcopy to build a library out of several libraries. Here's the actual source code I implemented:
#
# Build an object file out of several other source files, objects, and libraries
# Optionally execute objcopy on the resulting library, depending if objcopyFlags
# has been populated
#
# env - SCons Environment used to build, Mandatory arg
# target - resulting library name, without LIBPREFIX and LIBSUFFIX, ej 'nsp2p',
# Mandatory arg
# sourceFiles - list of '.cc' files that will be compiled and included in the
# resulting lib, Optional arg
# objects - list of already compiled object files to be included in resulting lib,
# Optional arg
# libraries - list of libraries to be included in resulting lib, Optional arg
# objcopyFlags - list of flags to pass to objcopy command. objcopy will only
# be executed if this list is populated, Optional arg
#
# One of [sourceFiles, objects, or libraries] must be specified, else nothing
# will be performed
#
# Not using a custom builder because I dont like the way SCons prints the
# entire command each time its called, even if its not going to actually
# build anything AND I need more method args than provided by custom builders
#
def buildWholeArchive(self, env, target, sourceFiles, objects, libraries, objcopyFlags):
if len(sourceFiles) == 0 and len(objects) == 0 and len(libraries) == 0:
print "Incorrect use of buildWholeArchive, at least one of [sourceFiles | objects | librarires] must be specified, no build action will be performed"
return None
# Compile each source file
objNodes = []
if len(sourceFiles) > 0:
objNodes = env.Object(source = sourceFiles)
cmdList = []
cmdList.append(env['CXX'])
cmdList.append('-nostdlib -r -o $TARGET -Wl,--whole-archive')
for obj in objNodes:
cmdList.append(env.File(obj).abspath)
for obj in objects:
cmdList.append(env.File(obj).abspath)
for lib in libraries:
cmdList.append(lib)
cmdList.append('-Wl,--no-whole-archive')
cmd = ' '.join(cmdList)
libTarget = '%s%s%s' % (env['LIBPREFIX'], target, env['LIBSUFFIX'])
if len(objcopyFlags) > 0:
# First create the library, then run objcopy on it
objTarget = '%s%s_preObjcopy%s' % (env['LIBPREFIX'], target, env['LIBSUFFIX'])
preObjcopyTarget = env.Command(target = objTarget, source = [], action = cmd)
env.Depends(preObjcopyTarget, [objNodes, sourceFiles, objects, libraries])
objCmdList = [env['OBJCOPY']]
objCmdList.extend(objcopyFlags)
objCmdList.append('$SOURCE $TARGET')
objcopyCmd = ' '.join(objCmdList)
archiveTarget = env.Command(target = libTarget, source = preObjcopyTarget, action = objcopyCmd)
else:
# Just create the library
archiveTarget = env.Command(target = libTarget, source = [], action = cmd)
env.Depends(archiveTarget, [objNodes, sourceFiles, objects, libraries])
return archiveTarget
And here is how I called it:
sourceFiles = ['file1.cc', 'file2.cc']
libSource = []
if 'OcteonArchitecture' in env:
libSource.append(lib1)
libSource.append(lib2)
libSource.append(lib3)
objcopy = []
if 'OcteonArchitecture' in env:
objcopy.extend([
'--redefine-sym calloc=ns_calloc',
'--redefine-sym free=ns_free',
'--redefine-sym malloc=ns_malloc',
'--redefine-sym realloc=ns_realloc'])
archiveTarget = clonedEnv.buildWholeArchive(target = libName,
sourceFiles = sourceFiles,
objects = [],
libraries = libSource,
objcopyFlags = objcopy)
env.Alias('libMyLib', archiveTarget)

How to put multiple documents into Berkeley-DB XML container?

I have a directory with a bunch of XML documents and want to put all of them into a container.
In other words, I need to do something like this:
dbxml> putDocument tests/*.xml
I have written a GUI program to do that but the host server does not have X-windows installed, so must be in command line.
I do a similar thing when reloading certain XML docs into my current application DB. It helps if all of the files sharing a common naming convention. In python you would could use the following script to add doc001.xml to doc009.xml:
from bsddb3.db import *
from dbxml import *
#Load source files 001 - 009
sourceDir = 'C:/directory-containing-xml-docs'
fileRange = range(1,10)
for x in fileRange:
mycontainer = mymgr.openContainer("myDB.dbxml")
xmlucontext = mymgr.createUpdateContext()
xmlinput = mymgr.createLocalFileInputStream(sourceDir + "doc00" + str(x) + ".xml")
mycontainer.putDocument("doc00" + str(x) + ".xml", xmlinput, xmlucontext)
print 'Added: ' + str(x)
del mycontainer
print '1 - 9 Added'
Hope that helps
You could have a shell script write the list of XML files to another file and then call dbxml_load_container with the -f option.
Ended up using a script that lists files and puts everything into the DB.

Resources