I am currently using TCOM to work on excel using TCL. I have 2 excel sheet files. What i need to do is compare the two files for differences and list them out in txt file/excel.
I would like to know whether this comparison between two excel files can be done using tcl/tcom.
If you working on a linux environment you could use some bash commands to help you out.
I think the best way to quickly process their data would be through csv files. You could use:
exec sort sheet1.csv
exec sort sheet2.csv
set diff [diff sheet1.csv sheet2.csv]
Editing since this isn't a pure Tcl:
Let's say both of your csv files looks like this:
sheet1.csv -> a,b,c,d
sheet2.csv -> a,d,c,e
You can load these files by passing them as argument to your Tcl file:
myTclFile sheet1.csv sheet2.csv
Inside your Tcl you could read them using argv:
set list1 [lindex $argv 0]
set list2 [lindex $argv 1]
It is good practice to check for input files before invoking them.
If order of the files aren't important but only the fact that they have the same data you could use lsort. Nevertheless, in order to turn this into a actual list of elements, instead of one big string use split:
set list1 [split $list1 ',']
set list2 [split $list2 ',']
Then you can iterate over these list the way you want. My suggestion would be using foreach. This would go more or less like this (exempla if you wanted to iterate over whole lists)
foreach element $list1 {
foreach element2 $list2 {
set hasMatch 0
if {$element == $element2} {
incr hasMatch
break
} else {
continue
}
}
if {!$hasMatch} {
set diff [lappend $diff $element]
} else {
continue
}
}
Related
In my Tcl/Tk project, i need to allow my users to mangle a string in a well-defined way.
The idea is, to allow people to declare a "string mangling" proc/expr/function/... in a configuration file, which then gets applied to the strings in question.
I'm a bit worried on how to properly implement that.
Possibilities I have considered so far:
regular expressions
That was my first thought, but there's two caveats:
search/replace with regular expressions in Tcl seems to be awkward. at least with regsub i need to pass the match and replacement parts separately (as opposed to how e.g. sed allows me to pass a single complicated string that does everything for me); there are sed implementations for Tcl, but they look naive and might break rather sooner than later
also regexes can be awkward by themselves; using them to mangle complicated strings is often more complicated than it should be
procs?
Since the target platform is Tcl anyhow, why not use the power of Tcl to do string mangling?
The "function" should have a single input and produce a single output, and ideally it the user should be nudged into doing it right (e.g. not being able to define a proc that requires two arguments) and it be (nigh) impossible to create side-effects (like changing the state of the application).
A simplistic approach would be to use proc mymangler s $body (with $body being the string defined by the user), but there are so many things that can go wrong:
$body assuming a different arg-name (e.g. $x instead of $s)
$body not returning anything
$body changing variables,... in the environment
expressions look more like it (always returning things, not allowing to modify the environment easily), but i cannot make them work on strings, and there's no way to pass a variable without agreeing its name.
So, the best I've come up with so far is:
set userfun {return $s} # user-defined string
proc mymangler s ${userfun}
set output [mymangler $input]
Are there better ways to achieve user-defined string-manglers in Tcl?
You can use apply -- the user provides a 2-element list: the second element is the "proc body", the code that does the mangling; the first element is the variable name to hold the string, this variable is used in the body.
For example:
set userfun {{str} {string reverse $str}}
set input "some string"
set result [apply $userfun $input] ;# => "gnirts emos"
Of course the code you get from the user is any arbitrary Tcl code. You can run it in a safe interpreter:
set userfun {{str} {exec some malicious code; return [string reverse $str]}}
try {
set interp [safe::interpCreate]
set result [$interp eval [list apply $userfun $input]]
puts "mangled string is: $result"
safe::interpDelete $interp
} on error e {
error "Error: $e"
}
results in
Error: invalid command name "exec"
Notes:
a standard Tcl command is used, apply
the user must specify the variable name used in the body.
this scheme does protect the environment:
set userfun {{str} {set ::env(SOME_VAR) "safe slave"; return $str$str}}
set env(SOME_VAR) "main"
puts $env(SOME_VAR)
try {
set interp [safe::interpCreate]
set result [$interp eval [list apply $userfun $input]]
puts "mangled string is: $result"
safe::interpDelete $interp
} on error e {
error "Error: $e"
}
puts $env(SOME_VAR)
outputs
main
mangled string is: some stringsome string
main
if the user does not return a value, then the mangled string is simply the empty string.
The "simplistic" approach is like foreach in that it requires the user to supply a variable name and a script to evaluate that uses that variable, and is a good approach. If you don't want it affecting the rest of the program, run it in a separate interpreter:
set x 0
proc mymangler {name body} {
set i [interp create -safe]
set s "some string to change"
try {
# Build the lambda used by apply here instead of making
# the user do it.
$i eval [list apply [list $name $body] $s]
} on error e {
return $e
} finally {
interp delete $i
}
}
puts [mymangler s { set x 1; string toupper $s }]
puts $x
outputs
SOME STRING TO CHANGE
0
If the person calling this says to use s as a variable and then uses something else in the body, it's on them. Same with providing a script that doesn't return anything.
I'd generally allow the user to specify a command prefix as a Tcl list (most simple command names are trivially suitable for this), which you would then apply to the argument by doing:
set mangled [{*}$commandPrefix $valueToMangle]
This lets people provide pretty much anything they want, especially as they can use apply and a lambda term to mangle things as required. Of course, if you're in a procedure then you're probably actually better off doing:
set mangled [uplevel 1 [list {*}$commandPrefix $valueToMangle]]
so that you're running in the caller's context (change 1 to #0 to use the global context instead) which can help protect your procedure against accidental changes and make using upvar within the mangler easier.
If the source of the mangling prefix is untrusted (what that means depends greatly on your application and deployment) then you can run the mangling code in a separate interpreter:
# Make the safe evaluation context; this is *expensive*
set context [interp create -safe]
# You might want to let them define extra procedures too
# interp invokehidden $context source /the/users/file.tcl
# Use the context
try {
set mangled [interp eval $context [list {*}$commandPrefix $valueToMangle]]
} on error {msg} {
# User supplied something bad; error message in $msg
}
There's various ways to support users specifying the transformation, but if you can expose the fact that you're working with Tcl to them then that's probably easiest and most flexible.
I have a shell file with the below SQL statements in it:
SELECT distinct vpi.pin_id_e
FROM MSSINT.V_DSLAMS vd,
MSSINT.v_pin_inventory_old vpi
where vd.dslam like '%#%'
and vd.dslam_id = vpi.dslam_id ;
select pa.circuit_design_id,pa.node_address,c.exchange_carrier_circuit_id,c.type,c.rate_code,c.status
from ASAP.port_address pa,
asap.circuit c
where pa.equipment_id = 4561233 and pa.circuit_design_id is not null
and pa.circuit_design_id = c.circuit_design_id;
In the above content of my shell file, I have to extract the table or view names alone (those between from and where keywords).
I have seen a lot of suggestions to get words based on position, but I don't want those since they will not work like between operators.
awk 'toupper($0) ~ /^FROM/ { getline;flag=1 } toupper($0) ~ /^WHERE/ { flag=0 }flag' filename
With awk, convert the string to upper case and then pattern match against FROM at the beginning of the line. If this exists, read in the next line and set flag to one. When WHERE is encountered at the beginning of the line, set the flag equal to 0. The complete line will then only print when flag is set to one i.e. between the from and where lines
What I want to do seems simple, but I don't know if the TCL interpreter has this functionality.
I have a tcl script that will have thousands of variables defined prior to running within its scope -- this is done by a pre-script that simply does a "global" on the thousands of variables to bring it into this current scope.
Is there an easy way to determine which of those thousands of variables were actually used during that script?
For instance, if the script has variables
a,b,c,d,e,
but only variable e was accessed (whether modified or just used), I would like to know.
You can use tcl's trace capability to keep track of variable access.
Something like:
# at the end of the pre-script:
array set var_stats {}
proc track_var {varname n1 n2 op} {
global var_stats
incr var_stats($varname.$op)
}
foreach var $list_of_varnames {
foreach op {array read write unset} {
set var_stats($var.$op) 0
trace add variable $var $op [list track_var $var]
}
}
The code above will increment the appropriate stats (array, read, write and unset) for the variables when they are accessed. At the end of the script just dump the array with either an array get or a parray.
Updated answer:
I just reread your question and realize that if you just want to know which variable is accessed then there is a simpler way to do it:
array set var_stats {}
proc track_var {varname n1 n2 op} {
global var_stats
set var_stats($varname) 1
}
foreach var $list_of_varnames {
trace add variable $var {array read write unset} [list track_var $var]
}
Then at the end of the script just do an array names to get a list of all variables accessed.
I have a manual list I created in a macro in stata, something like
global list1 "a b c d"
which I later iterate through with something like
foreach name in $list1 {
action
}
I am trying to change this to a DB driven list because the list is getting big and changing quickly, I create a new $list1 with the following commands
odbc load listitems=items, exec("SELECT items from my_table")
levelsof listitems
global list1=r(levels)
The items on each are the same, but this list seems to be different and when I have too many items it break on the for loop with the error
{ required
r(100);
Also, when I run only levelsof listitems I get the output
`"a"' `"b"' `"c"' `"d"'
Which looks a little bit different than the other macros.
I've been stuck in this for a while. Again, it only fails when the number of items becomes large (over 15), any help would be very appreciated.
Solution 1:
levelsof listitems, clean local(list1)
foreach name of local list1 {
...action with `name'...
}
Solution 2:
levelsof listitems, clean
global list1 `r(levels)'
foreach name of global list1 {
...action with `name'...
}
Explanation:
When you type
foreach name in $list1 {
then whatever is in $list1 gets substituted inline before Stata ever sees it. If global macro list1 contains a very long list of things, then Stata will see
foreach name in a b c d e .... very long list of things here ... {
It is more efficient to tell Stata that you have a list of things in a global or local macro, and that you want to loop over those things. You don't have to expand them out on the command line. That is what
foreach name of local list1 {
and
foreach name of global list1 {
are for. You can read about other capabilities of foreach in -help foreach-.
Also, you originally coded
levelsof listitems
global list1=r(levels)
and you noted that you saw
`"a"' `"b"' `"c"' ...
as a result. Those are what Stata calls "compound quoted" strings. A compound quoted string lets you effectively nest quoted things. So, you can have something like
`"This is a string with `"another quoted string"' inside it"'
You said you don't need that, so you can use the "clean" option of levelsof to not quote up the results. (See -help levelsof- for more info on this option.) Also, you were assigning the returned result of levelsof (which is in r(levels)) to a global macro afterward. It turns out -levelsof- actually has an option named -local()- where you can specify the name of a local (not global) macro to directly put the results in. Thus, you can just type
levelsof listitems, clean local(list1)
to both omit the compound quotes and to directly put the results in a local macro named list1.
Finally, if you for some reason don't want to use that local() option and want to stick with putting your list in a global macro, you should code
global list1 `r(levels)'
rather than
global list1=r(levels)
The distinction is that the latter treats r(levels) as a function and runs it through Stata's string expression parser. In Stata, strings (strings, not macros containing strings) have a limit of 244 characters. Macros containing strings on the other hand can have thousands of characters in them. So, if r(levels) had more than 244 characters in it, then
global list1=r(levels)
would end up truncating the result stored in list1 at 244 characters.
When you instead code
global list1 `r(levels)'
then the contents of r(levels) are expanded in-line before the command is executed. So, Stata sees
global list1 a b c d e ... very long list ... x y z
and everything after the macro name (list1) is copied into that macro name, no matter how long it is.
I have multiple folders. There are multiple txt files inside these folder. I need to extract data (just a single value: value --->554) from a particular type of txt file in this folder.(individual_values.txt)
No 100 Value 555 level match 0.443 top level 0.443 bottom 4343
There will be many folders with same txt file names but diff value. Can all these values be copyed to excel one below the other.
I have to extract a value from a txt file which i mentioned above. Its a same text file with same name located inside different folders. All i want to do is extract this value from all the text file and paste it in excel or txt one below the other in each row.
Eg: The above is a text file here I have to get the value of 555 and similarly from other diff values.
555
666
666
776
Yes.
(you might want to clarify your question )
Your question isn't very clear, I imagine you want to know how this can be done.
You probably need to write a script that traverses the folders, reads the individual files, parses them for the value you want, and generates a Comma Separated Values (CSV) file. CSV files can easily be imported to Excel.
There are two or three basic methods you can use to get stuff into a Excel Spreadsheet.
You can use OLE wrappers to manipulate Excel.
You can write the file in a binary form
You can use Excel's import methods to take delimited text in as a spreadsheet.
I chose the latter way, because 1) it is the simplest, and 2) your problem is so poorly stated as it does not require a more complex way. The solution below outputs a tab-delimited text file that Excel can easily support.
In Perl:
use IO::File;
my #field_names = split m|/|, 'No/Value/level match/top level/bottom';
#' # <-- catch runaway quote
my $input = IO::File->new( '<data.txt' );
die 'Could not open data.txt for input!' unless $input;
my #data_rows;
while ( my $line = <$input> ) {
my %fields = $line =~ /(level match|top level|bottom|Value|No)\s+(\d+\S*)/g;
push #data_rows, \%fields if exists $fields{Value};
}
$input->close();
my $tab_file = IO::File->new( '>data.tab' );
die 'Could not open data.tab for output!' unless $tab_file;
$tab_file->print( join( "\t", #field_names ), "\n" );
foreach my $data_ref ( #data ) {
$tab_file->print( join( "\t", #$data_ref{#field_names} ), "\n" );
}
$tab_file->close();
NOTE: Excel's text processing is really quite neat. Try opening the text below (replacing the \t with actual tabs) -- or even copying and pasting it:
1\t2\t3\t=SUM(A1:C1)
I chose c#, because i thought it would be fun to use a recursive lambda. This will create the csv file containing matches to the regex pattern.
string root_path = #"c:\Temp\test";
string match_filename = "test.txt";
Func<string,string,StringBuilder, StringBuilder> getdata = null;
getdata = (path,filename,content) => {
Directory.GetFiles(path)
.Where(f=>
Path.GetFileName(f)
.Equals(filename,StringComparison.OrdinalIgnoreCase))
.Select(f=>File.ReadAllText(f))
.Select(c=> Regex.Match(c, #"value[\s\t]*(\d+)",
RegexOptions.IgnoreCase))
.Where(m=>m.Success)
.Select(m=>m.Groups[1].Value)
.ToList()
.ForEach(m=>content.AppendLine(m));
Directory.GetDirectories(path)
.ToList()
.ForEach(d=>getdata(d,filename,content));
return content;
};
File.WriteAllText(
Path.Combine(root_path, "data.csv"),
getdata(root_path, match_filename, new StringBuilder()).ToString());
No.
just making sure you have a 50/50 chance of getting the right answer
(assuming it was a question answerable by Yes and No) hehehe
File_not_found
Gotta have all three binary states for the response.