I am looking for help with my REXX script. Which should open an existing Member and search for a specific string.
Here is my script:
/* REXX */
"ALLOC FILE(input) DA('.....(MEMBER)') SHR REUSE"
"EXECIO * DISKR "input" (STEM input. FINIS"
"FREE FILE(input)"
/* Parmlib werden ausgelesen */
do i =1 to input.0
if POS('met,', input.i) > 0 Then
/* Code if string is found */
say Zeile gefunden
else
/* Code if string is not found */
say Zeile nicht gefunden
end
As cschneid stated this looks like something you would just use SuperC for. ISPF Option 3.14 or 3.15 will search for a string and show results. Also you can issue SRCHFOR from a member list and have only the found members then filtered in the member list. Additionally ISPF LM services can be used to go thru members of a PDS and then run an Edit macro to do the find. EXEXIO could be used to write the results to an output file. Note that SuperC will already do this using 3.15
Under ISPF, edit macros seem like a good fit. You can set up an ISPF stack if you're not running one already, works in batch too.
If I'm reading your requirement correctly, maybe something like this might work:
/* REXX-ISPF V/E macro */
Address ISREDIT
"MACRO (needle,dest)"
"CURSOR = 1 0"
lastHit = 0
i = 0
"SEEK "needle
Do While RC=0
"(l#) = CURSOR"
/* Conditional for multiple hits on same line */
If l# > lastHit Then Do /* do */
"(this) = LINE "l#
i=i+1; out.i = this
lastHit = l#
End
"SEEK "needle
End
out.0=i
Address TSO
"ALLOCATE F(OUT) DA("dest"') OLD"
"EXECIO "out.0" DISKW OUT (FINIS STEM out."
Exit 0
You can do this with much fewer lines with more ISPF services in a Macro (X ALL -> F ALL needle -> DEL ALL X -> CREATE dest). Or through intermittent use of ISPF E clipboard. That has some risks, so not going into that.
Good thing about ISPF E/V Macros is that thy use almost the same command you'd normally use in ISPF E/V. Find is quick. It needs to fit the whole dataset in the Region which might be an issue sometimes.
You could also call ISRSUPC from Rexx. There was a nifty Rexx exec published in MVS Update long ago. It searches for a string across all members of a PDS and presents the "hit list" on an ISPF panel, so you can edit or view the members.
I still use it (a descendant of it anyway) on my systems. I found the MVS Update article, here it is :
https://manualzz.com/doc/10913425/mvs0207
Related
I have had to look up hundreds (if not thousands) of free-text answers on google, making notes in Excel along the way and inserting SAS-code around the answers as a last step.
The output looks like this:
This output contains an unnecessary number of blank spaces, which seems to confuse SAS's search to the point where the observations can't be properly located.
It works if I manually erase superflous spaces, but that will probably take hours. Is there an automated fix for this, either in SAS or in excel?
I tried using the STRIP-function, to no avail:
else if R_res_ort_txt=strip(" arild ") and R_kom_lan=strip(" skåne ") then R_kommun=strip(" Höganäs " );
If you want to generate a string like:
if R_res_ort_txt="arild" and R_kom_lan="skåne" then R_kommun="Höganäs";
from three variables, let's call them A B C, then just use code like:
string=catx(' ','if R_res_ort_txt=',quote(trim(A))
,'and R_kom_lan=',quote(trim(B))
,'then R_kommun=',quote(trim(C)),';') ;
Or if you are just writing that string to a file just use this PUT statement syntax.
put 'if R_res_ort_txt=' A :$quote. 'and R_kom_lan=' B :$quote.
'then R_kommun=' C :$quote. ';' ;
A saner solution would be to continue using the free-text answers as data and perform your matching criteria for transformations with a left join.
proc import out=answers datafile='my-free-text-answers.xlsx';
data have;
attrib R_res_ort_txt R_kom_lan length=$100;
input R_res_ort_txt ...;
datalines4;
... whatever all those transforms will be performed on...
;;;;
proc sql;
create table want as
select
have.* ,
answers.R_kommun_answer as R_kommun
from
have
left join
answers
on
have.R_res_ort_txt = answers.res_ort_answer
& have.R_kom_lan = abswers.kom_lan_answer
;
I solved this by adding quotes in excel using the flash fill function:
https://www.youtube.com/watch?v=nE65QeDoepc
I'm trying to write a script that would query specific resource profiles in a RACF class and later do a bit of logic to match a few things - not relevant.
The problem is that when I issue the command below I get the AUDIT TRAIL on the terminal. The script is meant to just return a 1 or a 0. All the logic works as it should but when I run the script I get the whole AUDIT TRAIL from RACF and at the bottom the result.
y = outtrap('resourceAccess.')
address tso 'RLIST CLASSX CLASSX.RESOURCE.LIST'
y = outtrap('off')
I already tried to create another outtrap after the one above with no success.
Is there a way to remove that AUDIT TRAIL bit?
It's possible that those lines of text are being issued in such a way that they cannot be trapped using outtrap and are instead being placed on the external data queue (EDQ) and then echoed to the terminal when the REXX exits. ACF2 does this with all output, making trapping command responses a bit tricky.
Try this:
/* Trap command response*/
y = outtrap('temp.')
address tso 'RLIST CLASSX CLASSX.RESOURCE.LIST'
y = outtrap('off')
/* Display anything put onto the EDQ */
do queued()
pull line
say line
end
Old answer:
If the output you are getting matches what's in the IBM docs you linked to (https://www.ibm.com/docs/en/szs/2.2?topic=effects-command-audit-trail), then what you need to do is after to have trapped the output, simply discard the first 2 lines, (which should be):
Command Audit Trail for USER IBMUSER
(one line of text and a blank line).
You could do this as follows:
y = outtrap('temp.')
address tso 'RLIST CLASSX CLASSX.RESOURCE.LIST'
y = outtrap('off')
/* Copy from the 3rd command response line into our 'real' response var */
do tempIndex = 3 to temp.0
desiredIndex = tempIndex - 2
resourceAccess.desiredIndex = temp.tempIndex
end
resourceAccess.0 = temp.0 - 2 /* Set number of lines */
How do I use a map on every match found after a search?
I have created various functions which I invoke using a map.
I would like to use the maps on every search matches found.
If I search for dates in my text, how would I apply a i/v/nmap on every search-match found?
something like this?
%s/search-pattern/=\normal mode map/g
%s/search-pattern/=\insert mode map/g
Is it possible also to combine maps?
Hope I made myself clear.
Vim is quite powerful, and I suspect insert mode/normal mode maps are not the most convenient approach here.
Some idioms that may get you started:
Edit: I've built on your earlier question (
How do I visual select a calculation backwards?
) and provided a demo, explained in
chat
1. Record a macro:
qqniMyText<Esc>q
This will insert 'MyText' at each match position. Now, repeat a hundred times: 100#q
(consider setting :se nowrapscan to avoid restarting from the top).
2. Use :global
:g/somepattern/norm! Aappended<Esc>
will append the text 'appended' to each line containing the search pattern
3. Use smart substitutions:
You can do some 'static' edit actions using replacement patterns:
:%s/\v(\d\d)-(\d\d)-(\d{4})/\3\2\1/g
To transform dd-mm-yyyy into yyyymmdd date stamps.
To do a dynamically evaluated substitution (using vimscript with \= in the replacement expression) you can do virtually anything (including, sending mail or printing a document, if you would really want to):
:%s/\v<DB_\w+>/\=substitute(submatch(0), '\v_?([^_])([^_]*)', '\U\1\L\2', 'g')/g
To transform 'database style' names like
var DB_USER_ID = f();
var DB_USER_FIRST_NAME = f();
var DB_USER_LAST_NAME = f();
var DB_USER_HOME_ADDRESS = f();
into 'camel case style names' like:
var DbUserId = f();
var DbUserFirstName = f();
var DbUserLastName = f();
var DbUserHomeAddress = f();
Live demo with expression evaluations
Edit In response to the comment/chat: You can use the approach #1 for this quite easily:
/\v\c\s*\zs(\s{-}(((sqrt|log|sin|cos|tan|exp)?\(.{-}\))|(-?[0-9,.]+(e-?[0-9]+)?)|([-+*/%^]+)))+(\s*\=?)?\s*
qqa<M-.><Esc>nq
Now you can repeat for all of the document:
:set nowrapscan
100#q
If there's only one match in every line, you could use :global instead of :s:
:%g/search-pattern/normal nrX
The :[range]normal positions the cursor at the beginning of the line, therefore the n to go to the first match before the mapping (I use rX as an example). You could write a custom command that would handle all matches in a line, but I would solve your use case with a recursive macro instead:
First, perform the search: /search-pattern, then record a macro containing your mapping, which jumps to the next match at the end: qarXnq. You can now manually apply the macro repeatedly via #a, or make it recursive via qA#aq, or :let #a .= '#a'. Execute this once #a, and it will run until it runs out of matches.
I'm sure this is an issue anyone who uses Stata for publications or reports has run into:
How do you conveniently export your output to something that can be parsed by a scripting language or Excel?
There are a few ado files that do this for specific commands. For example:
findit tabout
findit outreg2
But what about exporting the output of the table command? Or the results of an anova?
I would love to hear about how Stata users address this problem for either specific commands or in general.
After experimenting with this for a while, I've found a solution that works for me.
There are a variety of ADOs that handle exporting specific functions. I've made use of outreg2 for regressions and tabout for summary statistics.
For more simple commands, it's easy to write your own programs to save results automatically to plaintext in a standard format. Here are a few I wrote...note that these both display results (to be saved to a log file) and export them into text files – if you wanted to just save to text you could get rid of the di's and qui the sum, tab, etc. commands:
cap program drop sumout
program define sumout
di ""
di ""
di "Summary of `1'"
di ""
sum `1', d
qui matrix X = (r(mean), r(sd), r(p50), r(min), r(max))
qui matrix colnames X = mean sd median min max
qui mat2txt, matrix(X) saving("`2'") replace
end
cap program drop tab2_chi_out
program define tab2_chi_out
di ""
di ""
di "Tabulation of `1' and `2'"
di ""
tab `1' `2', chi2
qui matrix X = (r(p), r(chi2))
qui matrix colnames X = chi2p chi2
qui mat2txt, matrix(X) saving("`3'") replace
end
cap program drop oneway_out
program define oneway_out
di ""
di ""
di "Oneway anova with dv = `1' and iv = `2'"
di ""
oneway `1' `2'
qui matrix X = (r(F), r(df_r), r(df_m), Ftail(r(df_m), r(df_r), r(F)))
qui matrix colnames X = anova_between_groups_F within_groups_df between_groups_df P
qui mat2txt, matrix(X) saving("`3'") replace
end
cap program drop anova_out
program define anova_out
di ""
di ""
di "Anova command: anova `1'"
di ""
anova `1'
qui matrix X = (e(F), e(df_r), e(df_m), Ftail(e(df_m), e(df_r), e(F)), e(r2_a))
qui matrix colnames X = anova_between_groups_F within_groups_df between_groups_df P RsquaredAdj
qui mat2txt, matrix(X) saving("`2'") replace
end
The question is then how to get the output into Excel and format it. I found that the best way to import the text output files from Stata into Excel is to concatenate them into one big text file and then import that single file using the Import Text File... feature in Excel.
I concatenate the files by placing this Ruby code in the output folder and then running int from my Do file with qui shell cd path/to/output/folder/ && ruby table.rb:
output = ""
Dir.new(".").entries.each do |file|
next if file =~/\A\./ || file == "table.rb" || file == "out.txt"
if file =~ /.*xml/
system "rm #{file}"
next
end
contents = File.open(file, "rb").read
output << "\n\n#{file}\n\n" << contents
end
File.open("out.txt", 'w') {|f| f.write(output)}
Once I import out.txt into its own sheet in Excel, I use a bunch of Excel's built-in functions to pull the data together into nice, pretty tables.
I use a combination of vlookup, offset, match, iferror, and hidden columns with cell numbers and filenames to do this. The source .txt file is included in out.txt just above the contents of that file, which lets you look up the contents of the file using these functions and then reference specific cells using vlookup and offset.
This Excel business is actually the most complicated part of this system and there's really no good way to explain it without showing you the file, though hopefully you can get enough of an idea to figure it out for yourself. If not, feel free to contact me through http://maxmasnick.com and I can get you more info.
I have found that the estout package is the most developed and has good documentation.
This is an old question and a lot has happened since it was posted.
Stata now has several built-in commands and functions that allow anyone to
export customized output fairly easily:
putexcel
putexcel with advanced syntax
putdocx
putpdf
There are also equivalent Mata functions / classes, which offer greater flexibility:
_docx*()
Pdf*()
xl()
From my experience, there aren't 100% general solutions. Community-contributed commands such as estout are now mature enough to handle most basic operations. That said, if you have something that deviates even slightly from the template you will have to program this yourself.
Most tutorials throw in several packages where it would indeed nice to have only one exporting everything, which is what Max suggests above with his interesting method.
I personally use tabout for summary statistics and frequencies, estout for regression output, and am trying out mkcorr for correlation matrixes.
It's been a while, but I believe you can issue a log command to capture the output.
log using c:\data\anova_analysis.log, text
[commands]
log close
I use estpost-- part of the estout package-- to tabulate results from non-estimation commands. You can then store them and export easily.
Here's an example:
estpost corr varA varB varC varD, matrix
est store corrs
esttab corrs using corrs.rtf, replace
You can then add options to change formatting, etc.
You can use asdoc that is available on SSC. To download,
ssc install asdoc
asdoc works well with almost all Stata commands. Specifically, it produces publication quality tables for :
summarize command - to report summary statistics
cor or pwcorr command - to report correlations
tabstat - for flexible tables of descriptive statistics
tabulate - for one-way, two-way, three-way tabulations
regress - for detailed, nested, and wide regression tables
table - flexible tables
and many more. You can explore more about asdoc here
https://fintechprofessor.com/2018/01/31/asdoc/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
We have been asked by a client to incorporate ICD-9 codes into a system.
I'm looking for a good resource to get a complete listing of codes and descriptions that will end up in a SQL database.
Unfortunately a web service is out of the question as a fair amount of the time folks will be off line using the application.
I've found http://icd9cm.chrisendres.com/ and http://www.icd9data.com/ but neither offer downloads/exports of the data that I could find.
I also found http://www.cms.hhs.gov/MinimumDataSets20/07_RAVENSoftware.asp which has a database of the ICD-9 codes but they are not in the correct format and I'm not 100% sure how to properly convert (It shows the code 5566 which is really 556.6 but I can't find a rule as to how/when to convert the code to include a decimal)
I'm tagging this with medical and data since I'm not 100% sure where it should really be tagged...any help there would also be appreciated.
Just wanted to chime in on how to correct the code decimal places. First, there are four broad points to consider:
Standard codes have Decimal place XXX.XX
Some Codes Do not have trailing decimal places
V Codes also follow the XXX.XX format --> V54.31
E Codes follow XXXX.X --> E850.9
Thus the general logic of how to fix the errors is
If first character = E:
If 5th character = '':
Ignore
Else replace XXXXX with XXXX.X
Else If 4th-5th Char is not '': (XXXX or XXXXX)
replace XXXXX with XXX + . + remainder (XXX.XX or XXX.X)
(All remaining are XXX)
I implemented this with two SQL Update statements:
Number 1, for Non E-codes:
USE MainDb;
UPDATE "dbo"."icd9cm_diagnosis_codes"
SET "DIAGNOSIS CODE" = SUBSTRING("DIAGNOSIS CODE",1,3)+'.'+SUBSTRING("DIAGNOSIS CODE",4,5)
FROM "dbo"."icd9cm_diagnosis_codes"
WHERE
SUBSTRING("DIAGNOSIS CODE",4,5) != ''
AND
LEFT("DIAGNOSIS CODE",1) != 'E'
Number 2 - For E Codes:
UPDATE "dbo"."icd9cm_diagnosis_codes"
SET "DIAGNOSIS CODE" = SUBSTRING("DIAGNOSIS CODE",1,4)+'.'+SUBSTRING("DIAGNOSIS CODE",5,5)
FROM "dbo"."icd9_Diagnosis_table"
WHERE
LEFT("DIAGNOSIS CODE",1) = 'E'
AND
SUBSTRING("DIAGNOSIS CODE",5,5) != ''
Seemed to do the trick for me (Using SQL Server 2008).
I ran into this same issue a while back and ended up building my own solution from scratch. Recently, I put up an open API for the codes for others to use: http://aqua.io/codes/icd9/documentation
You can just download all codes in JSON (http://api.aqua.io/codes/beta/icd9.json) or pull an individual code (http://api.aqua.io/codes/beta/icd9/250-1.json). Pulling a single code not only gives you the ICD-10 "crosswalk" (equivalents), but also some extra goodies, like relevant Wikipedia links.
I finally found the following:
"The field for the ICD-9-CM Principal and Other Diagnosis Codes is six characters in length, with the decimal point implied between the third and fourth digit for all diagnosis codes other than the V codes. The decimal is implied for V codes between the second and third digit."
So I was able to get a hold of a complete ICD-9 list and reformat as required.
You might find that the ICD-9 codes follow the following format:
All codes are 6 characters long
The decimal point comes between the 3rd and 4th characters
If the code starts with a V character the decimal point comes between the 2nd and 3rd characters
Check this out: http://en.wikipedia.org/wiki/List_of_ICD-9_codes
I struggled with this issue myself for a long time as well. The best resource I have been able to find for these are the zip files here:
https://www.cms.gov/ICD9ProviderDiagnosticCodes/06_codes.asp
It's unfortunate because they (oddly) are missing the decimal places, but as several other posters have pointed out, adding them is fairly easy since the rules are known. I was able to use a regular expression based "find and replace" in my text editor to add them. One thing to watch out for if you go that route is that you can end up with codes that have a trailing "." but no zero after it. That's not valid, so you might need to go through and do another find/replace to clean those up.
The annoying thing about the data files in the link above is that there is no relationship to categories. Which you might need depending on your application. I ended up taking one of the RTF-based category files I found online and re-formatting it to get the ranges of each category. That was still doable in a text editor with some creative regular expressions.
I was able to use the helpful answers here an create a groovy script to decimalize the code and combine long and short descriptions into a tab separated list. In case this helps anyone, I'm including my code here:
import org.apache.log4j.BasicConfigurator
import org.apache.log4j.Level
import org.apache.log4j.Logger
import java.util.regex.Matcher
import java.util.regex.Pattern
Logger log = Logger.getRootLogger()
BasicConfigurator.configure();
Logger.getRootLogger().setLevel(Level.INFO);
Map shortDescMap = [:]
new File('CMS31_DESC_SHORT_DX.txt').eachLine {String l ->
int split = l.indexOf(' ')
String code = l[0..split].trim()
String desc = l[split+1..-1].trim()
shortDescMap.put(code, desc)
}
int shortLenCheck = 40 // arbitrary lengths, but provide some sanity checking...
int longLenCheck = 300
File longDescFile = new File('CMS31_DESC_LONG_DX.txt')
Map cmsRows = [:]
Pattern p = Pattern.compile(/^(\w*)\s+(.*)$/)
new File('parsedICD9.csv').withWriter { out ->
out.write('ICD9 Code\tShort Description\tLong Description\n')
longDescFile.eachLine {String row ->
Matcher m = row =~ p
if (m.matches()) {
String code = m.group(1)
String shortDescription = shortDescMap.get(code)
String longDescription = m.group(2)
if(shortDescription.size() > shortLenCheck){
log.info("Not short? $shortDescription")
}
if(longDescription.size() > longLenCheck){
log.info("${longDescription.size()} == Too long? $longDescription")
}
log.debug("Match 1:${code} -- 2:${longDescription} -- orig:$row")
if (code.startsWith('V')) {
if (code.size() > 3) {
code = code[0..2] + '.' + code[3..-1]
}
log.info("Code: $code")
} else if (code.startsWith('E')) {
if (code.size() > 4) {
code = code[0..3] + '.' + code[4..-1]
}
log.info("Code: $code")
} else if (code.size() > 3) {
code = code[0..2] + '.' + code[3..-1]
}
if (code) {
cmsRows.put(code, ['longDesc': longDescription])
}
out.write("$code\t$shortDescription\t$longDescription\n")
} else {
log.warn "No match for row: $row"
}
}
}
I hope this helps someone.
Sean