Thanks in advance for the help. I am new to kdb/q, coming from a Python and C++ background.
Just a simple syntax question: I have a string with fields and their corresponding values
pp_str: "field_1:abc field_2:xyz field_3:kdb"
I wrote an atomic (scalar) function to extract the value of a given field.
get_field_value: {[field; pp_str] pp_fields: " " vs pp_str; pid_field: pp_fields[where like[pp_fields; field,":*"]]; start_i: (pid_field[0] ss ":")[0] + 1; end_i: count pid_field[0]; indices: start_i + til (end_i - start_i); pid_field[0][indices]}
show get_field_value["field_1"; pp_str]
"abc"
show get_field_value["field_3"; pp_str]
"kdb"
Now how do I generalize this so that if I input a vector of fields, I get a vector of values? I want to input ("field_1"; "field_2"; "field_3") and output ("abc"; "xyz"; "kdb"). I tried multiple approaches (below) but I just don't understand kdb/q's syntax well enough to vectorize my function:
/ Attempt 1 - Fail
get_field_value[enlist ("field_1"; "field_2"); pp_str]
/ Attempt 2 - Fail
get_field_value[; pp_str] /. enlist ("field_1"; "field_3")
/ Attempt 3 - Fail
fields: ("field_1"; "field_2")
get_field_value[fields; pp_str]
To run your function for each you could project the pp_str variable and use each for the others
q)get_field_value[;pp_str]each("field_1";"field_3")
"abc"
"kdb"
Kdb actually has built-in functionality to handle this: https://code.kx.com/q/ref/file-text/#key-value-pairs
q){#[;x](!/)"S: "0:y}[`field_1;pp_str]
"abc"
q)
q){#[;x](!/)"S: "0:y}[`field_1`field_3;pp_str]
"abc"
"kdb"
I think this might be the syntax you're looking for.
q)get_field_value[; pp_str]each("field_1";"field_2")
"abc"
"xyz"
I am reading the text file consisting of bengali words. But I am unable to print the dependent vowels like KA,KI etc...
Here is my sample code and output
import unicodedata
bengali_phoneme_maplist={u'অ':'A',u'আ':'AA',u'ই':'I',u'ঈ':'II',u'উ':'U',u'ঊ ':'UU',u'ঋ ':'R',u'ঌ ':'L',u'এ ':'E',u'ঐ ':'AI',u'ও ':'O',u'ঔ ':'AU',u'ক':'KA',u'খ ':'KHA',u'গ ':'GA',u'ঘ':'GHA',u'ঙ ':'NGA',u'চ ':'CA',u'ছ':'CHA',u'জ ':'JA',u'ঝ':'JHA',u'ঞ':'NYA',u'ট ':'TTA',u'ঠ':'TTHA',u'ড ':'DDA',u'ঢ':'DDHA',u'ণ ':'NNA',u'ত ':'TA',u'ত ':'THA',u'দ':'DA',u'ধ':'DHA',u'ন':'NA',u'প':'PA',u'ফ':'PHA',u'ব':'BA',u'ভ':'BHA',u'ম ':'MA',u'য ':'YA',u'র':'RA',u'ল ':'LA',u'শ ':'SHA',u'ষ':'SSA',u'স ':'SA',u'হ':'ha',u' া ':'AAV',u' ি':'IV',u'ী':'IIV',u'ু':'UV',u'ূ':'UUV',u'ৃ':'RRV',u'ৄ ':'RR',u'ৄ':'EV',u' ৈ':'EV',u'়':'NUKTHA',u'ঽ':'AVAGRAHA'}
bengali_phoneme_maplist_normalise={unicodedata.normalize('NFKD',k):v
for k,v in bengali_phoneme_maplist.items()}
with open('bengali.txt','r')as infile:
lines=infile.readlines()
for index,line in enumerate(lines):
print('Phonemes in line{0}.total{1} symbols'.format(index,len(line)))
unknown=[]
words=line.split()
for word in words:
print(word,':',sep=' ', end='')
for character in word:
c=unicodedata.normalize('NFKD',character).casefold()
try:
print(bengali_phoneme_maplist_normalise[c],sep='',end='')
except KeyError:
print('_',sep='',end='')
if c not in unknown:
unknown.append(c)
print()
if unknown:
print('Unrecognised symbols:{0},total {1} symbols'.format(','.join(unknown),len(unknown)))
Sample input:
শিল্পাঞ্চলে ঢোকার মুখে, স্ন্যাক্সবারে খাবার কিনছিলেন, বহুজাতিক তথ্যপ্রযুক্তি সংস্থার কর্মী, শুভময় বন্দ্যোপাধ্যায়
Sample output:
Phonemes in line0.total129 symbols
text_000002 :___________
"শিল্পাঞ্চলে :_____PA_NYA____
ঢোকার :DDHA_KA_RA
মুখে, :_UV___
স্ন্যাক্সবারে :__NA___KA__BA_RA_
খাবার :__BA_RA
কিনছিলেন, :KA_NACHA___NA_
Unrecognisedsymbols:t,e,x,_,0,2,",শ,ি,ল,্,া,চ,ে,ো,ম,খ,,,স,য,জ,ত,থ,ং,য়,),
(Note that I know nohting about Bengali. :)
There are a few problems in your code:
There are many extra SPACE chars in the bengali_phoneme_maplist definition. For example, u'ঊ ' should be u'ঊ'. And it seems like it's not easy to input chars like u'া' in an text editor so I suggest you directly use unicode in the code, like '\u09be':'AAV'. (Actually I'd suggest you use '\uxxxx' for all chars and write the real chars in comments.)
u'ত':'TA',u'ত':'THA' should change to u'ত':'TA',u'থ':'THA'.
The chars in bengali_phoneme_maplist are not complete. For example there's no ো , ৌ , ্ and ং
After fixing these errors you will get the correct result.
In my code I have the following line:
fprintf(logfile,'Parameters: Size: %d\tH: %.4f\tF: %.1f\tI: %.3f\tR: %d\tSigma: %d\tDisp: %.1f\r\n',parameter_sets(ps,:));
which is too long, so I want to break it to:
fprintf(logfile,'Parameters: Size: %d\tH: %.4f\tF: %.1f\tI: %.3f\tR: ...
%d\tSigma: %d\tDisp: %.1f\r\n',parameter_sets(ps,:));
However, since the brake is within a string, MATLAB see the formatting %d sign in the second line as a start of a comment, and ignore this line (and produce an error...).
So I tried to make it clearer with a [] that warp the string:
fprintf(logfile,['Parameters: Size: %d\tH: %.4f\tF: %.1f\tI: %.3f\tR: ...
%d\tSigma: %d\tDisp: %.1f\r\n'],parameter_sets(ps,:));
but no help, it still interpret the second line as a comment. I also tried with and without the ellipsis (...) in different places, with no success.
So how can I write a line in a formatted way (i.e. a reasonable length) if it has a % sign in it?
Divide it in two lines like this:
fprintf(logfile,['Parameters: Size: %d\tH: %.4f\tF: %.1f\tI: %.3f\tR:', ...
'%d\tSigma: %d\tDisp: %.1f\r\n'],parameter_sets(ps,:));
% notice the apostrophe and comma(',) before ellpsis(...) at the end of first line
% and apostrophe(') at the start of the second line
Background:
I am working to migrate a Linux server to a newer one from Ubuntu 10.04 to 12.04
This server is responsible for executing several a number of Perl modules via crontabs.
These Perl Modules rely heavily on 30-40 perl extensions.
I have installed all Perl extensions and the crontabs are able to process successfully except for several Syntax errors caused by the newer versions of these Perl extensions.
I need some help with modifying the syntax to get the Perl script to process as intended.
Errors:
defined(%hash) is deprecated at pm/Alerts/Alerts.pm line 943.
(Maybe you should just omit the defined()?)
defined(%hash) is deprecated at pm/Alerts/Alerts.pm line 944.
(Maybe you should just omit the defined()?)
Code:
###
# Iterate the arrays deleting identical counts from each.
# If we found a mismatch then die.
# If either array is not empty when we are done then die
$logger->info('Comparing ' . (scalar keys %cms_rows) . ' CMS symbols to ' . (scalar keys %stats_rows) . ' STATS symbols');
foreach my $symbol ( keys %cms_rows ) {
my %cms_row = delete $cms_rows{$symbol};
my %stats_row = delete $stats_rows{$symbol};
##LINE 943## die("Error: NULL CMS counts for symbol '$symbol'") unless defined %cms_row;
##LINE 944## die("Error: NULL Stats counts for symbol '$symbol'") unless defined %stats_row;
my $cms_json = encode_json(\%cms_row);
my $stats_json = encode_json(\%stats_row);
$logger->debug("Comparing counts for '$symbol': CMS($cms_json), Stats($stats_json)");
die("Error: Up Counts Don't match for symbol '$symbol': CMS($cms_json), Stats($stats_json)") unless (!defined $cms_row{1} && !defined $stats_row{1}) || $cms_row{1} == $stats_row{1};
die("Error: Down Counts Don't match for symbol '$symbol': CMS($cms_json), Stats($stats_json)") unless (!defined $cms_row{-1} && !defined $stats_row{-1}) || $cms_row{-1} == $stats_row{-1};
}
###
Hopefully someone can help with this, any help is appreciated.
You must have upgraded from a seriously old version of Perl. The Perl 5.6.1 release notes say:
defined(%hash) is deprecated
(D) defined() is not usually useful on hashes because it checks for an
undefined scalar value. If you want to see if the hash is empty, just
use if (%hash) { # not empty } for example.
It was always a pretty stupid thing to do and Perl now warns you that you're doing something stupid. The warning is pretty clear about what you should do to fix this:
Maybe you should just omit the defined()?
So your lines would become:
die("Error: NULL CMS counts for symbol '$symbol'") unless %cms_row;
die("Error: NULL Stats counts for symbol '$symbol'") unless %stats_row;
I have the following code block in my .lhs file which uses inline TeX comments:
\begin{code}
main = print 0
{-"$\langle$Link$\rangle$"-}
\end{code}
However, after compiling with lhs2TeX, I get the following errors when compiling the generated .tex file:
! Missing $ inserted.
<inserted text>
$
l.269 \end{hscode}
\resethooks
I've inserted a begin-math/end-math symbol since I think
you left one out. Proceed, with fingers crossed.
! LaTeX Error: Bad math environment delimiter.
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.269 \end{hscode}
\resethooks
Your command was ignored.
Type I <command> <return> to replace it with another command,
or <return> to continue without it.
! Missing $ inserted.
<inserted text>
$
l.269 \end{hscode}
\resethooks
I've inserted a begin-math/end-math symbol since I think
you left one out. Proceed, with fingers crossed.
! LaTeX Error: Bad math environment delimiter.
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.269 \end{hscode}
\resethooks
Your command was ignored.
Type I <command> <return> to replace it with another command,
or <return> to continue without it.
! Missing $ inserted.
<inserted text>
$
l.269 \end{hscode}
\resethooks
I've inserted a begin-math/end-math symbol since I think
you left one out. Proceed, with fingers crossed.
! LaTeX Error: Bad math environment delimiter.
See the LaTeX manual or LaTeX Companion for explanation.
Type H <return> for immediate help.
...
l.269 \end{hscode}
\resethooks
Your command was ignored.
When I remove the " marks in the inline comment, the error disappears. Anyone know what's wrong?
P.S Here's the .tex file that lhs2TeX generates:
\documentclass{article}%% ODER: format == = "\mathrel{==}"
%% ODER: format /= = "\neq "
%
%
\makeatletter
\#ifundefined{lhs2tex.lhs2tex.sty.read}%
{\#namedef{lhs2tex.lhs2tex.sty.read}{}%
\newcommand\SkipToFmtEnd{}%
\newcommand\EndFmtInput{}%
\long\def\SkipToFmtEnd#1\EndFmtInput{}%
}\SkipToFmtEnd
\newcommand\ReadOnlyOnce[1]{\#ifundefined{#1}{\#namedef{#1}{}}\SkipToFmtEnd}
\usepackage{amstext}
\usepackage{amssymb}
\usepackage{stmaryrd}
\DeclareFontFamily{OT1}{cmtex}{}
\DeclareFontShape{OT1}{cmtex}{m}{n}
{<5><6><7><8>cmtex8
<9>cmtex9
<10><10.95><12><14.4><17.28><20.74><24.88>cmtex10}{}
\DeclareFontShape{OT1}{cmtex}{m}{it}
{<-> ssub * cmtt/m/it}{}
\newcommand{\texfamily}{\fontfamily{cmtex}\selectfont}
\DeclareFontShape{OT1}{cmtt}{bx}{n}
{<5><6><7><8>cmtt8
<9>cmbtt9
<10><10.95><12><14.4><17.28><20.74><24.88>cmbtt10}{}
\DeclareFontShape{OT1}{cmtex}{bx}{n}
{<-> ssub * cmtt/bx/n}{}
\newcommand{\tex}[1]{\text{\texfamily#1}} % NEU
\newcommand{\Sp}{\hskip.33334em\relax}
\newcommand{\Conid}[1]{\mathit{#1}}
\newcommand{\Varid}[1]{\mathit{#1}}
\newcommand{\anonymous}{\kern0.06em \vbox{\hrule\#width.5em}}
\newcommand{\plus}{\mathbin{+\!\!\!+}}
\newcommand{\bind}{\mathbin{>\!\!\!>\mkern-6.7mu=}}
\newcommand{\rbind}{\mathbin{=\mkern-6.7mu<\!\!\!<}}% suggested by Neil Mitchell
\newcommand{\sequ}{\mathbin{>\!\!\!>}}
\renewcommand{\leq}{\leqslant}
\renewcommand{\geq}{\geqslant}
\usepackage{polytable}
%mathindent has to be defined
\#ifundefined{mathindent}%
{\newdimen\mathindent\mathindent\leftmargini}%
{}%
\def\resethooks{%
\global\let\SaveRestoreHook\empty
\global\let\ColumnHook\empty}
\newcommand*{\savecolumns}[1][default]%
{\g#addto#macro\SaveRestoreHook{\savecolumns[#1]}}
\newcommand*{\restorecolumns}[1][default]%
{\g#addto#macro\SaveRestoreHook{\restorecolumns[#1]}}
\newcommand*{\aligncolumn}[2]%
{\g#addto#macro\ColumnHook{\column{#1}{#2}}}
\resethooks
\newcommand{\onelinecommentchars}{\quad-{}- }
\newcommand{\commentbeginchars}{\enskip\{-}
\newcommand{\commentendchars}{-\}\enskip}
\newcommand{\visiblecomments}{%
\let\onelinecomment=\onelinecommentchars
\let\commentbegin=\commentbeginchars
\let\commentend=\commentendchars}
\newcommand{\invisiblecomments}{%
\let\onelinecomment=\empty
\let\commentbegin=\empty
\let\commentend=\empty}
\visiblecomments
\newlength{\blanklineskip}
\setlength{\blanklineskip}{0.66084ex}
\newcommand{\hsindent}[1]{\quad}% default is fixed indentation
\let\hspre\empty
\let\hspost\empty
\newcommand{\NB}{\textbf{NB}}
\newcommand{\Todo}[1]{$\langle$\textbf{To do:}~#1$\rangle$}
\EndFmtInput
\makeatother
%
%
%
%
%
%
% This package provides two environments suitable to take the place
% of hscode, called "plainhscode" and "arrayhscode".
%
% The plain environment surrounds each code block by vertical space,
% and it uses \abovedisplayskip and \belowdisplayskip to get spacing
% similar to formulas. Note that if these dimensions are changed,
% the spacing around displayed math formulas changes as well.
% All code is indented using \leftskip.
%
% Changed 19.08.2004 to reflect changes in colorcode. Should work with
% CodeGroup.sty.
%
\ReadOnlyOnce{polycode.fmt}%
\makeatletter
\newcommand{\hsnewpar}[1]%
{{\parskip=0pt\parindent=0pt\par\vskip #1\noindent}}
% can be used, for instance, to redefine the code size, by setting the
% command to \small or something alike
\newcommand{\hscodestyle}{}
% The command \sethscode can be used to switch the code formatting
% behaviour by mapping the hscode environment in the subst directive
% to a new LaTeX environment.
\newcommand{\sethscode}[1]%
{\expandafter\let\expandafter\hscode\csname #1\endcsname
\expandafter\let\expandafter\endhscode\csname end#1\endcsname}
% "compatibility" mode restores the non-polycode.fmt layout.
\newenvironment{compathscode}%
{\par\noindent
\advance\leftskip\mathindent
\hscodestyle
\let\\=\#normalcr
\let\hspre\(\let\hspost\)%
\pboxed}%
{\endpboxed\)%
\par\noindent
\ignorespacesafterend}
\newcommand{\compaths}{\sethscode{compathscode}}
% "plain" mode is the proposed default.
% It should now work with \centering.
% This required some changes. The old version
% is still available for reference as oldplainhscode.
\newenvironment{plainhscode}%
{\hsnewpar\abovedisplayskip
\advance\leftskip\mathindent
\hscodestyle
\let\hspre\(\let\hspost\)%
\pboxed}%
{\endpboxed%
\hsnewpar\belowdisplayskip
\ignorespacesafterend}
\newenvironment{oldplainhscode}%
{\hsnewpar\abovedisplayskip
\advance\leftskip\mathindent
\hscodestyle
\let\\=\#normalcr
\(\pboxed}%
{\endpboxed\)%
\hsnewpar\belowdisplayskip
\ignorespacesafterend}
% Here, we make plainhscode the default environment.
\newcommand{\plainhs}{\sethscode{plainhscode}}
\newcommand{\oldplainhs}{\sethscode{oldplainhscode}}
\plainhs
% The arrayhscode is like plain, but makes use of polytable's
% parray environment which disallows page breaks in code blocks.
\newenvironment{arrayhscode}%
{\hsnewpar\abovedisplayskip
\advance\leftskip\mathindent
\hscodestyle
\let\\=\#normalcr
\(\parray}%
{\endparray\)%
\hsnewpar\belowdisplayskip
\ignorespacesafterend}
\newcommand{\arrayhs}{\sethscode{arrayhscode}}
% The mathhscode environment also makes use of polytable's parray
% environment. It is supposed to be used only inside math mode
% (I used it to typeset the type rules in my thesis).
\newenvironment{mathhscode}%
{\parray}{\endparray}
\newcommand{\mathhs}{\sethscode{mathhscode}}
% texths is similar to mathhs, but works in text mode.
\newenvironment{texthscode}%
{\(\parray}{\endparray\)}
\newcommand{\texths}{\sethscode{texthscode}}
% The framed environment places code in a framed box.
\def\codeframewidth{\arrayrulewidth}
\RequirePackage{calc}
\newenvironment{framedhscode}%
{\parskip=\abovedisplayskip\par\noindent
\hscodestyle
\arrayrulewidth=\codeframewidth
\tabular{#{}|p{\linewidth-2\arraycolsep-2\arrayrulewidth-2pt}|#{}}%
\hline\framedhslinecorrect\\{-1.5ex}%
\let\endoflinesave=\\
\let\\=\#normalcr
\(\pboxed}%
{\endpboxed\)%
\framedhslinecorrect\endoflinesave{.5ex}\hline
\endtabular
\parskip=\belowdisplayskip\par\noindent
\ignorespacesafterend}
\newcommand{\framedhslinecorrect}[2]%
{#1[#2]}
\newcommand{\framedhs}{\sethscode{framedhscode}}
% The inlinehscode environment is an experimental environment
% that can be used to typeset displayed code inline.
\newenvironment{inlinehscode}%
{\(\def\column##1##2{}%
\let\>\undefined\let\<\undefined\let\\\undefined
\newcommand\>[1][]{}\newcommand\<[1][]{}\newcommand\\[1][]{}%
\def\fromto##1##2##3{##3}%
\def\nextline{}}{\) }%
\newcommand{\inlinehs}{\sethscode{inlinehscode}}
% The joincode environment is a separate environment that
% can be used to surround and thereby connect multiple code
% blocks.
\newenvironment{joincode}%
{\let\orighscode=\hscode
\let\origendhscode=\endhscode
\def\endhscode{\def\hscode{\endgroup\def\#currenvir{hscode}\\}\begingroup}
%\let\SaveRestoreHook=\empty
%\let\ColumnHook=\empty
%\let\resethooks=\empty
\orighscode\def\hscode{\endgroup\def\#currenvir{hscode}}}%
{\origendhscode
\global\let\hscode=\orighscode
\global\let\endhscode=\origendhscode}%
\makeatother
\EndFmtInput
%
\begin{document}\section{}Precis 0
\newline{}\subsubsection*{\texttt{test0.tweave:}}\begin{hscode}\SaveRestoreHook
\column{B}{#{}>{\hspre}l<{\hspost}#{}}%
\column{E}{#{}>{\hspre}l<{\hspost}#{}}%
\>[B]{}\Varid{main}\mathrel{=}\Varid{print}\;\mathrm{0}{}\<[E]%
\\
\>[B]{}$\langle$Link$\rangle${}\<[E]%
\ColumnHook
\end{hscode}\resethooks
\section{}Precis 1
\newline{}\subsubsection*{\texttt{test0.tweave:}}\begin{hscode}\SaveRestoreHook
\column{B}{#{}>{\hspre}l<{\hspost}#{}}%
\column{E}{#{}>{\hspre}l<{\hspost}#{}}%
\>[B]{}\Varid{accum}\;\Varid{i}\mathrel{=}\Varid{id}{}\<[E]%
\ColumnHook
\end{hscode}\resethooks
\end{document}
The {-" ... "-} construct is quite low-level. It drops you in the environment TeX currently is in, which happens to be math mode already. So the solution to your problem is simple. Write the code to be inserted as if you are in math mode.
A different option is to use a normal comment, but make the comment characters invisible using \invisiblecomments. Normal comments are typeset as text by lhs2TeX.
The following complete lhs2TeX document demonstrates both options:
\documentclass{article}
%include polycode.fmt
\begin{document}
% Assume you are in math mode already:
\begin{code}
main = print 0
{-"\langle\text{Link}\rangle"-}
\end{code}
% This works, too:
\invisiblecomments
\begin{code}
main = print 0
{- $\langle$Link$\rangle$ -}
\end{code}
\end{document}