Adding a new lexer to scintilla/scite (...and eventually wxPython StyledTextCtrl) - lexer

Has anyone of you successfully added a lexer to scintilla?
I have been following the short instructions at http://www.scintilla.org/SciTELexer.html - and even discovered the secret extra instructions at http://www.scintilla.org/ScintillaDoc.html#BuildingScintilla (Changing Set of Lexers)
Everything compiles, and I can add the lexer to SciTE just fine, but my ColouriseMapfileDoc method just does not get called (a printf does not produce output). If I add the same code to e.g. the ColouriseLuaDoc lexer, everything is fine (a printf does produce output).
Specifically I have
In scintilla/include/Scintilla.iface, added val SCLEX_MAPFILE=99
And any lexical class IDs
In the scintilla/include directory run HFacer.py and confirmed that the SciLexer.h file has changed.
Created LexMapfile.cxx with a ColouriseMapfileDoc function
At the end of the file associated the lexer ID and name with the function:
LexerModule lmMapfile(SCLEX_MAPFILE, ColouriseMapfileDoc, "mapfile");
Run LexGen.py to generate all the makefiles (as per the secret instructions)
Created a new SciTE properties file by cloning scite/src/others.properties
Set up some styles
In scite/src/SciTEGlobal.properties added $(filter.conf) to the definition of open.filter.
Added this language to the Language menu of SciTE,
Built both Scintilla and SciTE.
Grumbled and cursed.
What am I doing wrong, except maybe step 12?

In case someone reads this question in the future - you will also have to add a line
import yourformat in SciTEGlobal.properties. That's the undocumented step 9b.

In case someone reads this question in the future - you will also have to add a line import
yourformat in SciTEGlobal.properties. That's the undocumented step 9b.
This step is no longer required. I compiled 3.2.2 and this was done with import *. The rest of the steps are still complete and relevant though.

I'm wring one lexer directly in scintilla/lexer/LexOthers.cxx as described in http://www.scintilla.org/SciTELexer.html.
For scite 3.2.3 the lacking step 5b is that you need to add LINK_LEXER(lmYouLexerMod); in scintilla/src/Catalogue.cxx.

Related

Converting a package backend from Rcpp to cpp11

I'm trying to use cpp11 with a fork of fixest (I need to use vendoring). I put a new branch at https://github.com/pachadotdev/fixest/tree/cpp11, then started committing step by step as cpp11's FAQ says, and I fixed all the errors one by one.
Now when I run devtools::install(), the process fails, and now it doesn't say "I didn't like line 10 in means.cpp" or similar.
I'm looking for any help to be able to solve this problem. I listed my doubts at https://github.com/pachadotdev/fixest/issues.
The problem appears to be in this file just to show diff in the file quf.cpp (https://github.com/pachadotdev/fixest/pull/8/files#diff-b9826484794eda5aaf5e9687abf2c971b0b71666b0c44c0b5fa985ce24f3ef26).

Adding content in file in specific place with node js (Like Angular Cli Modify app.module file)

I got the idea that node js its not just for web application for example I can create a console application with node (cli) .
and already I have an interest in how I can make a cli app that create files and modify existing files for example something like angular cli with one command "ng generate component" its :-
1- create a set of files
2- modify app.module file
a. add import statement for generated component
b. add generated component in declarations array
and after a lot of search I got that first step can be handled in some way with node file system module.
but i don't know how they modify "app.module" file by just adding some syntax in its right place for instance adding new import statement after all exists import statements also adding the component name in declarations array as a last item
I'm really appreciate any help maybe with some code example if possible and thanks in advance
After some searches i found this answer:
There a couple of ways of editing a file, the most reliable is perhaps
the most complex one which can be done by parsing the file (Generating
an abstract syntax tree) update the new ast and pass it to a code
generator which will output the new string (code) of the modified ast.
Another option is to use regular expressions to know where add to
certain statements. For example there would be a regex to match import
statements to lines, you will map the lines of the file to this regex
where you'll get an array of booleans denoting whether the line is an
import stmt. Or not, once the import stmts are finished you can insert
a new line with the new import statement in the original lines array.
A third option is to regenerate the file all at once everytime, but
this means that you'll have to a ctx of the project (ctx = object
contains some details.
and as reference i found that AST (Abstract Syntax Tree) is more reliable way also it's not that hard this is some links that helped me a lot to know what is AST in simple way and how to deal with it
1- What is AST and how to understand it and how to use it https://www.youtube.com/watch?v=tM_S-pa4xDk
2- and this is an amazing article about "Write Code to Rewrite Your Code with jscodeshift" https://www.toptal.com/javascript/write-code-to-rewrite-your-code
https://www.youtube.com/watch?v=tM_S-pa4xDk

How to get SMAC3 working for Python 3x on Windows

This is a great package for Bayesian optimization of hyperparameters (especially mixed integer/continuous/categorical...and has shown to be better than Spearmint in benchmarks). However, clearly it is meant for Linux. What do I do...?
First you need to download swig.exe (the whole package) and unzip it. Then drop it somewhere and add the folder to path so that the installer for SMAC3 can call swig.exe.
Next, the Resource module is going to cause issues because that is only meant for Linux. That is specifically used by Pynisher. You'll need to comment out import pynisher in the execute_func.py module. Then, set use_pynisher:bool=False in the def __init__(self...) in the same module. The default is true.
Then, go down to the middle of the module where an if self.use_pynisher....else statement exists. Obviously our code now enters the else part, but it is not setup correctly. Change result = self.ta(config, **obj_kwargs) to result = self.ta(list(config.get_dictionary().values())). This part may need to be adjusted yet depending on what kind of inputs your function handles, but essentially you can see that this will enable the basic example shown in the included branin_fmin.py module. If doing the random forest example, don't change at all...etc.

Trouble building R package wtih devtools when it uses RcppArmadillo

This is my first stackoverflow question, so please be kind, folks!
I have greatly enjoyed my recently-found power to build R packages using devtools. However, as soon as I try building a package that uses RcppArmadillo, my workflow of running devtools::document(), devtools::check(), and devtools::build() no longer works.
For example, I have a (hopefully pretty minimal + complete) test version of the package I'm trying to develop here: https://github.com/suztolwinskiward/fooR/. fooR contains only one functions, which is a C++ implementation of the rdist.earth function from the fields package.
Running devtools::document("fooR") spits out lots of messages (several alluding to "undefined references" to variables that do not live in my source co that are not interpretable to me, and then fails:
collect2: ld returned 1 exit status
no DLL was created
ERROR: compilation failed for package 'fooR'
* removing 'C:/Users/I53794/AppData/Local/Temp/RtmpWgC8nD/devtools_install_1ea473123086/fooR'
Error: Command failed (1)
One the other hand, when I source the C++ function that depends on RcppArmadillo, it seems to run just fine:
> Rcpp::sourceCpp('./src/rdist_earth_cpp.cpp')
> data('miami')
> data('new_orleans','katrina_path')
> rdist_earth_cpp(katrina_path,new_orleans)
[,1]
[1,] 1042.36073
[2,] 998.96793
[3,] 957.69315
[4,] 917.91486
[5,] 868.07791
[6,] 805.73485
[7,] 763.01476
[8,] 726.10133
[9,] 692.14482
[10,] 670.15133
[11,] 662.23353
[12,] 625.55592
[13,] 601.08682
[14,] 579.73940
[15,] 560.32660
[16,] 539.14192
[17,] 510.15438
[18,] 481.40037
[19,] 442.52322
[20,] 391.96619
[21,] 331.66378
[22,] 271.79088
[23,] 201.24749
[24,] 128.12647
[25,] 56.99198
[26,] 45.80297
[27,] 32.96609
[28,] 81.71237
[29,] 189.31050
[30,] 296.92104
[31,] 406.12593
[32,] 516.08458
[33,] 654.81113
[34,] 808.21670
This leads me to think there's something wrong with the way I'm trying to use RcppArmadillo in my package, but I haven't been able to figure out what. Any advice much appreciated!
P.S. I'm surprised there's no RcppArmadillo tag here....
In addition to the answers of jtilly and the comment from Dirk:
RcppArmadillo.package.skeleton() generates the correct namespace file, but after running roxygen2 via document() the namespace just contains one line
# Generated by roxygen2: do not edit by hand
and the DynLib/export directives are overwritten. To let roxygen2 automatically generate the correct namespace, add a new R file to the R-subdirectory of your package directory containing the following:
#' #useDynLib YourPackageName
#' #importFrom Rcpp evalCpp
#' #exportPattern "^[[:alpha:]]+"
NULL
The name of this file doesn't matter, but YourPackageName.r is usual for this (kind of) "main file".
When running "document()", the following namespace file is generated:
# Generated by roxygen2: do not edit by hand
exportPattern("^[[:alpha:]]+")
importFrom(Rcpp,evalCpp)
useDynLib(YourPackageName)
This is the same namespace which is generated via RcppArmadillo.package.skeleton() by RcppArmadillo 0.6.700.6.0.
Your NAMESPACE file is empty. It should contain something like this:
useDynLib(fooR)
exportPattern("^[[:alpha:]]+")
What eventually worked was to initialize a new package with RcppArmadillo.skeleton.package, move all my previous files therein, document manually, and then check and build with the GUI buttons in RStudio. This feels pretty kludgy and I really liked using roxygen2 much better for documentation-- but as a relative novice in package development with RcppAmadillo dependence, I am just happy for now to have found a way to build successfully!
'devtools::document()' compiles your code, honestly, I'm not sure why. This means that if the compilation fails, the documentation isn't completed. It seems in your case this means you don't get your NAMESPACE written.
Start as Dirk suggests, and add stuff in, but it will have to compile before the docs get done.

minko / lua issue : premake5.lua:3: attempt to index global 'minko' (a nil value)

I am working with minko and managed to compile MINKO SDK properly for 3 platforms (Linux, Android, HTML5) and build all tutorials / examples. Moving on to create my own project, I followed the instructions on how to use the existing skeleton project, then using an existing example project.
(I believe there is an error in the skeleton code at this line :
auto sceneManager = SceneManager::create(canvas->context()); //does not compile
where as the example file look like this :
auto sceneManager = SceneManager::create(canvas); //compile and generate binary
I was able to do so by modifying premake5.lua (to include more plugins) and calling script/solution_gmake_gcc.sh
to generate the make solution a week ago. Today, I tried to make a new project in a new folder but calling
script/solution_gmake_gcc.sh and script/clean failed with this error:
minko-master/skel_tut/mycode/premake5.lua:3: attempt to index global 'minko' (a nil value)
Now at premake5.lua line 3 there is this line : minko.project.solution(PROJECT_NAME),
however sine i am not familiar with lua at all, can anyone shed any light on the issue ?
What is supposed to be declared here, why is it failing suddenly... ?
(I can still modify,compile and run the code but i can't for example add more plug-ins)
PS: weirdly enough, the previously 'working' project is also failing at this point.
Thanks.
PROJECT_NAME = path.getname(os.getcwd())
minko.project.application("minko-tutorial-" .. PROJECT_NAME)
files { "src/**.cpp", "src/**.hpp", "asset/**" }
includedirs { "src" }
-- plugins
minko.plugin.enable("sdl")
minko.plugin.enable("assimp")
minko.plugin.enable("jpeg")
minko.plugin.enable("bullet")
minko.plugin.enable("png")
--html overlay
minko.plugin.enable("html-overlay")
Assuming that's indeed your project premake5.lua file (please us the code tags next time), you should have include "script" at the beginning of the file:
https://github.com/aerys/minko/blob/master/skeleton/premake5.lua#L1
If you don't have this line, it will not include script/premake5.lua which is in charge of including the SDK build system files that defines everything inside the minko Lua namespace/table. That's why you get that error.
I think you copy pasted one of the examples/tutorials premake5.lua file instead of modifying the one provided by the skeleton. The premake conf file of the examples/tutorials are different since they are included from the SDK premake files. But your app premake5.lua does the "opposite": it includes the SDK conf files rather than being included by them.
The best practice is to edit your app's copy of the skeleton's premake5.lua (instead of copy/pasting one from the examples/tutorials).
(I believe there is an error in the skeleton code at this line :
That's possible. Our build server doesn't test the skeleton code. That's a mistake we will fix ASAP to make sure it works properly.
script/solution_gmake_gcc.sh and script/clean failed with this error:
minko-master/skel_tut/mycode/premake5.lua:3: attempt to index global 'minko' (a nil value)
Could you copy/paste your premake5.lua file?
Also, what's the value you set for the MINKO_HOME env var? Maybe you've moved the SDK...
Note that instead of setting a global MINKO_HOME env var, you can also set the corresponding LUA constant at the very begining of your premake5.lua file.

Resources