I've been trying to write a simple lexer in Nim and I need to my Procedure to return a Table. I've been having some issues with making this happen, here is my code:
main.nim
# Code
import lexer
echo lexer.lex("hi")
lexer.nim
import tables
proc lex*(code: string): Table =
let variable = to_table({"1": "hi"})
return variable
Every time I try compile the code (nim c -r --outDir:"../bin" "main.nim"), I get the following error:
/path/to/main.nim(5, 15) template/generic instantiation of `lex` from here
/path/to/lexer.nim(3, 26) Error: cannot instantiate 'Table[A, B]' inside of type definition: 'lex'; Maybe generic arguments are missing?
I've probably made some simple error, but I have wasted too much time on this, so any help is appreciated.
Operating System: Solus
Nim Version: 1.2.0
There are several issues with the code but lets start trying to decipher the compiler error: Table[A, B]' inside of type definition: 'lex'; Maybe generic arguments are missing?.
Nim has type inference but lets say it is forward declared and fails at the first moment something is unknown. In this case, the compiler is going through your main.nim file and sees that you want to invoke a proc with the following signature:
proc lex*(code: string): Table
That's all it sees, and since Table is a generic type, it fails instantiation of the return type, because it doesn't know what the generic types A or B should be, based on just the proc signature. That's what the compiler is complaining about. You could argue the compiler could try to look into the proc and figure out the specific type of the generic signature, but this requires more work on part of the compiler and would slow down compiling times, a metric the Nim community cares a lot about.
There are two ways to fix that, the simple one, which is declaring the types in the proc signature, and the difficult one, which requires you to specify the types at the caller. The former is a matter of changing the proc signature to this:
proc lex*(code: string): Table[string, string]
The latter one is to specify the type at the caller, which means changing your main.nim code to this:
echo lexer.lex[Table[string, string]]("hi")
Unfortunately this second solution won't compile either because the main.nim module doesn't now anything about the Table type: Error: undeclared identifier: 'Table'. In order to keep using this solution you would need to either export the Table type from lexer.nim adding an export statement to that module, or importing the tables module in main.nim (and any other future calling modules).
You need to be explicit in the type returned by the proc. As Grzegorz says, changing to:
proc lex*(code: string): Table[string, string] =
is enough. No need to modify anything in the main.nim code.
Two minor things to take into consideration:
Nim procs already init a result variable with the return type, which is implicitly returned, so your proc can be rewritten as:
proc lex*(code: string): Table[string, string] =
result["1"] = "hi"
No need to namespace lex proc in main, unless there are name collisions:
import lexer
lex("hi")
Related
I've been using Python's type annotations in an unusual way: I have some code that inspects the annotations of a function's arguments at run-time, searches for values that match the types of the arguments, and calls the function with values of appropriate types (if found).
This has been working wonderfully, and was remarkably easy to implement, but the source file has grown fairly large, so today I tried breaking it into multiple files. I found that get_type_hints() can't evaluate a type annotation if it's from a module other than the current one, at least not without being given a global namespace where all the needed types are defined.
I'm using from __future__ import annotations everywhere, so the type annotations in each function's .__annotations__ attribute are stored as strings in need of evaluation. To evaluate them, I need the globals from the module where the function was defined. How can I get that? Or will that even work? I'm using if TYPE_CHECKING: to avoid circular imports; consequently some annotations won't be available at run-time in each module where they're applied to a function.
Here's the code that extracts the types of the function arguments, if that helps:
def params_of(func: Callable) -> Iterable[Tuple[str, TypeAnnotation]]:
type_hints = get_type_hints(func)
for param_name in inspect.signature(func).parameters:
if param_name == 'return':
continue # disregard return type
yield (param_name, type_hints.get(param_name, Any))
(TypeAnnotation is only for readability; it's defined to Any.)
The following code works as expected, but the os.path.join produces a type error using pyright in VSCode, where shown.
# python 3.6.9
# pyright 1.1.25
# windows 10
# vscode 1.42.1
import os
import tempfile
with tempfile.TemporaryDirectory() as tmpfolder:
name = "hello.txt"
path = os.path.join(tmpfolder, name)
# No overloads for 'os.path.join(tmpfolder, name)' match parameters
# Argument types: (TypeVar['AnyStr', str, bytes], Literal['hello.txt'])
print(path)
I think I understand the immediate cause of the problem, but contend it should not be happening. Given that, I have some questions:
Is this the idiomatic way to write this code?
Is the problem in tempfile, os, pyright, or me?
If I cannot upgrade Python, what is the best (i.e. least clunky) way to suppress the error?
This seems like a limitation of pyright.
In short, the tempfile.TemporaryDirectory class is typed to be generic with respect to AnyStr. However, your example code omits specifying the generic type, leaving it up to the type checker to infer something appropriate.
In this case, I think there are several reasonable things for a type checker to do:
Pick some default generic type based on the typevar, such as 'str' or 'Union[str, bytes]'. For example, mypy ends up picking 'str' by default, giving 'tmpfolder' a type of 'str'.
Pick some placeholder type like either 'Any', the dynamic type, or NoReturn (aka 'bottom' aka 'nothing'). Both types are a valid subtype of every type, so are guaranteed to be valid placeholders and not cause downstream errors. This is what pyre and pytype does -- they deduce 'tmpfolder' has type 'Any' and 'nothing' respectively.
Attempt to infer the correct type based on context. Some type checkers may attempt to do this, but I don't know of any that handles this particular case perfectly.
Report an error and ask the user to specify the desired generic type.
What pyright seems to do instead is to just "leak" the generic variable. There is perhaps a principled reason why pyright is deciding to do this that I'm overlooking, but IMO this seems like a bug.
To answer your other questions, your example program is idiomatic Python, and type checkers should ideally support it without modification.
Adding a # type: ignore comment to the line with the error is the PEP 484 sanctioned way of suppressing error messages. I'm not familiar enough with pyright to know if it has a different preferred way of suppressing errors.
I'm writing a class named "MyObject".
one of the class methods is:
addTo: aCodeString assertType: aTypeCollection
when the method is called with aCodeString, I want to add (in runtime) a new method to "MyObject" class which aCodeString is it's source code and inject type checking code into the source code.
for example, if I call addTo: assertType: like that:
a := MyObject new.
a addTo: 'foo: a boo:b baz: c
^(a*b+c)'
assertType: #(SmallInteger SmallInteger SmallInteger).
I expect that I could write later:
answer := (a foo: 2 boo: 5 baz: 10).
and get 20 in answer.
and if I write:
a foo: 'someString' boo: 5 baz: 10.
I get the proper message because 'someString' is not a SmallInteger.
I know how to write the type checking code, and I know that to add the method to the class in runtime I can use 'compile' method from Behavior class.
the problem is that I want to add the type checking code inside the source code.
I'm not really familiar with all of squeak classes so I'm not sure if I rather edit the aCodeString as a string inside addTo: assertType: and then use compile: (and I don't know how to do so), or that there is a way to inject code to an existing method in Behavior class or other squeak class.
so basically, what I'm asking is how can I inject string into an existing string or to inject code into an existing method.
There are many ways you could achieve such type checking...
The one you propose is to modify the source code (a String) so as to insert additional pre-condition type checks.
The key point with this approach is that you will have to insert the type checking at the right place. That means somehow parsing the original source (or at least the selector and arguments) so as to find its exact span (and the argument names).
See method initPattern:return: in Parser and its senders. You will find quite low level (not most beautiful) code that feed the block (passed thru return: keyword) with sap an Array of 3 objects: the method selector, the method arguments and the method precedence (a code telling if the method is connected to unary, binary or keyword message). From there, you'll get enough material for achieving source code manipulation (insert a string into another with copyReplace:from:to:with:).
Do not hesitate to write small snippets of code and execute in the Debugger (select code to debug, then use debug it menu or ALT+Shift+D). Also use the inspectors extensively to gain more insight on how things work!
Another solution is to parse the whole Abstract Syntax Tree (AST) of the source code, and manipulate that AST to insert the type checks. Normally, the Parser builds the AST, so observe how it works. From the modified AST, you can then generate new CompiledMethod (the bytecode instructions) and install it in methodDictionary - see the source code of compile: and follow the message sent until you discover generateMethodFromNode:trailer:. This is a bit more involved, and has a bad side effect that the source code is now not in phase with generated code, which might become a problem once you want to debug the method (fortunately, Squeak can used decompiled code in place of source code!).
Last, you can also arrange to have an alternate compiler and parser for some of your classes (see compilerClass and/or parserClass). The alternate TypeHintParser would accept modified syntax with the type hints in source code (once upon a time, it was implemented with type hints following the args inside angle brackets foo: x <Integer> bar: y <Number>). And the alternate TypeHintCompiler would arrange to compile preconditions automatically given those type hints. Since you will then be very advanced in Squeak, you will also create special mapping between source code index and bytecodes so as to have sane debugger and even special Decompiler class that could recognize the precondition type checks and transform them back to type hints just in case.
My advice would be to start with the first approach that you are proposing.
EDIT
I forgot to say, there is yet another way, but it is currently available in Pharo rather than Squeak: Pharo compiler (named OpalCompiler) does reify the bytecode instructions as objects (class names beginning with IR) in the generation phase. So it is also possible to directly manipulate the bytecode instructions by proper hacking at this stage... I'm pretty sure that we can find examples of usage. Probably the most advanced technic.
I wrote most of my unit test with the help of the unittest module, but I’m not sure how to use it for code that the compiler should reject at compile time. For example, if I want to write the following code and make sure the compiler always errors during compilation (the type and template would be in a separate module), how do I write a test case for this?
import macros
type
T[n:static[int]] = object
template foo3(n: int): expr =
static:
if n > 3: error "n > 3"
type T3 = T[n]
T3
var
bar: foo3(4)
You can do something similar with the compiles magic, provided by the system module.
Here is an example from the compiler test suite:
https://github.com/nim-lang/Nim/blob/devel/tests/metatype/tbindtypedesc.nim#L19
Notice how at the top of the file, we define accept and reject as simple static assertions using the compiles magic and we use them throughout the file to test valid and invalid overloaded calls.
Personally, I think failing at compile-time is better, but you can assign the result of compiles to a run-time value or use it in a check statement. The only advantage of this would be that failures will be reported in the standard way for the unittest library.
Just to add an example of how to combine check with compiles:
template notCompiles*(e: untyped): untyped =
not compiles(e)
# usage in unit tests:
check:
notCompiles:
does not compile
notCompiles:
let x = 1 # would fail
I'm using a template, because combining the not with the block directly is not possible, and I don't want to use parentheses.
In https://github.com/shaunc/cucumber_nim/blob/b1601a795dbf8ea0d0b5d96bf5b6a1cd90271327/tests/steps/dynmodule.nim
I have a wrapper that compiles and loads a nim source module. Of course, you won't want to run it, but the technique might work for you:
sourceFN = "foo.nim"
... (write source) ...
libFN = "foo.dll"
let output = execProcess(
"nim c --verbosity:0 --app:lib $1" % sourceFN,
options = {poStdErrToStdOut, poUsePath, poEvalCommand})
if not fileExists(libFN):
echo "COULDN'T COMPILE"
I have a user-editable Excel file in a document repository that defines some inputs for an F# program. I'd like to read it using something F#-y, so I thought I would try out FSharpX and their ExcelFile type provider.
What works
The provider is installed via NuGet and enabled, and this works:
open FSharpX
type Example = ExcelFile<"example.xlsx", "Worksheet1", true>
let file = new Example()
for row in File.Data do ......
What doesn't
However, when I try to initialize the constructor with a different file (one I pull out of a database at runtime and stash in a temporary location), I get a really strange type error.
let file = new Example(#"c:\temp\path\to.xlsx")
results in
The type provider 'FSharpx.TypeProviders.ExcelProvider+ExcelProvider'
reported an error in the context of provided type
'FSharpx.ExcelFile,filename="example.xlsx",sheetname="Worksheet1",forcestring="True"',
member '.ctor'. The error: Type mismatch when splicing expression into
quotation literal. The type of the expression tree being inserted
doesn't match the type expected by the splicing operation. Expected
'System.String', but received type
'System.Tuple`2[System.String,System.String]'. Consider
type-annotating with the expected expression type, e.g., (%% x :
string) or (%x : string). Parameter name: receivedType
Huh?
I have no idea where the tuple it's talking about could be coming from, and I don't have any other ideas about how to initialize this.
Bonus question: What if I wanted to vary over the worksheet name at runtime? The existing FSharpx provider does not seem to allow that.
That's a bug in the provider, not your code. The provided constructor is using the filename you passed in, but the underlying code expects a filename and a workbook name (that's where the tuple ought to come from).
Does appear to be a bug. The provided constructor accepting a single filename parameter does not call the internal constructor correctly in the quotation:
ty.AddMember(ProvidedConstructor([ProvidedParameter("filename", typeof<string>)], InvokeCode = fun [filename] -> <## ExcelFileInternal(%%filename) ##>))
....
type ExcelFileInternal(filename, sheetorrangename)
I forked the FSharpx.Excel type provider a while ago, to modify it so that uses ClosedXML (as opposed to office interop), if the file is >= office 2007 (which I note your workbook is). I Also made a few changes to the API, exposing the worksheets (and/or ranges) as provided types, a 'Rows' provided type, and 'Row{n}' types.
type exc = ExcelFile<"C:\\temp\\Template.xlsx",false>
let file = new exc(#"C:\\temp\\Book.xlsx")
for row in file.Sheet1.Rows do
printfn "%s" row.BID
let sht1Row1Col20 = file.Sheet1.Row1.BID
I'm afraid there is no 'Sheets' provided type, though, so you can't vary over the worksheet name at runtime. There is a string[] 'SheetAndRangeNames' type, but that's not going to be much use to you. Should not be particularly tricky to implement, however. You can find it here:
https://github.com/bennylynch/ExcelTypeProvider/