How to have custom enconding for struct using Jason? - struct

Background
I am trying to encode a structure into json format using the Jason library. However, this is not working as expected.
Code
Let's assume I have this struct:
defmodule Test do
defstruct [:foo, :bar, :baz]
end
And that when using Jason.enconde(%Test{foo: 1, bar: 2, baz:3 }) I want this json to be created:
%{"foo" => 1, "banana" => 5}
Error
It is my understanding that to achieve this I need to implement the Jason.Enconder protocol in my struct:
https://hexdocs.pm/jason/Jason.Encoder.html
defmodule Test do
defstruct [:foo, :bar, :baz]
defimpl Jason.Encoder do
#impl Jason.Encoder
def encode(value, opts) do
Jason.Encode.map(%{foo: Map.get(value, :foo), banana: Map.get(value, :bar, 0) + Map.get(value, :baz, 0)}, opts)
end
end
end
However, this will not work:
Jason.encode(%Test{foo: 1, bar: 2, baz: 3})
{:error,
%Protocol.UndefinedError{
description: "Jason.Encoder protocol must always be explicitly implemented.\n\nIf you own the struct, you can derive the implementation specifying which fields should be encoded to JSON:\n\n #derive {Jason.Encoder, only: [....]}\n defstruct ...\n\nIt is also possible to encode all fields, although this should be used carefully to avoid accidentally leaking private information when new fields are added:\n\n #derive Jason.Encoder\n defstruct ...\n\nFinally, if you don't own the struct you want to encode to JSON, you may use Protocol.derive/3 placed outside of any module:\n\n Protocol.derive(Jason.Encoder, NameOfTheStruct, only: [...])\n Protocol.derive(Jason.Encoder, NameOfTheStruct)\n",
protocol: Jason.Encoder,
value: %Test{bar: 2, baz: 3, foo: 1}
}}
From what I understand, it looks like I can only select/exclude keys to serialize, I cannot transform/add new keys.
Since I own the structure in question, using Protocol.derive is not necessary.
However I fail to understand how I can leverage the Jason.Encoder protocol to achieve what I want.
Questions
Is my objective possible using the Jason library, or is this a limitation?
Am I miss understanding the documentation and doing something incorrect?

My guess is, this is due to writing the protocol inside a test file. Protocol consolidation happens before the test file executes, so the protocol never becomes part of the compiled codebase.
To elaborate with an example...
I did the following in a Phoenix app
into the lib folder, I added foo.ex
defmodule Foo do
defstruct [:a, :b]
defimpl Jason.Encoder do
def encode(%Foo{a: a, b: b}, opts) do
Jason.Encode.map(%{"a" => a, "b" => b}, opts)
end
end
end
in the test folder, I added foo_test.exs
defmodule FooTest do
use ExUnit.Case
defmodule Bar do
defstruct [:c, :d]
defimpl Jason.Encoder do
def encode(%Bar{c: c, d: d}, opts) do
Jason.Encode.map(%{"c" => c, "d" => d}, opts)
end
end
end
test "encodes Foo" do
%Foo{a: 1, b: 2} |> Jason.encode!() |> IO.inspect()
end
test "encodes Bar" do
%Bar{c: 5, d: 6} |> Jason.encode!()
end
end
Running this test fule, results in the "encodes Foo" passing, but "encodes Bar" fails with a warning
warning: the Jason.Encoder protocol has already been consolidated, an implementation for FooTest.Bar has no effect. If you want to implement protocols after compilation or during tests, check the "Consolidation" section in the Protocol module documentation
followed by an error in the test
** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for %FooTest.Bar{c: 5, d: 6} of type FooTest.Bar (a struct), Jason.Encoder protocol must always be explicitly implemented.
This is because of protocol consolidation happening, causing the Bar protocol to not be compiled.
You can turn off protocol consolidation in the test environment, by adding the following to mix.exs
def project do
# ...
consolidate_protocols: Mix.env() != :test,
#...
end
If you do that, the protocol will compile and both tests will pass.
However, the solution is probably to just not write the struct/protocol directly in the test file.

Related

post test execution callbacks available?

I am looking to apply a callback post test execution that will check for an alarm flag. I don't see any listed here so I then checked the test interface and only see what looks like a flow level callback:
# This will be called at the end of every flow or sub-flow (at the end of every
# Flow.create block).
# Any options passed to Flow.create will be passed in here.
# The options will contain top_level: true, whenever this is called at the end of a
# top-level flow file.
def shutdown(options = {})
end
We need the ability to check the alarm flags after every test but still apply a common group ID to a list of tests like this:
group "func tests", id: :func do
[:minvdd, :maxvdd].each do |cond|
func :bin1_1200, ip: :cpu, testmode: :speed, cond: cond
end
end
Here is an example of the V93K alarm flow flag:
thx!
It is common when writing interfaces to funnel all test generation methods through a common single method to add them to the flow:
def func(name, options = {})
t = test_suites.add(name)
t.test_method = test_methods.origen.functional_test(options)
add_to_flow(t, options)
end
def para(name, options = {})
t = test_suites.add(name)
t.test_method = test_methods.origen.parametric_test(options)
add_to_flow(t, options)
end
def add_to_flow(test_obj, options = {})
# Here you can do anything you want before adding each test to the flow
flow.test(test_obj, options)
# Here you can do anything you want after adding each test to the flow
end
So while there is no per-test callback, you can generally achieve whatever you wanted to do with one via the above interface architecture.
EDIT:
With reference to the alarm flag flow structure you want to create, you would code it like this:
func :some_func_test, id: :sft1
if_failed :sft1 do
bin 10, if_flag: "Alarm"
bin 11, unless_flag: "Alarm"
end
Or, if you prefer, this is equivalent:
func :some_func_test, id: :sft1
bin 10, if_flag: "Alarm", if_failed: :sft1
bin 11, unless_flag: "Alarm", if_failed: :sft1
At the time of writing, that will generate something logically correct but with a sub-optimal branch structure.
In the next release that will be fixed, see the test case that has been added here and the output it generates here.
You can call all of the flow control methods from the interface the same way you can from within the flow, so you can inject such conditions in the add_to_flow method if you want.
Note also that in the test case both if_flag and if_enable are used. if_enable should generally be used if the flag is something that would be set at the start of the flow (e.g. by the operator) and would not change. if_flag should be used if it is a flag that is subject to modification by the flow at runtime.

Programmatically alter Elixir Code

I need to load, alter and write the code in a mix.exs file. I want to be able to load the file, write the dependencies and write the file.
I start with:
defmodule Elixir_2ndTest.Mixfile do
use Mix.Project
def project do
[app: :elixir_2nd_test,
version: "0.0.1",
elixir: "~> 1.2",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
description: description(),
deps: deps]
end
def application do
[applications: [:logger]]
end
defp deps do
[]
end
end
And I need to end up with (the only difference is in the deps fun):
defmodule Elixir_2ndTest.Mixfile do
use Mix.Project
def project do
[app: :elixir_2nd_test,
version: "0.0.1",
elixir: "~> 1.2",
build_embedded: Mix.env == :prod,
start_permanent: Mix.env == :prod,
description: description(),
deps: deps]
end
def application do
[applications: [:logger]]
end
defp deps do
[{:httpoison, "~> 0.8.3"}]
end
end
The dependencies come from a different build system (I cannot use hex directly form the public internet, so I use it in OFFLINE mode and drop the dependencies in .hex/
I know what teh depenencies and what the versions are an need to insert them in the deps function (in this case httpoison 0.8.3).
If my understanding is correct this should be possible by loading the file, quoting, altering, unquoting.
This is what I have up until this point:
{:ok, body} = File.read("mix.exs")
{:ok, ast} = Code.string_to_quoted(body)
Any pointer on how I can alter the ast and write it back would be appreciated.
It won't look exactly the same, but you can use Macro.to_string to convert the ast back to elixir code.
I was playing around with using my library PhStTransform to modify the ast and convert it back to code. Here's a very simple example from the PhStTransform test library.
test "transform quote do output" do
data = quote do: Enum.map(1..3, fn(x) -> x*x end)
data_transform = quote do: Enum.map(1..3, fn(y) -> y*y end)
replace_x = fn(a, _d ) ->
case a do
:x -> :y
atom -> atom
end
end
potion = %{ Atom => replace_x }
assert PhStTransform.transform(data, potion) == data_transform
end
What that does is convert all references to :x in the ast into :y. You'd need to be a bit more clever with writing the potion for PhStTransform, but I think it should be possible. PhStTransform is in hex.pm.
https://hex.pm/packages/phst_transform
I'm not an Elixir expert, but I know about transforming source code; see my bio.
If you have access to the AST as a data structure, you can always write procedural code to climb over it and hack at where you want something different. I assume if Elixir will give you the AST, it will give you access/modification procedures for working with it. This is compiler 101.
That's usually NOT pretty code to write or maintain. And, it may not be enough: you often need more than just the AST to do serious analysis and transformation. See my essay on Life After Parsingl. Think of this as compiler 102.
One the first stumbling blocks is regenerating text from the AST. Here is my SO discussion on how to prettyprint an AST, and why it is harder than it looks: https://stackoverflow.com/a/5834775/120163
(Sounds like Fred the Magic Wonder Dog didn't think what Elixir offered was enough and is inventing his own extensions to make this easier.).

How to avoid Dialyzer errors for protocols?

A simple protocol yields two kinds of dialyzer warnings:
defmodule Dtest do
defprotocol Valid do
#doc "Returns true if data is in a valid state"
def valid?(data)
end
defimpl Valid, for: Integer do
def valid?(_), do: true
end
end
The warning I can't figure out is this:
dtest.ex:2: The specification for
'Elixir.Dtest.Valid':'__protocol__'/1 states that the function might
also return 'true' but the inferred return is
'Elixir.Dtest.Valid' | 'false' | [{'valid?',1},...]
I also couldn't figure out a #spec that'd work here to silence the warning.
The other kind of warning has been discussed elsewhere – many "unknown functions" listed:
Unknown functions:
'Elixir.Dtest.Valid.Atom':'__impl__'/1
'Elixir.Dtest.Valid.BitString':'__impl__'/1
(etc.)
Is there a #spec that can be used with defprotocol's? I haven't found any examples. Or, is there a way, in the source code to mark the defprotocol to be ignored by dialyzer?
EDIT: Here's the full fix for the first error:
defmodule Dtest do
defprotocol Valid do
#doc "Returns true if data is in a valid state"
#dialyzer {:nowarn_function, __protocol__: 1}
def valid?(data)
end
defimpl Valid, for: Integer do
def valid?(_), do: true
end
end
I'm using
#dialyzer {:nowarn_function, __protocol__: 1}
in the protocol definition for now.

What does it mean by _._ in Groovy/Spock

I have seen _._ in the source code of a few Spock Specs. Does anyone know what that means?
For example,
def "test something"() {
given:
someClass = Mock(SomeClass)
and:
1 * someClass.someMethod() >> returnSomething
0 * _._
when:
String str = someClass.someMethod().toString()
then:
str == 'returnedValue'
}
_ is like a wildcard.
_.someMethod()
Means the method someMethod called on any mocked object, likewise
myDomain._
Means anything called on myDomain.
And following that logic _._ means anything called on any mock object. This can be useful if you don't want any more methods called:
0 * _._
Extra Reading
You should read the docs for some more information on the kinds of things _ can be used for.

Are there equivalents to Ruby's method_missing in other languages?

In Ruby, objects have a handy method called method_missing which allows one to handle method calls for methods that have not even been (explicitly) defined:
Invoked by Ruby when obj is sent a message it cannot handle. symbol is the symbol for the method called, and args are any arguments that were passed to it. By default, the interpreter raises an error when this method is called. However, it is possible to override the method to provide more dynamic behavior. The example below creates a class Roman, which responds to methods with names consisting of roman numerals, returning the corresponding integer values.
class Roman
def romanToInt(str)
# ...
end
def method_missing(methId)
str = methId.id2name
romanToInt(str)
end
end
r = Roman.new
r.iv #=> 4
r.xxiii #=> 23
r.mm #=> 2000
For example, Ruby on Rails uses this to allow calls to methods such as find_by_my_column_name.
My question is, what other languages support an equivalent to method_missing, and how do you implement the equivalent in your code?
Smalltalk has the doesNotUnderstand message, which is probably the original implementation of this idea, given that Smalltalk is one of Ruby's parents. The default implementation displays an error window, but it can be overridden to do something more interesting.
PHP objects can be overloaded with the __call special method.
For example:
<?php
class MethodTest {
public function __call($name, $arguments) {
// Note: value of $name is case sensitive.
echo "Calling object method '$name' "
. implode(', ', $arguments). "\n";
}
}
$obj = new MethodTest;
$obj->runTest('in object context');
?>
Some use cases of method_missing can be implemented in Python using __getattr__ e.g.
class Roman(object):
def roman_to_int(self, roman):
# implementation here
def __getattr__(self, name):
return self.roman_to_int(name)
Then you can do:
>>> r = Roman()
>>> r.iv
4
I was looking for this before, and found a useful list (quickly being overtaken here) as part of the Merd project on SourceForge.
Construct Language
----------- ----------
AUTOLOAD Perl
AUTOSCALAR, AUTOMETH, AUTOLOAD... Perl6
__getattr__ Python
method_missing Ruby
doesNotUnderstand Smalltalk
__noSuchMethod__(17) CoffeeScript, JavaScript
unknown Tcl
no-applicable-method Common Lisp
doesNotRecognizeSelector Objective-C
TryInvokeMember(18) C#
match [name, args] { ... } E
the predicate fail Prolog
forward Io
With footnotes:
(17) firefox
(18) C# 4, only for "dynamic" objects
JavaScript has noSuchMethod, but unfortunately this is only supported by Firefox/Spidermonkey.
Here is an example:
wittyProjectName.__noSuchMethod__ = function __noSuchMethod__ (id, args) {
if (id == 'errorize') {
wittyProjectName.log("wittyProjectName.errorize has been deprecated.\n" +
"Use wittyProjectName.log(message, " +
"wittyProjectName.LOGTYPE_ERROR) instead.",
this.LOGTYPE_LOG);
// just act as a wrapper for the newer log method
args.push(this.LOGTYPE_ERROR);
this.log.apply(this, args);
}
}
Perl has AUTOLOAD which works on subroutines & class/object methods.
Subroutine example:
use 5.012;
use warnings;
sub AUTOLOAD {
my $sub_missing = our $AUTOLOAD;
$sub_missing =~ s/.*:://;
uc $sub_missing;
}
say foo(); # => FOO
Class/Object method call example:
use 5.012;
use warnings;
{
package Shout;
sub new { bless {}, shift }
sub AUTOLOAD {
my $method_missing = our $AUTOLOAD;
$method_missing =~ s/.*:://;
uc $method_missing;
}
}
say Shout->bar; # => BAR
my $shout = Shout->new;
say $shout->baz; # => BAZ
Objective-C supports the same thing and calls it forwarding.
This is accomplished in Lua by setting the __index key of a metatable.
t = {}
meta = {__index = function(_, idx) return function() print(idx) end end}
setmetatable(t, meta)
t.foo()
t.bar()
This code will output:
foo
bar
In Common Lisp, no-applicable-method may be used for this purpose, according to the Common Lisp Hyper Spec:
The generic function no-applicable-method is called when a generic function is invoked and no method on that generic function is applicable. The default method signals an error.
The generic function no-applicable-method is not intended to be called by programmers. Programmers may write methods for it.
So for example:
(defmethod no-applicable-method (gf &rest args)
;(error "No applicable method for args:~% ~s~% to ~s" args gf)
(%error (make-condition 'no-applicable-method :generic-function gf :arguments args) '()
;; Go past the anonymous frame to the frame for the caller of the generic function
(parent-frame (%get-frame-ptr))))
C# now has TryInvokeMember, for dynamic objects (inheriting from DynamicObject)
Actionscript 3.0 has a Proxy class that can be extended to provide this functionality.
dynamic class MyProxy extends Proxy {
flash_proxy override function callProperty(name:*, ...rest):* {
try {
// custom code here
}
catch (e:Error) {
// respond to error here
}
}
Tcl has something similar. Any time you call any command that can't be found, the procedure unknown will be called. While it's not something you normally use, it can be handy at times.
In CFML (ColdFusion, Railo, OpenBD), the onMissingMethod() event handler, defined within a component, will receive undefined method calls on that component. The arguments missingMethodName and missingMethodArguments are automatically passed in, allowing dynamic handling of the missing method call. This is the mechanism that facilitated the creation of implicit setter/getter schemes before they began to be built into the various CFML engines.
Its equivalent in Io is using the forward method.
From the docs:
If an object doesn't respond to a message, it will invoke its "forward" method if it has one....
Here is a simple example:
Shout := Object clone do (
forward := method (
method_missing := call message name
method_missing asUppercase
)
)
Shout baz println # => BAZ
/I3az/
Boo has IQuackFu - there is already an excellent summary on SO at how-can-i-intercept-a-method-call-in-boo
Here is an example:
class XmlObject(IQuackFu):
_element as XmlElement
def constructor(element as XmlElement):
_element = element
def QuackInvoke(name as string, args as (object)) as object:
pass # ignored
def QuackSet(name as string, parameters as (object), value) as object:
pass # ignored
def QuackGet(name as string, parameters as (object)) as object:
elements = _element.SelectNodes(name)
if elements is not null:
return XmlObject(elements[0]) if elements.Count == 1
return XmlObject(e) for e as XmlElement in elements
override def ToString():
return _element.InnerText

Resources