Haskell FFI stack size with a C++ shared library - linux

I am trying to send a very large amount of data, on the order of several Gigabytes to Haskell via a shared library. Everything seems to work fine up to a point, but I eventually get the error:
Stack space overflow: current size 8388608 bytes.
Use `+RTS -Ksize' to increase it.
So basically, my question is: How do I increase the Haskell stack size?
I have tried:
1) Passing the appropriate -rtsopts flags to the linker when building the shared library -- No luck, but this is expected because I am using the Haskell as a shared library.
2) Calling hs_init with the RTS flags as described in this link:
Haskell User Guide FFI/GHC...which did not work either.
int argc = 2;
char *argv[] = { "+RTS", "-K100M", NULL };
char **pargv = argv;
// Initialize Haskell runtime
hs_init(&argc, &pargv);
Basically I think that these arguments are also ignored because I am not using a Haskell main, which is also described in the user guide link I provided.
NOTE: I have also tried this approach adding a "-RTS" to the list before NULL and increasing argc but it made no difference.
3) I suspect that the true solution to this problem would be by following section 8.2.1.1 in the given link and making the special call to hs_init_ghc as so:
#include "HsFFI.h"
#include "Rts.h"
RtsConfig conf = defaultRtsConfig;
conf.rts_opts_enabled = RtsOptsAll;
hs_init_ghc(&argc, &argv, conf);
However when I try to do this as described in the guide, my C code that calls:
hs_init_ghc
will NOT compile due to a whole bunch of errors which I have pasted into the bottom of this question. It seems that including RTS.h will not compile, but I am following the example exactly, and compiling with:
ghc -c foo.cpp
I am wondering if anyone has any advice on how to get around this compilation error or to set the stack-size to a bigger value when using the Haskell as a shared library from C++?
I have also tried setting the corresponding environment variable, which also did not work. I even got a message saying it will be ignored.
I am using GHC version 7.4.1
Thanks very much for your help.
--JL
Compilation errors are:
>
In file included from /usr/lib/ghc/include/Rts.h:222:0:
0, from foo.cpp:6:
/usr/lib/ghc/include/rts/storage/Block.h:227:26:
error: expected ‘,’ or ‘...’ before ‘new’
dbl_link_replace(bdescr *new, bdescr *old, bdescr **list)
^
/usr/lib/ghc/include/rts/storage/Block.h: In function ‘void dbl_link_replace(bdescr*)’:
/usr/lib/ghc/include/rts/storage/Block.h:229:8:
error: expected type-specifier before ‘->’ token
new->link = old->link;
^
/usr/lib/ghc/include/rts/storage/Block.h:229:8:
error: expected ‘;’ before ‘->’ token
/usr/lib/ghc/include/rts/storage/Block.h:230:8:
error: expected type-specifier before ‘->’ token
new->u.back = old->u.back;
^
/usr/lib/ghc/include/rts/storage/Block.h:230:8:
error: expected ‘;’ before ‘->’ token
/usr/lib/ghc/include/rts/storage/Block.h:231:9:
error: ‘old’ was not declared in this scope
if (old->link) {
^
/usr/lib/ghc/include/rts/storage/Block.h:232:32:
error: expected type-specifier before ‘;’ token
old->link->u.back = new;
^
/usr/lib/ghc/include/rts/storage/Block.h:234:9:
error: ‘old’ was not declared in this scope
if (old->u.back) {
^
/usr/lib/ghc/include/rts/storage/Block.h:235:32:
error: expected type-specifier before ‘;’ token
old->u.back->link = new;
^
/usr/lib/ghc/include/rts/storage/Block.h:237:10:
error: ‘list’ was not declared in this scope
*list = new;
^
/usr/lib/ghc/include/rts/storage/Block.h:237:20:
error: expected type-specifier before ‘;’ token
*list = new;
^
In file included from /usr/lib/ghc/include/Rts.h:218:0:
0,
from foo.cpp:6:
/usr/lib/ghc/include/rts/storage/ClosureMacros.h: In function ‘rtsBool
More compile errors removed...but available upon request.

Related

Undeclared ptep_t in linux_kernel system call

I'm working with the following system call in the Linux kernel that takes a virtual address of a process and outputs the following information:
If the data in this address is in memory or on disk.
If the page which this address belongs to has been referenced or not.
If the page which this address belongs to is dirty or not.
#include<linux/kernel.h>
#include<linux/sched.h>
#include<asm/page.h>
#include<asm/pgtable.h>
#include<linux/mm_types.h>
asmlinkage int sys_vma_props(unsigned long mem,int pid)
{
struct task_struct *task=find_task_by_vpid(pid);
struct mm_struct *memory=task->active_mm;
int data=0;
int ref=0;
int dirty =0;
pgd_t *pgd=pgd_offset(memory,mem);
pud_t *pud=pud_offset(pgd,mem);
pmd_t *pmd=pmd_offset(pud,mem);
ptet_t *ptep=pte_offset_kernel(pmd,mem);
pte_t pte=*ptep;
data=pte_present(pte);
printk("present flag: %i\n",data?1:0);
ref=pte_young(pte);
printk("referenced flag: %i\n",ref?1:0);
dirty=pte_dirty(pte);
printk("dirty flag: %i\n",dirty?1:0);
return 0;
}
However, I'm getting the following error for the ptep_t variable and the ptep variable. I've researched and this makes sense to me so I'm not really sure what the problem is. What's causing the error? Any advice would be much appreciated.
address/sys_vadd.c:19: error: ‘ptep_t’ undeclared (first use in this function)
address/sys_vadd.c:19: error: (Each undeclared identifier is reported only once
address/sys_vadd.c:19: error: for each function it appears in.)
address/sys_vadd.c:19: error: ‘ptep’ undeclared (first use in this function)
address/sys_vadd.c:20: warning: ISO C90 forbids mixed declarations and code
The problem is that you've used the wrong type for the ptep variable. It should be pte_t not ptet_t.

Inline C Varnish (VCL_deliver)

I am using Varnish 4.0.
My backend is adding to some responses an http header "x-count"
I would like to log the value of "x-count" into a file with a line break.
I assumed i should do it in VCL deliver.
Here is what i have so far :
sub vcl_deliver {
if (resp.http.x-count-this:) {
set resp.http.X-infodbg = "xx";
C{
FILE *fp;
fp = fopen("/tmp/test.txt", "w+");
fputs(VRT_GetHdr(sp, HDR_OBJ, "\013x-count-this:"), fp);
fputs("\n", fp);
fclose(fp);
}C
}
}
Of course it doesnt work and there is a couple of errors ..
./vcl.gK2lu7uM.c: In function ‘VGC_function_vcl_deliver’:
./vcl.gK2lu7uM.c:1049:22: error: ‘sp’ undeclared (first use in this
function) ./vcl.gK2lu7uM.c:1049:22: note: each undeclared identifier
is reported only once for each function it appears in
./vcl.gK2lu7uM.c:1049:5: error: passing argument 2 of ‘VRT_GetHdr’
makes pointer from integer without a cast [-Werror]
./vcl.gK2lu7uM.c:330:7: note: expected ‘const struct gethdr_s *’ but
argument is of type ‘int’ ./vcl.gK2lu7uM.c:1049:5: error: too many
arguments to function ‘VRT_GetHdr’ ./vcl.gK2lu7uM.c:330:7: note:
declared here
I have to say that i simply copy/pasted "sp" from some examples, but i have no idea where it comes from (i suppose the inline C was in a different context and therefore it was declared there but not in vcl_deliver)
So the probably undocumented differences between Varnish 4 and 3 in the above examples are :
VRT_GetHdr is now VRT_GetHdr(context, struct gethdr_s)
sp doesn't exist, but there is a "ctx" variable
Found this, there :
http://jan.bogutzki.de/Artikel/395/set-ttl-in-varnish-4.html
char *stuffid;
const struct gethdr_s hdr = { HDR_BERESP, "\015x-count-this:" };
stuffid = VRT_GetHdr(ctx, &hdr);
And now a different story: Varnish is crashing as soon as the backend sends back "count-this", but that is a different problem :p (my crappy C code probably)
I don't have Varnish 4.0 handy to test this out, but I was able to get your example working with Varnish 3.0. When I tried the VCL as is, I wasn't getting the exact error you are though. The first change:
if (resp.http.x-count-this:) {
needs to be:
if (resp.http.x-count-this) {
The colon should be left off of the header name when referred to this way. Next:
fputs(VRT_GetHdr(sp, HDR_OBJ, "\013x-count-this:"), fp);
needs to be:
fputs(VRT_GetHdr(sp, HDR_OBJ, "\015x-count-this:"), fp);
The length value in that string needs to be in octal for some reason, and 13 in octal is 15. Making those changes got this to work for me. That being said, you many want to look into using open and fcntl instead of fopen since without file locking I'm not sure what the effect of multiple requests contending for that file would be.

can't build wxwidgets on Cygwin: ambiguous overload for ‘operator[]’ basedll_apebase.o error

I'm willing to bet I'm not configuring it right, but it's not brief on how I need to for wxWidgets_3_0
I get the following error:
$ make
/home/Bill/WX_3_0_BRANCH/build-debug/bk-deps g++ -c -o basedll_appbase.o -D__WXM SW__ -DWXBUILDING -I/home/Tom/WX_3_0_BRANCH/build-debug/src/tiff/libtiff -I ../src/tiff/libtiff -I../src/jpeg -I../src/png -I../src/zlib -I../src/regex -I.. /src/expat/lib -DwxUSE_GUI=0 -DWXMAKINGDLL_BASE -DwxUSE_BASE=1 -Wall -Wundef -W unused-parameter -Wno-ctor-dtor-privacy -Woverloaded-virtual -D_FILE_OFFSET_BITS =64 -I/home/Tom/WX_3_0_BRANCH/build-debug/lib/wx/include/msw-unicode-3.0 -I../in clude -O2 -fno-strict-aliasing ../src/common/appbase.cpp
In file included from ../src/common/appbase.cpp:42:0:
../include/wx/filename.h: In static member function ‘static wxUniChar wxFileName ::GetPathSeparator(wxPathFormat)’:
../include/wx/filename.h:473:43: error: ambiguous overload for ‘operator[]’ (ope rand types are ‘wxString’ and ‘unsigned int’)
{ return GetPathSeparators(format)[0u]; }
In file included from ../include/wx/memory.h:15:0,
from ../include/wx/object.h:19,
from ../include/wx/list.h:32,
from ../src/common/appbase.cpp:30:
../include/wx/string.h:1544:15: note: wxUniChar wxString::operator[](int) const
wxUniChar operator[](int n) const
^
../include/wx/string.h:1546:15: note: wxUniChar wxString::operator[](long int) c onst
wxUniChar operator[](long n) const
^
../include/wx/string.h:1548:15: note: wxUniChar wxString::operator[](size_t) con st
wxUniChar operator[](size_t n) const
^
../include/wx/string.h:1556:18: note: wxUniCharRef wxString::operator[](int)
wxUniCharRef operator[](int n)
^
../include/wx/string.h:1558:18: note: wxUniCharRef wxString::operator[](long int )
wxUniCharRef operator[](long n)
^
../include/wx/string.h:1560:18: note: wxUniCharRef wxString::operator[](size_t)
wxUniCharRef operator[](size_t n)
^
Makefile:28650: recipe for target 'basedll_appbase.o' failed
make: *** [basedll_appbase.o] Error 1
This is what's in the makefile on that line:
basedll_appbase.o: $(srcdir)/src/common/appbase.cpp $(BASEDLL_ODEP)
$(CXXC) -c -o $# $(BASEDLL_CXXFLAGS) $(srcdir)/src/common/appbase.cpp
This is a known bug, but I thought it affected only wxGTK and hence it had a low priority (because subset of people using wxGTK under Cygwin is vanishingly small). If it affects wxMSW as well, it would be nice to fix it, especially as it shouldn't be difficult, and we'll try to do it for the next 3.0.1 release.
Seems that long, unsigned int, and size_t are all expected to exist as different function signatures
So the fix for this case was to add a cast of size_t to any ambiguous overload errors with 0u or 1u.
Here's on example: in the filename.h I edited its source code:
// get the canonical path separator for this format
static wxUniChar GetPathSeparator(wxPathFormat format = wxPATH_NATIVE)
{ return GetPathSeparators(format)[0u]; }
to
// get the canonical path separator for this format
static wxUniChar GetPathSeparator(wxPathFormat format = wxPATH_NATIVE)
{ return GetPathSeparators(format)[(size_t)0u]; }
The second thing I had to do was configure for my 64 bit windows
../configure --host=i686-w64-mingw32 --build=i686-pc-cygwin

c2664 in Visual Studio 2012 when using make_pair

I dig up an old project and wanted to compile it, but received several errors, a few of those being a c2664:
error C2664: 'std::make_pair' : cannot convert parameter 1 from 'CUser *' to 'CUser *&&'
error C2664: 'std::make_pair' : cannot convert parameter 1 from 'unsigned long' to ' unsigned long &&'
The relevant code parts are:
//typedef for the userdata map
typedef std::map<unsigned long, std::pair<CUser*,userstatus*>> UserDataMapType;
//...
Inc::incret CUserManager::AddUser(unsigned long ID, CUser* pUser, userstatus* pUserStatus)
{
//...
std::pair<UserDataMapType::iterator, bool> ret = m_mapUserData.insert(std::make_pair<unsigned long, std::pair<CUser*, userstatus*>>(ID, std::make_pair<CUser*, userstatus*>(pUser, pUserStatus)));
//...
}
I tried to make the function parameters const, but that did not help.
It did compile just fine in VS2010.
Please help me find what causes this and how to solve it.
make_pair() has been changed in VS2012 to support a new C++11 feature called move semantics and I suspect that explicitly specifying the types for make_pair() is getting in the way.
Remember that make_pair() does not need any template parameters to be explicitly specified. It deduces them from the type of each argument.
Try removing the explicit template arguments from both calls to make_pair() like so...
std::pair<UserDataMapType::iterator, bool> ret = m_mapUserData.insert(std::make_pair(ID, std::make_pair(pUser, pUserStatus)));
Explicitly providing them like this would have worked fine pre-VS2012 because of a new C++11 feature added called move semantics. You'll want to read up on that subject later since you have a shiny new compiler that supports it.

Linux kernel compilation error: undefined reference to `__udivdi3' & `__umoddi3'

Here is the error I've got:
http://pastebin.com/VadUW6fy
drivers/built-in.o: In function `gem_rxmac_reset':
clkdev.c:(.text+0x212238): undefined reference to `__bad_udelay'
drivers/built-in.o: In function `divide.part.4':
clkdev.c:(.text.unlikely+0x7214): undefined reference to `__udivdi3'
clkdev.c:(.text.unlikely+0x7244): undefined reference to `__umoddi3'
I googled and found this patch: https://lkml.org/lkml/2008/4/7/82
--- a/include/linux/time.h
+++ b/include/linux/time.h
## -174,6 +174,10 ## static inline void timespec_add_ns(struct timespec *a, u64 ns)
{
ns += a->tv_nsec;
while(unlikely(ns >= NSEC_PER_SEC)) {
+ /* The following asm() prevents the compiler from
+ * optimising this loop into a modulo operation. */
+ asm("" : "+r"(ns));
+
ns -= NSEC_PER_SEC;
a->tv_sec++;
}
but failed to apply (may be due to new version of the file).
patching file linux/time.h
Hunk #1 FAILED at 174.
1 out of 1 hunk FAILED -- saving rejects to file linux/time.h.rej
surprisingly, the file time.h.rej is not present!
I should have read more closely. The patch is for timespec_add_ns(), and you have gem_rxmac_reset() and divide.part.4 functions failing. Probably unrelated to the patch you found -- instead, probably standard 64-bit div / mod functions don't have an implementation on your target platform.
Do you have a Sun GEM or Apple GMAC NIC? If not, you can probably just disable that driver and get rid of the first error message.
For the second, you might need to implement a similar asm trick in the clkdev.c file -- when I skimmed my copy for a repeated subtraction operation I didn't spot one -- but maybe you can simply steal a newer clkdev.c or clkdev.h to fix this problem? (It's a long shot, there's only one entry in git log drivers/clk/clkdev.c.)

Resources