I would like to wrap a data file (~1MB) to golang app and then use that data in os.exec. The app runs on Linux.
How to define the data in the app, as a string or []byte, variable, or Const?
Should be defined in a global scope, or wrapped in a func?
How to pass the data from the app memory to the executed process ?
For building the data file(s) in to your program, you have a number of choices. You are correct in that you could manually copy/paste the data file(s) in to the program as types string, []byte as variables, but there are other packages/applications for go that handle this for you already that can also minimize your app's memory footprint. One example that comes to mind is go-bindata (link to repo) which does just this.
As for using the file in os/exec, I'm assuming you're trying to either pass the entire file to the program using either a file path or a raw string. For file paths, you would have to write the entire file to disk first. If you're trying to pass the entire file contents as a raw string, you can still use go-bindata or a string of the data file as arg in os/exec.Command.
P.S. - go-bindata has not seen updates in a while, so I would encourage you to seek more active forks or alternatives if you're having trouble using it.
Related
I am writing a nodejs app which works with fonts. One action it performs is that it downloads a .ttf font from the web, converts it to a base64 string, deletes the .ttf and uses that string in other stuff. I need the .ttf file stored somewhere, so I convert it. This process takes like 1-2 seconds. I know heroku has an ephemeral file system but I need to store stuff for such a short time. Is there any way I can store my files? Using fs.writeFile currently returns this error:
Error: EROFS: read-only file system, open '/app\test.txt']
I had idea how about you make an action, That would get font, convert it and store it on a global variable before used by another task.
When you want to use it again, make sure you check that global variable already filled or not with that font buffer.
Reference
Singleton
I didn't know that you could store stuff in /tmp directory. It is working for the moment but according to the dyno/ephemeral system, it gets cleaned frequently so I don't know if it may cause other problems in the long run.
I am trying to read a file content after executing a class GetContentsAPI, basically this class GetContentsAPI will write into the file /etc/api/token.
class Main{
require GetContentsAPI
file("/etc/api/token")
}
When I did the above steps, its says Evaluation Error: Error while evaluating a Function Call, Could not find any files from /etc/api. Not sure how to make sure the file is already created before trying to read.
Thanks James
The file() function reads the contents of a file during catalog building. You don't present any details of class GetContentsApi, but all of the standard puppet facilities that write to files (especially, but not limited to, File resources) write during catalog application. Unless you've cooked up something highly customized, the file() function will always read before GetContentsApi writes.
Moreover, in a master / agent setup (which is the only kind supported in current Puppet), catalog building happens on the master, whereas catalog application happens on the target node, which is usually a different machine, so you're unlikely even to be able to read what was written during a previous catalog-building run.
Also, file() just returns the file contents as a string, so it's not very useful to call it without using the return value somehow.
It's not at all clear what you're trying to achieve, but from what I can see, you are not going in a fruitful direction. Perhaps you should take a step back and ask a different question about that.
I am using the latest version of pyRevit, v45.
I'm writing some info in temporary files with
myTempFile = script.get_instance_data_file("id")
This creates a file named pyRevit_2018_xxxx_id.tmp in which I store useful info. If I'm not mistaken, the "xxxx" part is changing every time I reload Revit. Now, I need to get access to this information from another pyRevit script.
How can I retrieve the name of the temp file I need to read? In other words, how do I access "myTempFile" from within the second script, which has no idea of the name of "myTempFile"?
I guess I can share somehow that variable between my script, but what's the proper way to do this? I know this must be a very basic programming question, but I'm indeed not a programmer ;)
Thanks a lot,
Arnaud.
Ok, I realise now that my variables in the 1st script cease to exist after its execution.
So for now I wrote the file name in another file, of which I know the name.. That works.
But if there's a cleaner way to do this, I'd be glad to learn ;)
Arnaud
pyrevit.script module provides 4 different methods for creating temporary files based on their use case:
get_instance_data_file:
for data files marked with Revit instance pid. This means that scripts running on another instance will not see this temp file.
http://pyrevit.readthedocs.io/en/latest/pyrevit/script.html#pyrevit.script.get_instance_data_file
get_universal_data_file:
for temp files accessible to all Revit instances and versions
http://pyrevit.readthedocs.io/en/latest/pyrevit/script.html#pyrevit.script.get_universal_data_file
get_data_file:
Base method to get a standard temp file for current revit version
http://pyrevit.readthedocs.io/en/latest/pyrevit/script.html#pyrevit.script.get_data_file
get_document_data_file:
temp file marked with active document (so scripts working on another document will not see this)
http://pyrevit.readthedocs.io/en/latest/pyrevit/script.html#pyrevit.script.get_document_data_file
Each method uses a pattern to create the temp file name. So as long as the call to the method is the same of different scripts, the method generates the same file name.
Example:
Script 1:
from pyrevit import script
tfile = script.get_data_file('mydata')
Script 2:
from pyrevit import script
tempfile = script.get_data_file('mydata')
In this example tempfile = tfile since the file id is the same.
There is documentation on each so make sure you take a look at those and pick the flavor that serves your purpose.
I have a rather peculiar file format to work with:
Every line begins with the checksum of its content, followed by a new-line-character.
It looks like this:
[CHECKSUM OF LINE_1][LINE_1]\n
[CHECKSUM OF LINE_2][LINE_2]\n
[CHECKSUM OF LINE_3][LINE_3]\n
...
My goal: To allow any application to work with these files like they would work with any other text file - unaware of the additional checksums at the beginning of each line.
Since I work on a linux machine with debian wheezy (kernel 3.18.26) I want to use the LD_PRELOAD-mechanism to override the relevant file functions.
I have seen something like this with zlibc on https://zlibc.linux.lu/index.html - with an explanation of how it works ( https://zlibc.linux.lu/zlibc.html#SEC8 ).
But I dont get it. They only replace the file-opening functions. No read. No write. no fseek. Nothing. So how does it work?
Or - which functions would I have to intercept to handle every read or write operation on this file and handle them accordingly?
I didn't exactly check how it works but the reason seems to be quite simple.
Possible implementation:
zlibc open:
uncompress file you wanted to open to some temporary file
open this temporary file instead of yours
zlibc close:
Compress temporary file
Override original file
In this case you don't need to override read/write/etc because you can use original ones.
In your case you have two possible solutions:
open, that make a copy of your file with striped checksums. close that calculates checksums and override original file
read and write that are able to skip/calculate checksums.
Ad 2.
From What is the difference between read() and fread()?:
fread() is part of the C library, and provides buffered reads. It is
usually implemented by calling read() in order to fill its buffer
In this case I believe that overriding open and close will be less error prone because you can safely reuse original read, write, fread, fseek etc.
I'll explain the task requested from me:
I have two containers in Azure, one called "data" and one called "script". In the "data" container there's a txt file with data, and in the "script" container there's a script file.
Now, I need programatically (with WorkerRole) to execute the script file, with the content of the data file as parameters (Example: a script file that accepts a string 's' and returns to the screen "Hello, 's'", when 's' in the string given, and in the data file there's a string), and save the result of the run into another file which needs to be saved in another container called "result".
How do I do all these? I've already uploaded the files and created the blobs programatically, but I can't seem to understand how to execute the file of how to save its result to another file?
Can I please have some help?
Thanks in advance
Here are the steps in pseudo code:
Retrieve the script from the blob(using DownloadToStream())
Compile the script(I will leave this to you as I have no idea what
format your script is)
Load parameters from blob(same as step 1)
Execute script with those parameters.
If your script's can be written as lambda expressions then this becomes a lot easier as you can turn them into Action's
Edit based on your questiions:
DownloadText() is no longer included in Azure Storage 2.0, you only have access to DownloadToStream(). Even if you are using an older version(say 1.7) I would recommend using DownloadToStream() in the event you ever upgrade in the future. This will prevent having to refactor your code.
In terms of executing your script, depending on what type of script it is(if it is c# code you can use this example: Is it possible to dynamically compile and execute C# code fragments?. If you need to execute a different type of script you would need to run it using Process.Start and you can look at this example: http://www.dotnetperls.com/process-start
I do not have much experience with point number 2 but those are the processes I have heard and seen used.