I am trying to filter down the gnomad INFO section to just a few fields (AC_AFR,AC_AMR,AC_FIN,AC_ASJ, etc.). I tried following their example and did
bcftools annotate -x FORMAT,^INFO/AC_AFR,INFO/AC_ASJ.... gnomad_exomes.vcf
But it returned an unknown file type error.
Related
In the documentation, is it not clear how can I use this option?
Is it for telling arangoimport : "Hey, please use this field as _from/_to field when you import" ?
define string… Define key=value for a #key# entry in config file
This has nothing to do with data import. arangod, arangosh etc. all support --define to set environment variables, which can be used in configuration files using placeholders like #FOO# and be set like --define FOO=something on the command line.
This is briefly explained here: https://www.arangodb.com/docs/stable/administration-configuration.html#environment-variables-as-parameters
Example configuration file example.conf:
[server]
endpoint = tcp://127.0.0.1:#PORT#
Example invocation:
arangosh --config example.conf --define PORT=8529
For delimited source files (CSV, TSV) you can use the option --translate to map columns to different attributes, e.g. --translate "child=_from" --translate "parent=_to".
https://www.arangodb.com/docs/stable/programs-arangoimport-examples-csv.html#attribute-name-translation
If the references are just keys, then you may use --from-collection-prefix and to-collection-prefix to prepend the collection name.
--translate is not supported for JSON inputs. You can do the translation and import using a driver, or edit the source file somehow, or import into a collection and then use AQL to adjust the fields.
The variables var1 and var2 in the below command needs to be replaced and replicated with over 500 unique attributes. Complete beginner in python, suggestions ?
(Plan is to read a file with 500 attributes and loop it to replicate the command with all the different attributes found in the file to a second file or print the command output in console.)
command:
dsconfig create-proxy-transformation --transformation-name Attr-Mapping_proxy01 --type attribute-mapping --set enabled:true --set source-attribute:"var1" --set target-attribute:"var2"
file = open('open.txt')
for line in file:
fields = line.strip().split()
print fields[0], fields[1]
Assign values to variables var1 and var2 in above command with for loop with appropriate field values, it will do the rest of stuff.
considering in text file
AA11 BB11 CC11 DD11
AA22 BB22 CC22 DD22
AA33 BB44 CC44 DD33
Python is so easy :)
To be more specific, split() splits the contents of a string into fields delimited by some delimiter (by default any blank character, e.g. space, tab etc.), and returns an array with the split fields. strip() strips all blank characters from the beginning and end of a line. And a file in python is an iterable object which when iterated over by the keyword in, gives the lines in the file one by one. For more information on these, you can look at http://docs.python.org/2/library/stdtypes.html#str.split , http://docs.python.org/2/library/stdtypes.html#str.strip , http://docs.python.org/2/library/stdtypes.html#bltin-file-objects .
link : Reading a file and storing values into variables in python
I've downloaded some JSON data from Shodan, and only want to retain some fields from it. To explore what I want, I'm running the following, which works-
shodan parse --fields ip,port --separator , "data.json.gz"
However, I now want to output/ export the data; I'm trying to run the following -
shodan parse --fields ip,port -O "data_processed.json.gz" "data.json.gz"
It's requiring me to specify a filter parameter, which I don't need. If I do add an empty filter as so, it tells me data_processes.json.gz doesn't exist.
shodan parse --fields ip,port -f -O "data_processed.json.gz" "data.json.gz"
I'm a bit stumped on how to export only certain fields of my data; how do I go about doing so?
If you only want to output those 2 properties then you can simply pipe them to a file:
shodan parse --fields ip,port --separator , data.json.gz > data_processed.csv
A few things to keep in mind:
You probably want to export the ip_str property as it's a more user-friendly version of the IP address. The ip property is a numeric version of the IP address and aimed at users storing the information in a database.
You can convert your data file into Excel or CSV format using the shodan convert command. For example: shodan convert data.json.gz csv See here for a quick guide: https://help.shodan.io/guides/how-to-convert-to-excel
I have some special custom attributes with my ldap setup. I have a custom attribute called "GroupCode". I have bunch of entries with this special attribute that I was able to write to the ldap database. Lets say that I have an entry with attribute "xyz" and another attribute with "wasd". I search with the filter "(GroupCode=xyz)" and "(GroupCode=wasd)" neither one of these search return anything. However, if I change the filter to "(GroupCode=*)", then it would return all the entries that have the GroupCode attribute. I have examined the attribute properties, and it looks normal, the apache directory studio shows it to be of "String" value, do I don't know why it isn't searching with the filter I provided. My knowledge with ldap structure is fairly limited as it is fairly complexed. Anyone have any idea, please let me know. Much appreciated. Thanks.
can you see if you can formulate the same search criteria into an ldapsearch command in the command line?
ldapsearch -H ldap://LDAP_SERVER -D LDAP_AUTH_LOGIN -b LDAP_BASE -w PW -x "CRITERIA"
if so, then you can then also experiment with your criteria.
ldapsearch -H ldap://LDAP_SERVER -D LDAP_AUTH_LOGIN -b LDAP_BASE -w PW -x "(GroupCode=xyz)"
One possible reason for your issue is that you forgot to specify the EQUALITY and SUSBSTR properties of your custom attribute.
Here is an example for a custom attribute called sAMAccountName:
attributeTypes: ( 1.2.840.113556.1.4.221
NAME 'sAMAccountName'
EQUALITY caseIgnoreMatch
SUBSTR caseIgnoreSubstringsMatch
SYNTAX '1.3.6.1.4.1.1466.115.121.1.15'
SINGLE-VALUE )
Is it possible to insert set type data (column) in usergrid (as cassandra support set type column). I tried
curl -XPOST http://localhost:8080/<org>/<app>/<collection>
-d '{"name":"1974", "category":{"a","b","c"}}'
but it reply json_parse error.
Response to answer: I knew that payload in above request isn't valid JSON, I only tried to tell that is there any way I could make set type column (I need to prevent duplicate entries on single column record). with square brackets, It create list type column which don't prevent duplicate entries.
One of core member reply that on current Version (1.0), usergrid don't support set type column.
It certainly is - but your payload isn't valid JSON; in JSON, you use square brackets to specify an Array: [].
Try instead:
curl -X POST http://localhost:8080/<org>/<app>/<collection>
-d '{"name":"1974", "category":["a","b","c"]}'
# ^ ^