I'm trying to watch spotify packages on the emulator, but the data sent and received are corrupted. How can I solve this problem?
İmages:
I tried reading the data many times but it always looks like this. I want to see the data properly in JSON form.
This data is not corrupted, it's just not in the format you want.
You can see the format by looking at the content-type header, which says application/protobuf.
This is not JSON data. Instead it's Protobuf, which is a general-purpose serialization format, similar to JSON or XML, but designed to be faster to process and smaller to transfer, in part by being sent as raw binary data, instead of readable strings.
To deserialize this 100% correctly, you will need the Protobuf schema for the API you're talking to (a .proto file). In many cases, unless this is your own or a publicly documented API, that's not going to be available.
You can still try to decode the data into raw data types though, although that might not allow you to decode all information immediately. There's more info on that here: raw decoder for protobufs format. Decoding data like this works best using the protoc command-line tool, but you may also be able to decode this data using https://protobuf-decoder.netlify.app/. Note that that takes hex data, not a raw string like you have here, so you'll have to pick the 'Hex' option for the body in HTTP Toolkit to copy the hex codes over instead.
I can only extract data from here, how can I use them with python requests? I want to convert them to dictionary data type. Or can we solve this using https://github.com/spotify/proto-requests? I'm trying to write a program on Spotify.
Related
I am using turtle files containing biographical information for historical research. Those files are provided by a major library and most of the information in the files is not explicit. While people's professions, for instance, are sometimes stated alongside links to the library's URIs, I only have URIs in the majority of cases. This is why I will need to retrieve the information behind them at some point in my workflow, and I would appreciate some advice.
I want to use Python's RDFLib for parsing the .ttl files. What is your recommended workflow? Should I read the prefixes I am interested in first, then store the results in .txt (?) and then write a script to retrieve the actual information from the web, replacing the URIs?
I have also seen that there are ways to convert RDFs directly to CSV, but although CSV is nice to work with, I would get a lot of unwanted "background noise" by simply converting all the data.
What would you recommend?
RDFlib's all about working with RDF data. If you have RDF data, my suggestion is to do as much RDF-native stuff that you can and then only export to CSV if you want to do something like print tabular results or load into Pandas DataFrames. Of course there are always more than one way to do things, so you could manipulate data in CSV, but RDF, by design, has far more information in it than a CSV file can so when you're manipulating RDF data, you have more things to get hold of.
most of the information in the files is not explicit
Better phrased: most of the information is indicated with objects identified by URIs, not given as literal values.
I want to use Python's RDFLib for parsing the .ttl files. What is your recommended workflow? Should I read the prefixes I am interested in first, then store the results in .txt (?) and then write a script to retrieve the actual information from the web, replacing the URIs?
No! You should store the ttl files you can get and then you may indeed retrieve all the other data referred to by URI but, presumably, that data is also in RDF form so you should download it into the same graph you loaded the initial ttl files in to and then you can have the full graph with links and literal values it it as your disposal to manipulate with SPARQL queries.
I work with a tool that contains everything within XML inside the database.
Some reports that are stored in the database use a third party tool to load, and store the main data to configure the 'report' definition in what is not a human-readable format.
I'd post it here, but it's some 130,000 bytes.
I have attempted to decode it using popular methods that I assumed it would have been encoded in, such as base64, base 32, etc, but none have been able to decode the string.
Is there a way to identify what encoding a given string has, using a tool available online?
I don't have the benefit of access to the developer that built this functionality, the source code generating this string, or any documentation on it.
To give some context around what I'm trying to do - I need to reverse-engineer how a specific definition in a system is generated, so that it can be modified slightly (manually) in a text editor to support an operation that would otherwise require manually re-creating the report.
I apologize is if this may be the wrong exchange site for this question - I realize it's not specific to a 'programming' issue and I haven't tried to solve it using a programming language. If so - please redirect me to the appropriate place and I'll be happy to ask there instead.
Update: The text consists of strictly A-Z, 0-9 characters.
You can check amongst known encoding formats with this tool only if you are sure data is not encrypted
Cassandra newbie question. I'm collecting some data from a social networking site using REST calls. So I end up with the data coming back in JSON format.
The JSON is only one of the columns in my table. I'm trying to figure out what the "best practice" is for storing the JSON string.
First I thought of using the map type, but the JSON contains a mix of strings, numerical types, etc. It doesn't seem like I can declare wildcard types for the map key/value. The JSON string can be quite large, probably over 10KB in size. I could potentially store it as a string, but it seems like that would be inefficient. I would assume this is a common task, so I'm sure there are some general guidelines for how to do this.
I know Cassandra has native support for JSON, but from what I understand, that's mostly used when the entire JSON map matches 1-1 with the database schema. That's not the case for me. The schema has a bunch of columns and the JSON string is just a sort of "payload". Is it better to store the JSON string as a blob or as text? BTW, the Cassandra version is 2.1.5.
Any hints appreciated. Thanks in advance.
In the Cassandra Storage engine there's really not a big difference between a blob and a text, since Cassandra stores text as blobs essentially. And yes the "native" JSON support you speak of is only for when your data model matches your JSON model, and it's only in Cassandra 2.2+.
I would store it as a text type, and you shouldn't have to implement anything to compress your JSON data when sending the data (or handle uncompressing). Since Cassandra's Binary Protocol supports doing transport compression. Also make sure your table is storing the data compressed with the same compression algorithm (I suggest using LZ4 since it's the fastest algo implmeneted) to save on doing compression for each read request. Thus if you configure storing the data compressed and use transport compression, you don't even have to implement either yourself.
You didn't say which Client Driver you're using, but here's the documentation on how to setup Transport Compression for Datastax Java Client Driver.
It depends on how to want to query your JSON. There are 3 possible strategies:
Store as a string
Store as a compressed blob
Store as a blob
Option 1 has the advantage of being human readable when you query your data on command line with cqlsh or if you want to debug data directly live. The drawback is the size of this JSON column (10k)
Option 2 has the advantage to keep the JSON payload small because text elements have a pretty decent compression ration. Drawbacks are: a. you need to take care of compression/decompression client side and b. it's not human readable directly
Option 3 has drawbacks of option 1 (size) and 2 (not human readable)
I'm looking for a good strategy to include an image into a service stack model (if it is possible). I've searched for examples or tips, but haven't had much luck.
Something like
class House
{
string Address;
//Image could be anything as long as it can be returned and viewed as a picture in a
browser.
Image Picture;
}
Can this be returned as a part of a Response and properly displayed?
It is rarely a good idea to include binary blob data directly in a DTO payload.
Send Binary Data with Binary Formats
The only formats where it makes sense to do is when using binary formats like ServiceStack's support for ProtoBuf or Message Pack formats which wont penalize you for encoding binary data. In which case you would serialize the image into a byte[] property on the DTO. If you try to do this in a text format you would need to encode the binary data with a Base64-like format, un-necessary bloating the payload and adding computational overhead.
Embed urls to images or binary files instead
The recommended approach is instead of embedding the Image directly in the DTO, to only include a Url that references the image. ServiceStack has great support for returning raw binary or text data where you can simply return any byte[], Image, Stream, IStreamWriter, raw strings, etc and it will serialize them as expected.
ServiceStack's support for raw binary or text responses
A good example of this can be seen in the new Image Resizer imgur.servicestack.net or the RestFiles example demos.
Here's some more info about custom HTTP Responses on the wiki:
Return any response
Customizing HTTP Responses
Consuming raw data with typed service clients
Consuming raw data from 3rd Party APIs with built-in HTTP Utils
Why some applications base 64 encode HMACs even if it's not for mail purposes?
Is it a vestige from older implementations or practices?
If you do a little research in google with the words 'HMAC', 'base' and '64' you will see many people do it, and i don't know why. I'm pretty sure that it has nothing to do neither with storing nor outputting it.
If you want to send binary data encoded into SOAP envelope or as JSON data, you would risk that the part of the raw data is interpreted as the structural elements of these formats (e.g. '<', '[', '{').
Therefore you need to encode it as Base64 to preserve its contents on the wire.