I'm looking for a good strategy to include an image into a service stack model (if it is possible). I've searched for examples or tips, but haven't had much luck.
Something like
class House
{
string Address;
//Image could be anything as long as it can be returned and viewed as a picture in a
browser.
Image Picture;
}
Can this be returned as a part of a Response and properly displayed?
It is rarely a good idea to include binary blob data directly in a DTO payload.
Send Binary Data with Binary Formats
The only formats where it makes sense to do is when using binary formats like ServiceStack's support for ProtoBuf or Message Pack formats which wont penalize you for encoding binary data. In which case you would serialize the image into a byte[] property on the DTO. If you try to do this in a text format you would need to encode the binary data with a Base64-like format, un-necessary bloating the payload and adding computational overhead.
Embed urls to images or binary files instead
The recommended approach is instead of embedding the Image directly in the DTO, to only include a Url that references the image. ServiceStack has great support for returning raw binary or text data where you can simply return any byte[], Image, Stream, IStreamWriter, raw strings, etc and it will serialize them as expected.
ServiceStack's support for raw binary or text responses
A good example of this can be seen in the new Image Resizer imgur.servicestack.net or the RestFiles example demos.
Here's some more info about custom HTTP Responses on the wiki:
Return any response
Customizing HTTP Responses
Consuming raw data with typed service clients
Consuming raw data from 3rd Party APIs with built-in HTTP Utils
Related
I'm trying to watch spotify packages on the emulator, but the data sent and received are corrupted. How can I solve this problem?
İmages:
I tried reading the data many times but it always looks like this. I want to see the data properly in JSON form.
This data is not corrupted, it's just not in the format you want.
You can see the format by looking at the content-type header, which says application/protobuf.
This is not JSON data. Instead it's Protobuf, which is a general-purpose serialization format, similar to JSON or XML, but designed to be faster to process and smaller to transfer, in part by being sent as raw binary data, instead of readable strings.
To deserialize this 100% correctly, you will need the Protobuf schema for the API you're talking to (a .proto file). In many cases, unless this is your own or a publicly documented API, that's not going to be available.
You can still try to decode the data into raw data types though, although that might not allow you to decode all information immediately. There's more info on that here: raw decoder for protobufs format. Decoding data like this works best using the protoc command-line tool, but you may also be able to decode this data using https://protobuf-decoder.netlify.app/. Note that that takes hex data, not a raw string like you have here, so you'll have to pick the 'Hex' option for the body in HTTP Toolkit to copy the hex codes over instead.
I can only extract data from here, how can I use them with python requests? I want to convert them to dictionary data type. Or can we solve this using https://github.com/spotify/proto-requests? I'm trying to write a program on Spotify.
The TipTap editor and its progeny quasar-tiptap can export user created content in the browser in both HTML and JSON formats. If I plan on allowing round trips of the user data to my server what are the pros and cons of using either format for storage.
I would assume HTML has a greater likelihood of XSS attacks, and indeed such a vulnerability has been found (and rectified) in the past. And using JSON would be easier for backend parsing should it ever be required.
Beyond this, are there any major benefits of using either format? Preserving fidelity of user input is important. Size is important (any differences in image storage?). Editor performance is important. Scripting attack vulnerability is extremely important.
Which to choose?
I am sure at 12 months later you have made your choice. The usual caveats that this question is not really answerable and can only really elicit opinions. However, the best I can offer (and I am definately no expert):
User entered text is a risk (cross-site scripting, SQL injection attack, potentially creating a false alert dialog with disinformation for the user reading another user's content) regardless of how the rich text data is sent to the server and/or stored. This is because eventually the content will be a) very likely stored in a database b) sent to and interpreted by a browser in HTML format. Whether it goes through an intermediary JSON format is largely irrelevant. Stripping out all unwanted tags, attributes and SQL commands is going to be an extremely important server responsibility (known as sanitising), whatever the format. Many more well tested sanitising libraries exist for HTML in many more languages/server side technologies than for the relatively niche ProseMirror JSON format used by TipTap.
The TipTap documentation talks about the advantages of the 2 formats here
The ProseMirror JSON format is reasonably verbose, and inspecting both the JSON and HTML side by side, the JSON format seems to require more characters. Obviously it will depend on the rich text itself - how many formatting marks etc, but I would not worry about this in terms of server storage. Given you are likely using Vue or React with TipTap, it is likely you can set up an editor and have 2 seperate divs displaying the output side by side. You can see the 2 formats for the same text by scrolling to the bottom of this page
Editor performance - if you store the data as HTML you can pass it straight to the client. If you store it as JSON you will have to parse it and create the HTML. If you do this client side, the client will have to download and execute a library to perform this task - on NPM this is currently 14kB, so not a big issue.
Both formats will almost certainly send image data to the server as Base64 strings, so no bandwidth will be saved with either format for image files. Base64 is less efficient for DB storage as compared to saving as binary objects, particularly if compression is used, so you could strip these out at the server, but this will have a cost in time spent developing and testing the backend which may well be better spent on other things
I'm working on a project which requires an image database. Many of the images are rather larger than they need to be for 90% of the use cases of the application, however, there are cases where I need the original quality image.Therefore, I cannot reduce the image before uploading/saving to the filesystem (MongoDB database).
My plan was to reduce the size of the image after retrieving the original from MongoDB but before actually sending it from the Node server to the Angular client.
I cannot find an example of this using either Sharp or Jimp (which seem to be leading contenders) and I'm struggling to make either work:
When I use Jimp, I get errors about the constructor.
When I use Sharp it throws errors because it doesn't like the input - it seems to want the
original JPEG or PNG as input, which obviously I don't have as the returned image is an object with a buffer binary field.
Could anyone point me in the right direction about how to achieve this?
In the end, after discussing multiple strategies with my superior, I saved each image with a high definition image (base64) string, and a reduced image string. I was fortunate in that the only time I needed high definition images I was calling a single image by id, therefore I queried the database to return only the image string which was required by the front end.
If anyone does query this or something similar in future I used Sharp on the backend to facilitate the image resizing.
The errors in Jimp and Sharp initially came from attempting to work with the binary. In the end it was simpler to convert the image to a string from the offset.
In my backend, I have data attributes labeled in camelcase:
customerStats: {
ownedProducts: 100,
usedProducts: 50,
},
My UI code is set up in a way that an array of ["label", data] works best most of the time i.e. most convenient for frontend coding. In my frontend, I need these labels to be in proper english spelling so they can be used as is in the UI:
customerStats: [
["Owned products", 100],
["Used products", 50],
],
My question is about best practices or standards in web development. I have been inconsistent in my past projects where I would convert the data at random places, sometimes client-side, sometimes right on the backend, sometimes converting it one way and then back again because I needed the JSON data structure.
Is there a coding convention how the data should be supplied to the frontend?
Right now all my data is transfered as JSON to the frontend. Is it best practice to convert the data to the form that is need on the frontend or backend? What if I need the JSON attributes to do further calculations right on the client?
Technologies I am using:
Frontend: Javascript / React
Backend: Javascript / Node.js + Java / Java Spring
Is there a coding convention for how to transfer data to the front end
If your front end is JavaScript based, then JSON (Java Script Object Notation) is the simplest form to consume, it is a stringified version of the objects in memory. See this healthy discussion for more information on JSON
Given that the most popular front end development language is JavaScript these days, (see the latest SO Survey on technology) It is very common and widely accepted to use JSON format to transfer data between the back and front end of solutions. The decision to use JSON in non-JavaScript based solutions is influenced by the development and deployment tools that you use, seeing more developers are using JavaScript, most of our tools are engineered to support JavaScript in some capacity.
It is however equally acceptable to use other structured formats, like XML.
JSON is generally more light-weight than XML as there is less provision made to transfer meta-data about the schema. For in-house data streams, it can be redundant to transfer a fully specced XML schema with each data transmission, so JSON is a good choice where the structure of the data is not in question between the sender and receiver.
XML is a more formal format for data transmission, it can include full schema documentation that can allow receivers to utilize the information with little or no additional documentation.
CSV or other custom formats can reduce the bytes sent across the wire, but make it hard to visually inspect the data when you need to, and there is an overhead at both the sending and receiving end to format and parse the data.
Is it best practice to convert the data to the form that is need on the frontend or backend?
The best practice is to reduce the number of times that a data element needs to be converted. Ideally you never have to convert between a label and the data property name... This is also the primary reason for using JSON as the data transfer format.
Because JSON can be natively interpreted in a JavaScript front end, in a JavaScript front end we can essentially reduce conversion to just the server-side boundary where data is serialized/deserialized. There is in effect no conversion in the front end at all.
How to refer to data by the property name and not the visual label
The general accepted convention in this space is to separate the concerns between the data model and the user experience, the view. Importantly the view is the closest layer to the user, it represents a given 'point of view' of the data model.
It is hard to tailor a code solution for OP without any language or code provided for context, in an abstract sense, to apply this concept means to not try and have the data model carry the final information about how the data should be displayed, instead you have another piece of code that provides the information needed to present the data.
In different technologies and platforms we refer to this in different ways but the core concept of separating the Model from the View or Presentation is consistently represented through these design patterns:
Exploring the MVC, MVP, and MVVM design patterns
MVP vs MVC vs MVVM vs VIPER
For OP's specific scenario, this might involve a mapping structure like the following:
customerStatsLabels: {
ownedProducts: "Owned products",
usedProducts: "Used products",
}
If this question is updated with some code around how the UI is constructed I will update this response with something more specific.
NOTE:
In JavaScript, objects are simply arrays of arrays, and as such it is very easy to tweak existing code that is based on arrays, into code based on objects and vice-versa.
Cassandra newbie question. I'm collecting some data from a social networking site using REST calls. So I end up with the data coming back in JSON format.
The JSON is only one of the columns in my table. I'm trying to figure out what the "best practice" is for storing the JSON string.
First I thought of using the map type, but the JSON contains a mix of strings, numerical types, etc. It doesn't seem like I can declare wildcard types for the map key/value. The JSON string can be quite large, probably over 10KB in size. I could potentially store it as a string, but it seems like that would be inefficient. I would assume this is a common task, so I'm sure there are some general guidelines for how to do this.
I know Cassandra has native support for JSON, but from what I understand, that's mostly used when the entire JSON map matches 1-1 with the database schema. That's not the case for me. The schema has a bunch of columns and the JSON string is just a sort of "payload". Is it better to store the JSON string as a blob or as text? BTW, the Cassandra version is 2.1.5.
Any hints appreciated. Thanks in advance.
In the Cassandra Storage engine there's really not a big difference between a blob and a text, since Cassandra stores text as blobs essentially. And yes the "native" JSON support you speak of is only for when your data model matches your JSON model, and it's only in Cassandra 2.2+.
I would store it as a text type, and you shouldn't have to implement anything to compress your JSON data when sending the data (or handle uncompressing). Since Cassandra's Binary Protocol supports doing transport compression. Also make sure your table is storing the data compressed with the same compression algorithm (I suggest using LZ4 since it's the fastest algo implmeneted) to save on doing compression for each read request. Thus if you configure storing the data compressed and use transport compression, you don't even have to implement either yourself.
You didn't say which Client Driver you're using, but here's the documentation on how to setup Transport Compression for Datastax Java Client Driver.
It depends on how to want to query your JSON. There are 3 possible strategies:
Store as a string
Store as a compressed blob
Store as a blob
Option 1 has the advantage of being human readable when you query your data on command line with cqlsh or if you want to debug data directly live. The drawback is the size of this JSON column (10k)
Option 2 has the advantage to keep the JSON payload small because text elements have a pretty decent compression ration. Drawbacks are: a. you need to take care of compression/decompression client side and b. it's not human readable directly
Option 3 has drawbacks of option 1 (size) and 2 (not human readable)