I am fairly new to Haskell and Yesod so I've been looking through many tutorials and documentation. I found another StackOverflow about parsing url queries: Lookup query parameters in Yesod
That said, the solution there is not working for arrays of parameters in these url queries.
For example:
http://localhost:3000/models/product?field[]=test1&field[]=test2&field[]=test3&domain[]=["id","name",True]&limit=25
As far as I can tell, the /models/product can be handled by defining a route, and I am able to even get the domain[] parameter. However if I do a lookup on field[] I can only get the first value test1 and I have no way to get the rest. Additionally, assuming that I have no foreknowledge of the size of this array, how would I accomplish this?
Thanks!
I solved this via some hack code, but I'm sure there's probably a better solution out there. Just posting this for other's reference:
getProductR :: Handler Value
getProductR = do
getParameters <- reqGetParams <$> getRequest
let fields = [ b | (a,b) <- getParameters , a == "field[]"]
let domains = cleanseInput getParameters "domain[]"
let limit = Import.lookup (Import.pack "limit") getParameters
return $ object [
(Import.pack "fields") .= fields,
(Import.pack "domains") .= domains,
(Import.pack "limit") .= limit
]
cleanseInput params key = [ Data.List.filter (/= "") [Data.Text.filter (/= ',' ) x | x<-(Data.Text.splitOn "\"" (Data.Text.tail (Data.Text.init b)))] | (a,b) <- params , a == key]
To my knowledge, there isn't an HTTP standard way of handling arrays in query parameters. Several tools, like Ruby on Rails or PHP, have their own conventions for expressing arrays in query parameters.
Yesod doesn't have its own format for arrays (or dictionaries, or other data structures) in query parameters, so I don't think there's any official API to handle this.
If you can, I would recommend doing a POST request with a JSON body for this case. If it must be a GET request, you could also do a GET with JSON in the query parameters and parse that. This allows you to avoid writing custom parser code, leverages the power of existing aeson instances, will be more portable across HTTP clients like Android/iOS/the web. This advice is fairly non-standard, admittedly, but it's worked great for me in practice.
Related
How can I get the label of an equation? I'm attempting to reprocess an equation with a label, but I have to delete the label from MathJax.Extension["TeX/AMSmath"].labels first, for which the label must be known...
I know I can scan through the source text for the label MathJax.Hub.getAllJax("mathDiv")[0}.SourceElement().find("\label(") (...), but this seems needlessly complicated. Is there a better way?
There's no built-in API for this.
If you don't need to keep labels, then the reset in the comment above is probably the best way to go about it:
MathJax.Extension["TeX/AMSmath"].labels = {}
A quick and dirty way to get the IDs is to leverage the fact that they end up in the output. So you can just get all the IDs in the output, e.g.,
const math = MathJax.Hub.getAllJax()[0];
const nodesWithIds = document.getElementById(math.root.inputID).previousSibling.querySelectorAll('[id]');
const ids = [];
for (node of nodesWithIds) ids.push(node.id);
A cleaner and perhaps conceptually easier way would be to leverage MathML (which is essentially the internal format): the \label{} always ends up on an mlabeledtr. The trouble is that you'd have to re-parse that, e.g.,
const temp = document.createElement('span');
temp.innerHTML = math.root.toMathML();
const nodesWithIds = temp.querySelectorAll('mlabeledtr [id]');
const ids = [];
for (node of nodesWithIds) ids.push(node.id);
This will make sure the array only has relevant IDs in them (and the contents of the nodes should correspond to \label{}.
I suppose with helper libraries it might be easier to dive into the math.root object directly and look for IDs recursively (in its data key).
EDIT: For those with similar ailments, I found this is related to the "Extensible Records Problem", something I will personally research more into.
EDIT2: I have started to solve this (weeks later now) by being pretty explicit about data types, and having multiple data types per semantic unit of data. For example, if the database holds an X, my code has an XAction for representing things I want to do with an X, and XResponse for relaying Xs to an http client. And then I need to build the supporting code for shuttling bits between instances. Not ideal, but, I like that it's explicit, and hopefully when my models crystallize, it shouldn't really need much up keep, and should be very reliable.
I'm not sure what the correct level of abstraction is for tackling this problem (ie records? or Yesod?) So I'll just lay out the simple case.
Simple Case / TL;DR
I want to decode a request body into a type
data Comment = Comment {userid :: ..., comment :: ...}
but actually I don't want the request body to contain userid, the server will supply that based on their Auth Headers, (or wherever I want to get data to default fill a field).
So they actually pass me something like:
data SimpleComment = SimpleComment {comment :: ...} deriving (Generic, FromJSON)
And I turn it into a Comment. But maintaining both nearly-identical types simultaneously is a hassle, and not DRY.
How do I solve this problem?
Details on Problem
I have a record type:
data Comment = Comment {userid :: ..., comment :: ...}
I have a POST route:
postCommentR :: Handler Value
postCommentR = do
c <- requireJsonBody :: (Handler Comment)
insertedComment <- runDB ...
returnJson insertedComment
Notice that the Route requires that the user supply their userid (in the Comment type, which is at least redundant since their id is associated with their auth headers. At worst, it means I need to check that users are adding their own id, or throwing away their supplied id, in which case why did they supply it in the first case.
So, I want a record type that's Comment minus userid, but I don't know how to do that intelligently.
My Current (awful but working) Solution
So I made a custom type with derived FromJSON (for the request body) which is almost completely redundant with the Comment type.
data SimpleComment = SimpleComment {comment :: ...} deriving (Generic, FromJSON)
Then my new route needs to decode the request body according to this, and then merge a SimpleComment with a userid field to make it a Comment:
postComment2R :: Handler Value
postComment2R = do
c <- requireJsonBody :: (Handler SimpleComment)
(uid, _) requireAuthPair
insertedComment <- runDB $ insertEntity (Comment { commentUserid = uid
, commentComment = comment c})
returnJson ...
Talk about boilerplate. And my use case is more complex than this simple Comment type.
If it factors in, you might be able to tell, I'm using the Yesod Scaffolding.
What I usually do to get a type minus a field is just to have a function which take that field and return the type. In your case you just need to declare an JSON instance for UserId -> Comment. Ok it doesn't seem natural and you have to go it manually but it actually works really well, especially as there is only one field of type UserId in Comment.
A solution I like is to use a wrapper for things that come from/go to the DB:
data Authenticated a = Authenticated
{ uid :: Uid
, thing :: a
} deriving (Show)
Then you can have Comment be just SimpleComment and turn it into an Authenticated Comment once you know the user id.
I'm also looking for a nice way to solve this. :-)
What I usually do in my code is to operate directly on the Aeson's type Value. This is some of the sample code taken from my current project:
import qualified Data.HashMap.Strict as HM
removeKey :: Text -> Value -> Value
removeKey key (Object xs) = Object $ HM.delete key xs
removeKey _ ys = ys
I directly operate on the value Object and remove the particular key present in the javascript object.
And in the Yesod handler code, I do this processing:
myHandler :: Handler RepJson
myHandler = do
userId <- insert $ User "sibi" 23
guser <- getJuser user
let guser' = removeKey "someId" $ toJSON guser
return $ repJson $ object [ "details" .= guser' ]
In some cases, I actually want to add some specific key to the outgoing JSON object. For those, I have specific helper functions defined which operate on the type Value. While this is not perfect, it has been helping me to avoid a lot of boilerplate code.
I used a code, generated from slick code generator.
My table has more than 22 columns, hence it uses HList
It generates 1 type and 1 function:
type AccountRow
def AccountRow(uuid: java.util.UUID, providerid: String, email: Option[String], ...):AccountRow
How do I write compiled insert code from generated code?
I tried this:
val insertAccountQueryCompiled = {
def q(uuid:Rep[UUID], providerId:Rep[String], email:Rep[Option[String]], ...) = Account += AccountRow(uuid, providerId, email, ...)
Compiled(q _)
}
I need to convert Rep[T] to T for AccountRow function to work. How do I do that?
Thank you
;TLDR; Not possible
Explanation
There are two levels of abstraction in Slick: Querys and DBIOActions.
When you're dealing with Querys, you have to access your schema definitions, and rows, Reps and, basically, it's very constrained as it's the closest level of abstraction to the actual DB you're using. A Rep refers to an hypothetical value in the database, not in your program.
Then you have DBIOActions, which are the next level... not just some definition of a query, but the execution of it. You usually get DBIOActions when getting information out of a query, like with the result method or (TADAN!) when inserting rows.
Inserts and Updates are not queries and so what you're trying to do is not possible. You're dealing with DBIOAction (the += method), and Query stuff (the Rep types). The only way to get a Rep inside a DBIOAction is by executing a Query and obtaining a DBIOAction and then composing both Actions using flatMap or for comprehensions (which is the same).
I have fairly deep urls with IDs and I want to see if I can convert them into something nicer looking. I tried looking into how Slugs are done for Yesod Blog (https://github.com/yesodweb/yesod/wiki/Slugs) but not sure if I know how to translate that to what I am looking for here.
Suppose let's say I want to display Top Fiction Books, I have a resource that looks like this:
/topbooks/bookcategory/#BookCategoryId
If I go to /topbooks/bookcategory/1 I may get Fiction books, If I got to /topbooks/bookcategory/2 I may get Non-fiction, etc.
All my handlers use the #BookCategoryId input parameter in the database queries to get the appropriate records.
Ideally I would like to create a url that looks like: /topbooks/fiction, /topbooks/non-fictionetc. If I create my route as /topbooks/#Text, I can pattern match the string and return a Key back. However, I will have to manually transform it in every handler using #BookCategoryId. Note that the IDs are used as Foreign keys so it makes a bit cumbersome to rely on getBy like how it is done in Slug example.
So I am wondering if there is a better way to do it: Is it possible to define a custom type similar to Slug but instead of just converting values to/from Text / String, actually output IDs? That way I can just use the parameter directly in my queries.
Update:
To clarify given Michael's comment:
I understand we cannot get the IDs without doing a database lookup. In fact for this example, I am ok hard coding the look-up mechanism. I was just trying to see if the PathPiece mechanism will somehow simplify the conversion process.
For example, if something like this worked then it will be fine but of course I will get a type error since I am trying to return a Key when the compiler is expecting BookCategories.
data BookCategories = FICTION | NONFICTION
instance PathPiece BookCategories where
toPathPiece (FICTION) = T.pack "fiction"
toPathPiece (NONFICTION) = T.pack "nonfiction"
fromPathPiece s =
let ups = map toUpper $ T.unpack s
in
case reads ups of
[(FICTION, "")] -> Just $ Key $ PersistInt64 1
[(NONFICTION, "")] -> Just $ Key $ PersistInt64
[] -> Nothing
otherwise -> Nothing
Of course I could just return Just FICTION and unwrap it in my handler. This is not conceptually very different from actually pattern matching on Text directly with a function with a signature Text -> BookCategoryId.
getBookCategoryR :: BookCategoryId -> Handler Html
getBookCategoryR bcId = do
-- Normal use case when IDs are used in the URL
books <- runDB $ selectList [ModelBookCategory ==. bcId] []
If I swtich to Text input
getBookCategoryR :: Text -> Handler Html
getBookCategoryR bc = do
bcId = convertToId (bc) -- This is the line I am trying to avoid everywhere
books <- runDB $ selectList [ModelBookCategory ==. bcId] []
The one line conversion code is what I am trying to avoid. PathPiece has been handling it nicely for id-based-urls and kept the code clean. If there was a way to get Ids returned through some Type magic then it will be great. With limited knowledge of Haskell, I have no idea if it is even feasible.
Hope my question is clearer now.
Thanks!
No, there's no such way to do that, and the reason is simple: without consulting the database, there's no way to know if foo exists as a slug at all and, if it does, which ID it relates to. You'll always have to perform some database action to convert a slug into an ID.
UPDATE I'm still not certain I understand what you're looking for, but the short answer regarding PathPiece is that it only works on pure conversions, nothing which has side effects. If you're looking to write a function like Text -> Handler BookCategoryId, you can certainly do so. And if you really wanted to, you could even abstract this with a typeclass, though I'm not sure if you'll gain anything.
This may be barking up the wrong tree, but here's a short idea that might inspire you a bit: you could creating different newtype wrappers for each textual slug field, and then create a typeclass to convert a textual slug field into the appropriate entity, e.g.:
newtype BookCatSlug = BookCatSlug Text
deriving PathPiece
BookCategory
slug BookCatSlug
title Text
...
UniqueBookCat slug
class Slug slug where
type SlugEntity slug
lookupSlug :: slug -> YesodDB App (Maybe (Entity (SlugEntity slug)))
instance Slug BookCatSlug where
type SlugEntity BookCatSlug = BookCategory
lookupSlug = getBy . UniqueBookCat
lookupSlug404 slug = runDB (lookupSlug slug) >>= maybe notFound return
myHandler slug = do
Entity bookCatId bookCat <- lookupSlug404 slug
Something along these lines should work, but I'm not sure if the "type magic" is worthwhile, since having a helper function and manually passing in the appropriate Unique constructor would be almost as easy for the call site and result in much simpler error messages.
I am quite new to the FuelPHP framework. Right now I'm implementing an "autocomplete" for a list of locations.
My code looks like this:
public function action_search($term=null){
$clean_query = Security::clean($term);
$data["locations"] = array();
if ($clean_query != "") {
$data["locations"] = Model_Orm_Location::query()
->where("title", "like", $clean_query."%")
->get();
}
$response = Response::forge(View::forge("location/search", $data));
$response->set_header("Content-Type","application/json");
return $response;
}
As you can see, I'm concatenating a LIKE statement and it sort of feels bad to me. Is this code safe against SQL injections ? If yes, then is it because:
Security::clean will remove all mess;
where() in the ORM query will do the filtering?
Looking at the implementation of Security::clean in the source code of core/class/security.php, in your case the applied filters depend the configuration security.input_filter, which is empty by default. So no filter is applied.
But when you dig deep into the database abstraction, you will see, that when the query is compiled just before execution, the query builder will apply quote on the value that was supplied in the where condition, which will then apply escape on string values. The implementations of that escape method depend on the DBMS connection:
mysql_real_escape_string for mysql,
mysqli::real_escape_string for mysqli, and
PDO::quote for PDO.
This reflects today’s best practices. So, yes, this is safe against SQL injections.