I've an elastic search cluster with "elasticsearch-head" plugin installed properly.
I want to use the _bulk API to insert several values at once but the specific format of _bulk request body seems to be causing troubles to the plugin.
I use the "Any query" panel to specify my request with the following setup:
query: /_bulk
body:
{ "create" : { "_index" : "eco", "_type" : "usage" } }
{ "index": 1, name" : "my_value" }
I get the following when validating the json and the request won't be executed:
JSON.parse: unexpected non-whitespace character after JSON data at line 2 column 1 of the JSON data
Does anyone knows if elasticsearch-head plugin can handle the _bulk API ?
Or is there something wrong with my request ?
elasticsearch-head plugin does not support _bulk, will cause JSON error...
hope the author can support it as soon as possible, you can submit an issue to this below website.
https://github.com/TravisTX/elasticsearch-head-chrome/issues
You are missing the quotes before name.
{ "index": 1, "name" : "my_value" }
Now 2021 Sep. 27, ES 7.15.0 .
I tried to use the "head" plugin send _bulk , it failed too.
but I found that using Postman or Curl , both are ok.
[PUT] http://localhost:9200/customer/external/_bulk
// Postman > Body > raw > JSON
{"index":{"_id":"1"}}
{"name":"a"}
{"index":{"_id":"2"}}
{"name":"b"}
Notice
Keep every json statement stay at each one line, do not expand them.
You need a blank line at the bottom.
With curl, you need save statements in a file, then #it:
$ curl -XPUT 'http://localhost:9200/customer/external/_bulk' -H "Content-Type:application/json" --data-binary #esSQL.json
Related
Here is one field of the JSON API from accuweather :
data.list[i].rain.3h
It doesn't work when using Javascript (from cloudpebble). This line produces an error :
var rain =1;
rain = data.list[i].rain.3h;
The error is :
[PHONE] pebble-app.js:?: SyntaxError: Unexpected token ILLEGAL
[PHONE] pebble-app.js:?: JS failed.
The problem comes from the "3" because it works for all others field. Do you know a workaround to make it work ?
PS : API description is here (not sure you need it):
http://openweathermap.org/forecast5
Somebody gave me the solution : I can use this syntax to make it work when it starts with a digit :
rain=data.list[i].rain['3h']
I am trying to update a calendar event in Domino Mail Calendar by using the REST data service "calendar" from the latest xpages extlib release "ExtensionLibraryOpenNTF-901v00_13.20150611-0803".
Has anybody done this with a successful return?
Unfortunately I haven't had any success trying to update a calendar event. I was successful getting the list of events, creating events, deleting an event, but to update an event seems to be somehow special. The PDF documentation for the calendar service is quite short on this point. My domino server is accepting all protocols including PUT. I am using the JSON format for my REST calls. The UPDATE I tried as described in the documentation with iCAL as well, but getting the same error.
I am using the Firefox REST Plugin for checking out the service, before I am implementing it.
Im using PUT, with content-type "text/calendar" as well "application/json".
My URL:
http://sitlap55.xyzgmbh.de:8080/mail/padmin.nsf/api/calendar/events/4D750E2B8159D254C1257E9C0066D48D
My Body looks like this, which is the easiest event type, a reminder (but I tried it with meeting and appointment as well):
{"events":[{"UID:"4D750E2B8159D254C1257E9C0066D48D","summary":"Ein Reminder update","start":{"date":"2015-08-13","time":"13:00:00","utc":true}}]}
This is how I return the event by a GET:
{
"href": "/mail/padmin.nsf/api/calendar/events/4D750E2B8159D254C1257E9C0066D48D-Lotus_Notes_Generated",
"id": "4D750E2B8159D254C1257E9C0066D48D-Lotus_Notes_Generated",
"summary": "Ein Reminder",
"start": {
"date": "2015-08-12",
"time": "07:00:00",
"utc": true
},
"class": "public",
"transparency": "transparent",
"sequence": 0,
"x-lotus-summarydataonly": {
"data": "TRUE"
},
"x-lotus-appttype": {
"data": "4"
}
}
This is the error I get:
{
"code": 400,
"text": "Bad Request",
"cserror": 1028,
"message": "Error updating event",
"type": "text",
"data": "com.ibm.domino.calendar.store.StoreException: Error updating event\r\n\tat com.ibm.domino.calendar.dbstore.NotesCalendarStore.updateEvent(NotesCalendarStore.java:229)\r\n\tat ... 65 more\r\n"
}
The attributes in the body I tried a lot of different things, using the id, no id, an UID like in the calendar service doumentation, ...
What am I doing wrong here?
The solution:
Using the PUT method, the URL which worked looks like this:
http://sitlap55.xyzgmbh.de:8080/mail/padmin.nsf/api/calendar/events/4D750E2B8159D254C1257E9C0066D48D-Lotus_Notes_Generated
the BODY looks like this:
{"events":[{"id":"4D750E2B8159D254C1257E9C0066D48D-Lotus_Notes_Generated","summary":"Some Reminder update #6","start":{"date":"2015-08-13","time":"10:00:00","utc":true}}]}
What I figured out was, that the "id" attribute is required ! a bit strange, because it is already in the URL.
I just checked against the documentation for Domino Access Services (DAS) 9.0.1 - and the example they have there actually works.
I tried a few things before that, e.g. if I could PATCH (change individual fields) or PUT (change ALL fields) just specifying the non-system fields. None of these worked. But taking the response from creating (or getting) the event and put it into a PUT request and adjust e.g. start time works fine.
Looking at your example I think the issue is similar as you do not include the end time in the request. But even so you seem to have to include the entire record as it is returned from the service - and please note that the url must end on the ENTIRE id (i.e. including "...-Lotus_Auto_Generated") :-)
/John
Edit:
It seems you don't need to add all fields... But be aware of the side effects of not specifying fields... You need to test it for yourself!
I have a use case in which concurrent update requests make hit my Elasticsearch cluster. In order to make sure that a stale event (one that is made irrelevant by a newer request) does not update a document after a newer event has already reached the cluster, I would like to pass a script with my update requests to compare a field to determine if the incoming request is relevant or not. The request would look like this:
curl -XPOST 'localhost:9200/test/type1/1/_update' -d '
{
"script": " IF ctx._source.user_update_time > my_new_time THEN do not update ELSE proceed with update",
"params": {
"my_new_time": "2014-09-01T17:36:17.517""
},
"doc": {
"name": "new_name"
},
"doc_as_upsert": true
}'
Is the pseudo code I wrote in the "script" field possible in Elasticsearch ? If so, I would love some help with the syntax (groovy, python or javascript).
Any alternative approach suggestions would be greatly appreciated too.
Elasticsearch has built-in optimistic concurrency control (+ here and here).
The way it works is that the Update API allows you two use the version parameter in order to control whether the update should proceed or not.
So taking your above example, the first index/update operation would create a document with version: 1. Then take the case where you have two concurrent requests. Both components A and B will send an updated document, they initially have both retrieved the document with version: 1 and will specify that version in their request (see version=1 in the query string below). Elasticsearch will update the document if and only if the provided version is the same as the current one
Component A and B both send this, but A's request is the first to make it:
curl -XPOST 'localhost:9200/test/type1/1/_update?version=1' -d '{
"doc": {
"name": "new_name"
},
"doc_as_upsert": true
}'
At this point the version of the document will be 2 and B's request will end up with HTTP 409 Conflict, because B assumed the document was still at version 1, even though the version increased in the meantime due to A's request.
B can definitely retrieve the document with the new version (i.e. 2) and try its update again, but this time with ?version=2in the URL. If it's the first one to reach ES, the update will succeed.
I think the script should be like this:
"script": "if(ctx._source.user_update_time > my_new_time) ctx._source.user_update_time=my_new_time;"
or
"script": "ctx._source.user_update_time > my_new_time ? ctx.op=\"none\" : ctx._source.user_update_time=my_new_time"
Is it possible to only retrieve the _source document(s) when I execute a search query with the (official) nodejs-elasticsearch library? According to the documentation, there seems to be a way, sort of:
Use the /{index}/{type}/{id}/_source endpoint to get just the _source field of the document, without any additional content around it. For example:
curl -XGET 'http://localhost:9200/twitter/tweet/1/_source'
And the corresponding API call in the nodejs library is:
client.getSource([params, [callback]])
However, this method only seems to be able to retrieve documents on an ID basis. I need to issue a full search body (with filters and query_strings and whatnot), which this method doesn't support.
I'm running ES 1.4
You can use "fields" for this. See a simplified example below. Go ahead and customize your query as per your requirement:
{
"fields": [
"_source"
],
"query": {
"match_all": {}
}
}
The value of fields _index, _type, _id and _score will always be present in the response of Search API.
I am trying to use a curl command to tag a commit. Is this possible? I went through the create-a-tag-object link from GitHub, but it doesn't work.
Creating a tag is a bit complicated, but if you just follow the API docs you should be fine. Note that the API docs say:
Note that creating a tag object does not create the reference that makes a tag in Git. If you want to create an annotated tag in Git, you have to do this call to create the tag object, and then create the refs/tags/[tag] reference. If you want to create a lightweight tag, you simply have to create the reference - this call would be unnecessary.
So, before you continue with creating a tag, you should know what kind of tag you want to create -- annotated or lightweight. Basically, an annotated tag is the same as a lightweight, but it also contains a message of the tag, info about the author of the tag, and the date and time when the tag was created. A lightweight tag is just a named pointer to a specific commit in your history.
Ok, so, what the API docs basically say is: if you want to create an annotated tag - you will have to make 2 API calls, and if you want to create a lightweight tag - you will have to make just 1 call. So, I'll give an example of creating an annotated tag with the 2 API calls, and if you want to create a lightweight tag - just skip the first API call and go to the second.
To create an annotated tag, you have to:
Step 1
Create a tag object using the tags API. The API docs are a bit unclear here how the parameters should be passed. What's missing is an example of the message that you need to send to the server. So, create a file called tag_object_req.json on your local disk, and put the following JSON document in it:
{
"tag": "v0.0.1",
"object": "c5f8759ffd808d4a57ea36c63960f3e2cc6fcc2b",
"message": "creating a tag",
"tagger": {
"name": "Ivan Zuzak",
"email": "izuzak#gmail.com",
"date": "2012-06-17T14:53:35-07:00"
},
"type": "commit"
}
Obviously, you have to replace the information in the document to reflect your situation. The meaning of the parameters are described in the API docs here.
After you have saved the file, you can make an API call using curl to create the tag object:
curl -v -X POST -d #tag_object_req.json --header "Content-Type:application/json" -u izuzak "https://api.github.com/repos/izuzak/test/git/tags"
So, the -v part will force curl to output all HTTP headers, the -X POST part means that an HTTP POST request must be made, -d #tag_object_req.json specifies which file will be used as the content (body) of the POST request, --header "Content-Type:application/json" specifies the media type of the request (JSON message), and -u izuzak specifies your username for authorization (and curl will ask you for your password when you make the request).
The response you get should be a 201 Created HTTP response, with the JSON message structured as this:
{
"sha": "e6d9fb6b9a13cab11923345e2400d5cf8df97267",
"url": "https://api.github.com/repos/izuzak/test/git/tags/e6d9fb6b9a13cab11923345e2400d5cf8df97267",
"tagger": {
"name": "Ivan Zuzak",
"email": "izuzak#gmail.com",
"date": "2012-06-17T21:53:35Z"
},
"object": {
"sha": "c5f8759ffd808d4a57ea36c63960f3e2cc6fcc2b",
"type": "commit",
"url": "https://api.github.com/repos/izuzak/test/git/commits/c5f8759ffd808d4a57ea36c63960f3e2cc6fcc2b"
},
"tag": "v0.0.1",
"message": "creating a tag"
}
Before continuing, notice the sha attribute of the object you just created (e6d9fb6b9a13cab11923345e2400d5cf8df97267) because you will use this value in the next step.
Step 2
Create a tag reference using the references API. The API docs are much clearer here about what the request should look like. So, you first have to create another file on the disk, named tag_ref_req.json, and put this content inside:
{
"ref": "refs/tags/v0.0.1",
"sha": "e6d9fb6b9a13cab11923345e2400d5cf8df97267"
}
However, notice that the value of the sha in this JSON depends on the type of tag you are creating. If you are creating an annotated tag, the value of the sha is the same value you got from the previous step - the sha of the tag object (e6d9fb6b9a13cab11923345e2400d5cf8df97267). However, if you are creating a lightweight tag, the value of the sha is the sha of the commit object you are tagging with the tag, because you have not created a tag object. In my case, you can see in step 1 that the commit object I am tagging is c5f8759ffd808d4a57ea36c63960f3e2cc6fcc2b, and this will be different in your case of course if you are creating a lightweight tag.
Ok, so after you have created this file and defined the sha and the name of the tag, you can make an API request in a similar way as in the previous step, using curl:
curl -v -X POST -d #tag_ref_req.json --header "Content-Type:application/json" -u izuzak "https://api.github.com/repos/izuzak/test/git/refs"
Notice that we are now making a request to https://api.github.com/repos/izuzak/test/git/refs with the second file as the content.
The response should again be a 201 Created HTTP response, and the body will be a JSON document looking like this:
{
"ref": "refs/tags/v0.0.1",
"url": "https://api.github.com/repos/izuzak/test/git/refs/tags/v0.0.1",
"object": {
"sha": "e6d9fb6b9a13cab11923345e2400d5cf8df97267",
"type": "tag",
"url": "https://api.github.com/repos/izuzak/test/git/tags/e6d9fb6b9a13cab11923345e2400d5cf8df97267"
}
}
And now you can navigate to your project on GitHub and go to "Switch branches/tags" and see your tag there.
Hope this helps!
You can also try the new Releases API
http://developer.github.com/v3/repos/releases/#create-a-release
curl \
--user <username> \
--header "Accept: application/vnd.github.manifold-preview" \
--data "tag_name=mytagname" \
"https://api.github.com/repos/<username>/<repository>/releases"
This is curl will create Releases. (But as ChucK mentioned it may create only lightweight tag, I haven't personally verified this )
curl \
--user <Your Github username> \
--header "Accept: application/vnd.github.manifold-preview" \
--data '{"tag_name": "v1.0.0", "target_commitish": "master", "name": "v1.0.0", "body": "Description of the release", "draft": false, "prerelease": false }' \
"https://api.github.com/repos/<OrganizationName>/<RepoName>/releases" -X POST
Adding this here, incase anybody come looking for this, like me.